International Journal of Innovative Research in Computer and Communication Engineering

International Journal of Innovative Research in Computer and Communication Engineering

Menu

ISSN ONLINE(2320-9801) PRINT (2320-9798)

All submissions of the EM system will be redirected to Online Manuscript Submission System. Authors are requested to submit articles directly to Online Manuscript Submission System of respective journal.

An Approach to Evaluate Subjective Questions for Online Examination System

Sheeba Praveen
Assistant Professor, Dept. CSE, Integral University, Lucknow ,U.P, India
Related article at Pubmed, Scholar Google

Visit for more related articles at International Journal of Innovative Research in Computer and Communication Engineering

International Journal of Innovative Research in Computer and Communication Engineering

Abstract

In recent years we have seen that a number of governments, semi government examinations are gone online, for example [IBPS Common Written Examination (CWE)]. This system or any other such systems are advantages in terms of saving resources. However we have observed that these systems cater only multiple choice questions and there is no provision to extend these systems to subjective questions. Our objective is to design an algorithm for the automatic evaluation of single sentence descriptive answer. The paper presents an approach to check the degree of learning of the student/learner, by evaluating their descriptive exam answer sheets. By representing the descriptive answer in the form of graph and comparing it with standard answer are the key steps in our approach.

Keywords
Subjective answer, Preprocessing,Post Tagging, parse Tree, Grading System, Inference System, Knowledge Representation System, graphical representation, similarity measures, subjective evaluation, word Net

INTRODUCTION
For ever-growing population governments strive to provide a good education system, proper drinking water & food and other amenities for the betterment of their people.
For countries like India, ever-growing population and poor infrastructure hampers quality education system. The number of students appearing for board examination in state of Utter Pradesh itself [1]. It can be imagined the amount of pressure that is subjected on education system and teachers to evaluate the number of answer copies. Further according to the figure [2] as per the Annual Status of Education Report (ASER) 2012, 96.5% of all rural children between the ages of 6-14 were enrolled in school. This is the fourth annual survey to report enrollment above 96%. 83% of all rural 15-16 year olds were enrolled in school. However, going forward, India will need to focus more on quality. Gross enrollment at the tertiary level has crossed 20% (as per an Ernst & Young Report cited in Jan 2013 in Education News/minglebox.com) As per the latest (2013) report issued by the All India Council of Technical Education (AICTE), there are more than 3524 diploma and post-diploma offering institutions in the country with an annual intake capacity of over 1.2 million. The AICTE also reported 3495 degree-granting engineering colleges in India with an annual student intake capacity of over 1.76 million with actual enrollment crossing 1.2 million. Capacity for Management Education crossed 385000, and post graduate degree slots in Computer Science crossed 100,000. Pharmacy slots reached over 121,000.Total annual intake capacity for technical diplomas and degrees exceeded 3.4 million in 2012. According to the University Grants Commission (UGC) total enrollment in Science, Medicine, Agriculture and Engineering crossed 6.5 million in 2010. Charu Sudan Kasturi reported in the Hindustan Times (New Delhi, January 10, 2011) that the number of women choosing engineering has more than doubled since 2001.We propose an Approach to Evaluate Subjective Questions for Online Examination System.

RELATED WORK
In recent years we have seen that a number of governments, semi government examinations are gone online, for example [IBPS Common Written Examination (CWE)]. This system or any other such systems are advantages in terms of saving resources. However we have observed that these systems cater only multiple choice questions and there is no provision to extend these systems to subjective questions. We have studied a number of problems for which these systems can not be used in board examination, university examination where student writes subjective answers, these problems are as follows:
• Students coming from various educational backgrounds have difficulties like grammatical mistakes, formation of correct and complete sentences.
• Natural language is very rich in expressibility; therefore the same meaning can be conveyed in different forms using different set of word.
• Similarly for evaluating answers for the same question different teachers may have different views for example: Biology (biology is always full of exception)
• Use of formulas and mathematical expressions are other classes of difficulties for these systems.
• The multiple choice online examination systems so far have been design in such a way that a continuous link to the main server is to be maintain, in situations where link gets disturbed or bracken (temporally).
So far we have reviewed a lot of literature related to above field and we have found a) questions like who, why, how, when, can be asked by the system (..) an answers given can be evaluated) and nobody have tried to evaluate subjective answers given by various people c) We have found that there is no standardized graving system to award marks/grades for the answer given d)We have found some literature on generating simple questions for mathematical problems and formula.Many architectures and features have been proposed for descriptive answer evaluation. The approaches are mainly based on keyword matching, sequence matching and quantitative analysis , but semantic analysis of descriptive answer is still an open problem. Considering the general structure of text analysis in natural language processing, most of the work has been done for morphological and syntactic analysis [7], [8], [9], [10], but semantic, pragmatic and discourse are still being explored.
Online tools that support managing of online assessments such as Moodle and Zoho are based on string matching technique for short answers but long answer evaluation is still handled manually by most systems [9],[10]. Features which are available currently in online assessment are [9], [10]:
• Question paper setting
• Online Evaluation of objective type questions
• Question bank editor
• Spell checker
• Grammar checker
• Report generation of result
• Descriptive answer evaluation is still an open problem. [6]
A. Benefits of Online Subjective Examination System
• Can be adopted and implemented quickly
Reduce/eliminate faculty time demands in instrument development and grading (i.e., relatively low “frontloading” and “backloading” effort)
• Objective scoring
• Provide for externality of measurement (i.e., external validity is the degree to which the conclusions in your study would hold for other persons in other places and at other times – ability to generalize the results beyond the original test group.)
• Provide norm reference group(s) comparison often required by mandates.
• May be beneficial or required in instances where state or national standards exist for the discipline or profession.
• Very valuable for benchmarking and cross-institutional comparison studies
B. Issues in Online Subjective Examination System
• Main issue of subjective examination is the explanation, example and description given by students which may have different words i.e. synonym used to frame the sentence but they must have the same meaning and point’s necessary for the answer to be correct.
• Second issue is the size or the length of the sentences in an answer. Indirectly the answer vary person to person, which requires huge efforts to put them into the category according to context.

PROPOSED MODEL FOR ONLINE SUBJECTIVE EXAMINATION SYSTEM
In this work we will propose solution of aforementioned problems, precisely our system will solve the problem of deducing knowledge represented by partially or grammatically incorrect sentences, we will interpret the meaning conveyed by the student in different forms and sentences, propose a normalized strategy for grading the answers, ways to interpret the mathematical formulas and expression however our system will be limited to non mathematical subjects only. The proposed architecture of the system is:
In order to calculate the function point from UML diagrams, we use the sequence diagrams and class diagrams. Because these diagrams includes the information about all functions and data manipulated in the system.
i. Pre-processing: It is normalization of the text includes throwing unwanted words, stemming etc. However, this is domain dependent process and manual rules are, created to suit ones need. All relational operators are separated by white space. The preprocessing is dependent upon the parsing algorithm’s ability to recognize the sentence tags in desired format. Split compound sentences to simple sentences.
ii. Part-of-Speech Tagging: It assigning a part-of-speech to each word in a sentence, useful in information retrieval, word sense disambiguation and it helps in parsing by assigning unique tags to each word thus reducing number of parses . We use POS tagger that uses a model trained on English data from the Wall Street Journal and the Brown corpus. The latest model created achieved more that 96% accuracy on unseen data. [3].
iii. Grading System: The Indian Institutes of Technology are under control of the Government of India and therefor have strict rules for grades. Depending on the course the evaluation is based on participation in class, attendance, quiz, exam and/or paper. Continuous evaluation is done by course instructors. The Evaluation System of IIT Madras is the Cumulative Grade Point Average with a scale from 0 to 10 which is converted to letters [4].
iv. Parse tree: “A natural language parser takes a sentence as input and determines the labeled syntactic tree structure that corresponds to the interpretation of the sentence” [Ratnaparkhi, A. (1998). Maximum Entropy Models for Natural Language Ambiguity Resolution. Ph.D. thesis, University of Pennsylvania.]. We use shift reduce style parser based on Adwait Ratnaparki's 1998 thesis [Ratnaparkhi, A. (1998). Maximum Entropy Models for Natural Language Ambiguity Resolution. Ph.D. thesis, University of Pennsylvania.] with the help of tool based on Penn Treebank to generate the parse trees of each statement[5].

CONCLUSION
Automatic Evaluation of Subjective Questions for Online Examination System would be beneficial for the universities, schools and colleges for academic purpose by providing ease to faculties and the examination evaluation cell. So this Proposed model will convert the student’s subjective answer and standard answer into its graphical form and then, to apply some of the similarity measures such as string match, wordNet and spreading process for the calculation of similarity score. This proposed model provides a solution for the automation of subjective answer evaluation process.

FUTURE WORK
More analysis would be required for similarity matching. Derive a method to check the domain ontology of two phrases .Find out an appropriate technique to minimize the gap between human and computer assessor. Upgrade the algorithm for the evaluation of multiple sentences answer.

Figures at a glance
Figure 1
Figure 1

References