Combining Computational Models of Short Essay Grading for Conceptual Physics Problems
The difficulties of grading essays with natural language processing tools are addressed. The present project investigated the effectiveness of combining multiple measures of text similarity to grade essays on conceptual physics problems. Latent semantic analysis (LSA) and a new text similarity metric called Union of Word Neighbors (UWN) were used with other measures to predict expert grades. It appears that the best strategy for grading essays is to use student derived ideal answers and statistical models that accommodate inferences. LSA and the UWN gave near equivalent performance in predicting expert grades when student derived ideal answers served as a comparison for student answers. However, if ideal expert answers are used, explicit symbolic models involving word matching are more suitable to predict expert grades. This study identified some computational constraints on models of natural language processing in intelligent tutoring systems.
KeywordsTarget Word Latent Semantic Analysis Intelligent Tutoring System Cognitive Tutor Student Answer
Unable to display preview. Download preview PDF.
- 6.Graesser, A.C., Lu, S., Jackson, G.T., Mitchell, H., Ventura, M., Olney, A., Louwerse, M.M.: AutoTutor: A tutor with dialogue in natural language. In: Behavioral Research Methods, Instruments, and Computers (in press)Google Scholar
- 7.Graesser, A.C., McNamara, D.S., Louwerse, M.M., Cai, Z.: Coh-Metrix: Analysis of text on cohesion and language. In: Behavioral Research Methods, Instruments, and Computers (in press)Google Scholar
- 8.Graesser, A.C., Person, N., Harter, D., TRG: Teaching tactics and dialog in AutoTutor. International Journal of Artificial Intelligence in Education 12, 257–279 (2001)Google Scholar
- 11.Hewitt, P.G.: Conceptual physics, 8th edn. Addison-Wesley, Reading (1998)Google Scholar
- 16.Wiemer-Hastings, P., Wiemer-Hastings, K., Graesser, A.: Improving an intelligent tutor’s comprehension of students with Latent Semantic Analysis. In: Lajoie, S.P., Vivet, M. (eds.) Artificial Intelligence in Education, pp. 535–542. IOS Press, Amsterdam (1999)Google Scholar