Abstract
We explored the possibility of predicting the quality of student answers (error-ridden, vague, partially-correct, and correct) to tutor questions by examining their linguistic patterns in 50 tutoring sessions with expert human tutors. As an alternative to existing computational linguistic methods that focus on domain-dependent content words (e.g., velocity, RAM, speed) in interpreting a student’s response, we focused on function words (e.g., I, you, but) and domain-independent content words (e.g., think, because, guess). Proportional incidence of these word categories in over 6,000 student responses to tutor questions was automatically computed using Linguistic Inquiry and Word Count (LIWC), a computer program for analyzing text. Multiple regression analyses indicated that two parameter models consisting of pronouns (e.g., I, they, those) and discrepant terms (e.g., should, could, would) were effective in predicting the conceptual quality of student responses. Furthermore, the classification accuracy of discriminant functions derived from the domain-independent LIWC features competed with conventional domain-dependent assessment methods. We discuss the possibility of a composite assessment algorithm that focuses on both domain-dependent and domain-independent words for dialogue-based ITSs.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Anderson, J., Corbett, A., Koedinger, K., Pelletier, R.: Cognitive Tutors: Lessons Learned. The Journal of the Learning Sciences 4(2), 167–207 (1995)
VanLehn, K., Lynch, C., Taylor, L., Weinstein, A., Shelby, R., Schulze, K.: Minimally Invasive Tutoring of Complex Physics Problem Solving. In: Cerri, S.A., Gouardéres, G., Paraguaçu, F. (eds.) ITS 2002. LNCS, vol. 2363, pp. 367–376. Springer, Heidelberg (2002)
Koedinger, K., Anderson, J., Hadley, W., Mark, M.: Intelligent Tutoring Goes to School in the Big City. Journal of Artificial Intelligence in Education 8, 30–43 (1997)
Kurup, M., Greer, J.E., McCalla, G.: The Fawlty Article Tutor. In: Frasson, C., McCalla, G.I., Gauthier, G. (eds.) ITS 1992. LNCS, vol. 608, pp. 84–91. Springer, Heidelberg (1992)
Heift, T., Schulze, M.: Error Diagnosis and Error Correction in Computer-Assisted Language Learning. CALICO 20(3), 433–436 (2003)
Graesser, A., Penumatsa, P., Ventura, M., Cai, Z., Hu, X.: Using LSA in AutoTutor: Learning Through Mixed-Initiative Dialogue in Natural Language. In: Landauer, T., McNamara, D., Dennis, S., Kintsch, W. (eds.) Handbook of Latent Semantic Analysis, pp. 243–262. Erlbaum, Mahwah (2007)
Graesser, A., Lu, S., Jackson, G., Mitchell, H., Ventura, M., Olney, A., Louwerse, M.M.: AutoTutor: A Tutor with Dialogue in Natural Language. Behavioral Research Methods, Instruments, and Computers 36, 180–193 (2004)
Landauer, T., McNamara, D., Dennis, S., Kintsch, W. (eds.): Handbook of Latent Semantic Analysis. Erlbaum, Mahwah (2007)
Campbell, R.S., Pennebaker, J.W.: The Secret Life of Pronouns: Flexibility in Writing Style and Physical Health. Psychological Science 14, 60–65 (2003)
D’Mello, S., Dowell, N., Graesser, A.: Cohesion Relationships in Tutorial Dialogue as Predictors of Affective States. In: Dimitrova, V., Mizoguchi, R., du Boulay, B., Graesser, A. (eds.) Proceedings of the 14th International Conference on Artificial Intelligence in Education, pp. 9–16. IOS Press, Amsterdam (2009)
Hancock, J.T., Curry, L., Goorha, S., Woodworth, M.T.: On Lying and Being Lied to: A Linguistic Analysis of Deception. Discourse Processes 45, 1–23 (2008)
Mairesse, F., Walker, M.A., Mehl, M.R., Moore, R.K.: Using Linguistic Cues for the Automatic Recognition of Personality in Conversation and Text. Journal of Artificial Intelligence Research 30, 457–500 (2007)
Pennebaker, J.W., Stone, L.D.: Words of Wisdom: Language Use Over the Life Span. Journal of Personality and Social Psychology 83(2), 291–301 (2003)
Pennebaker, J.W.: Writing About Emotional Experiences as a Therapeutic Process. Psychological Sciences 8, 162–166 (1997)
Pennebaker, J.W., Francis, M.E., Booth, R.J.: Linguistic Inquiry and Word Count (LIWC). Erlbaum, Mahway (2001)
Stiles, W.B.: In Describing Talk: A Taxonomy of Verbal Response Modes. Sage, Newbury Park (1992)
Pennebaker, J.W., King, L.A.: Linguistic Styles: Language Use as an Individual Difference. Journal of Personality and Social Psychology 77, 1296–1312 (1999)
Boroditsky, L.: Does Language Shape Thought?: Mandarin and English Speakers’ Conception of Time. Cognitive Psychology 43, 1–22 (2001)
Whorf, B.: In Language, Thought, and Reality: Selected Writings of Benjamin Lee Whorf. MIT Press, Cambridge (1956)
Zhang, T., Hasegawa-Johnson, M., Levinson, S.E.: Cognitive State Classification in a Spoken Tutorial Dialogue System. Speech Communication 48, 616–632 (2006)
Person, N., Lehman, B., Ozbun, R.: Pedagogical and Motivational Dialogue Moves Used by Expert Tutors. In: 17th Annual Meeting of the Society for Text and Discourse, Glasgow, Scotland (2007)
Groom, C.J., Pennebaker, J.W.: Words. Journal of Research in Personality 36, 615–621 (2002)
Pennebaker, J.W., Lay, T.C.: Language Use and Personality During Crises: Analyses of Mayor Rudolph Giuliani’s Press Conference. Journal of Research in Personality 36, 271–282 (2002)
Kitayama, S.: Interaction Between Affect and Cognition in Word Perception. Journal of Personality and Social Psychology 58(2), 209–217 (1990)
Cohen, J.: A Power Primer. Psychological Bulletin 112(1), 155–159 (1992)
Chipman, P.: An Analysis and Optimization of AutoTutor’s Student Model (Unpublished Master’s Thesis). The University of Memphis, Memphis (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Williams, C., D’Mello, S. (2010). Predicting Student Knowledge Level from Domain-Independent Function and Content Words. In: Aleven, V., Kay, J., Mostow, J. (eds) Intelligent Tutoring Systems. ITS 2010. Lecture Notes in Computer Science, vol 6095. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13437-1_7
Download citation
DOI: https://doi.org/10.1007/978-3-642-13437-1_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13436-4
Online ISBN: 978-3-642-13437-1
eBook Packages: Computer ScienceComputer Science (R0)