Abstract
Identifying effective tutorial dialogue strategies is a key issue for intelligent tutoring systems research. Human-human tutoring offers a valuable model for identifying effective tutorial strategies, but extracting them is a challenge because of the richness of human dialogue. This paper addresses that challenge through a machine learning approach that 1) learns tutorial strategies from a corpus of human tutoring, and 2) identifies the statistical relationships between student outcomes and the learned strategies. We have applied hidden Markov modeling to a corpus of annotated task-oriented tutorial dialogue to learn one model for each of two effective human tutors. We have identified significant correlations between the automatically extracted tutoring modes and student learning outcomes. This work has direct applications in authoring data-driven tutorial dialogue system behavior and in investigating the effectiveness of human tutoring.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Lepper, M.R., Woolverton, M., Mumme, D.L., et al.: Motivational Techniques of Expert Human Tutors: Lessons for the Design of Computer-Based Tutors. In: Lajoie, S.P., Derry, S.J. (eds.) Computers as Cognitive Tools. Lawrence Erlbaum Associates, Hillsdale (1993)
Fox, B.A.: The Human Tutorial Dialogue Project. Lawrence Erlbaum Associates, Hillsdale (1993)
Graesser, A.C., Person, N., Magliano, J.: Collaborative Dialogue Patterns in Naturalistic One-to-One Tutoring. Jl. of Applied Cog. Psych. 9, 269–306 (2004)
Chi, M.T.H., Siler, S.A., Jeong, H., et al.: Learning from Human Tutoring. Cog. Sci. 25, 471–533 (2001)
Ohlsson, S., Di Eugenio, B., Chow, B., Fossati, D., Lu, X., Kershaw, T.C.: Beyond the Code-and-Count Analysis of Tutoring Dialogues. In: 13th International Conference on AI in Education, pp. 349–356 (2007)
Chi, M., Jordan, P., VanLehn, K., Litman, D.: To Elicit Or to Tell: Does it Matter? In: 14th International Conference on AI in Education, pp. 197–204 (2009)
Litman, D., Forbes-Riley, K.: Correlations between Dialogue Acts and Learning in Spoken Tutoring Dialogues. Nat. Lang. Eng. 12, 161–176 (2006)
Boyer, K.E., Ha, E.Y., Wallis, M.D., Phillips, R., Vouk, M.A., Lester, J.C.: Discovering Tutorial Dialogue Strategies with Hidden Markov Models. In: 14th International Conference on AI in Education, pp. 141–148 (2009)
Chi, M., Jordan, P., VanLehn, K., Hall, M.: Reinforcement Learning-Based Feature Selection for Developing Pedagogically Effective Tutorial Dialogue Tactics. In: 1st International Conference on Educational Data Mining, pp. 258–265 (2008)
Boyer, K.E., Phillips, R., Wallis, M.D., Vouk, M.A., Lester, J.C.: Balancing Cognitive and Motivational Scaffolding in Tutorial Dialogue. In: Woolf, B.P., Aïmeur, E., Nkambou, R., Lajoie, S. (eds.) ITS 2008. LNCS, vol. 5091, pp. 239–249. Springer, Heidelberg (2008)
Forbes-Riley, K., Litman, D.: Adapting to Student Uncertainty Improves Tutoring Dialogues. In: 14th International Conference on AI and Education, pp. 33–40 (2009)
Evens, M., Michael, J.: One-on-One Tutoring by Humans and Computers. Lawrence Erlbaum Associates, Mahwah (2006)
Cade, W., Copeland, J., Person, N., D’Mello, S.: Dialog Modes in Expert Tutoring. In: Woolf, B.P., Aïmeur, E., Nkambou, R., Lajoie, S. (eds.) ITS 2008. LNCS, vol. 5091, pp. 470–479. Springer, Heidelberg (2008)
Rosé, C.P., Moore, J.D., VanLehn, K., et al.: A Comparative Evaluation of Socratic Versus Didactic Tutoring. #LRDC-BEE-1 (2000)
Graesser, A.C., Person, N.K., Magliano, J.P.: Collaborative Dialogue Patterns in Naturalistic One-to-One Tutoring. Applied Cog. Psych. 9, 495–522 (1995)
D’Mello, S., Taylor, R.S., Graesser, A.: Monitoring Affective Trajectories during Complex Learning. In: 29th Annual Cognitive Science Society, pp. 203–208 (2007)
Chi, M.T.H., Roy, M., Hausmann, R.G.M.: Observing Tutorial Dialogues Collaboratively: Insights about Human Tutoring Effectiveness from Vicarious Learning. Cog. Sci. 32, 301–341 (2008)
Core, M.G., Moore, J.D., Zinn, C.: The Role of Initiative in Tutorial Dialogue. In: 10th Conference of the European Chapter of the Association for Computational Linguistics, pp. 67–74 (2003)
Katz, S., Allbritton, D., Connelly, J.: Going Beyond the Problem Given: How Human Tutors use Post-Solution Discussions to Support Transfer. International Journal of Artificial Intelligence in Education 13, 79–116 (2003)
Rosé, C., Bhembe, D., Siler, S., Srivastava, R., VanLehn, K.: The Role of Why Questions in Effective Human Tutoring. In: International Conference on AI in Education, pp. 55–62 (2003)
Litman, D., Moore, J., Dzikovska, M., Farrow, E.: Using Natural Language Processing to Analyze Tutorial Dialogue Corpora Across Domains and Modalities. In: 14th International Conference on AI in Education, pp. 149–156 (2009)
Tetreault, J.R., Litman, D.J.: A Reinforcement Learning Approach to Evaluating State Representations in Spoken Dialogue Systems. Speech Comm. 50, 683–696 (2008)
Beal, C., Mitra, S., Cohen, P.R.: Modeling Learning Patterns of Students with a Tutoring System using Hidden Markov Models. In: 13th International Conference on AI in Education, pp. 238–245 (2007)
Jeong, H., Gupta, A., Roscoe, R., Wagster, J., Biswas, G., Schwartz, D.: Using Hidden Markov Models to Characterize Student Behaviors in Learning-by-Teaching Environments. In: Woolf, B.P., Aïmeur, E., Nkambou, R., Lajoie, S. (eds.) ITS 2008. LNCS, vol. 5091, pp. 614–625. Springer, Heidelberg (2008)
Soller, A., Stevens, R.: Applications of Stochastic Analyses for Collaborative Learning and Cognitive Assessment. In: Hancock, G.R., Samuelsen, K.M. (eds.) Advances in Latent Variable Mixture Models, pp. 217–253. Information Age Publishing (2007)
Boyer, K.E., Vouk, M.A., Lester, J.C.: The Influence of Learner Characteristics on Task-Oriented Tutorial Dialogue. In: Proceedings of the 13th International Conference on AI in Education, pp. 365–372 (2007)
Boyer, K.E., Phillips, R., Wallis, M.D., Vouk, M.A., Lester, J.C.: The Impact of Instructor Initiative on Student Learning through Assisted Problem Solving. In: 40th Tech. Symposium on Computer Science Education, pp. 14–18 (2009)
Boyer, K.E., Phillips, R., Ha, E.Y., Wallis, M.D., Vouk, M.A., Lester, J.C.: Modeling Dialogue Structure with Adjacency Pair Analysis and Hidden Markov Models. In: NAACL HLT, Short Papers, pp. 49–52 (2009)
Landis, J.R., Koch, G.: The Measurement of Observer Agreement for Categorical Data. Biometrics 33, 159–174 (1977)
Rabiner, L.R.: A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceedings of the IEEE 77, 257–286 (1989)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Boyer, K.E. et al. (2010). Characterizing the Effectiveness of Tutorial Dialogue with Hidden Markov Models. In: Aleven, V., Kay, J., Mostow, J. (eds) Intelligent Tutoring Systems. ITS 2010. Lecture Notes in Computer Science, vol 6094. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13388-6_10
Download citation
DOI: https://doi.org/10.1007/978-3-642-13388-6_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13387-9
Online ISBN: 978-3-642-13388-6
eBook Packages: Computer ScienceComputer Science (R0)