Advertisement

AutoTutor: A tutor with dialogue in natural language

  • Arthur C. GraesserEmail author
  • Shulan Lu
  • George Tanner Jackson
  • Heather Hite Mitchell
  • Mathew Ventura
  • Andrew Olney
  • Max M. Louwerse
Article

Abstract

AutoTutor is a learning environment that tutors students by holding a conversation in natural language. AutoTutor has been developed for Newtonian qualitative physics and computer literacy. Its design was inspired by explanation-based constructivist theories of learning, intelligent tutoring systems that adaptively respond to student knowledge, and empirical research on dialogue patterns in tutorial discourse. AutoTutor presents challenging problems (formulated as questions) from a curriculum script and then engages in mixed initiative dialogue that guides the student in building an answer. It provides the student with positive, neutral, or negative feedback on the student’s typed responses, pumps the student for more information, prompts the student to fill in missing words, gives hints, fills in missing information with assertions, identifies and corrects erroneous ideas, answers the student’s questions, and summarizes answers. AutoTutor has produced learning gains of approximately .70 sigma for deep levels of comprehension.

Keywords

Latent Semantic Analysis Computer Literacy Intelligent Tutoring System World Knowledge Discourse Process 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. Aleven, V., &Koedinger, K. R. (2002). An effective metacognitive strategy: Learning by doing and explaining with a computer-based cognitive tutor. Cognitive Science,26,147–179.CrossRefGoogle Scholar
  2. Allen, J. (1995).Natural language understanding. Redwood City, CA: Benjamin/Cummings.Google Scholar
  3. Anderson, J. R., Corbett, A. T., Koedinger, K. R., &Pelletier, R. (1995). Cognitive tutors: Lessons learned.Journal of the Learning Sciences,4,167–207.CrossRefGoogle Scholar
  4. Atkinson, R. K. (2002). Optimizing learning from examples using animated pedagogical agents.Journal of Educational Psychology,94,416–427.CrossRefGoogle Scholar
  5. Bloom, B. S. (Ed.) (1956).Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York: Longmans, Green.Google Scholar
  6. Burgess, C., Livesay, K., &Lund, K. (1998). Explorations in context space: Words, sentences, and discourse.Discourse Processes,25, 211–257.CrossRefGoogle Scholar
  7. Cassell, J., &Thorisson, K. (1999). The power of a nod and a glance: Envelope vs. emotional feedback in animated conversational agents.Applied Artificial Intelligence,13,519–538.CrossRefGoogle Scholar
  8. Chi, M. T. H., de Leeuw, N., Chiu, M., &LaVancher, C. (1994). Eliciting self-explanations improves understanding.Cognitive Science,18,439–477.Google Scholar
  9. Chi, M. T. H., Siler, S. A., Jeong, H., Yamauchi, T., &Hausmann, R. G. (2001). Learning from human tutoring.Cognitive Science,25,471–533.CrossRefGoogle Scholar
  10. Cohen, P. A., Kulik, J. A., &Kulik, C. C. (1982). Educational outcomes of tutoring: A meta-analysis of findings.American Educational Research Journal,19,237–248.Google Scholar
  11. Colby, K. M., Weber, S., &Hilf, F. D. (1971). Artificial paranoia.Artificial Intelligence,2, 1–25.CrossRefGoogle Scholar
  12. Collins, A., Brown, J. S., &Newman, S. E. (1989). Cognitive apprenticeship: Teaching the craft of reading, writing, and mathematics. In L. B. Resnick (Ed.),Knowing, learning, and instruction: Essays in honor of Robert Glaser (pp. 453–494). Hillsdale, NJ: Erlbaum.Google Scholar
  13. Collins, A., Warnock, E. H., &Passafiume, J. J. (1975). Analysis and synthesis of tutorial dialogues. In G. H. Bower (Ed.),The psychology of learning and motivation: Advances in research and theory (Vol. 8, pp. 49–87). New York: Academic Press.Google Scholar
  14. DARPA (1995).Proceedings of the Sixth Message Understanding Conference (MUC-6). San Francisco: Morgan Kaufman.Google Scholar
  15. Foltz, P. W., Gilliam, S., &Kendall, S. (2000). Supporting content-based feedback in on-line writing evaluation with LSA.Interactive Learning Environments,8,111–128.CrossRefGoogle Scholar
  16. Fox, B. (1993).The human tutorial dialogue project. Hillsdale, NJ: Erlbaum.Google Scholar
  17. Graesser, A. C., Gernsbacher, M. A., &Goldman, S. (Eds.) (2003).Handbook of discourse processes. Mahwah, NJ: Erlbaum.Google Scholar
  18. Graesser, A. C., Hu, X., & McNamara, D. S. (in press). Computerized learning environments that incorporate research in discourse psychology, cognitive science, and computational linguistics. In A. F. Healy (Ed.),Experimental cognitive psychology and its applications: Festschrift in honor of Lyle Bourne, Walter Kintsch, and Thomas Landauer. Washington, DC: American Psychological Association.Google Scholar
  19. Graesser, A. C., Jackson, G. T., Mathews, E. C., Mitchell, H. H., Olney, A., Ventura, M., &Chipman, P. (2003). Why/AutoTutor: A test of learning gains from a physics tutor with natural language dialog. In R. Alterman & D. Hirsh (Eds.),Proceedings of the Twenty-Fifth Annual Conference of the Cognitive Science Society (pp. 1–6). Mahwah, NJ: Erlbaum.Google Scholar
  20. Graesser, A. C., Moreno, K., Marineau, J., Adcock, A., Olney, A., Person, N., &the Tutoring Research Group (2003). AutoTutor improves deep learning of computer literacy: Is it the dialogue or the talking head? In U. Hoppe, F. Verdejo, & J. Kay (Eds.),Proceedings of artificial intelligence in education (pp. 47–54). Amsterdam: IOS Press.Google Scholar
  21. Graesser, A. C., &Olde, B. A. (2003). How does one know whether a person understands a device? The quality of the questions the person asks when the device breaks down.Journal of Educational Psychology,95, 524–536.CrossRefGoogle Scholar
  22. Graesser, A. C., &Person, N. K. (1994). Question asking during tutoring.American Educational Research Journal,31104–137.Google Scholar
  23. Graesser, A. C., Person, N. K., Harter, D., &the Tutoring Research Group (2001). Teaching tactics and dialogue in AutoTutor.International Journal of Artificial Intelligence in Education,12, 257–279.Google Scholar
  24. Graesser, A. C., Person, N. K., &Magliano, J. P. (1995). Collaborative dialogue patterns in naturalistic one-on-one tutoring.Applied Cognitive Psychology,9, 359–387.CrossRefGoogle Scholar
  25. Graesser, A. C., VanLehn, K., Rosé, C. P., Jordan, P. W., &Harter, D. (2001). Intelligent tutoring systems with conversational dialogue.AI Magazine,22(4), 39–52.Google Scholar
  26. Graesser, A. C., Wiemer-Hastings, K., Wiemer-Hastings, P., Kreuz, R., &the Tutoring Research Group (1999). AutoTutor: A simulation of a human tutor.Journal of Cognitive Systems Research,1,35–51.CrossRefGoogle Scholar
  27. Graesser, A. C., Wiemer-Hastings, P., Wiemer-Hastings, K., Harter, D., Person, N. K., &the Tutoring Research Group (2000). Using latent semantic analysis to evaluate the contributions of students in AutoTutor.Interactive Learning Environments,8,129–148.CrossRefGoogle Scholar
  28. Gratch, J., Rickel, J., André, E., Cassell, J., Petajan, E., &Badler, N. (2002). Creating interactive virtual humans: Some assembly required.IEEE Intelligent Systems,17,54–63.Google Scholar
  29. Harabagiu, S. M., Maiorano, S. J., &Pasca, M. A. (2002). Open-domain question answering techniques.Natural Language Engineering,1, 1–38.Google Scholar
  30. Heffernan, N. T., &Koedinger, K. R. (1998). A developmental model for algebra symbolization: The results of a difficulty factor assessment. In M. A. Gernsbacher & S. J. Derry (Eds.),Proceedings of the Twentieth Annual Conference of the Cognitive Science Society (pp. 484–489). Mahwah, NJ: Erlbaum.Google Scholar
  31. Hu, X., Cai, Z., Graesser, A. C., Louwerse, M. M., Penumatsa, P., Olney, A., &the Tutoring Research Group (2003). An improved LSA algorithm to evaluate student contributions in tutoring dialogue. In G. Gottlob & T. Walsh (Eds.),Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (pp. 1489–1491). San Francisco: Morgan Kaufmann.Google Scholar
  32. Hume, G. D., Michael, J. A., Rovick, A., &Evens, M. W. (1996). Hinting as a tactic in one-on-one tutoring.Journal of the Learning Sciences,5, 23–47.CrossRefGoogle Scholar
  33. Johnson, W. L., Rickel, J. W., &Lester, J. C. (2000). Animated pedagogical agents: Face-to-face interaction in interactive learning environments.International Journal of Artificial Intelligence in Education,11, 47–78.Google Scholar
  34. Jurafsky, D., &Martin, J. H. (2000).Speech and language processing: An introduction to natural language processing, computational linguistics, and speech recognition. Upper Saddle River, NJ: Prentice Hall.Google Scholar
  35. Kintsch, E., Steinhart, D., Stahl, G., &the LSA Research Group (2000). Developing summarization skills through the use of LSA-based feedback.Interactive Learning Environments,8,87–109.CrossRefGoogle Scholar
  36. Kintsch, W. (1998).Comprehension: A paradigm for cognition. New York: Cambridge University Press.Google Scholar
  37. Landauer, T. K., &Dumais, S. T. (1997). An answer to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge.Psychological Review,104,211–240.CrossRefGoogle Scholar
  38. Landauer, T. K., Foltz, P. W., &Laham, D. (1998). An introduction to latent semantic analysis.Discourse Processes,25, 259–284.CrossRefGoogle Scholar
  39. Lehnert, W. G., &Ringle, M. H. (Eds.) (1982). Strategies for natural language processing. Hillsdale, NJ: Erlbaum.Google Scholar
  40. Lenat, D. B. (1995). CYC: A large-scale investment in knowledge infrastructure.Communications of the ACM,38,33–38.CrossRefGoogle Scholar
  41. Louwerse, M. M., Graesser, A. C., Olney, A., &the Tutoring Research Group (2002). Good computational manners: Mixed-initiative dialogue in conversational agents. In C. Miller (Ed.),Etiquette for human—computer work: Papers from the 2002 fall symposium, Technical Report FS-02-02 (pp. 71–76). North Falmouth, MA: AAAI Press.Google Scholar
  42. Louwerse, M. M., &Mitchell, H. H. (2003). Towards a taxonomy of a set of discourse markers in dialog: A theoretical and computational linguistic account.Discourse Processes,35, 199–239.CrossRefGoogle Scholar
  43. Louwerse, M. M., &van Peer, W. (Eds.) (2002).Thematics: Interdisciplinary studies. Philadelphia: John Benjamins.Google Scholar
  44. Manning, C. D., &Schütze, H. (1999).Foundations of statistical natural language processing. Cambridge, MA: MIT Press.Google Scholar
  45. Massaro, D. W., &Cohen, M. M. (1995). Perceiving talking faces.Current Directions in Psychological Science,4, 104–109.CrossRefGoogle Scholar
  46. Moore, J. D. (1995).Participating in explanatory dialogues. Cambridge, MA: MIT Press.Google Scholar
  47. Moore, J. D., &Wiemer-Hastings, P. (2003). Discourse in computational linguistics and artificial intelligence. In A. C. Graesser, M. A. Gernsbacher, & S. R. Goldman (Eds.),Handbook of discourse processes (pp. 439–486). Mahwah, NJ: Erlbaum.Google Scholar
  48. Moreno, K. N., Klettke, B., Nibbaragandla, K., Graesser, A. C., &the Tutoring Research Group (2002). Perceived characteristics and pedagogical efficacy of animated conversational agents. In S. A. Cerri, G. Gouarderes, & F. Paraguacu (Eds.),Proceedings of the Sixth International Conference on Intelligent Tutoring Systems (pp. 963–971). Berlin: Springer-Verlag.Google Scholar
  49. Moreno, R., Mayer, R. E., Spires, H. A., &Lester, J. C. (2001). The case for social agency in computer-based teaching: Do students learn more deeply when they interact with animated pedagogical agents?Cognition & Instruction,19,177–213.CrossRefGoogle Scholar
  50. Norman, D. A., &Rumelhart, D. E. (1975).Explorations in cognition. San Francisco: Freeman.Google Scholar
  51. Olde, B. A., Franceschetti, D. R., Karnavat, A., Graesser, A. C., &the Tutoring Research Group (2002). The right stuff: Do you need to sanitize your corpus when using latent semantic analysis? In W. D. Gray & C. D. Schunn (Eds.),Proceedings of the Twenty-Fourth Annual Meeting of the Cognitive Science Society (pp. 708–713). Mahwah, NJ: Erlbaum.Google Scholar
  52. Olney, A., Louwerse, M. M., Mathews, E. C., Marineau, J., Mitchell, H. H., &Graesser, A. C. (2003). Utterance classification in AutoTutor. In J. Burstein & C. Leacock (Eds.),Building educational applications using natural language processing: Proceedings of the Human Language Technology, North American chapter of the Association for Computational Linguistics Conference 2003 Workshop (pp. 1–8). Philadelphia: Association for Computational Linguistics.Google Scholar
  53. Palincsar, A. S., &Brown, A. (1984). Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities.Cognition & Instruction,1,117–175.CrossRefGoogle Scholar
  54. Person, N. K., Graesser, A. C., Bautista, L., Mathews, E. C., &the Tutoring Research Group (2001). Evaluating student learning gains in two versions of AutoTutor. In J. D. Moore, C. L. Redfield, & W. L. Johnson (Eds.),Artificial intelligence in education: AI-ED in the wired and wireless future (pp. 286–293). Amsterdam: IOS Press.Google Scholar
  55. Person, N. K., Graesser, A. C., Kreuz, R. J., Pomeroy, V., &the Tutoring Research Group (2001). Simulating human tutor dialogue moves in AutoTutor.International Journal of Artificial Intelligence in Education,12, 23–39.Google Scholar
  56. Person, N. K., Graesser, A. C., &the Tutoring Research Group (2002). Human or computer? AutoTutor in a bystander Turing test. In S. A. Cerri, G. Gouarderes, & F. Paraguacu (Eds.),Proceedings of the Sixth International Conference on Intelligent Tutoring Systems (pp. 821–830). Berlin: Springer-Verlag.Google Scholar
  57. Picard, R. W. (1991).Affective computing. Cambridge, MA: MIT Press.Google Scholar
  58. Rich, C., &Sidner, C. L. (1998). COLLAGEN: A collaborative manager for software interface agents.User Modeling & User-Adapted Interaction,8,315–350.CrossRefGoogle Scholar
  59. Rickel, J., Lesh, N., Rich, C., Sidner, C. L., &Gertner, A. S. (2002). Collaborative discourse theory as a foundation for tutorial dialogue. In S. A. Cerri, G. Gouarderes, & F. Paraguacu (Eds.),Proceedings of the Sixth International Conference on Intelligent Tutoring Systems (pp. 542–551). Berlin: Springer-Verlag.Google Scholar
  60. Schank, R. C., &Riesbeck, C. K. (Eds.) (1982).Inside computer understanding: Five Programs Plus Miniatures. Hillsdale, NJ: Erlbaum.Google Scholar
  61. Sekine, S., &Grishman, R. (1995). A corpus-based probabilistic grammar with only two non-terminals. In H. Bunt (Ed.),Fourth international workshop on parsing technology (pp. 216–223). Prague: Association for Computational Linguistics.Google Scholar
  62. Shah, F., Evens, M. W., Michael, J., &Rovick, A. (2002). Classifying student initiatives and tutor responses in human keyboard-to-keyboard tutoring sessions.Discourse Processes,33, 23–52.CrossRefGoogle Scholar
  63. Sleeman, D., &Brown, J. S. (Eds.) (1982).Intelligent tutoring systems. New York: Academic Press.Google Scholar
  64. Song, K., Hu, X., Olney, A., Graesser, A. C., & the Tutoring Research Group (in press). A framework of synthesizing tutoring conversation capability with Web based distance education courseware.Computers & Education.Google Scholar
  65. Susarla, S., Adcock, A., Van Eck, R., Moreno, K., Graesser, A. C., &the Tutoring Research Group (2003). Development and evaluation of a lesson authoring tool for AutoTutor. In V. Aleven, U. Hoppe, J. Kay, R. Mizoguchi, H. Pain, F. Verdejo, & K. Yacef (Eds.),AIED2003 supplemental proceedings (pp. 378–387). Sydney: University of Sydney School of Information Technologies.Google Scholar
  66. VanLehn, K., & Graesser, A. C. (2002).Why2 report: Evaluation of Why/Atlas, Why/AutoTutor, and accomplished human tutors on learning gains for qualitative physics problems and explanations. Unpublished report prepared by the University of Pittsburgh CIRCLE group and the University of Memphis Tutoring Research Group.Google Scholar
  67. VanLehn, K., Jones, R. M., &Chi, M. T. H. (1992). A model of the self-explanation effect.Journal of the Learning Sciences,2, 1–60.CrossRefGoogle Scholar
  68. VanLehn, K., Jordan, P., Rosé, C. P., Bhembe, D., Bottner, M., Gaydos, A., Makatchev, M., Pappuswamy, U., Ringenberg, M., Roque, A., Siler, S., &Srivastava, R. (2002). The architecture of Why2-Atlas: A coach for qualitative physics essay writing. In S. A. Cerri, G. Gouarderes, & F. Paraguacu (Eds.),Proceedings of the Sixth International Conference on Intelligent Tutoring Systems (pp. 158–167). Berlin: Springer-Verlag.Google Scholar
  69. VanLehn, K., Lynch, C., Taylor, L., Weinstein, A., Shelby, R., Schulze, K., Treacy, D., &Wintersgill, M. (2002). Minimally invasive tutoring of complex physics problem solving. In S. A. Cerri, G. Gouarderes, & F. Paraguacu (Eds.),Proceedings of the Sixth International Conference on Intelligent Tutoring Systems (pp. 367–376). Berlin: Springer-Verlag.Google Scholar
  70. Voorhees, E. M. (2001). The TREC question answering track.Natural Language Engineering,7,361–378.CrossRefGoogle Scholar
  71. Weizenbaum, J. (1966). ELIZA—A computer program for the study of natural language communication between man and machine.Communications of the ACM,9, 36–45.CrossRefGoogle Scholar
  72. Whittaker, S. (2003). Theories and methods in mediated communication. In A. C. Graesser, M. A. Gernsbacher, & S. R. Goldman (Eds.),Handbook of discourse processes (pp. 243–286). Mahwah, NJ: Erlbaum.Google Scholar
  73. Winograd, T. (1972).Understanding natural language. New York: Academic Press.Google Scholar
  74. Woods, W. A. (1977). Lunar rocks in natural English: Explorations in natural language question answering. In A. Zampoli (Ed.),Linguistic structures processing (pp. 201–222). New York: Elsevier.Google Scholar

Copyright information

© Psychonomic Society, Inc. 2004

Authors and Affiliations

  • Arthur C. Graesser
    • 1
    Email author
  • Shulan Lu
    • 1
  • George Tanner Jackson
    • 1
  • Heather Hite Mitchell
    • 1
  • Mathew Ventura
    • 1
  • Andrew Olney
    • 1
  • Max M. Louwerse
    • 1
  1. 1.Department of PsychologyUniversity of MemphisMemphis

Personalised recommendations