Skip to main content
Download PDF

AutoTutor: A tutor with dialogue in natural language

Behavior Research Methods, Instruments, & Computers volume 36, pages 180–192 (2004)Cite this article

Abstract

AutoTutor is a learning environment that tutors students by holding a conversation in natural language. AutoTutor has been developed for Newtonian qualitative physics and computer literacy. Its design was inspired by explanation-based constructivist theories of learning, intelligent tutoring systems that adaptively respond to student knowledge, and empirical research on dialogue patterns in tutorial discourse. AutoTutor presents challenging problems (formulated as questions) from a curriculum script and then engages in mixed initiative dialogue that guides the student in building an answer. It provides the student with positive, neutral, or negative feedback on the student’s typed responses, pumps the student for more information, prompts the student to fill in missing words, gives hints, fills in missing information with assertions, identifies and corrects erroneous ideas, answers the student’s questions, and summarizes answers. AutoTutor has produced learning gains of approximately .70 sigma for deep levels of comprehension.

References

  • Aleven, V., &Koedinger, K. R. (2002). An effective metacognitive strategy: Learning by doing and explaining with a computer-based cognitive tutor. Cognitive Science,26,147–179.

    Article  Google Scholar 

  • Allen, J. (1995).Natural language understanding. Redwood City, CA: Benjamin/Cummings.

    Google Scholar 

  • Anderson, J. R., Corbett, A. T., Koedinger, K. R., &Pelletier, R. (1995). Cognitive tutors: Lessons learned.Journal of the Learning Sciences,4,167–207.

    Article  Google Scholar 

  • Atkinson, R. K. (2002). Optimizing learning from examples using animated pedagogical agents.Journal of Educational Psychology,94,416–427.

    Article  Google Scholar 

  • Bloom, B. S. (Ed.) (1956).Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York: Longmans, Green.

    Google Scholar 

  • Burgess, C., Livesay, K., &Lund, K. (1998). Explorations in context space: Words, sentences, and discourse.Discourse Processes,25, 211–257.

    Article  Google Scholar 

  • Cassell, J., &Thorisson, K. (1999). The power of a nod and a glance: Envelope vs. emotional feedback in animated conversational agents.Applied Artificial Intelligence,13,519–538.

    Article  Google Scholar 

  • Chi, M. T. H., de Leeuw, N., Chiu, M., &LaVancher, C. (1994). Eliciting self-explanations improves understanding.Cognitive Science,18,439–477.

    Google Scholar 

  • Chi, M. T. H., Siler, S. A., Jeong, H., Yamauchi, T., &Hausmann, R. G. (2001). Learning from human tutoring.Cognitive Science,25,471–533.

    Article  Google Scholar 

  • Cohen, P. A., Kulik, J. A., &Kulik, C. C. (1982). Educational outcomes of tutoring: A meta-analysis of findings.American Educational Research Journal,19,237–248.

    Google Scholar 

  • Colby, K. M., Weber, S., &Hilf, F. D. (1971). Artificial paranoia.Artificial Intelligence,2, 1–25.

    Article  Google Scholar 

  • Collins, A., Brown, J. S., &Newman, S. E. (1989). Cognitive apprenticeship: Teaching the craft of reading, writing, and mathematics. In L. B. Resnick (Ed.),Knowing, learning, and instruction: Essays in honor of Robert Glaser (pp. 453–494). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Collins, A., Warnock, E. H., &Passafiume, J. J. (1975). Analysis and synthesis of tutorial dialogues. In G. H. Bower (Ed.),The psychology of learning and motivation: Advances in research and theory (Vol. 8, pp. 49–87). New York: Academic Press.

    Google Scholar 

  • DARPA (1995).Proceedings of the Sixth Message Understanding Conference (MUC-6). San Francisco: Morgan Kaufman.

    Google Scholar 

  • Foltz, P. W., Gilliam, S., &Kendall, S. (2000). Supporting content-based feedback in on-line writing evaluation with LSA.Interactive Learning Environments,8,111–128.

    Article  Google Scholar 

  • Fox, B. (1993).The human tutorial dialogue project. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Graesser, A. C., Gernsbacher, M. A., &Goldman, S. (Eds.) (2003).Handbook of discourse processes. Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Graesser, A. C., Hu, X., & McNamara, D. S. (in press). Computerized learning environments that incorporate research in discourse psychology, cognitive science, and computational linguistics. In A. F. Healy (Ed.),Experimental cognitive psychology and its applications: Festschrift in honor of Lyle Bourne, Walter Kintsch, and Thomas Landauer. Washington, DC: American Psychological Association.

  • Graesser, A. C., Jackson, G. T., Mathews, E. C., Mitchell, H. H., Olney, A., Ventura, M., &Chipman, P. (2003). Why/AutoTutor: A test of learning gains from a physics tutor with natural language dialog. In R. Alterman & D. Hirsh (Eds.),Proceedings of the Twenty-Fifth Annual Conference of the Cognitive Science Society (pp. 1–6). Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Graesser, A. C., Moreno, K., Marineau, J., Adcock, A., Olney, A., Person, N., &the Tutoring Research Group (2003). AutoTutor improves deep learning of computer literacy: Is it the dialogue or the talking head? In U. Hoppe, F. Verdejo, & J. Kay (Eds.),Proceedings of artificial intelligence in education (pp. 47–54). Amsterdam: IOS Press.

    Google Scholar 

  • Graesser, A. C., &Olde, B. A. (2003). How does one know whether a person understands a device? The quality of the questions the person asks when the device breaks down.Journal of Educational Psychology,95, 524–536.

    Article  Google Scholar 

  • Graesser, A. C., &Person, N. K. (1994). Question asking during tutoring.American Educational Research Journal,31104–137.

    Google Scholar 

  • Graesser, A. C., Person, N. K., Harter, D., &the Tutoring Research Group (2001). Teaching tactics and dialogue in AutoTutor.International Journal of Artificial Intelligence in Education,12, 257–279.

    Google Scholar 

  • Graesser, A. C., Person, N. K., &Magliano, J. P. (1995). Collaborative dialogue patterns in naturalistic one-on-one tutoring.Applied Cognitive Psychology,9, 359–387.

    Article  Google Scholar 

  • Graesser, A. C., VanLehn, K., Rosé, C. P., Jordan, P. W., &Harter, D. (2001). Intelligent tutoring systems with conversational dialogue.AI Magazine,22(4), 39–52.

    Google Scholar 

  • Graesser, A. C., Wiemer-Hastings, K., Wiemer-Hastings, P., Kreuz, R., &the Tutoring Research Group (1999). AutoTutor: A simulation of a human tutor.Journal of Cognitive Systems Research,1,35–51.

    Article  Google Scholar 

  • Graesser, A. C., Wiemer-Hastings, P., Wiemer-Hastings, K., Harter, D., Person, N. K., &the Tutoring Research Group (2000). Using latent semantic analysis to evaluate the contributions of students in AutoTutor.Interactive Learning Environments,8,129–148.

    Article  Google Scholar 

  • Gratch, J., Rickel, J., André, E., Cassell, J., Petajan, E., &Badler, N. (2002). Creating interactive virtual humans: Some assembly required.IEEE Intelligent Systems,17,54–63.

    Google Scholar 

  • Harabagiu, S. M., Maiorano, S. J., &Pasca, M. A. (2002). Open-domain question answering techniques.Natural Language Engineering,1, 1–38.

    Google Scholar 

  • Heffernan, N. T., &Koedinger, K. R. (1998). A developmental model for algebra symbolization: The results of a difficulty factor assessment. In M. A. Gernsbacher & S. J. Derry (Eds.),Proceedings of the Twentieth Annual Conference of the Cognitive Science Society (pp. 484–489). Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Hu, X., Cai, Z., Graesser, A. C., Louwerse, M. M., Penumatsa, P., Olney, A., &the Tutoring Research Group (2003). An improved LSA algorithm to evaluate student contributions in tutoring dialogue. In G. Gottlob & T. Walsh (Eds.),Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (pp. 1489–1491). San Francisco: Morgan Kaufmann.

    Google Scholar 

  • Hume, G. D., Michael, J. A., Rovick, A., &Evens, M. W. (1996). Hinting as a tactic in one-on-one tutoring.Journal of the Learning Sciences,5, 23–47.

    Article  Google Scholar 

  • Johnson, W. L., Rickel, J. W., &Lester, J. C. (2000). Animated pedagogical agents: Face-to-face interaction in interactive learning environments.International Journal of Artificial Intelligence in Education,11, 47–78.

    Google Scholar 

  • Jurafsky, D., &Martin, J. H. (2000).Speech and language processing: An introduction to natural language processing, computational linguistics, and speech recognition. Upper Saddle River, NJ: Prentice Hall.

    Google Scholar 

  • Kintsch, E., Steinhart, D., Stahl, G., &the LSA Research Group (2000). Developing summarization skills through the use of LSA-based feedback.Interactive Learning Environments,8,87–109.

    Article  Google Scholar 

  • Kintsch, W. (1998).Comprehension: A paradigm for cognition. New York: Cambridge University Press.

    Google Scholar 

  • Landauer, T. K., &Dumais, S. T. (1997). An answer to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge.Psychological Review,104,211–240.

    Article  Google Scholar 

  • Landauer, T. K., Foltz, P. W., &Laham, D. (1998). An introduction to latent semantic analysis.Discourse Processes,25, 259–284.

    Article  Google Scholar 

  • Lehnert, W. G., &Ringle, M. H. (Eds.) (1982). Strategies for natural language processing. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Lenat, D. B. (1995). CYC: A large-scale investment in knowledge infrastructure.Communications of the ACM,38,33–38.

    Article  Google Scholar 

  • Louwerse, M. M., Graesser, A. C., Olney, A., &the Tutoring Research Group (2002). Good computational manners: Mixed-initiative dialogue in conversational agents. In C. Miller (Ed.),Etiquette for human—computer work: Papers from the 2002 fall symposium, Technical Report FS-02-02 (pp. 71–76). North Falmouth, MA: AAAI Press.

    Google Scholar 

  • Louwerse, M. M., &Mitchell, H. H. (2003). Towards a taxonomy of a set of discourse markers in dialog: A theoretical and computational linguistic account.Discourse Processes,35, 199–239.

    Article  Google Scholar 

  • Louwerse, M. M., &van Peer, W. (Eds.) (2002).Thematics: Interdisciplinary studies. Philadelphia: John Benjamins.

    Google Scholar 

  • Manning, C. D., &Schütze, H. (1999).Foundations of statistical natural language processing. Cambridge, MA: MIT Press.

    Google Scholar 

  • Massaro, D. W., &Cohen, M. M. (1995). Perceiving talking faces.Current Directions in Psychological Science,4, 104–109.

    Article  Google Scholar 

  • Moore, J. D. (1995).Participating in explanatory dialogues. Cambridge, MA: MIT Press.

    Google Scholar 

  • Moore, J. D., &Wiemer-Hastings, P. (2003). Discourse in computational linguistics and artificial intelligence. In A. C. Graesser, M. A. Gernsbacher, & S. R. Goldman (Eds.),Handbook of discourse processes (pp. 439–486). Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Moreno, K. N., Klettke, B., Nibbaragandla, K., Graesser, A. C., &the Tutoring Research Group (2002). Perceived characteristics and pedagogical efficacy of animated conversational agents. In S. A. Cerri, G. Gouarderes, & F. Paraguacu (Eds.),Proceedings of the Sixth International Conference on Intelligent Tutoring Systems (pp. 963–971). Berlin: Springer-Verlag.

    Google Scholar 

  • Moreno, R., Mayer, R. E., Spires, H. A., &Lester, J. C. (2001). The case for social agency in computer-based teaching: Do students learn more deeply when they interact with animated pedagogical agents?Cognition & Instruction,19,177–213.

    Article  Google Scholar 

  • Norman, D. A., &Rumelhart, D. E. (1975).Explorations in cognition. San Francisco: Freeman.

    Google Scholar 

  • Olde, B. A., Franceschetti, D. R., Karnavat, A., Graesser, A. C., &the Tutoring Research Group (2002). The right stuff: Do you need to sanitize your corpus when using latent semantic analysis? In W. D. Gray & C. D. Schunn (Eds.),Proceedings of the Twenty-Fourth Annual Meeting of the Cognitive Science Society (pp. 708–713). Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Olney, A., Louwerse, M. M., Mathews, E. C., Marineau, J., Mitchell, H. H., &Graesser, A. C. (2003). Utterance classification in AutoTutor. In J. Burstein & C. Leacock (Eds.),Building educational applications using natural language processing: Proceedings of the Human Language Technology, North American chapter of the Association for Computational Linguistics Conference 2003 Workshop (pp. 1–8). Philadelphia: Association for Computational Linguistics.

    Google Scholar 

  • Palincsar, A. S., &Brown, A. (1984). Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities.Cognition & Instruction,1,117–175.

    Article  Google Scholar 

  • Person, N. K., Graesser, A. C., Bautista, L., Mathews, E. C., &the Tutoring Research Group (2001). Evaluating student learning gains in two versions of AutoTutor. In J. D. Moore, C. L. Redfield, & W. L. Johnson (Eds.),Artificial intelligence in education: AI-ED in the wired and wireless future (pp. 286–293). Amsterdam: IOS Press.

    Google Scholar 

  • Person, N. K., Graesser, A. C., Kreuz, R. J., Pomeroy, V., &the Tutoring Research Group (2001). Simulating human tutor dialogue moves in AutoTutor.International Journal of Artificial Intelligence in Education,12, 23–39.

    Google Scholar 

  • Person, N. K., Graesser, A. C., &the Tutoring Research Group (2002). Human or computer? AutoTutor in a bystander Turing test. In S. A. Cerri, G. Gouarderes, & F. Paraguacu (Eds.),Proceedings of the Sixth International Conference on Intelligent Tutoring Systems (pp. 821–830). Berlin: Springer-Verlag.

    Google Scholar 

  • Picard, R. W. (1991).Affective computing. Cambridge, MA: MIT Press.

    Google Scholar 

  • Rich, C., &Sidner, C. L. (1998). COLLAGEN: A collaborative manager for software interface agents.User Modeling & User-Adapted Interaction,8,315–350.

    Article  Google Scholar 

  • Rickel, J., Lesh, N., Rich, C., Sidner, C. L., &Gertner, A. S. (2002). Collaborative discourse theory as a foundation for tutorial dialogue. In S. A. Cerri, G. Gouarderes, & F. Paraguacu (Eds.),Proceedings of the Sixth International Conference on Intelligent Tutoring Systems (pp. 542–551). Berlin: Springer-Verlag.

    Google Scholar 

  • Schank, R. C., &Riesbeck, C. K. (Eds.) (1982).Inside computer understanding: Five Programs Plus Miniatures. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Sekine, S., &Grishman, R. (1995). A corpus-based probabilistic grammar with only two non-terminals. In H. Bunt (Ed.),Fourth international workshop on parsing technology (pp. 216–223). Prague: Association for Computational Linguistics.

    Google Scholar 

  • Shah, F., Evens, M. W., Michael, J., &Rovick, A. (2002). Classifying student initiatives and tutor responses in human keyboard-to-keyboard tutoring sessions.Discourse Processes,33, 23–52.

    Article  Google Scholar 

  • Sleeman, D., &Brown, J. S. (Eds.) (1982).Intelligent tutoring systems. New York: Academic Press.

    Google Scholar 

  • Song, K., Hu, X., Olney, A., Graesser, A. C., & the Tutoring Research Group (in press). A framework of synthesizing tutoring conversation capability with Web based distance education courseware.Computers & Education.

  • Susarla, S., Adcock, A., Van Eck, R., Moreno, K., Graesser, A. C., &the Tutoring Research Group (2003). Development and evaluation of a lesson authoring tool for AutoTutor. In V. Aleven, U. Hoppe, J. Kay, R. Mizoguchi, H. Pain, F. Verdejo, & K. Yacef (Eds.),AIED2003 supplemental proceedings (pp. 378–387). Sydney: University of Sydney School of Information Technologies.

    Google Scholar 

  • VanLehn, K., & Graesser, A. C. (2002).Why2 report: Evaluation of Why/Atlas, Why/AutoTutor, and accomplished human tutors on learning gains for qualitative physics problems and explanations. Unpublished report prepared by the University of Pittsburgh CIRCLE group and the University of Memphis Tutoring Research Group.

  • VanLehn, K., Jones, R. M., &Chi, M. T. H. (1992). A model of the self-explanation effect.Journal of the Learning Sciences,2, 1–60.

    Article  Google Scholar 

  • VanLehn, K., Jordan, P., Rosé, C. P., Bhembe, D., Bottner, M., Gaydos, A., Makatchev, M., Pappuswamy, U., Ringenberg, M., Roque, A., Siler, S., &Srivastava, R. (2002). The architecture of Why2-Atlas: A coach for qualitative physics essay writing. In S. A. Cerri, G. Gouarderes, & F. Paraguacu (Eds.),Proceedings of the Sixth International Conference on Intelligent Tutoring Systems (pp. 158–167). Berlin: Springer-Verlag.

    Google Scholar 

  • VanLehn, K., Lynch, C., Taylor, L., Weinstein, A., Shelby, R., Schulze, K., Treacy, D., &Wintersgill, M. (2002). Minimally invasive tutoring of complex physics problem solving. In S. A. Cerri, G. Gouarderes, & F. Paraguacu (Eds.),Proceedings of the Sixth International Conference on Intelligent Tutoring Systems (pp. 367–376). Berlin: Springer-Verlag.

    Google Scholar 

  • Voorhees, E. M. (2001). The TREC question answering track.Natural Language Engineering,7,361–378.

    Article  Google Scholar 

  • Weizenbaum, J. (1966). ELIZA—A computer program for the study of natural language communication between man and machine.Communications of the ACM,9, 36–45.

    Article  Google Scholar 

  • Whittaker, S. (2003). Theories and methods in mediated communication. In A. C. Graesser, M. A. Gernsbacher, & S. R. Goldman (Eds.),Handbook of discourse processes (pp. 243–286). Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Winograd, T. (1972).Understanding natural language. New York: Academic Press.

    Google Scholar 

  • Woods, W. A. (1977). Lunar rocks in natural English: Explorations in natural language question answering. In A. Zampoli (Ed.),Linguistic structures processing (pp. 201–222). New York: Elsevier.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, University of Memphis, 202 Psychology Building, 38152-3230, Memphis, TN

    Arthur C. Graesser, Shulan Lu, George Tanner Jackson, Heather Hite Mitchell, Mathew Ventura, Andrew Olney & Max M. Louwerse

Authors
  1. Arthur C. Graesser
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shulan Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. George Tanner Jackson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Heather Hite Mitchell
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mathew Ventura
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Andrew Olney
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Max M. Louwerse
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arthur C. Graesser.

Additional information

The Tutoring Research Group is an interdisciplinary research team comprised of approximately 35 researchers with backgrounds in psychology, computer science, physics, and education (for more information, visithttp://www.autotutor.org). The research on AutoTutor was supported by National Science Foundation (NSF) Grants SBR 9720314, REC 0106965, REC 0126265, and ITR 0325428 and by Grant N00014-00-1-0600 of the Department of Defense Multidisciplinary University Research Initiative administered by the Office of Naval Research (ONR). Any opinions, findings, and conclusions or recommendations expressed in this article are those of the authors and do not necessarily reflect the views of the Department of Defense, the ONR, or the NSF. Kurt VanLehn, Carolyn Rosé, Pam Jordan, and others at the University of Pittsburgh collaborated with us in preparing AutoTutor materials on conceptual physics.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Graesser, A.C., Lu, S., Jackson, G.T. et al. AutoTutor: A tutor with dialogue in natural language. Behavior Research Methods, Instruments, & Computers 36, 180–192 (2004). https://doi.org/10.3758/BF03195563

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.3758/BF03195563

Keywords

  • Latent Semantic Analysis
  • Computer Literacy
  • Intelligent Tutoring System
  • World Knowledge
  • Discourse Process