Skip to main content
SpringerLink
Log in

AutoTutor: Incorporating Back-Channel Feedback and Other Human-Like Conversational Behaviors into an Intelligent Tutoring System

  • Published:
International Journal of Speech Technology Aims and scope Submit manuscript

Abstract

This paper describes our recent attempts to incorporate human-like conversational behaviors into the dialog moves delivered by an animated pedagogical agent that simulates human tutors. We first present a brief overview of the modules comprising AutoTutor, an intelligent tutoring system. The second section describes a set of conversational behaviors that are being incorporated into AutoTutor. The behaviors of interest involve variations in intonation, head movements, arm and hand movements, facial expressions, eye blinking, gaze direction, and back-channel feedback. The final section presents a recent empirical study concerned with back-channel feedback events during human-to-human tutoring sessions. The back-channel feedback events emitted by tutors are mostly positive (63%), mostly verbal (77%), and immediately follow speech-act boundaries or noun-phrase boundaries (83%). Tutors also deliver back-channelevents at a very high rate when students are emitting dialog, about 13 events per minute. Conversely, 88% of students' back-channel feedback events are head nods, and they occur at unbounded locations (63%).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Allen, J. (1995). Natural Language Understanding. Redwood City, CA: Benjamin/Cummings.

    Google Scholar 

  • Anderson, J.R., Corbett, A.T., Koedinger, K.R., and Pelletier, R. (1995). Cognitive tutors: Lessons learned. The Journal of the Learning Sciences, 4:167-297.

    Google Scholar 

  • Argyle, M. and Cook, M. (1976). Gaze and Mutual Gaze. Cambridge: Cambridge University Press.

    Google Scholar 

  • Bloom, B.S. (1956). Taxonomy of Educational Objectives: The Classification of Educational Goals. Handbook I: Cognitive Domain. New York: McKay.

    Google Scholar 

  • Bloom, B.S. (1984). The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher, 13:4-16.

    Google Scholar 

  • Bransford, J.D., Goldman, S.R., and Vye, N.J. (1991). Making a difference in people's ability to think: Reflections on a decade of work and some hopes for the future. In R.J. Sternberg and L. Okagaki (Eds.), Influences on Children. Hillsdale, NJ: Erlbaum, pp. 147-180.

    Google Scholar 

  • Collins, A. (1985). Teaching reasoning skills. In S.F. Chipman, J.W. Segal, and R. Glaser (Eds.), Thinking and Learning Skills, Vol. 2. Hillsdale, NJ: Erlbaum, pp. 579-586.

    Google Scholar 

  • Collins, A., Brown, J.S., and Newman, S.E. (1989). Cognitive apprenticeship: Teaching the craft of reading, writing, and mathematics. In L.B. Resnick (Ed.), Knowing, learning, and instruction: Essays in honor of Robert Glaser. Hillsdale, NJ: Erlbaum, pp. 453-494.

    Google Scholar 

  • Cassell, J., Bickmore, J., Billinghurst, M., Campbell, L., Chang, K., Vilhjálmsson, H., and Yan, H. (1999). Embodiment in conversational interfaces: REA. CHI'99 Conference Proceedings. Marina del Rey: ACM Press, pp. 520-527.

    Google Scholar 

  • Cassell, J., Pelachaud, C., Badler, N., Steedman, M., Achorn, B., Becket, T., Douville, B., Prevost, S., and Stone, M. (1994). Animated conversation: Rule-based generation of facial expression, AutoTutor 125 gesture and spoken intonation for multiple conversational agents. Computational Graphics, 28:413-420.

    Google Scholar 

  • Cassell, J. and Stone, M. (1999). Living hand to mouth: Psychological theories about speech and gesture in interactive dialogue systems.Paper present at AAAI 1999Fall Symposium on Narrative Intelligence, Cape Cod, MA.

  • Cassell, J. and Thórisson, K.R. (1999). The power of a nod and a glance: Envelope vs. emotional feedback in animated conversational agents. Applied Artificial Intelligence, 13:519-538.

    Google Scholar 

  • Chapanis, A. (1973). The communication of factual information through various channels. Information Storage and Retrieval, 9:215.

    Google Scholar 

  • Chapanis, A. (1975). Interactive human communication. Scientific American, 232:36-42.

    Google Scholar 

  • Chapanis, A., Ochsman, R.B., Parrish, R.N., and Weeks, G.D. (1972). Studies in interactive communication. I. The effects of four communication modes on the behavior of teams during co-operative problem-solving. Human Factors, 14:487.

    Google Scholar 

  • Cohen, P.A., Kulik, J.A., and Kulik, C.C. (1982). Educational outcomes of tutoring: A meta-analysis of findings. American Educational Research Journal, 19:237-248.

    Google Scholar 

  • Collier, G. (1985). Emotional expression. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Criag, S.D., Gholson, B., Ventura, M., Graesser, A.C., and The Tutoring Research Group (2000). Overhearing dialogues and monologues in virtual tutoring sessions: Effects on questioning and vicarious learning. International Journal of Artificial Intelligence in Education, 11:242-253.

    Google Scholar 

  • DARPA (1995). In Proceedings of the Sixth Message Understanding Conference (MUC-6). San Francisco: Morgan Kaufman Publishers.

    Google Scholar 

  • Duncan, S. (1974). Some signals and rules for taking speaking turns in conversations. In S. Weitz (Ed.), Nonverbal Communication. New York: Oxford University Press.

    Google Scholar 

  • Ekman, P. (1979). About brows: Emotional and conversational signals. In M. von Cranach, K. Froppa, W. Lepenies, and D. Ploog (Eds.), Human Ethology: Claims and Limits of a New Discipline: Contributions to the Colloquium. Cambridge: Cambridge University Press, pp. 169-248.

    Google Scholar 

  • Ekman, P. and Friesen, W.V. (1984). Unmasking the Face. Palo Alto, CA: Consulting Psychologists Press.

    Google Scholar 

  • Elliott, C., Rickel, J., and Lester, J. (1999). Lifelike pedagogical agents and affective computing: An exploratory synthesis. In M. Wooldridge and M. Veloso (Eds.), Artificial Intelligence Today. Lecture Notes in Computer Science. Berlin: Springer-Verlag, pp. 195-212.

    Google Scholar 

  • Foltz, P.W. (1996). Latent semantic analysis for text-based research. Behavior Research Methods, Instruments, and Computers, 28:197-202.

    Google Scholar 

  • Fox, B. (1993). The Human Tutorial Dialog Project. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Gagne, R.M. (1977). The Conditions of Learning, 3rd ed. New York: Holdt,Rinehart, and Winston.

    Google Scholar 

  • Graesser, A.C., Franklin, S., Wiemer-Hastings, P., and The TutoringResearch Group (1998). Simulating smooth tutorial dialog with pedagogical value. In Proceedings of the American Association for Artificial Intelligence. Menlo Park, CA: AAAI Press, pp. 163-167.

    Google Scholar 

  • Graesser, A.C. and Person, N.K. (1994). Question asking during tutoring. American Educational Research Journal, 31:104-137.

    Google Scholar 

  • Graesser, A.C., Swamer, S., and Hu, X. (1997). Quantitative discourse psychology. Discourse Processes, 23:229-264.

    Google Scholar 

  • Graesser, A.C., Wiemer-Hastings, P., Wiemer-Hastings, K., Harter, D., Person, N., and The Tutoring Research Group (2000). Using Latent Semantic Analysis to Evaluate the Contributions of Students in AutoTutor. Interactive Learning Environments, 8:129-148.

    Google Scholar 

  • Graesser, A.C., Wiemer-Hastings, K., Wiemer-Hastings, P., Kreuz, R., and The Tutoring Research Group (2000). AutoTutor: A simulation of a human tutor. Journal of Cognitive Systems Research, 1:35-51.

    Google Scholar 

  • Hu, X., Graesser, A.C., and The Tutoring Research Group (1998). UsingWordNet and latent semantic analysis to evaluate the conversational contributions of learners in the tutorial dialog. In Proceedings of the International Conference on Computers in Education, Vol. 2. Beijing, China: Springer, pp. 337-341.

    Google Scholar 

  • Hume, G.D., Michael, J.A., Rovick, A., and Evens, M.W. (1996). Hinting as a tactic in one-on-one tutoring. The Journal of the Learning Sciences, 5:23-47.

    Google Scholar 

  • Johnson, W.L. and Rickel, J. (1998). STEVE: An animated pedagogical agent for procedural training in virtual environments. SIGART Bulletin, 8:16-21.

    Google Scholar 

  • Johnson, W.L., Rickel, J., and Lester, J.C. (2000). Animated pedagogical agents: Face-to-face interaction in interactive learning environments. International Journal of Artificial Intelligence in Education, 11:47-78.

    Google Scholar 

  • Jurafsky, D. and Martin, J.H. (2000). Speech and language processing: An introduction to natural language processing, computational linguistics, and speech recognition. Upper Saddle River, NJ: Prentice Hall.

    Google Scholar 

  • Just, M.A. and Carpenter, P.A. (1987). The psychology of reading and language comprehension. Boston: Allyn and Bacon.

    Google Scholar 

  • Just, M.A. and Carpenter, P.A. (1992). A capacity theory of comprehension: Individual differences inworking memory. Psychological Review, 99:122-149.

    Google Scholar 

  • Landauer, T.K. and Dumais, S.T. (1997). A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104:211-240.

    Google Scholar 

  • Landauer, T.K., Foltz, P.W., and Laham, D. (1998). An introduction to latent semantic analysis. Discourse Processes, 25:259-284.

    Google Scholar 

  • Lepper, M.R., Woolverton, M., Mumme, D.L., and Gurtner, J.L. (1991). Motivational techniques of expert human tutors: Lessons for the design of computer-based tutors. In S.P. Lajoie and S.J. Derry (Eds.), Computers as Cognitive Tools. Hillsdale, NJ: Erlbaum, pp. 75-105.

    Google Scholar 

  • Lesgold, A., Lajoie, S., Bunzo, M., and Eggan, G. (1992). SHERLOCK: A coached practice environment for an electronics troubleshooting job. In J.H. Larkin and R.W. Chabay (Eds.), Computer-Assisted Instruction and Intelligent Tutoring Systems. Hillsdale, NJ: Erlbaum, pp. 201-238.

    Google Scholar 

  • Lester, J.C., Stone, B.A., and Stelling, G.D. (1999). Life-like pedagogical agents for mixed initiative problem solving in constructivist learning environments. User Modeling and User-Adapted Interaction, 9:1-44.

    Google Scholar 

  • Lester, J.C., Voerman, J.L., Towns, S.G., and Callaway, C.B. (1999). Deictic believability: Coordinating gesture, locomotion, and speech in lifelike pedagogical agents. Applied Artificial Intelligence, 13:383-414.

    Google Scholar 

  • Massaro, D.W. and Egan, P.B. (1996). Perceiving affect from the voice and the face. Psychonomic Bulletin and Review, 3:215-221.

    Google Scholar 

  • McCauley, L., Gholson, B., Hu, X., Graesser, A.C., and The Tutoring Research Group (1998). Delivering smooth tutorial dialog using a talking head. In Proceedings of the Workshop on Embodied Conversation Characters. Tahoe City, CA: AAAI and ACM, pp. 31-38.

    Google Scholar 

  • Merrill, D.C., Reiser, B.J., Ranney, M., and Trafton, J.G. (1992). Effective tutoring techniques: A comparison of human tutors and intelligent tutoring systems. The Journal of the Learning Sciences, 2:277-305.

    Google Scholar 

  • Moore, J.D. (1995). Participating in Explanatory Dialogues. Cambridge, MA: MIT Press.

    Google Scholar 

  • Ochsman, R.B. and Chapanis, A. (1974). The effects of 10 communication modes on behavior of teams during co-operative problem soling. Journal of Man-Machine Studies, 6:579-619.

    Google Scholar 

  • Olde, B.A., Hoeffner, J., Chipman, P., Graesser, A.C., and The Tutoring Research Group (1999). A connectionist model for part of speech tagging. In Proceedings of the American Association for Artificial Intelligence. Menlo Park, CA: AAAI Press, pp. 172-176.

    Google Scholar 

  • Palinscar, A.S. and Brown, A. (1984). Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities. Cognition and Instruction, 1:117-175.

    Google Scholar 

  • Pelachaud, C., Badler, N.I., and Steedman, M. (1996). Generating facial expressions for speech. Cognitive Science, 20:1-46.

    Google Scholar 

  • Person, N.K., Craig, S., Price, P., Hu, X., Gholson, B., Graesser, A.C., and The Tutoring Research Group (2000). Incorporating humanlike conversational behaviors into AutoTutor. The Fourth International Conference on Autonomous Agents. Barcelona, Spain, June.

  • Person, N.K., Gholson, B., Criag, S., Stewart, C.O., and Graesser, A.C. (2001). A web-based intelligent agent who manages mixed initiative dialog to optimize information retrieval. In Proceedings of theWorld Conference on Educational Multimedia, Hypermedia and Telecommunications. Tamper, Finland, in press.

  • Person, N.K. and Graesser, A.C. (1999). Evolution of discourse in cross-age tutoring. In A.M. O'Donnell and A. King (Eds.), Cognitive Perspectives on Peer Learning. Mahwah, NJ: Erlbaum, pp. 69-86.

    Google Scholar 

  • Person, N.K., Graesser, A.C., and The Tutoring Research Group (in press). Designing AutoTutor to be an effective conversational partner. In Proceedings for the 4th International Conference of the Learning Sciences. Ann Arbor, MI.

  • Person, N.K., Graesser, A.C., Kreuz, R.J., Pomeroy, V., and The TRG (in press). Simulating human tutor dialog moves in AutoTutor. International Journal of Artificial Intelligence in Education, 12.

  • Person, N.K., Klettke, B., Link, K., Kreuz, R.J., and The Tutoring Research Group (1999). The integration of affective responses into AutoTutor. In Proceeding of the International Workshop on Affect in Interactions. Siena, Italy, pp. 167-178.

  • Pierrehumbert, J. and Hirschberg, J. (1990). The meaning of intonational contours in the interpretation of discourse. In P. Cohen, J. Morgan, and M. Pollack (Eds.), Intentions in Communication. Cambridge, MA: MIT Press, pp. 271-312.

    Google Scholar 

  • Putnam, R.T. (1987). Structuring and adjusting content for students: A study of live and simulated tutoring of addition. American Educational Research Journal, 24:13-48.

    Google Scholar 

  • Rajan, S., Harter, D., Graesser, A.C., and The Tutoring ResearchGroup (2000). Backchannel feedback in human and intelligent tutoring systems. In Proceedings of the Tenth Annual Meeting of the Society for Text and Discourse. Hillsdale, NJ. pp. 102-103.

  • Reeves, B. and Nass, C. (1996). The Media Equation: How People Treat Computers, Television and New Media Like Real People and Places. New York: CSLI.

    Google Scholar 

  • Rickel, J. and Johnson, W.L. (1999). Animated agents for procedural training in virtual reality: Perception, cognition, and motor control. Applied Artificial Intelligence, 13:343-382.

    Google Scholar 

  • Rogoff, B. (1990). Apprenticeship in Thinking. New York: Oxford University Press.

    Google Scholar 

  • Takeuchi, A. and Nagao, K. (1993). Communicative facial displays as a newconversational modality. In Proceedings of the International CHI'93 Conference. Amsterdam, Netherlands. pp. 187-193.

  • VanLehn, K. (1990). Mind Bugs: The Origins of Procedural Misconceptions. Cambridge, MA: MIT Press.

    Google Scholar 

  • Vygotsky, L.S. (1978). Mind in Society. Cambridge, MA: Harvard University Press.

    Google Scholar 

  • Wiemer-Hastings, P., Graesser, A.C., Harter, D., and The Tutoring Research Group (1998). The foundations and architecture of AutoTutor. In Proceedings of the 4th International Conference on Intelligent Tutoring Systems. Berlin, Germany: Springer-Verlag, pp. 334-343.

    Google Scholar 

  • Wolfe, M.B.W., Schreiner, M.E., Rehder, B., Laham, D., Foltz, P.W., Kintsch, W., and Landauer, T.K. (1998). Learning from text: Matching readers and texts by latent semantic analysis. Discourse Processes, 25:309-336.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, University of Memphis, Memphis, TN, 38152

    Sonya Rajan, Scotty D. Craig & Barry Gholson

  2. Department of Psychology, Rhodes College, 2000 North Parkway, Memphis, TN, 38112

    Natalie K. Person

  3. Department of Psychology, Institute for Intelligent Systems, University of Memphis, Memphis, TN, 38152

    Arthur C. Graesser

Authors
  1. Sonya Rajan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Scotty D. Craig
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Barry Gholson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Natalie K. Person
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Arthur C. Graesser
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

The Tutoring Research Group

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rajan, S., Craig, S.D., Gholson, B. et al. AutoTutor: Incorporating Back-Channel Feedback and Other Human-Like Conversational Behaviors into an Intelligent Tutoring System. International Journal of Speech Technology 4, 117–126 (2001). https://doi.org/10.1023/A:1017319110294

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1017319110294

Search

Navigation