Educational Psychology Review

, Volume 19, Issue 3, pp 239–264 | Cite as

Exploring the Assistance Dilemma in Experiments with Cognitive Tutors

  • Kenneth R. KoedingerEmail author
  • Vincent Aleven
Original Article


Intelligent tutoring systems are highly interactive learning environments that have been shown to improve upon typical classroom instruction. Cognitive Tutors are a type of intelligent tutor based on cognitive psychology theory of problem solving and learning. Cognitive Tutors provide a rich problem-solving environment with tutorial guidance in the form of step-by-step feedback, specific messages in response to common errors, and on-demand instructional hints. They also select problems based on individual student performance. The learning benefits of these forms of interactivity are supported, to varying extents, by a growing number of results from experimental studies. As Cognitive Tutors have matured and are being applied in new subject-matter areas, they have been used as a research platform and, particularly, to explore interactive methods to support metacognition. We review experiments with Cognitive Tutors that have compared different forms of interactivity and we reinterpret their results as partial answers to the general question: How should learning environments balance information or assistance giving and withholding to achieve optimal student learning? How best to achieve this balance remains a fundamental open problem in instructional science. We call this problem the “assistance dilemma” and emphasize the need for further science to yield specific conditions and parameters that indicate when and to what extent to use information giving versus information withholding forms of interaction.


Cognitive tutors Intelligent tutoring system Self-explanation Desirable difficulties Cognitive load Worked examples Tutoring Problem solving Interaction Cognitive psychology Experimental studies Cognitive theory 


  1. Aleven, V., & Koedinger, K. R. (2000a). Limitations of student control: Do students know when they need help? In G. Gauthier, C. Frasson, & K. VanLehn (Eds.), Proceedings of the 5th International Conference on Intelligent Tutoring Systems, ITS 2000 (pp. 292–303). Berlin: Springer.Google Scholar
  2. Aleven, V., & Koedinger, K. R. (2000b). The need for tutorial dialog to support self-explanation. In C. P. Rose & R. Freedman (Eds.), Building dialogue systems for tutorial applications, papers of the 2000 AAAI Fall Symposium (pp. 65–73). Technical Report FS-00-01. Menlo Park, CA: AAAI.Google Scholar
  3. Aleven, V., & Koedinger, K. R. (2002). An effective metacognitive strategy: Learning by doing and explaining with a computer-based Cognitive Tutor. Cognitive Science, 26(2):147–179.CrossRefGoogle Scholar
  4. Aleven, V., McLaren, B., Roll, I., & Koedinger, K. (2006). Toward meta-cognitive tutoring: A model of help seeking with a Cognitive Tutor. International Journal of Artificial Intelligence and Education, 16, 101–128.Google Scholar
  5. Aleven, V., Stahl, E., Schworm, S., Fischer, F., & Wallace, R. M. (2003). Help seeking and help design in interactive learning environments. Review of Educational Research, 73(2), 277–320.CrossRefGoogle Scholar
  6. Anderson, J. R. (1993). Rules of the mind. Hillsdale, NJ: Erlbaum.Google Scholar
  7. Anderson, J. R., Conrad, F. G., & Corbett, A. T. (1989). Skill acquisition and the Lisp Tutor. Cognitive Science, 13, 467–505.CrossRefGoogle Scholar
  8. Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. The Journal of the Learning Sciences, 4(2), 167–207.CrossRefGoogle Scholar
  9. Anderson, J. R., & Lebiere, C. (1998). The Atomic Components of Thought. Hillsdale, NJ: Erlbaum.Google Scholar
  10. Anderson, J. R., Reder, L. M., & Simon, H. A. (1996). Situated learning and education. Educational Researcher, 25(4), 5–11.Google Scholar
  11. Anderson, J. R., Reder, L. M., & Simon, H. A. (1998). Radical constructivism and cognitive psychology. In D. Ravitch (Ed.), Brookings papers on education policy 1998. Washington, DC: Brookings Institute.Google Scholar
  12. Atkinson, R. K., Derry, S. J., Renkl, A., & Wortham, D. (2000). Learning from examples: Instructional principles from the worked examples research. Review of Educational Research, 70(2), 181–214.Google Scholar
  13. Atkinson, R. K., Renkl, A., & Merrill, M. M. (2003). Transitioning from studying examples to solving problems: Effects of self-explanation prompts and fading worked-out steps. Journal of Educational Psychology, 95(4), 774–783.CrossRefGoogle Scholar
  14. Baker, R., Corbett, A., Koedinger, K. R., Evenson, S., Roll, I., Wagner, A., et al. (2006). Adapting to when students game an intelligent tutoring system. In M. Ikeda, K. D. Ashley, & T.-W. Chan (Eds.), Proceedings of the 8th International Conference on Intelligent Tutoring Systems (pp. 392–401). Berlin: Springer.Google Scholar
  15. Baker, R. S., Corbett, A. T., Koedinger, K. R., & Wagner, A. Z. (2004). Off-task behavior in the Cognitive Tutor classroom: When students “game the system.” Proceedings of ACM CHI 2004: Computer-Human Interaction (pp. 383–390). New York: ACM.Google Scholar
  16. Bloom, B. S. (1984). The 2-sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher, 13, 4–16.Google Scholar
  17. Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13(2), 145–182.CrossRefGoogle Scholar
  18. Chi, M. T. H., de Leeuw, N., Chiu, M. H., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18(3), 439–477.CrossRefGoogle Scholar
  19. Clark, R. C., & Mayer, R. E. (2003). e-Learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning. San Francisco: Jossey-Bass.Google Scholar
  20. Corbett, A. (2001). Cognitive computer tutors: solving the two-sigma problem. In M. Bauer, P. J. Gmytrasiewicz, & J. Vassileva (Eds.), Proceedings of the 2001 International Conference on User Modeling (pp. 137–147). Berlin: Springer.Google Scholar
  21. Corbett, A., & Anderson, J. R. (1995). Knowledge tracing: modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4, 253–278.CrossRefGoogle Scholar
  22. Crowley, R. S., Legowski, E., Medvedeva, O., & Tseytlin, E. (2005). An ITS for medical classification problem-solving: Effects of tutoring and representations. In C. K. Looi, G. McCalla, B. Bredeweg, & J. Breuker (Eds.), Proceedings of the 12th International Conference on Artificial Intelligence. Amsterdam, The Netherlands: AIED.Google Scholar
  23. Crowley, R. S., & Medvedeva, O. (2006). An intelligent tutoring system for visual classification problem solving. Artificial Intelligence in Medicine, 36(1), 85–117.PubMedCrossRefGoogle Scholar
  24. Duch, B., Gron, S., & Allen, D. (2001). The Power of Problem-Based Learning; A Practical “How To” For Teaching Undergraduate Courses in Any Discipline. Sterling, Virginia: Stylus.Google Scholar
  25. Eberts, R. E. (1997). Computer-based instruction. In M. G. Helander, T. K. Landauer, & P. V. Prabhu (Eds.), Handbook of Human–Computer Interaction (pp. 825–847). Amsterdam, The Netherlands: Elsevier.Google Scholar
  26. Guskey, T. R. (1987). The essential elements of mastery learning. Journal of Classroom Interaction, 22(2), 19–22.Google Scholar
  27. Kalyuga, S., Chandler, P., Tuovinen, J., & Sweller, J. (2001). When problem solving is superior to studying worked examples. Journal of Educational Psychology, 93(3), 579–588.CrossRefGoogle Scholar
  28. Krajcik, J., & Starr, M. (2001). Learning science content in a project-based environment. In R. Tinker & J. S. Krajcik (Eds.), Portable technologies: Science learning in context. The Netherlands: Kluwer.Google Scholar
  29. Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75–86.CrossRefGoogle Scholar
  30. Klahr, D., & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. Psychological Science, 15(10), 661–667.PubMedCrossRefGoogle Scholar
  31. Kluger, A. N., & DeNisi, A. (1996). The effects of feedback intervention on performance: A historical review, a meta-analysis and a preliminary feedback intervention theory. Psychological Bulletin, 112(2), 254–284.CrossRefGoogle Scholar
  32. Koedinger, K. R. (2002). Toward evidence for instructional design principles: Examples from Cognitive Tutor Math 6. In D. S. Mewborn, P. Sztajn, D. Y. White, H. G. Wiegel, R. L. Bryant, & K. Nooney (Eds.), Proceedings of twenty-fourth annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education Vol. 1 (pp. 21–49). Columbus, OH: ERIC Clearinghouse for Science, Mathematics, and Environmental Education.Google Scholar
  33. Koedinger, K. R., & Anderson, J. R. (1993). Reifying implicit planning in geometry: Guidelines for model-based intelligent tutoring system design. In S. Lajoie & S. Derry (Eds.), Computers as cognitive tools. Hillsdale, NJ: Erlbaum.Google Scholar
  34. Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30–43.Google Scholar
  35. Koedinger, K. R., & Corbett, A. T. (2006). Cognitive tutors: Technology bringing learning science to the classroom. In K. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences. Cambridge, MA: Cambridge University Press.Google Scholar
  36. Lepper, M. R., & Malone, T. W. (1987). Intrinsic motivation and instructional effectiveness in computer-based education. In R. E. Snow & M. J. Farr (Eds.), Aptitude, learning and instruction: Volume III Conative and affective process analyses. Hillsdale, NJ: Erlbaum.Google Scholar
  37. Mathan, S. A. (2003). Recasting the feedback debate: Benefits of tutoring error detection and correction skills. Unpublished dissertation, Carnegie Mellon University, Pittsburgh, PA.Google Scholar
  38. Mathan, S. A., & Koedinger, K. R. (2005) Fostering the Intelligent Novice: Learning from errors with metacognitive tutoring. Educational Psychologist, 40(4), 257–265.CrossRefGoogle Scholar
  39. McKendree, J. E. (1990). Effective feedback content for tutoring complex skills. Human Computer Interaction, 5, 381–414.CrossRefGoogle Scholar
  40. McLaren, B. M., Lim, S., Gagnon, F., Yaron, D., & Koedinger, K. R. (2006). Studying the effects of personalized language and worked examples in the context of a web-based intelligent tutor. In M. Ikeda, K. D. Ashley, & T.-W. Chan (Eds.), Proceedings of the 8th International Conference on Intelligent Tutoring Systems (pp. 318–328). Berlin: Springer.Google Scholar
  41. Merrill, D. C., Reiser, B. J., Ranney, M., & Trafton, J. G. (1992). Effective tutoring techniques: Comparison of human tutors and intelligent tutoring systems. Journal of the Learning Sciences, 2(3), 277–305.CrossRefGoogle Scholar
  42. Morgan, P., & Ritter, S. (2002). An experimental study of the effects of Cognitive Tutor® Algebra I on student knowledge and attitude (Available from Carnegie Learning, Inc., 1200 Penn Avenue, Suite 150, Pittsburgh, PA 15222). Available at:
  43. Nathan, M. J. (1998). Knowledge and situational feedback in a learning environment for algebra story problem solving. Interactive Learning Environments, 5, 135–159.CrossRefGoogle Scholar
  44. Ohlsson, S. (1996). Learning from performance errors. Psychological Review, 103(2), 241–262.CrossRefGoogle Scholar
  45. Ohlsson, S., & Mitrovic, A. (2006). Constraint-based knowledge representation for individualized instruction. Computer Science and Information Systems, 3(1), 1–22.CrossRefGoogle Scholar
  46. Paas, F., & Van Merrienboer, J. (1994). Variability of worked examples and transfer of geometry problem-solving skills: A cognitive-load approach. Journal of Educational Psychology, 86, 122–133.CrossRefGoogle Scholar
  47. Pavlik, Jr., P. I. (2007). Timing is an order: Modeling order effects in the learning of information. In F. E. Ritter, J. Nerb, T. O’Shea & E. Lehtinen (Eds.), In order to learn: How the sequences of topics affect learning, (pp. 137–150). New York, NY: Oxford University Press.Google Scholar
  48. Plano, G. S. (2004). The Effects of the Cognitive Tutor Algebra on student attitudes and achievement in a 9th grade Algebra course. Unpublished doctoral dissertation, Seton Hall University, South Orange, NJ.Google Scholar
  49. Renkl, A. (2002). Learning from worked-out examples: Instructional explanations supplement self-explanations. Learning & Instruction, 12, 529–556.CrossRefGoogle Scholar
  50. Renkl, A., Atkinson, R. K., & Große, C. S. (2004) How fading worked solution steps works—A cognitive load perspective. Instructional Science, 32, 59–82.CrossRefGoogle Scholar
  51. Renkl, A., Stark, R., Gruber, H., & Mandl, H. (1998). Learning from worked-out examples: The effects of example variability and elicited self-explanations. Contemporary Educational Psychology, 23, 90–108.PubMedCrossRefGoogle Scholar
  52. Roll, I., Aleven, V., McLaren, B. M., Ryu, E., Baker, R., & Koedinger, K. R. (2006). The help tutor: Does metacognitive feedback improve students’ help-seeking actions, skills and learning? In M. Ikeda, K. D. Ashley, & T.-W. Chan (Eds.), Proceedings of the 8th International Conference on Intelligent Tutoring Systems (pp. 360–369). Berlin: Springer.Google Scholar
  53. Sarkis, H. (2004). Cognitive Tutor Algebra 1 program evaluation, Miami-Dade County Public Schools. Lighthouse Point, FL: The Reliability Group.Google Scholar
  54. Schmidt, R. A., & Bjork, R. A. (1992). New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychological Science, 3(4), 207–217.CrossRefGoogle Scholar
  55. Schoenfeld, A. (1983). Beyond the purely cognitive: Belief systems, social cognitions, and metacognitions as driving forces in intellectual performance. Cognitive Science, 7, 329–363.CrossRefGoogle Scholar
  56. Schwartz, D. L., & Bransford, J. D. (1998). A time for telling. Cognition and Instruction, 16(4), 475–522.CrossRefGoogle Scholar
  57. Schwonke, R., Wittwer, J., Aleven, V., Salden, R. J. C. M., Krieg, C., & Renkl, A. (2007). Can tutored problem solving benefit from faded worked-out examples? Paper presented at the European Cognitive Science Conference 2007, May 23–;27. Delphi, Greece.Google Scholar
  58. Schworm, S., & Renkl, A. (2002). Learning by solved example problems: Instructional explanations reduce self-explanation activity. In W. D. Gray & C. D. Schunn (Eds.), Proceeding of the 24th Annual Conference of the Cognitive Science Society (pp.816–821). Mahwah, NJ: Erlbaum.Google Scholar
  59. Shneyderman, A. (2001). Evaluation of the Cognitive Tutor Algebra I program, Miami-Dade County Public Schools, Office Of Evaluation And Research. Retrieved from:
  60. Siegler, R. S. (2002). Microgenetic studies of self-explanation. In N. Garnott & J. Parziale (Eds.), Microdevelopment: A process-oriented perspective for studying development and learning (pp. 31–58). Cambridge, MA: Cambridge University Press.Google Scholar
  61. Slamecka, N. J., & Graf, P. (1978). The generation effect: Delineation of a phenomenon. Journal of Experimental Psychology: Human Learning and Memory, 4, 592–604.CrossRefGoogle Scholar
  62. Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285.CrossRefGoogle Scholar
  63. Sweller, J., & Cooper, G. A. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and Instruction, 2, 59–89.CrossRefGoogle Scholar
  64. Trafton, J. G., & Reiser, B. J. (1993). The contributions of studying examples and solving problems to skill acquisition. In M. Polson (Ed.), Proceedings of the Fifteenth annual conference of the Cognitive Science Society (pp. 1017–1022). Hillsdale, N.J.: Erlbaum.Google Scholar
  65. VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16, 227–265.Google Scholar
  66. VanLehn, K., Jones, R. M., & Chi, M. T. H. (1992). A model of the self-explanation effect. The Journal of the Learning Sciences, 2, 1–59.CrossRefGoogle Scholar
  67. Vanlehn, K., Lynch, C., Schultz, K., Shapiro, J. A., Shelby, R. H., Taylor, L., et al. (2005). The Andes physics tutoring system: Lessons learned. International Journal of Artificial Intelligence in Education, 15(3), 147–204.Google Scholar
  68. Vygotsky, L. S. (1978). Mind in society. Cambridge, MA: Harvard University Press.Google Scholar
  69. Ward, M., & Sweller, J. (1990). Structuring effective worked examples. Cognition and Instruction, 7, 1–39.CrossRefGoogle Scholar
  70. White, B. Y., & Frederiksen, J. R. (1998). Inquiry, modeling, and metacognition: Making science accessible to all students. Cognition and Instruction, 16(1), 3–118.CrossRefGoogle Scholar
  71. Zhu, X., & Simon, H. A. (1987). Learning mathematics from examples and by doing. Cognition and Instruction, 4(3), 137–166.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.Human-Computer Interaction Institute and Psychology DepartmentCarnegie Mellon UniversityPittsburghUSA
  2. 2.Human-Computer Interaction InstituteCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations