The Cognitive Tutor Authoring Tools (CTAT): Preliminary Evaluation of Efficiency Gains

  • Vincent Aleven
  • Bruce M. McLaren
  • Jonathan Sewall
  • Kenneth R. Koedinger
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4053)


Intelligent Tutoring Systems have been shown to be effective in a number of domains, but they remain hard to build, with estimates of 200-300 hours of development per hour of instruction. Two goals of the Cognitive Tutor Authoring Tools (CTAT) project are to (a) make tutor development more efficient for both programmers and non-programmers and (b) produce scientific evidence indicating which tool features lead to improved efficiency. CTAT supports development of two types of tutors, Cognitive Tutors and Example-Tracing Tutors, which represent different trade-offs in terms of ease of authoring and generality. In preliminary small-scale controlled experiments involving basic Cognitive Tutor development tasks, we found efficiency gains due to CTAT of 1.4 to 2 times faster. We expect that continued development of CTAT, informed by repeated evaluations involving increasingly complex authoring tasks, will lead to further efficiency gains.


Cognitive Model Production Rule Efficiency Gain Rule Activation Intelligent Tutor System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ainsworth, S.E., Fleming, P.F.: Evaluating a mixed-initiative authoring environment: is redeem for real? In: Proceedings of the 12th International Conference on Artificial Intelligence in Education, pp. 9–16. IOS Press, Amsterdam (2005)Google Scholar
  2. 2.
    Aleven, V., Koedinger, K.: An effective metacognitive strategy: learning by doing and explaining with a computer-based Cognitive Tutor. Cognitive Science 26, 147–179 (2002)CrossRefGoogle Scholar
  3. 3.
    Aleven, V., Rosé, C.: Authoring plug-in tutor agents by demonstration: Rapid, rapid tutor development. In: Proceedings of the 12th International Conference on Artificial Intelligence in Education, AIED 2005, pp. 735–737. IOS Press, Amsterdam (2005)Google Scholar
  4. 4.
    Anderson, J.R.: Rules of the Mind. Lawrence Erlbaum, Mahwah (1993)Google Scholar
  5. 5.
    Friedman-Hill, E.: Jess in Action. Manning Publications Co. (2003)Google Scholar
  6. 6.
    Koedinger, K.R., Aleven, V., Heffernan, N.T., McLaren, B.M., Hockenberry, M.: Opening the Door to Non-programmers: Authoring Intelligent Tutor Behavior by Demonstration. In: Lester, J.C., Vicari, R.M., Paraguaçu, F. (eds.) ITS 2004. LNCS, vol. 3220, pp. 162–174. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  7. 7.
    Koedinger, K.R., Aleven, V., Heffernan, N.T.: Toward a rapid development environment for Cognitive Tutors. In: Proceedings of the 11th International Conference on Artificial Intelligence in Education, AI-ED 2003, pp. 455–457. IOS Press, Amsterdam (2003)Google Scholar
  8. 8.
    Koedinger, K.R., Anderson, J.R., Hadley, W.H., Mark, M.A.: Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education 8, 30–43 (1997)Google Scholar
  9. 9.
    Matsuda, N., Cohen, W.W., Koedinger, K.R.: Applying programming by demonstration in an intelligent authoring tool for Cognitive Tutors. In: AAAI Workshop on Human Comprehensible Machine Learning, pp. 1–8. AAAI Association, Menlo Park (2005)Google Scholar
  10. 10.
    McLaren, B.M., Bollen, L., Walker, E., Harrer, A., Sewall, J.: Cognitive Tutoring of collaboration: developmental and empirical steps toward realization. In: Proceedings of the Conference on Computer Supported Collaborative Learning (CSCL 2005), Taipei, Taiwan (2005)Google Scholar
  11. 11.
    McLaren, B.M., Lim, S.-J., Gagnon, F., Yaron, D., Koedinger, K.R.: Studying the Effects of Personalized Language and Worked Examples in the Context of a Web-Based Intelligent Tutor. In: Ikeda, M., Ashley, K.D., Chan, T.-W. (eds.) ITS 2006. LNCS, vol. 4053, pp. 318–328. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  12. 12.
    Murray, T.: Authoring intelligent tutoring systems: An analysis of the state of the art. International Journal of Artificial Intelligence in Education 10, 98–129 (1999)Google Scholar
  13. 13.
    Murray, T., Blessing, S., Ainsworth, S.E. (eds.): Tools for Advanced Technology Learning Environments. Kluwer Academic Publishers, Amsterdam (2003)Google Scholar
  14. 14.
    Nuzzo-Jones, G., Walonoski, J.A., Heffernan, N.T., Livak, T.: The eXtensible Tutor Architecture: A new foundation for ITS. In: Proceedings of the 12th Artificial Intelligence In Education, pp. 902–904. IOS Press, Amsterdam (2005)Google Scholar
  15. 15.
    Ogan, A., Aleven, V., Jones, C.: Improving intercultural competence by predicting in French film. In: Proceedings of E-Learn 2005. AACE, Norfolk (2005)Google Scholar
  16. 16.
    Razzaq, L., Feng, M., Nuzzo-Jones, G., Heffernan, N.T., Koedinger, K.R., et al.: The Assistment Project: Blending Assessment and Assisting. In: Proceedings of the 12th Artificial Intelligence in Education, pp. 555–562. IOS Press, Amsterdam (2005)Google Scholar
  17. 17.
    Suraweera, P., Mitrovic, A., Martin, B.: A Knowledge Acquisition System for Constraint-based Intelligent Tutoring Systems. In: Proceedings of the 12th international conference on Artificial Intelligence in Education, pp. 638–645. IOS Press, Amsterdam (2005)Google Scholar
  18. 18.
    VanLehn, K., Lynch, C., Schulze, K., Shapiro, J.A., Shelby, R., et al.: The Andes physics tutoring system: five years of evaluations. In: Proceedings of the 12th international conference on Artificial Intelligence in Education. IOS Press, Amsterdam (2005)Google Scholar
  19. 19.
    Woolf, B.P., Cunningham, P.: Building a community memory for intelligent tutoring systems, pp. 82–89. AAAI Press, Menlo Park (1987)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Vincent Aleven
    • 1
  • Bruce M. McLaren
    • 1
  • Jonathan Sewall
    • 1
  • Kenneth R. Koedinger
    • 1
  1. 1.Human-Computer Interaction InstituteCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations