Evaluating an Authoring Tool for Model-Tracing Intelligent Tutoring Systems
We have been creating an authoring tool, the Cognitive Model SDK, which allows non-cognitive scientists and non-programmers to produce a cognitive model for model-tracing tutors [1, 2]. The SDK is in use by developers at Carnegie Learning to produce their commercial Cognitive Tutors for math. However, it has never been evaluated with regards to the strong claim that non-cognitive scientists and non-programmers could, without much effort, produce useful cognitive models with it. The research presented here shows that this can be done, using a task that past researchers have used . The models are evaluated across several metrics to see what characteristics of either them or their creators may distinguish better models from worse models. The goal of this work is to establish a baseline for future work examining how cognitive modeling can be opened up to a wider class of people.
Unable to display preview. Download preview PDF.
- 1.Blessing, S.B., Gilbert, S., Ritter, S.: Developing an authoring system for cognitive models within commercial-quality ITSs. In: Proceedings of the Nineteenth International FLAIRS Conference, pp. 497–502. AAAI Press, Melbourne (2006)Google Scholar
- 2.Blessing, S., Gilbert, S., Ourada, S., Ritter, S.: Lowering the bar for creating model-tracing intelligent tutoring systems. In: Proceedings of the 13th International Conference on Artificial Intelligence in Education, Marina del Rey, CA, pp. 443–450. IOS Press, Amsterdam (2007)Google Scholar
- 3.Suraweera, P., Mitrovic, A., Martin, B.: Constraint authoring system: An empirical evaluation. In: Proceedings of the 13th International Conference on Artificial Intelligence in Education, Marina del Rey, CA, pp. 451–458. IOS Press, Amsterdam (2007)Google Scholar
- 5.Koedinger, K.R., Anderson, J.R., Hadley, W.H., Mark, M.A.: Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education 8, 30–43 (1997)Google Scholar
- 6.VanLehn, K., Lynch, C., Schulze, K., Shapiro, J.A., Shelby, R., Taylor, L., Treacy, D., Weinstein, A., Wintersgill, M.: The andes physics tutoring system: Lessons learned. International Journal of Artificial Intelligence and Education 15(3) (2005)Google Scholar
- 7.Murray, T., Blessing, S., Ainsworth, S.: Authoring tools for advanced technology educational software. Kluwer Academic Publishers (2003)Google Scholar
- 8.Woolf, B.P., Cunningham, P.: Building a community memory for intelligent tutoring systems. In: AAAI 1987, pp. 82–89 (1987)Google Scholar
- 9.Anderson, J.R.: Rules of the Mind. Erlbaum, Hillsdale (1993)Google Scholar
- 10.Ainsworth, S.E., Fleming, P.F.: Evaluating a mixed-initiative authoring environment: is redeem for real? In: Proceedings of the 12th International Conference on Artificial Intelligence in Education, pp. 9–16. IOS Press, Amsterdam (2005)Google Scholar
- 11.Halff, H.M., Hsieh, P.Y., Wenzel, B.M., Chudanov, T.J., Dirnberger, M.T., Gibson, E.G., Redfield, C.L.: Requiem for a development system: Reflections on knowledge-based, generative instruction. In: Murray, T., Blessing, S., Ainsworth, S. (eds.) Authoring tools for advanced technology educational software, Kluwer Academic Publishers (2003)Google Scholar
- 12.Mathan, S., Koedinger, K., Corbett, A., Hyndman, A.: Effective strategies for bridging gulfs between users and computer systems. In: Proceedings of HCI-Aero 2000: International Conference on Human Computer Interaction in Aeronautics, Toulouse, France, September 27- 29, pp. 197–202 (2000)Google Scholar
- 13.Aleven, V., Sewall, J., McLaren, B.M., Koedinger, K.R.: Rapid authoring of intelligent tutors for real-world and experimental use. In: Kinshuk, R., Koper, P., Kommers, P., Kirschner, D.G. (eds.) Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (ICALT 2006), pp. 847–851. IEEE Computer Society, Los Alamitos (2006)CrossRefGoogle Scholar