User Modeling and User-Adapted Interaction

, Volume 12, Issue 4, pp 371–417

Using Bayesian Networks to Manage Uncertainty in Student Modeling

  • Cristina Conati
  • Abigail Gertner
  • Kurt VanLehn

Abstract

When a tutoring system aims to provide students with interactive help, it needs to know what knowledge the student has and what goals the student is currently trying to achieve. That is, it must do both assessment and plan recognition. These modeling tasks involve a high level of uncertainty when students are allowed to follow various lines of reasoning and are not required to show all their reasoning explicitly. We use Bayesian networks as a comprehensive, sound formalism to handle this uncertainty. Using Bayesian networks, we have devised the probabilistic student models for Andes, a tutoring system for Newtonian physics whose philosophy is to maximize student initiative and freedom during the pedagogical interaction. Andes’ models provide long-term knowledge assessment, plan recognition, and prediction of students’ actions during problem solving, as well as assessment of students’ knowledge and understanding as students read and explain worked out examples. In this paper, we describe the basic mechanisms that allow Andes’ student models to soundly perform assessment and plan recognition, as well as the Bayesian network solutions to issues that arose in scaling up the model to a full-scale, field evaluated application. We also summarize the results of several evaluations of Andes which provide evidence on the accuracy of its student models.

student modelling Intelligent Tutoring Systems dynamic Bayesian networks 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Albacete, P. and Van Lehn, K.: 2000, The conceptual helper: An intelligent tutoring system for teaching fundamental physics concepts. Proceedings of Intelligent Tutoring Systems, 5th International Conference, ITS2000, Montreal, Canada, Lecture Notes in Computer Science 1839, Springer, pp. 564-573.Google Scholar
  2. Albrecht, D.W., Zukerman, I. and Nicholson, A. E.: 1999, Bayesian Models for Keyhole Plan Recognition in an Adventure Game. User Modeling and User-Adapted Interaction, 8(1-2), 5-47.CrossRefGoogle Scholar
  3. Anderson, J. R., Corbett, A. T., Koedinger, K. R. and Pelletier, R.: 1995, Cognitive Tutors: Lessons Learned. The Journal of the Learning Sciences, 4(2), 167-207.CrossRefGoogle Scholar
  4. Bauer, M.: 1995, A Dempster-Shafer approach to modeling agents preferences in plan recognition. User Modeling and User-Adapted Interaction, 5(3-4), 317-348.CrossRefGoogle Scholar
  5. Breese, J., Goldman, R. and Wellman, P.: 1994, Introduction to the special section on knowledge-based construction of probabilistic and decision models. IEEE Transactions on Systems, Man, and Cybernetics, 24, pp. 1577-1579.Google Scholar
  6. Bunt, A., Conati, C., Hugget, M. and Muldner, K.: 2001, On Improving the Effectiveness of Open Learning Environments through Tailored Support for Exploration. Proceedings of AIED 2001, 10thWorld Conference of Artificial Intelligence and Education, SanAntonio, TX, U.S.A., pp. 365-376.Google Scholar
  7. Calistri-Yeh, R. J.: 1991, An {A*} approach to robust plan recognition for intelligent interfaces. N. G. Bourbakis (ed.): Applications of Learning and Planning Methods. World Scientific Publishing Co., pp. 227-251.Google Scholar
  8. Carberrry, S.: 2001, Techniques for Plan Recognition. User Modeling and User-Adapted Interaction, 11, 31-48.CrossRefGoogle Scholar
  9. Carberry, S.: 1990, Incorporating default inferences into plan recognition. Proceedings of the Eighth National Conference on Artificial Intelligence, Menlo Park, CA, MIT Press, pp. 471-478.Google Scholar
  10. Charniak, E. and Goldman, R. P.: 1992, A Bayesian model of plan recognition. Artificial Intelligence, 64, 53-79.CrossRefGoogle Scholar
  11. Chi, M. T. H.: in press, Self-Explaining: The dual process of generating inferences and repairing mental models. Advances in Instructional Psychology.Google Scholar
  12. Conati, C. and Carenini, G.: 2001, Generating Tailored Examples to Support Learning via Self-explanation. Proceedings of IJCAI’01, 17th International Joint Conference on Artificial Intelligence, Seattle, WA, U.S.A., pp. 1301-1306.Google Scholar
  13. Conati, C. and VanLehn, K.: 1996a, POLA: A student modeling framework for probabilistic on-line assessment of problem solving performance. Proceedings of UM-96, 5th International Conference on User Modeling, Kailua-Kona, Hawaii, U.S.A., UserModeling, Inc., pp. 75-82.Google Scholar
  14. Conati, C. and VanLehn, K.: 1996b, Probabilistic plan recognition for cognitive apprenticeship. Proceedings of the 18th Annual Meeting of the Cognitive Science Society, San Diego, CA. U.S.A., Erlbaum, pp. 403-408.Google Scholar
  15. Conati, C., Larkin, J., VanLehn, K.: 1997, A computer framework to support self-explanation. In duBouley B, Mizoguchi R. (eds) Artificial Intelligence in Education: Proceedings of the 8th World WNF, IOS Press, Ohmsha, pp. 279-286.Google Scholar
  16. Conati, C. and VanLehn, K.: 2000, Toward Computer-based Support of Meta-cognitive Skills: A Computational Framework to Coach Self-Explanation. International Journal of Artificial Intelligence in Education, 11(4), 389-415.Google Scholar
  17. Corbett, A., McLaughlin, M. and Scarpinatto, K. C.: 2000, Modeling Student Knowledge: Cognitive Tutors in High School and College. User Modeling and User-Adapted Interaction, 10(2-3), pp. 81-108.CrossRefGoogle Scholar
  18. Corbett, A. T. and Anderson, J. R.: 1995, Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4(4), 253-278.CrossRefGoogle Scholar
  19. Corbett, A. T. and Bhatnagar, A.: 1997, Student modeling in the ACT programming tutor: Adjusting a procedural learning model with declarative knowledge. User Modeling: Proceedings of the Sixth International Conference, UM97, Chia Laguna, Italy, Springer, Wien, New York, pp. 243-254.Google Scholar
  20. Dean, T. and Kanazawa, K.: 1989, A Model forReason ing about Persistence and Causation. Computational Intelligence, 5(3), 142-150.Google Scholar
  21. Gertner, A., Conati, C. and VanLehn, K.: 1998, Procedural help in Andes: Generating hints using a Bayesian network student model. Proceedings of the 15th National Conference on Artificial Intelligence, Madison, Wisconsin, U.S.A., pp. 106-111.Google Scholar
  22. Gertner, A. and VanLehn, K.: 2000, Andes: a coached problem solving environment for physics. Proceedings of Intelligent Tutoring Systems, 5th International Conference, ITS2000, Montreal, Canada, Lecture Notes in Computer Science 1839, Springer, pp. 131-142.Google Scholar
  23. Henrion, M.: 1989, Some practical issues in constructing belief networks. Proceedings of the 3rd Conference on Uncertainty in Artificial Intelligence, Elsevier Scien ce, pp. 161-173.Google Scholar
  24. Horvitz, E. and Barry, M.: 1995, Display of Information for Time-Critical Decision Making. Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence, Montreal, Canada, Morgan Kaufmann: San Francisco, pp. 296-305.Google Scholar
  25. Horvitz, E., Breese, J., Heckerman, D., Hovel, D. and Rommelse, R.: 1998, The Lumiere Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users. Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence, Madison, WI, U.S.A., Morgan Kaufmann: San Francisco, pp. 256-265.Google Scholar
  26. Huber, M. J., Durfee, E. H. and Wellman, M. P.: 1994, The automated mapping of plans for plan recognition. Proceedings of the 10th Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann, pp. 344-351.Google Scholar
  27. Jameson, A.: 1996, Numerical uncertainty management in user and student modeling: An overview of systems and issues. User Modeling and User-Adapted Interaction, 5(3-4), 193-251.CrossRefGoogle Scholar
  28. Just, M. and Carpenter, P.: 1986, The Psychology of Reading and Language Comprehension. Boston.Google Scholar
  29. Katz, S., Lesgold, A., Eggan, G. and Gordin, M.: 1992, Modelling the student in Sherlock II. Journal of Artificial Intelligence in Education, 3(4), 495-518.Google Scholar
  30. Koedinger, K. and Anderson, J. R.: 1993, Reifying implicit planning ingeometry: Guidelines for model-based intelligent tutoring system design. S. P. Lajoie, and S. J. Derry (eds.). Computers as cognitive tools. Hillsdale, NJ, Lawrence Erlbaum Associates, pp. 15-46.Google Scholar
  31. Koedinger, K. R., Anderson, J. R., Hadley, W. H. and Mark, M. A.: 1995, Intelligent tutoring goes to school in the big city. Proceedings of the 7thWorld Conference on Artificial Intelligence and Education, Charlottesville, NC, AACE, pp. 421-428.Google Scholar
  32. Mahoney, S. M. and Laskey, K. B.: 1998, Constructing situation-specific Belief networks. Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence, S. Francisco, CA, U.S.A., Morgan Kaufmann, pp. 370-378.Google Scholar
  33. Martin, J. and VanLehn, K.: 1993, OLAE: Progress toward a multi-activity, Bayesian student modeller. Artificial Intelligence in Education, 1993: Proceedings of AI-ED 93, Charlottesville, VA, Association for the Advancement of Computing in Education, pp. 410-417.Google Scholar
  34. Martin, J. and VanLehn, K.: 1994, Discrete factor analysis: Learning hidden variables in Bayesian networks, LRDC, University of Pittsburgh: Technical report.Google Scholar
  35. Martin, J. and VanLehn, K.: 1995, Student assessment using Bayesian nets. International Journal of Human-Computer Studies, 42, 575-591.CrossRefGoogle Scholar
  36. Mislevy, R.: 1995, Probability-based inference in cognitive diagnosis. P. Nichols, S. Chipman, and R. Brennan (eds.). Cognitive Diagnostic Assessment. Hillsdale, NJ., Erlbaum, pp. 43-71.Google Scholar
  37. Mislevy, R. J. and Gitomer, D. H.: 1996, The Role of Probability-Based Inference in an Intelligent Tutoring System. User Modeling and User-Adapted Interaction, 5(3-4), 253-282.CrossRefGoogle Scholar
  38. Murray, C. and VanLehn, K.: 2000, DT Tutor: A decision-theoretic dynamic approach for optimal selection of tutorial actions. Proceedings of Intelligent Tutoring Systems, 5th International Conference, ITS2000, Montreal, Canada, Lecture Notes in Computer Science 1839, Springer, pp. 153-162.Google Scholar
  39. Norman, D. A.: 1981, Categorization of action slips. Psychological Review, 88(1), 1-15.CrossRefGoogle Scholar
  40. Pearl, J.: 1988, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Mateo, CA, Morgan-Kaufmann.Google Scholar
  41. Petrushin, V. A., Sinitsa, K.M. and Zherdienko, V.: 1995, Probabilistic approach to adaptive student knowledge assessment: methodology and experiment. Artificial Intelligence in Education: Proceedings of AI-ED ‘95, Washington, DC, U.S.A., pp. 51-58.Google Scholar
  42. Polk, T. A., VanLehn, K. and Kalp, D.: 1995, ASPM2: Progress toward the analysis of symbolic parameter models. P. D. Nichols, S. F. Chipman, and R. L. Brennan (eds.). Cognitively Diagnostic Assessment. Mahwah, NH, Erlbaum, pp. 127-141.Google Scholar
  43. Reggia, J. A. and D’Autrechy, C. L.: 1990, Parsimonious covering theory in cognitive diagnosis and adaptive instruction. N. Frederiksen, R. Glaser, A. Lesgold and M. G. Shafto (eds.). Diagnostic Monitoring of Skill and Knowledge Acquisition. Hillsdale, NJ, Lawrence Erlbaum Associates, pp. 191-216.Google Scholar
  44. Renkl, A.: 1997, Learning from worked-examples: A study on individual differences. Cognitive Science, 21(1), 1-30.CrossRefGoogle Scholar
  45. Reye, J.: 1998, Two-phase updating of student models based on dynamic belief networks. Proceedings of the 4th International Conference on Intelligent Tutoring Systems, ITS ‘98, San Antonio, TX, U.S.A., Lecture Notes in Computer Science 1452, Springer, pp. 274-283.Google Scholar
  46. Russell, S. and Norvig, P.: 1995, Artificial Intelligence: A Modern Approach. Los Altos, CA, Morgan-Kaufman.Google Scholar
  47. Schulze, K.G., Correll, D., Shelby, R.N., Wintersgill, M. C. and Gertner, A.: 1998, ACLIPS problem solver for Newtonian physics force problems. C. Giarratano, and G. Riley (eds.). Expert Systems Principles and Programming. Boston, MA, PWS Publishing Company.Google Scholar
  48. Schulze, K. G., Shelby, R. N., Treacy, D. J., Wintersgill, M. C., VanLehn, K. and Gertner, A.: 2000, Andes: An intelligent tutorforclassica l physics. The Journal of Electronic Publishing 6(1), The University of Michigan Press.Google Scholar
  49. Shelby, R. N., Schulze, K. G., Treacy, D. J., Wintersgill, M. C. and VanLehn, K.: 2001, The Andes Intelligent Tutor: an Evaluation. Proceedings of the Physics Education Research Conference, Rochester, NY.Google Scholar
  50. Shute, V. J.: 1995, SMART: Student Modeling Approach for Responsive Tutoring. User Modeling and User-Adapted Interaction, 5(1), 1-44.CrossRefGoogle Scholar
  51. Shute, V. J. and Glaser, R.: 1990, A large-scale evaluation of an intelligent discovery world. Interactive Learning Environments, 1, 51-76.Google Scholar
  52. Shute, V. J. and Psotka, J.: 1996, Intelligent Tutoring Systems: Past, Present and Future. D. Jonassen (ed.) Handbook of Research on Educational Communications and Technology. Scholastic Publications.Google Scholar
  53. Singley, M. K. and Anderson, J. R.: 1989, Transfer of Cognitive Skill. Cambridge, MA., Harvard University Press.Google Scholar
  54. Tulving, E. and Thomson, D.M.: 1973, Encoding specificity and retrieval processes in episodic memory. Psychological Review, 80, 352-373.Google Scholar
  55. Van Mulken, S.: 1996, Reasoning about the user’s decoding of presentations in an intelligent multimedia presentation system. Proceedings of UM ‘96, 5th International Conference on User Modeling, Kailua Kona, HW, U.S.A., pp. 67-74.Google Scholar
  56. VanLehn, K.: 1988, Student modeling. M. Polson and M. Richardson (eds.). Foundations of Intelligent Tutoring Systems. Hillsdale, NJ, Lawrence Erlbaum Associates, pp. 55-78.Google Scholar
  57. VanLehn, K.: 1996, Conceptual and meta learning during coached problem solving. Proceedings of Intelligent Tutoring Systems, 3rd International Conference, ITS ‘96, Montreal, Canada, Lecture Notes in Computer Science 1086, Springer, pp. 29-47.Google Scholar
  58. VanLehn, K., Freedman, R., Jordan, P., Murray, C., Osan, R., Ringenberg, M., Rose, C. P., Shultze, K., Shelby, R., Treacy, D., Weinstein, A. and Wintersgill, M.: 2000, Fading and deepening: The next steps forAndes and othermodel-tracing tutors. Proceedings of Intelligent Tutoring Systems, 5th International Conference, ITS2000, Montreal, Canada, Lecture Notes in Computer Science 1839, Springer, pp. 474-483.Google Scholar
  59. VanLehn, K. and Martin, J.: 1998, Evaluation of an assessment system based on Bayesian student modeling. International Journal of Artificial Intelligence in Education, 8(2), 179-221.Google Scholar
  60. VanLehn, K. and Niu, Z. 2001, Bayesian student modeling, useri nterfaces and feedback: A sensitivity analysis. International Journal of Artificial Intelligence in Education, 12, 154-184.Google Scholar
  61. VanLehn, K., Niu, Z., Siler, S. and Gertner, A. S.: 1998, Student modeling from conventional test data: A Bayesian approach without priors. Proceedings of the 4th International Conference on Intelligent Tutoring Systems, ITS ‘98, San Antonio, TX, U.S.A., LectureNotes in Computer Science 1452, Springer Verlag, pp. 434-443.Google Scholar
  62. Zukerman, I. and Albrecht, D. V.: 2001, Predictive Statistical Models for User Modeling. User-Modeling and User-Adapted Interaction, 11(1-2), 5-18.CrossRefGoogle Scholar
  63. Zukerman, I., Albrecht, D. and Nicholson, A.: 1999, Predicting Users’ Requests on the WWW. Proceedings of UM’99, the 7th International Conference on User Modeling, Banff, Canada, Springer-Verlag, pp. 275-284.Google Scholar

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  • Cristina Conati
    • 1
  • Abigail Gertner
    • 2
  • Kurt VanLehn
    • 3
  1. 1.Department of Computer ScienceUniversity of British ColumbiaVancouverCanada
  2. 2.The MITRE CorporationBedfordUSA
  3. 3.Department of Computer Science and Learning and Research Development CenterUniversity of PittsburghPittsburghUSA

Personalised recommendations