Advertisement

A Bayesian Diagnostic Algorithm for Student Modeling and its Evaluation

  • Eva Millán
  • José Luis Pérez-de-la-Cruz
Article

Abstract

In this paper, we present a new approach to diagnosis in student modeling based on the use of Bayesian Networks and Computer Adaptive Tests. A new integrated Bayesian student model is defined and then combined with an Adaptive Testing algorithm. The structural model defined has the advantage that it measures students' abilities at different levels of granularity, allows substantial simplifications when specifying the parameters (conditional probabilities) needed to construct the Bayesian Network that describes the student model, and supports the Adaptive Diagnosis algorithm. The validity of the approach has been tested intensively by using simulated students. The results obtained show that the Bayesian student model has excellent performance in terms of accuracy, and that the introduction of adaptive question selection methods improves its behavior both in terms of accuracy and efficiency.

adaptive testing Bayesian networks student modeling 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Birnbaum, A.: 1968, Some latent trait models and their use in inferring an examinee's mental ability. In: F. M. Lord and M. R. Novick (eds.), Statistical Theories of Mental Test Scores. Reading, MA: Addison-Wesley.Google Scholar
  2. Bloom, B.: 1984, The 2 Sigma Problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher 13, 4–15.Google Scholar
  3. Castillo, E., Gutiérrez, J. M. and Hadi, A.: 1997, Expert Systems and Probabilistic Network Models. New York: Springer Verlag.Google Scholar
  4. Charniak, E.: 1991, Bayesian Networks Without Tears. AI Magazine 12(4), 50–63.Google Scholar
  5. Collins, J. A., Greer, J. E. and Huang, S. H.: 1996, Adaptive assessment using granularity hierarchies and Bayesian nets. In: Lecture Notes in Computer Science: Vol. 1086. Proceedings of 3rd International Conference ITS'96, Berlin Heidelberg: Springer Verlag, pp. 569–577.Google Scholar
  6. Conati, C., Gertner, A., VanLehn, K. and Druzdzel, M.: 1997, On-line student modelling for coached problem solving using Bayesian networks. Proceedings of the 6th International Conference on User Modelling UM'97, Vienna, New York: Springer Verlag, pp. 231–242.Google Scholar
  7. Conati, C. and VanLehn, K.: 1996a, POLA: A student modeling framework for probabilistic on-line assessment of problem solving performance. Proceedings of the 5th International Conference on User Modeling UM'96, User Modeling Inc., pp. 75–82.Google Scholar
  8. Flaugher, R.: 1990, Item pools. In H. Wainer (ed.), Computerized Adaptive Testing: A Primer. Hillsdale, NJ: Lawrence Erlbaum Associates Publishers.Google Scholar
  9. Hambleton, R. K.: 1989, Principles and selected applications of item response Theory. In: R. L. Linn (ed.), Educational Measurement. New York: MacMillan.Google Scholar
  10. Jameson, A.: 1996, Numerical uncertainty management in user and student modeling: An overview of systems and issues. User Modeling and User-Adapted Interaction 5, 193–251.Google Scholar
  11. Kingsbury, G. and Weiss, D. J.: 1983, A comparison of IRT-based adaptive mastery testing and sequential mastery testing procedure. In: D. J. Weiss (ed.), New Horizons in Testing: Latent Trait Test Theory and Computerized Adaptive Testing. New York: Academic Press.Google Scholar
  12. Martin, J. and Van Lehn, K.: 1995b, Student assessment using Bayesian nets. International Journal of Human-Computer Studies 42, 575–591.Google Scholar
  13. Mayo, M. and Mitrovic, A.: 2000, Using a probabilistic student model to control problem difficulty. In: Lecture Notes in Computer Science, Proceedings of 3rd International Conference on Intelligent Tutoring Systems ITS'2000, Berlin Heidelberg: Springer Verlag, pp. 525–533.Google Scholar
  14. Millán, E., Pérez-de-la-Cruz, J. L. and Suárez, E.: 2000, An adaptive Bayesian network for multilevel student modelling. In: Lecture Notes in Computer Science. Proceedings of 3rd International Conference on Intelligent Tutoring Systems ITS'2000, Berlin Heidelberg: Springer Verlag, pp. 534–543.Google Scholar
  15. Mislevy, R. and Gitomer, D. H.: 1996, The role of probability-based inference in an intelligent tutoring system. User Modeling and User-Adapted Interaction 5, 253–282.Google Scholar
  16. Mitrovic, A.: 1998, Experiences in implementing constraint-based modeling in SQL-Tutor. In: Lecture Notes in Computer Science: Vol. 1452. Intelligent Tutoring Systems. Proceedings of 4th International Conference ITS'98, Berlin Heidelberg: Springer Verlag, pp. 414–423.Google Scholar
  17. Mitrovic, A., Brent, M., Mayo. M.: 2002, Using evaluation to shape ITS design: Results and experiences with SQL-tutor. User Modeling and User-Adapted Interaction, in this issue.Google Scholar
  18. Murray, W.: 1998, A practical approach to Bayesian student modelling. In: Lecture Notes in Computer Science: Vol. 1452. Intelligent Tutoring Systems. Proceedings of 4th International Conference ITS'98, Berlin Heidelberg: Springer Verlag, pp. 424–433.Google Scholar
  19. Ohlsson, S.: 1994, Constraint-based student modelling. In: J. E. Greer and G. McCalla (eds.), Student Modelling: The Key to Individualized Knowledge-Based Instruction. Vol. 125, Berlin Heidelberg: Springer Verlag, pp. 167–190.Google Scholar
  20. Olea, J. and Ponsoda, V.: 1996, Tests adaptativos informatizados. In: J. Muñiz (ed.), Psicometría, Madrid: Universitas, pp. 731–783.Google Scholar
  21. Pearl, J.: 1988, Probabilistic Reasoning in Expert Systems: Networks of Plausible Inference. San Francisco: Morgan Kaufmann Publishers, Inc.Google Scholar
  22. Reye, J.: 1996, A belief net backbone for student modeling. In: Lecture Notes in Computer Science: Vol. 1086. Proceedings of 3rd International Conference ITS'96, Berlin Heidelberg: Springer Verlag, pp. 596–604.Google Scholar
  23. Reye, J.: 1998, Two-phase updating of student models based on dynamic belief networks. In: B. P. Goettl, J.M. Half, C. L. Redfield and V. J. Shutte, (eds.), Lecture Notes in Computer Science: Vol. 1452. Intelligent Tutoring Systems. Proceedings of 4th International Conference ITS'98, Berlin Heidelberg: Springer Verlag, pp. 6–15.Google Scholar
  24. Ríos, A., Millán, E., Trella, M., Pérez-de-la-Cruz, J. L. and Conejo, R.: 1999, Internet based evaluation system. In: Open Learning Environments: New Computational Technologies to Support Learning, Exploration and Collaboration. Proceedings of the 9th World Conference of Artificial Intelligence and Education AIED'99, Amsterdam: IOS Press, pp. 387–394.Google Scholar
  25. Rudner, L.: 1998, An on-line, interactive, computer adaptive testing mini-tutorial. http://ericae.net/scripts/cat/catdemo.Google Scholar
  26. Shute, V. J.: 1995, Intelligent tutoring systems: Past, present and future. In: D. Jonassen (ed.), Handbook of Research on Educational Communications and Technology. Scholastic Publications.Google Scholar
  27. Stern, M., Beck, J. and Woolf, B. P.: 1996, Adaptation of problem presentation and feedback in an intelligent mathematics tutor. In: C. Frasson, G. Gauthier and A. Lesgold (eds.), Intelligent Tutoring Systems. New York: Springer Verlag, pp. 603–613.Google Scholar
  28. Thissen, D. and Mislevy, R.: 1990, Testing algorithms. In: H. Wainer (ed.), Computerized Adaptive Testing: A Primer, Hillsdale, NJ: Lawrence Erlbaum Associates Publishers, pp. 103–136.Google Scholar
  29. Van der Linden, W. and Hambleton, R.: 1997, Handbook of Modern Item Response Theory. New York: Springer Verlag.Google Scholar
  30. Van Lehn, K.: 1988, Student modelling. In: M. C. Polson and J. J. Richardson (eds.), Foundations of Intelligent Tutoring Systems. Hillsdale, NJ: Lawrence Erlbaum Associates Publishers, pp. 55–76.Google Scholar
  31. Van Lehn, K.: 1996, Conceptual and meta learning during coached problem solving. In: Lecture Notes in Computer Science: Vol. 1086. Proceedings of 3rd International Conference ITS'96, Berlin Heidelberg: Springer Verlag, pp. 29–47.Google Scholar
  32. Van Lehn, K., Niu, Z., Siler, S. and Gertner, A. S.: 1998, Student modeling from conventional test data: A Bayesian approach without priors. In: Lecture Notes in Computer Science: Vol. 1452. Intelligent Tutoring Systems. Proceedings of 4th International Conference ITS'98, Berlin Heidelberg: Springer Verlag, pp. 434–443.Google Scholar
  33. Van Lehn, K., Ohlsson, S. and Nason, R.: 1995, Applications of Simulated Students: An Exploration. Journal of Artificial Intelligence and Education 5(2), 135–175.Google Scholar
  34. Wainer, H.: 1990, Computerized Adaptive Testing: a Primer. Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
  35. Weiss, D. and Kingsbury, G.: 1984, Application of computerized adaptive testing to educational problems. Journal of Educational Measurement 12, 361–375.Google Scholar

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  • Eva Millán
    • 1
  • José Luis Pérez-de-la-Cruz
    • 1
  1. 1.Departamento de Lenguajes y Ciencias de la Computación E.T.S.I. InformáticaUniversidad de MálagaMálagaSpain

Personalised recommendations