Skip to main content

A Bayesian Diagnostic Algorithm for Student Modeling and its Evaluation

User Modeling and User-Adapted Interaction volume 12, pages 281–330 (2002)Cite this article

Abstract

In this paper, we present a new approach to diagnosis in student modeling based on the use of Bayesian Networks and Computer Adaptive Tests. A new integrated Bayesian student model is defined and then combined with an Adaptive Testing algorithm. The structural model defined has the advantage that it measures students' abilities at different levels of granularity, allows substantial simplifications when specifying the parameters (conditional probabilities) needed to construct the Bayesian Network that describes the student model, and supports the Adaptive Diagnosis algorithm. The validity of the approach has been tested intensively by using simulated students. The results obtained show that the Bayesian student model has excellent performance in terms of accuracy, and that the introduction of adaptive question selection methods improves its behavior both in terms of accuracy and efficiency.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

References

  • Birnbaum, A.: 1968, Some latent trait models and their use in inferring an examinee's mental ability. In: F. M. Lord and M. R. Novick (eds.), Statistical Theories of Mental Test Scores. Reading, MA: Addison-Wesley.

    Google Scholar 

  • Bloom, B.: 1984, The 2 Sigma Problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher 13, 4–15.

    Google Scholar 

  • Castillo, E., Gutiérrez, J. M. and Hadi, A.: 1997, Expert Systems and Probabilistic Network Models. New York: Springer Verlag.

    Google Scholar 

  • Charniak, E.: 1991, Bayesian Networks Without Tears. AI Magazine 12(4), 50–63.

    Google Scholar 

  • Collins, J. A., Greer, J. E. and Huang, S. H.: 1996, Adaptive assessment using granularity hierarchies and Bayesian nets. In: Lecture Notes in Computer Science: Vol. 1086. Proceedings of 3rd International Conference ITS'96, Berlin Heidelberg: Springer Verlag, pp. 569–577.

    Google Scholar 

  • Conati, C., Gertner, A., VanLehn, K. and Druzdzel, M.: 1997, On-line student modelling for coached problem solving using Bayesian networks. Proceedings of the 6th International Conference on User Modelling UM'97, Vienna, New York: Springer Verlag, pp. 231–242.

    Google Scholar 

  • Conati, C. and VanLehn, K.: 1996a, POLA: A student modeling framework for probabilistic on-line assessment of problem solving performance. Proceedings of the 5th International Conference on User Modeling UM'96, User Modeling Inc., pp. 75–82.

  • Flaugher, R.: 1990, Item pools. In H. Wainer (ed.), Computerized Adaptive Testing: A Primer. Hillsdale, NJ: Lawrence Erlbaum Associates Publishers.

    Google Scholar 

  • Hambleton, R. K.: 1989, Principles and selected applications of item response Theory. In: R. L. Linn (ed.), Educational Measurement. New York: MacMillan.

    Google Scholar 

  • Jameson, A.: 1996, Numerical uncertainty management in user and student modeling: An overview of systems and issues. User Modeling and User-Adapted Interaction 5, 193–251.

    Google Scholar 

  • Kingsbury, G. and Weiss, D. J.: 1983, A comparison of IRT-based adaptive mastery testing and sequential mastery testing procedure. In: D. J. Weiss (ed.), New Horizons in Testing: Latent Trait Test Theory and Computerized Adaptive Testing. New York: Academic Press.

    Google Scholar 

  • Martin, J. and Van Lehn, K.: 1995b, Student assessment using Bayesian nets. International Journal of Human-Computer Studies 42, 575–591.

    Google Scholar 

  • Mayo, M. and Mitrovic, A.: 2000, Using a probabilistic student model to control problem difficulty. In: Lecture Notes in Computer Science, Proceedings of 3rd International Conference on Intelligent Tutoring Systems ITS'2000, Berlin Heidelberg: Springer Verlag, pp. 525–533.

    Google Scholar 

  • Millán, E., Pérez-de-la-Cruz, J. L. and Suárez, E.: 2000, An adaptive Bayesian network for multilevel student modelling. In: Lecture Notes in Computer Science. Proceedings of 3rd International Conference on Intelligent Tutoring Systems ITS'2000, Berlin Heidelberg: Springer Verlag, pp. 534–543.

    Google Scholar 

  • Mislevy, R. and Gitomer, D. H.: 1996, The role of probability-based inference in an intelligent tutoring system. User Modeling and User-Adapted Interaction 5, 253–282.

    Google Scholar 

  • Mitrovic, A.: 1998, Experiences in implementing constraint-based modeling in SQL-Tutor. In: Lecture Notes in Computer Science: Vol. 1452. Intelligent Tutoring Systems. Proceedings of 4th International Conference ITS'98, Berlin Heidelberg: Springer Verlag, pp. 414–423.

    Google Scholar 

  • Mitrovic, A., Brent, M., Mayo. M.: 2002, Using evaluation to shape ITS design: Results and experiences with SQL-tutor. User Modeling and User-Adapted Interaction, in this issue.

  • Murray, W.: 1998, A practical approach to Bayesian student modelling. In: Lecture Notes in Computer Science: Vol. 1452. Intelligent Tutoring Systems. Proceedings of 4th International Conference ITS'98, Berlin Heidelberg: Springer Verlag, pp. 424–433.

    Google Scholar 

  • Ohlsson, S.: 1994, Constraint-based student modelling. In: J. E. Greer and G. McCalla (eds.), Student Modelling: The Key to Individualized Knowledge-Based Instruction. Vol. 125, Berlin Heidelberg: Springer Verlag, pp. 167–190.

    Google Scholar 

  • Olea, J. and Ponsoda, V.: 1996, Tests adaptativos informatizados. In: J. Muñiz (ed.), Psicometría, Madrid: Universitas, pp. 731–783.

    Google Scholar 

  • Pearl, J.: 1988, Probabilistic Reasoning in Expert Systems: Networks of Plausible Inference. San Francisco: Morgan Kaufmann Publishers, Inc.

    Google Scholar 

  • Reye, J.: 1996, A belief net backbone for student modeling. In: Lecture Notes in Computer Science: Vol. 1086. Proceedings of 3rd International Conference ITS'96, Berlin Heidelberg: Springer Verlag, pp. 596–604.

    Google Scholar 

  • Reye, J.: 1998, Two-phase updating of student models based on dynamic belief networks. In: B. P. Goettl, J.M. Half, C. L. Redfield and V. J. Shutte, (eds.), Lecture Notes in Computer Science: Vol. 1452. Intelligent Tutoring Systems. Proceedings of 4th International Conference ITS'98, Berlin Heidelberg: Springer Verlag, pp. 6–15.

    Google Scholar 

  • Ríos, A., Millán, E., Trella, M., Pérez-de-la-Cruz, J. L. and Conejo, R.: 1999, Internet based evaluation system. In: Open Learning Environments: New Computational Technologies to Support Learning, Exploration and Collaboration. Proceedings of the 9th World Conference of Artificial Intelligence and Education AIED'99, Amsterdam: IOS Press, pp. 387–394.

    Google Scholar 

  • Rudner, L.: 1998, An on-line, interactive, computer adaptive testing mini-tutorial. http://ericae.net/scripts/cat/catdemo.

  • Shute, V. J.: 1995, Intelligent tutoring systems: Past, present and future. In: D. Jonassen (ed.), Handbook of Research on Educational Communications and Technology. Scholastic Publications.

  • Stern, M., Beck, J. and Woolf, B. P.: 1996, Adaptation of problem presentation and feedback in an intelligent mathematics tutor. In: C. Frasson, G. Gauthier and A. Lesgold (eds.), Intelligent Tutoring Systems. New York: Springer Verlag, pp. 603–613.

    Google Scholar 

  • Thissen, D. and Mislevy, R.: 1990, Testing algorithms. In: H. Wainer (ed.), Computerized Adaptive Testing: A Primer, Hillsdale, NJ: Lawrence Erlbaum Associates Publishers, pp. 103–136.

    Google Scholar 

  • Van der Linden, W. and Hambleton, R.: 1997, Handbook of Modern Item Response Theory. New York: Springer Verlag.

    Google Scholar 

  • Van Lehn, K.: 1988, Student modelling. In: M. C. Polson and J. J. Richardson (eds.), Foundations of Intelligent Tutoring Systems. Hillsdale, NJ: Lawrence Erlbaum Associates Publishers, pp. 55–76.

    Google Scholar 

  • Van Lehn, K.: 1996, Conceptual and meta learning during coached problem solving. In: Lecture Notes in Computer Science: Vol. 1086. Proceedings of 3rd International Conference ITS'96, Berlin Heidelberg: Springer Verlag, pp. 29–47.

    Google Scholar 

  • Van Lehn, K., Niu, Z., Siler, S. and Gertner, A. S.: 1998, Student modeling from conventional test data: A Bayesian approach without priors. In: Lecture Notes in Computer Science: Vol. 1452. Intelligent Tutoring Systems. Proceedings of 4th International Conference ITS'98, Berlin Heidelberg: Springer Verlag, pp. 434–443.

    Google Scholar 

  • Van Lehn, K., Ohlsson, S. and Nason, R.: 1995, Applications of Simulated Students: An Exploration. Journal of Artificial Intelligence and Education 5(2), 135–175.

    Google Scholar 

  • Wainer, H.: 1990, Computerized Adaptive Testing: a Primer. Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Weiss, D. and Kingsbury, G.: 1984, Application of computerized adaptive testing to educational problems. Journal of Educational Measurement 12, 361–375.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Departamento de Lenguajes y Ciencias de la Computación E.T.S.I. Informática, Universidad de Málaga, Apdo. 4114, Málaga, 29080, Spain

    Eva Millán & José Luis Pérez-de-la-Cruz

Authors
  1. Eva Millán
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. José Luis Pérez-de-la-Cruz
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Millán, E., Pérez-de-la-Cruz, J.L. A Bayesian Diagnostic Algorithm for Student Modeling and its Evaluation. User Modeling and User-Adapted Interaction 12, 281–330 (2002). https://doi.org/10.1023/A:1015027822614

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1015027822614

  • adaptive testing
  • Bayesian networks
  • student modeling