Reliable Knowledge-Based Adaptive Tests by Credal Networks

  • Francesca Mangili
  • Claudio Bonesana
  • Alessandro Antonucci
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10369)

Abstract

An adaptive test is a computer-based testing technique which adjusts the sequence of questions on the basis of the estimated ability level of the test taker. We suggest the use of credal networks, a generalization of Bayesian networks based on sets of probability mass functions, to implement adaptive tests exploiting the knowledge of the test developer instead of training on databases of answers. Compared to Bayesian networks, these models might offer higher expressiveness and hence a more reliable modeling of the qualitative expert knowledge. The counterpart is a less straightforward identification of the information-theoretic measure controlling the question-selection and the test-stopping criteria. We elaborate on these issues and propose a sound and computationally feasible procedure. Validation against a Bayesian-network approach on a benchmark about German language proficiency assessments suggests that credal networks can be reliable in assessing the student level and effective in reducing the number of questions required to do it.

References

  1. 1.
    Pollard, E., Hillage, J.: Exploring e-learning. Inst. for Empl., Studies Brighton (2001)Google Scholar
  2. 2.
    Burns H., Luckhardt, C.A., Parlett, J.W., Redfield, C.L.: Intelligent Tutoring Systems: Evolutions in Design. Psychology Press (2014)Google Scholar
  3. 3.
    Koller, D., Friedman, N., Models, P.G.: Principles and Techniques. MIT Press, Cambridge (2009)Google Scholar
  4. 4.
    Almond, R.G., Mislevy, R.J., Steinberg, L., Yan, D., Williamson, D.: Bayesian Networks in Educational Assessment. Springer, New York (2015)CrossRefGoogle Scholar
  5. 5.
    Badaracco, M., Martínez, L.: A fuzzy linguistic algorithm for adaptive test in intelligent tutoring system based on competences. Expert Syst. Appl. 40(8), 3073–3086 (2013)CrossRefGoogle Scholar
  6. 6.
    Renooij, S., Witteman, C.: Talking probabilities: communicating probabilistic information with words and numbers. Int. J. Approx. Reason. 22(3), 169–194 (1999)CrossRefMATHGoogle Scholar
  7. 7.
    Walley, P.: Statistical Reasoning with Imprecise Probabilities. Chapman & Hall/CRC Monographs on Statistics & Applied Probability. Taylor & Francis (1991)Google Scholar
  8. 8.
    Piatti, A., Antonucci, A., Zaffalon, M.: Building knowledge-based systems by credal networks: a tutorial. In: Baswell, A.R. (ed.) Advances in Mathematics Research, vol. 11. Nova Science Publishers, New York (2010)Google Scholar
  9. 9.
    Troffaes, M.: Decision making under uncertainty using imprecise probabilities. Int. J. Approx. Reason. 45(1), 17–29 (2007)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Klir, G., Wierman, M.: Uncertainty-Based Information: Elements of Generalized Information Theory. STUDFUZZ, vol. 15. Springer, Heidelberg (1999)MATHGoogle Scholar
  11. 11.
    Mauá, D., de Campos, C., Benavoli, A., Antonucci, A.: Probabilistic inference in credal networks: new complexity results. J. Artif. Intell. Res. 50, 603–637 (2014)MathSciNetMATHGoogle Scholar
  12. 12.
    Hambleton, R.K., Swaminathan, H.: Item Response Theory: Principles and Applications, vol. 7. Springer Science & Business Media, New York (1985)CrossRefGoogle Scholar
  13. 13.
    Vomlel, J.: Building adaptive tests using Bayesian networks. Kybernetika 40(3), 333–348 (2004)MathSciNetMATHGoogle Scholar
  14. 14.
    Plajner, M., Vomlel, J.: Bayesian network models for adaptive testing, arXiv preprint arXiv:1511.08488
  15. 15.
    Antonucci, A., Piatti, A.: Modeling unreliable observations in bayesian networks by credal networks. In: Godo, L., Pugliese, A. (eds.) SUM 2009. LNCS (LNAI), vol. 5785, pp. 28–39. Springer, Heidelberg (2009). doi:10.1007/978-3-642-04388-8_4 CrossRefGoogle Scholar
  16. 16.
    Cozman, F.G.: Credal networks. Artif. Intell. 120, 199–233 (2000)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Antonucci, A., de Campos, C., Zaffalon, M., Huber, D.: Approximate credal network updating by linear programming with applications to decision making. Int. J. Approx. Reason. 58, 25–38 (2014)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Zaffalon, M., Corani, G., Mauá, D.: Evaluating credal classifiers by utility-discounted predictive accuracy. Int. J. Approx. Reason. 53(8), 1282–1301 (2012)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Abellan, J., Moral, S.: Maximum of entropy for credal sets. Int. J. Uncertain. Fuzz. 11(05), 587–597 (2003)MathSciNetCrossRefMATHGoogle Scholar
  20. 20.
    Antonucci, A., Corani, G.: The multilabel naive credal classifier. Int. J. Approx. Reason. 83, 320–336 (2016)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Francesca Mangili
    • 1
  • Claudio Bonesana
    • 1
  • Alessandro Antonucci
    • 1
  1. 1.Istituto Dalle Molle di Studi sull’Intelligenza ArtificialeLuganoSwitzerland

Personalised recommendations