An Empirical Study About Calibration of Adaptive Hints in Web-Based Adaptive Testing Environments

  • Ricardo Conejo
  • Eduardo Guzmán
  • José-Luis Pérez de-la Cruz
  • Eva Millán
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4018)


In this paper we present a proposal for introducing hint adaptive selection in an adaptive web-based testing environment. To this end, a discussion of some aspects concerning the adaptive selection mechanism for hints is presented, which results in the statement of two axioms that such hints must fulfil. Then, an empirical study with real students is presented, whose goal is to evaluate a tentative bank of items with their associated hints to determine the usefulness of such hints for different knowledge levels and to calibrate both test items and hints.


Item Response Theory Latent Trait Knowledge Level Item Bank Computerize Adaptive Test 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Rudner, L.M.: An Examination of Decision-Theory Adaptive Testing Procedures. In: Annual meeting of the American Educational Research Association (2002)Google Scholar
  2. 2.
    Chua Abdullah, S.: Student Modelling by Adaptive Testing - A Knowledge-based Approach. PhD thesis, University of Kent, Canterbury (2003)Google Scholar
  3. 3.
    Hambleton, R.K., Swaminathan, J., Rogers, H.J.: Fundamentals of Item Response Theory. Sage publications, Thousand Oaks (1991)Google Scholar
  4. 4.
    Wainer, H.: Computerized Adaptive Testing: A Primer. Lawrence Erlbaum, Hillsdale (1990)Google Scholar
  5. 5.
    Conejo, R., Guzmán, E., Millán, E., Trella, M., Pérez de la Cruz, J.L., Ríos, A.: SIETTE: a Web-based tool for adaptive testing. Journal of Artificial Intelligence in Education 14, 29–61 (2004)Google Scholar
  6. 6.
    Guzmán, E., Conejo, R.: A brief introduction to the new architecture of SIETTE. In: De Bra, P.M.E., Nejdl, W. (eds.) AH 2004. LNCS, vol. 3137, pp. 405–408. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  7. 7.
    Vygotskii, L.: Mind in Society: The Development of Higher Psychological Processes. Harvard University Press, Cambridge (1978)Google Scholar
  8. 8.
    Wells, G.: Dialogic Inquiry: Towards a Socio-Cultural Practice and Theory of Education. Cambridge University Press, New York (1999)CrossRefGoogle Scholar
  9. 9.
    Luckin, R., Hammerton, L.: Getting to Know Me: Helping Learners Understand Their Own Learning Needs through Metacognitive Scaffolding. In: Cerri, S.A., Gouardéres, G., Paraguaçu, F. (eds.) ITS 2002. LNCS, vol. 2363, pp. 759–771. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  10. 10.
    Aleven, V., McLaren, B., Roll, I., Koedinger, K.: Toward tutoring help seeking: Applying cognitive modeling to meta-cognitive skills. In: Lester, J.C., Vicari, R.M., Paraguaçu, F. (eds.) ITS 2004. LNCS, vol. 3220, pp. 227–239. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  11. 11.
    Gertner, A.S., Conati, C., VanLehn, K.: Procedural Help in Andes: Generating Hints Using a Bayesian Network Student Model. In: Proceedings of the 15th National Conference on Artificial Intelligence, Madison, Wisconsin (1998)Google Scholar
  12. 12.
    Arroyo, I., Beck, J.E., Woolf, B.P., Beal, C.R., Schultz, K.: Macroadapting Animalwatch to gender and cognitive differences with respect to hint interactivity and symbolism. In: Gauthier, G., VanLehn, K., Frasson, C. (eds.) ITS 2000. LNCS, vol. 1839, pp. 574–614. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  13. 13.
    Hume, G.D., Michael, J., Rovick, A., Evens, M.W.: Hinting as a tactic in one-on-one tutoring. Journal of Learning Sciences 5(1), 23–47 (1996)CrossRefGoogle Scholar
  14. 14.
    Birnbaum, A.: Some Latent Trait Models and Their Use in Inferring an Examinee’s Mental Ability. In: Statistical Theories of Mental Test Scores. Addison-Wesley, Reading (1968)Google Scholar
  15. 15.
    Thissen, D.: MULTILOG: Multiple, categorical item analysis and test scoring using item response theory (version 5.1) (1988)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Ricardo Conejo
    • 1
  • Eduardo Guzmán
    • 1
  • José-Luis Pérez de-la Cruz
    • 1
  • Eva Millán
    • 1
  1. 1.Departamento de Lenguajes y Ciencias de la ComputaciónUniversidad de MálagaMálagaSpain

Personalised recommendations