Learners’ Perceived Level of Difficulty of a Computer-Adaptive Test: A Case Study
A computer-adaptive test (CAT) is a software application that makes use of Item Response Theory (IRT) to create a test that is tailored to individual learners. The CAT prototype introduced here comprised a graphical user interface, a question database and an adaptive algorithm based on the Three-Parameter Logistic Model from IRT. A sample of 113 Computer Science undergraduate students participated in a session of assessment within the Human-Computer Interaction subject domain using our CAT prototype. At the end of the assessment session, participants were asked to rate the level of difficulty of the overall test from 1 (very easy) to 5 (very difficult). The perceived level of difficulty of the test and the CAT scores obtained by this group of learners were subjected to a Spearman’s rank order correlation. Findings from this statistical analysis suggest that the CAT prototype was effective in tailoring the assessment to each individual learner’s proficiency level.
KeywordsItem Response Theory Proficiency Level Educational Software Assessment Session Computer Adaptive Testing
- 1.Barker, T., Lilley, M.: Are Individual Learners Disadvantaged by the Use of Computer-Adaptive Testing? In: Proceedings of the 8th Learning Styles Conference, University of Hull. European Learning Styles Information Network (ELSIN), pp. 30–39 (2003)Google Scholar
- 2.Brusilovsky, P.: Knowledge Tree: A Distributed Architecture for Adaptive E-Learning. In: Proceedings of the 13th World Wide Web Conference, New York, USA, May 17-22, pp. 104–113 (2004)Google Scholar
- 5.Jettmar, E., Nass, C.: Adaptive testing: effects on user performance. In: Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, Minneapolis, Minnesota, USA, pp. 129–134 (2002)Google Scholar
- 7.Lord, F.M.: Applications of Item Response Theory to practical testing problems. Lawrence Erlbaum Associates, New Jersey (1980)Google Scholar
- 8.Wainer, H.: Computerized Adaptive Testing (A Primer), 2nd edn. Lawrence Erlbaum Associates, New Jersey (2000)Google Scholar