Abstract
In recent years, fixed forms and computerized adaptive testing (CAT) forms have coexisted in many testing programs and often used interchangeably on the premise that both testing formats meet the same test specifications. In conventional CAT, however, items are selected through computer algorithms to meet primarily statistical criteria, whereas fixed forms are often created focusing heavily on content, non-statistical, and practical requirements. Founded on the optimal test design framework, the shadow-test approach to CAT and its generalization allows for constructing fixed and adaptive test forms to satisfy the same objective and (possibly complex) set of constraints. This approach can render a variety of testing formats with different levels of adaptation and relative efficiency. This paper provides an overview of the optimal test assembly approach and the development of the TestDesign package in R. Several illustrations from real-world testing scenarios are also presented.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Berkelaar M et al (2020) lpSolve: interface to Lp\_solve v. 5.5 to solve linear/integer programs. https://CRAN.R-project.org/package=lpSolve
Birnbaum A (1968) Some latent trait models and their use in inferring an examinee’s ability. In: Lord FM, Novick MR (eds) Statistical theories of mental test scores. Addison-Wesley Pub. Co., Reading
Chang HH, Ying Z (1999) A-stratified multistage computerized adaptive testing. Appl Psychol Meas 23(3):211–222
Choi SW, van der Linden WJ (2018) Ensuring content validity of patient-reported outcomes: a shadow-test approach to their adaptive measurement. Qual Life Res 27(7):1683–1693. https://doi.org/10.1007/s11136-017-1650-1
Choi SW, Moellering KT, Li J, van der Linden WJ (2016) Optimal reassembly of shadow tests in CAT. Appl Psychol Meas 40(7):469–485. https://doi.org/10.1177/0146621616654597
Gurobi Optimization and LLC (2019) gurobi: Gurobi Optimizer 9.0 Interface. https://www.gurobi.com
Harter R, Hornik K, Theussl S (2017) Rsymphony: SYMPHONY in R. https://CRAN.R-project.org/package=Rsymphony
Kim V (2019) lpsymphony: symphony integer linear programming solver in R. http://R-Forge.R-project.org/projects/rsymphony. https://projects.coin-or.org/SYMPHONY. http://www.coin-or.org/download/source/SYMPHONY/
van der Linden WJ (2005) Linear models for optimal test design. Statistics for social and behavioral sciences. Springer, New York
van der Linden WJ (2010) Constrained adaptive testing with shadow tests. In: van der Linden WJ, Glas CAW (eds) Elements of adaptive testing. Springer, New York, pp 31–55
van der Linden WJ (2018) Optimal test design. In: van der Linden WJ (ed) Handbook of item response theory, volume three: applications. Chapman and Hall/CRC, pp 167–195
van der Linden WJ, Choi SW (2019) Improving item-exposure control in adaptive testing. J Educ Meas. https://doi.org/10.1111/jedm.12254
van der Linden WJ, Diao Q (2014) Using a universal shadow-test assembler with multistage testing. In: Yan D, von Davier AA, Lewis C (eds) Computerized multistage testing: theory and applications. CRC Press, New York, pp 101–118
van der Linden WJ, Reese LM (1998) A model for optimal constrained adaptive testing. Appl Psychol Meas 22(3):259–270
van der Linden WJ, Veldkamp BP (2004) Constraining item exposure in computerized adaptive testing with shadow tests. J Educ Behav Stat 29(3):273–291
van der Linden WJ, Veldkamp BP (2007) Conditional item-exposure control in adaptive testing using item-ineligibility probabilities. J Educ Behav Stat 32(4):398–418. https://doi.org/10.3102/1076998606298044
Swanson L, Stocking ML (1993) A model and heuristic for solving very large item selection problems. Appl Psychol Meas 17(2):151–166. https://doi.org/10.1177/014662169301700205
Theunissen TJJM (1985) Binary programming and test design. Psychometrika 50(4):411–420. https://doi.org/10.1007/BF02296260
Theussl S, Hornik K (2019) Rglpk: R/GNULinear programming kit interface. https://CRAN.R-project.org/package=Rglpk
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Maomi Ueno.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Choi, S.W., Lim, S. & van der Linden, W.J. TestDesign: an optimal test design approach to constructing fixed and adaptive tests in R. Behaviormetrika 49, 191–229 (2022). https://doi.org/10.1007/s41237-021-00145-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41237-021-00145-9