On Separable Tests, Correlated Priors, and Paradoxical Results in Multidimensional Item Response Theory
This paper presents a study of the impact of prior structure on paradoxical results in multidimensional item response theory. Paradoxical results refer to the possibility that an incorrect response could be beneficial to an examinee. We demonstrate that when three or more ability dimensions are being used, paradoxical results can be induced by using priors in which all abilities are positively correlated where they would not occur if the abilities were modeled as being independent. In the case of separable tests, we demonstrate the mathematical causes of paradoxical results, develop a computationally feasible means to check whether they can occur in any given test, and demonstrate a class of prior covariance matrices that can be guaranteed to avoid them.
Keywordsitem response theory multidimensional posterior paradoxical result MAP EAP
Unable to display preview. Download preview PDF.
- Finkelman, M., Hooker, G., & Wang, Z. (2010, in press). Prevalence and severity of paradoxical results in multidimensional item response theory. Journal of Educational and Behavioural Statistics. doi: 10.3102/1076998610381402.
- Mardia, K.V., Kent, J.T., & Bibby, J.M. (1979). Multivariate analysis. London: Academic Press. Google Scholar
- Segall, D.O. (2000). Principles of multidimensional adaptive testing. In W.J. van der Linden, & C.A.W. Glas (Eds.), Computerized adaptive testing: theory and practice (pp. 53–73). Boston: Kluwer Academic. Google Scholar
- Wainer, H., Vevea, J., Camacho, F., Reeve, B., Rosa, K., Nelson, L., Swygert, K., & Thissen, D. (2001). Augmented scores—“borrowing strength” to compute scores based on small numbers of items. In D. Thissen, & H. Wainer (Eds.), Test scoring (pp. 343–387). Mahwah: Erlbaum. Google Scholar