Psychometrika

, Volume 75, Issue 4, pp 694–707 | Cite as

On Separable Tests, Correlated Priors, and Paradoxical Results in Multidimensional Item Response Theory

Article

Abstract

This paper presents a study of the impact of prior structure on paradoxical results in multidimensional item response theory. Paradoxical results refer to the possibility that an incorrect response could be beneficial to an examinee. We demonstrate that when three or more ability dimensions are being used, paradoxical results can be induced by using priors in which all abilities are positively correlated where they would not occur if the abilities were modeled as being independent. In the case of separable tests, we demonstrate the mathematical causes of paradoxical results, develop a computationally feasible means to check whether they can occur in any given test, and demonstrate a class of prior covariance matrices that can be guaranteed to avoid them.

Keywords

item response theory multidimensional posterior paradoxical result MAP EAP 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Finkelman, M., Hooker, G., & Wang, Z. (2010, in press). Prevalence and severity of paradoxical results in multidimensional item response theory. Journal of Educational and Behavioural Statistics. doi:10.3102/1076998610381402.
  2. Hooker, G., & Finkelman, M. (2010). Paradoxical results and item bundles. Psychometrika, 75, 249–271. CrossRefGoogle Scholar
  3. Hooker, G., Finkelman, M., & Schwartzman, A. (2009). Paradoxical results in multidimensional item response theory. Psychometrika, 74(3), 419–442. CrossRefGoogle Scholar
  4. Mardia, K.V., Kent, J.T., & Bibby, J.M. (1979). Multivariate analysis. London: Academic Press. Google Scholar
  5. R Development Core Team (2008). R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. ISBN 3-900051-07-0. http://www.R-project.org. Google Scholar
  6. Segall, D.O. (2000). Principles of multidimensional adaptive testing. In W.J. van der Linden, & C.A.W. Glas (Eds.), Computerized adaptive testing: theory and practice (pp. 53–73). Boston: Kluwer Academic. Google Scholar
  7. Wainer, H., Vevea, J., Camacho, F., Reeve, B., Rosa, K., Nelson, L., Swygert, K., & Thissen, D. (2001). Augmented scores—“borrowing strength” to compute scores based on small numbers of items. In D. Thissen, & H. Wainer (Eds.), Test scoring (pp. 343–387). Mahwah: Erlbaum. Google Scholar

Copyright information

© The Psychometric Society 2010

Authors and Affiliations

  1. 1.Cornell UniversityIthacaUSA

Personalised recommendations