Abstract
Classic item response models assume that all items with the same difficulty have the same response probability among all respondents with the same ability. These assumptions, however, may very well be violated in practice, and it is not straightforward to assess whether these assumptions are violated, because neither the abilities of respondents nor the difficulties of items are observed. An example is an educational assessment where unobserved heterogeneity is present, arising from unobserved variables such as cultural background and upbringing of students, the quality of mentorship and other forms of emotional and professional support received by students, and other unobserved variables that may affect response probabilities. To address such violations of assumptions, we introduce a novel latent space model which assumes that both items and respondents are embedded in an unobserved metric space, with the probability of a correct response decreasing as a function of the distance between the respondent’s and the item’s position in the latent space. The resulting latent space approach provides an interaction map that represents interactions of respondents and items, and helps derive insightful diagnostic information on items as well as respondents. In practice, such interaction maps enable teachers to detect students from underrepresented groups who need more support than other students. We provide empirical evidence to demonstrate the usefulness of the proposed latent space approach, along with simulation results.
Similar content being viewed by others
Notes
Different subsets or versions of the data have been used in the literature. We used the data pre-processed by Skrondal and Rabe-Hesketh (2004), which include responses from 734 respondents. We analyzed the version after deleting respondents with no item responses assuming missing at random (Skrondal and Rabe-Hesketh 2004).
The MCMC estimates of the Rasch model were very similar to the ML estimates obtained from the R lme4 package (Bates et al. 2015). The results are also shown in “Appendix D” of the supplement.
References
Agarwal, D., & Chen, B.-C. (2009). Regression-based latent factor models. Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 19-28.
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01.
Bernaards, C. A., & Jennrich, R. I. (2005). Gradient projection algorithms and software for arbitrary rotation criteria in factor analysis. Educational and Psychological Measurement, 65, 676–696.
Borsboom, D. (2008). Psychometric perspectives on diagnostic systems. Journal of Clinical Psychology, 64(9), 1089–1108.
Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1–29.
Chen, W.-H., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22, 265–289.
Draney, K. (2007). The Saltus model applied to proportional reasoning data. Journal of Applied Measurement, 8, 438–455.
Epskamp, S., Borsboom, D., & Fried, E. I. (2018). Estimating psychological networks and their accuracy: A tutorial paper. Behavior Research Methods, 50, 195–212.
Fox, J. P., & Glas, C. A. (2001). Bayesian estimation of a multilevel IRT model using Gibbs sampling. Psychometrika, 66, 271–288.
Furr, D. C., Lee, S.-Y., Lee, J.-H., & Rabe-Hesketh, S. (2016). Two-parameter logistic item response model - STAN. https://mc-stan.org/users/documentation/case-studies/tutorial twopl.html.
Gelman, A., & Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Iterative simulation using multiple sequences. Statistical Science, 7, 457–472.
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.
Gower, J. C. (1975). Generalized procrustes analysis. Psychometrika, 40, 33–51.
Hoff, P. (2005). Bilinear mixed-effects models for dyadic data. Journal of the American Statistical Association, 286–295.
Hoff, P. (2020). Additive and multiplicative effects network models. Statistical Science (to appear).
Hoff, P., Raftery, A., & Handcock, M. S. (2002). Latent space approaches to social network analysis. Journal of the American Statistical Association, 97, 1090–1098.
Ishwaran, H., & Rao, J. S. (2005). Spike and slab variable selection: Frequentist and Bayesian strategies. The Annals of Statistics, 33, 730–773.
Jennrich, R. I. (2002). A simple general method for oblique rotation. Psychometrika, 67, 7–19.
Jin, I. H., & Jeon, M. (2019). A doubly latent space joint model for local item and person dependence in the analysis of item response data. Psychometrika, 84, 236–260.
Jin, I. H., Jeon, M., Schweinberger, M., & Lin, L. (2018). Hierarchical network item response modeling for discovering differences between innovation and regular school systems in Korea. Available at. arxiv.org/abs/1810.07876.
Lauritzen, S. (1996). Graphical models. Oxford: Oxford University Press.
Markovits, H., Fleury, M.-L., Quinn, S., & Venet, M. (1998). The development of conditional reasoning and the structure of semantic memory. Child Development, 69, 742–755.
Marsman, M., Borsboom, D., Kruis, J., Epskamp, S., van Bork, R., Waldorp, L. J., et al. (2018). An introduction to network psychometrics: Relating ising network models to item response theory models. Multivariate Behavioral Research, 53, 15–35.
McCullagh, P., & Nelder, J. A. (1983). Generalized linear models. London: Chapman & Hall.
Pearl, J. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference. San Francisco: Morgan Kaufmann.
Rasch, G. (1961). On general laws and meaning of measurement in psychology. In Proceedings of the fourth Berkeley symposium on mathematical statistics and probability (volume 4) (pp. 321–333).
Revelle, W. (2019). psych: Procedures for psychological, psychometric, and personality research [Computer software manual]. Evanston, Illinois. Retrieved from https://CRAN.R-project.org/package=psych (R package version 1.9.12).
Rost, J. (1990). Rasch models in latent classes: An integration of two approaches to item analysis. Applied Psychological Measurement, 14, 271–282.
Schweinberger, M., & Snijders, T. A. B. (2003). Settings in social networks: A measurement model. In R. M. Stolzenberg (Ed.), Sociological methodology (Vol. 33, pp. 307–341). Boston & Oxford: Basil Blackwell.
Sewell, D. K., & Chen, Y. (2015). Latent space models for dynamic networks. Journal of the American Statistical Association, 110, 1646–1657.
Skrondal, A., & Rabe-Hesketh, S. (2004). Generalized latent variable modeling: Multilevel, longitudinal, and structural equation models. Boca Raton, FL: Chapman & Hall/CRC.
Smith, A. L., Asta, D. M., & Calder, C. A. (2019). The geometry of continuous latent space models for network data. Statistical Science, 34, 428–453.
Social and community planning research. (1987). British social attitude, the 1987 report. Aldershot: Gower Publishing.
Spiel, C., & Gluck, J. (2008). A model based test of competence profile and competence level in deductive reasoning. In J. Hartig, E. Klieme, & D. Leutner (Eds.), Assessment of competencies in educational contexts: State of the art and future prospects (pp. 41–60). Gottingen: Hogrefe.
Spiel, C., Gluck, J., & Gossler, H. (2001). Stability and change of unidimensionality: The sample case of deductive reasoning. Journal of Adolescent Research, 16, 150–168.
Wainer, H., & Kiely, G. L. (1987). Item clusters and computerized adaptive testing: A case for testlets. Journal of Educational Measurement, 24, 185–201.
Wasserman, S., & Faust, K. (1994). Social network analysis: Methods and applications. Cambridge: Cambridge University Press.
Wilson, M., & Adams, R. J. (1995). Rasch models for item bundles. Psychometrika, 60, 181–198.
Acknowledgements
Ick Hoon Jin was partially supported by the Yonsei University Research Fund 2019-22-0210 and the Basic Science Research Program through the National Research Foundation of Korea (NRF 2020R1A2C1A01009881). Michael Schweinberger was partially supported by NSF award DMS-1812119.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Minjeong Jeon, Ick Hoon Jin, and Michael Schweinberger are co-first authors with equal contribution.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Jeon, M., Jin, I.H., Schweinberger, M. et al. Mapping Unobserved Item–Respondent Interactions: A Latent Space Item Response Model with Interaction Map. Psychometrika 86, 378–403 (2021). https://doi.org/10.1007/s11336-021-09762-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11336-021-09762-5