Advertisement

On a Generalization of Local Independence in Item Response Theory Based on Knowledge Space Theory

  • Stefano NoventaEmail author
  • Andrea Spoto
  • Jürgen Heller
  • Augustin Kelava
Article
  • 85 Downloads

Abstract

Knowledge space theory (KST) structures are introduced within item response theory (IRT) as a possible way to model local dependence between items. The aim of this paper is threefold: firstly, to generalize the usual characterization of local independence without introducing new parameters; secondly, to merge the information provided by the IRT and KST perspectives; and thirdly, to contribute to the literature that bridges continuous and discrete theories of assessment. In detail, connections are established between the KST simple learning model (SLM) and the IRT General Graded Response Model, and between the KST Basic Local Independence Model and IRT models in general. As a consequence, local independence is generalized to account for the existence of prerequisite relations between the items, IRT models become a subset of KST models, IRT likelihood functions can be generalized to broader families, and the issues of local dependence and dimensionality are partially disentangled. Models are discussed for both dichotomous and polytomous items and conclusions are drawn on their interpretation. Considerations on possible consequences in terms of model identifiability and estimation procedures are also provided.

Keywords

knowledge space theory item response theory Rasch models local independence BLIM Graded Response Model 

Abbreviations

nPNO

n Parameters normal ogive model, for the 4PNO see Eq. (3)

nPL

n Parameters logistic model, for the 4PL see Eqs. (1) and (2)

BLIM

Basic Local Independence Model, see Eq. (15)

CRF

Category response function

CRMs

Continuation ratio models

GRM

Graded Response Model, see Eq. (5)

ICC

Item characteristic curve

IRT

Item response theory

KST

Knowledge space theory

LD

Local dependence

LKS

Logistic knowledge structure, see Definition 1

PCM

Partial credit model

PKS

Probabilistic knowledge structure, see Eq. (14)

SLM

Simple learning model, see Eq. (19)

SCC

State characteristic curve (or subtest characteristic curve).

Notes

Acknowledgements

We wish to thank the two anonymous reviewers of the journal for their insight into the work and their helpful comments and suggestions. In particular, we would like to thank the second reviewer for pointing out the connection with Holland (1981).

References

  1. Albert, D. (Ed.). (1994). Knowledge structures. New York: Springer.Google Scholar
  2. Barton, M. A., & Lord, F. M. (1981). An upper asymptote for the three-parameter logistic item-response model. Princeton: Educational Testing Service.Google Scholar
  3. Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37(1), 29–51.CrossRefGoogle Scholar
  4. Bradlow, E., Wainer, H., & Wang, X. (1999). A Bayesian random effects model for testlets. Psychometrika, 64(2), 153–168.CrossRefGoogle Scholar
  5. Braeken, J., Tuerlinckx, F., & De Boeck, P. (2007). Copulas for residual dependencies. Psychometrika, 72(3), 393–411.CrossRefGoogle Scholar
  6. Chen, W.-H., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22(3), 265–289.CrossRefGoogle Scholar
  7. Doignon, J.-P., & Falmagne, J.-C. (1985). Spaces for the assessment of knowledge. International Journal of Man-Machine Studies, 23(2), 175–196.CrossRefGoogle Scholar
  8. Falmagne, J.-C., & Doignon, J.-P. (2011). Learning spaces. Berlin: Springer.CrossRefGoogle Scholar
  9. Falmagne, J.-C., Koppen, M., Villano, M., Doignon, J.-P., & Johanneses, L. (1990). Introduction to knowledge spaces: How to build, test, and search them. Psychological Review, 97(2), 201–224.CrossRefGoogle Scholar
  10. Fischer, G. H., & Molenaar, I. W. (1995). Rasch models: foundations, recent developments, and applications. New York: Springer.CrossRefGoogle Scholar
  11. Guttman, L. (1950). The basis for scalogram analysis. In S. A. Stouffer, L. Guttman, E. A. Suchman, P. F. Lazarsfeld, S. A. Star, & J. A. Clausen (Eds.), Measurement and prediction [studies in social psychology in World War II] (Vol. IV). Princeton, NJ: Princeton University Press.Google Scholar
  12. Heller, J. (2017). Identifiability in probabilistic knowledge structures. Journal of Mathematical Psychology, 77, 46–57.CrossRefGoogle Scholar
  13. Heller, J., Stefanutti, L., Anselmi, P., & Robusto, E. (2015). On the link between cognitive diagnostic models and knowledge space theory. Psychometrika, 80(4), 995–1019.CrossRefGoogle Scholar
  14. Hemker, B. T., van der Ark, L. A., & Sijtsma, K. (2001). On Measurement properties of continuation ratio models. Psychometrika, 66(4), 487–506.CrossRefGoogle Scholar
  15. Holland, P. W. (1981). When are item response models consistent with observed data? Psychometrika, 46(1), 79–92.CrossRefGoogle Scholar
  16. Ip, E. H. (2002). Locally dependent latent trait model and the Dutch identity revisited. Psychometrika, 67(3), 367–386.CrossRefGoogle Scholar
  17. Ip, E. H. (2010). Empirically indistinguishable multidimensional IRT and locally dependent unidimensional item response models. British Journal of Mathematical and Statistical Psychology, 63, 395–416.CrossRefGoogle Scholar
  18. Irtel, H. (1995). An extension of the concept of specific objectivity. Psychometrika, 60(1), 115–118.CrossRefGoogle Scholar
  19. Irtel, H. (1996). Entscheidungs- und testtheoretische Grundlagen der Psychologischen Diagnostik. Frankfurt am Main: Verlag Peter Lang.Google Scholar
  20. Jannarone, R. J. (1986). Conjunctive item response theory kernels. Psychometrika, 51(3), 357–373.CrossRefGoogle Scholar
  21. Jannarone, R. J. (1997). Models for locally dependent responses: Conjunctive item response theory. In W. J. van der Linden, J. Updike, & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 465–479). New York: Springer.CrossRefGoogle Scholar
  22. Johnson, M. S., & Sinharay, S. (2016). Bayesian estimation. In W. J. van der Linden (Ed.), Handbook of item response theory. Vol. II: Statistical tools (pp. 237–257). Boca Raton, FL: CRC Press.Google Scholar
  23. Liu, Y., & Maydeu-Olivares, A. (2012). Local dependence diagnostics in IRT modeling of binary data. Educational and Psychological Measurement, 73(2), 254–274.CrossRefGoogle Scholar
  24. Loken, E., & Rulison, K. L. (2010). Estimation of a 4-parameter item response theory model. The British Journal of Mathematical and Statistical Psychology, 63, 509–525.CrossRefGoogle Scholar
  25. Lord, F. M., & Novik, M. R. (1968). Statistical theories of mental test scores. London: Addison-Wesley Publishing Company.Google Scholar
  26. Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47(2), 149–174.CrossRefGoogle Scholar
  27. Mokken, R. J. (1971). A theory and procedure of scale analysis with applications in political research. New York: Walter de Gruyter, Mouton.CrossRefGoogle Scholar
  28. Noventa, S., & Stefanutti, L. (2016). Some considerations on the factorization of knowledge state probabilities in knowledge structures. Manuscript submitted for publication.Google Scholar
  29. Noventa, S., Stefanutti, L., & Vidotto, G. (2014). An analysis of item response theory and Rasch models based on the most probable distribution method. Psychometrika, 79(3), 377–402.CrossRefGoogle Scholar
  30. Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenaghen: Nielsen & Lydiche.Google Scholar
  31. Rusch, A., & Wille, R. (1996). Knowledge spaces and formal concept analysis. In H.-H. Bock & W. Polasek (Eds.), Data analysis and information systems (pp. 427–436). Berlin: Springer.CrossRefGoogle Scholar
  32. Rosenbaum, P. R. (1984). testing conditional independence and monotonicity assumptions of item response theory. Psychometrika, 49(3), 425–435.CrossRefGoogle Scholar
  33. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores (psychometric monograph No. 17). Richmond, VA: Psychometric Society.Google Scholar
  34. Samejima, F. (1972). A general model for free-response data (psychometric monograph no. 18). Richmond, VA: Psychometric Society.Google Scholar
  35. Samejima, F. (1973). A comment of Birnbaum’s three parameter logistic model in the latent trait theory. Psychometrika, 38(2), 221–233.CrossRefGoogle Scholar
  36. Samejima, F. (1995). Acceleration model in the heterogeneous case of the general graded response model. Psychometrika, 60(4), 549–572.CrossRefGoogle Scholar
  37. Scheiblechner, H. (1995). Isotonic ordinal probabilistic models. Psychometrika, 60(2), 281–304.CrossRefGoogle Scholar
  38. Schrepp, M. (2003). A method for the analysis of hierarchical dependencies between items of a questionnaire. Methods of Psychological Research, 19, 43–79.Google Scholar
  39. Schrepp, M. (2005). About the connection between knowledge structures and latent class models. Methodology, 1(3), 92–102.CrossRefGoogle Scholar
  40. Sijtsma, K. (1998). Methodology review: Nonparametric IRT approaches to the analysis of dichotomous item scores. Applied Psychological Measurement, 22, 3–31.CrossRefGoogle Scholar
  41. Spoto, A., Stefanutti, L., & Vidotto, G. (2012). On the unidentifiability of a certain class of skill multi map based probabilistic knowledge structures. Journal of Mathematical Psychology, 56, 248–255.CrossRefGoogle Scholar
  42. Spoto, A., Stefanutti, L., & Vidotto, G. (2013). Considerations about the identification of forward- and backward-graded knowledge structures. Journal of Mathematical Psychology, 57, 249–254.CrossRefGoogle Scholar
  43. Spoto, A., Stefanutti, L., & Vidotto, G. (2016). An iterative procedure for extracting skillmaps from data. Behaviour Research Methods, 48, 729–741.CrossRefGoogle Scholar
  44. Stefanutti, L. (2006). A logistic approach to knowledge structures. Journal of Mathematical Psychology, 50, 545–561.CrossRefGoogle Scholar
  45. Stefanutti, L., Spoto, A., & Vidotto, G. (2018). Detecting and explaining BLIM’s unidentifiability: Forward and backward parameter transformation groups. Journal of Mathematical Psychology, 82, 38–51.CrossRefGoogle Scholar
  46. Thissen, D., & Steinberg, L. (1986). A taxonomy of item response models. Psychometrika, 51(4), 567–577.CrossRefGoogle Scholar
  47. Tutz, G. (1990). Sequential item response models with an ordered response. British Journal of Mathematical and Statistical psychology, 43, 39–55.CrossRefGoogle Scholar
  48. Tutz, G. (2016). Sequential models for ordered responses. In W. J. van der Linden (Ed.), Handbook of item response theory. Vol. I: Models (pp. 139–151). Boca Raton, FL: CRC Press.Google Scholar
  49. Ünlü, A. (2006). Estimation of careless error and lucky guess probabilities for dichotomous test items: A psychometric application of a biometric latent class model with random effects. Journal of Mathematical Psychology, 50, 309–328.CrossRefGoogle Scholar
  50. Ünlü, A. (2007). Nonparametric item response theory axioms and properties under nonlinearity and their exemplification with knowledge space theory. Journal of Mathematical Psychology, 51, 383–400.CrossRefGoogle Scholar
  51. Ünlü, A. (2011). A Note on the connection between knowledge structures and latent class models. Methodology, 7(2), 63–67.CrossRefGoogle Scholar
  52. Van der Linden, W. J., & Barrett, M. D. (2016). Linking item response model parameters. Psychometrika, 81(3), 650–673.CrossRefGoogle Scholar
  53. Van leeuwe, J. F. J. (1974). Item tree analysis. Nederlands Tijdschrift voor de Psychologie, 29, 475–484.Google Scholar
  54. Verhelst, N. D., Glas, C. A. W., & de Vries, H. H. (1997). A steps model to analyze partial credit. In W. J. van der Linden, J. Updike, & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 123–138). New York: Springer.CrossRefGoogle Scholar
  55. Wilson, M., & Adams, R. J. (1995). Rasch models for item bundles. Psychometrika, 60(2), 181–198.CrossRefGoogle Scholar
  56. Wood, R. (1978). Fitting the Rasch model—A heady tale. British Journal of Mathematical and Statistical Psychology, 31, 27–32.CrossRefGoogle Scholar
  57. Yen, W. M. (1993). Scaling performance assessments: Strategies for managing local item independence. Journal of Educational Measurement, 30(3), 187–213.CrossRefGoogle Scholar

Copyright information

© The Psychometric Society 2018

Authors and Affiliations

  1. 1.Methods CenterUniversität TübingenTübingenGermany
  2. 2.Department of General PsychologyUniversity of PadovaPaduaItaly
  3. 3.Faculty of PsychologyUniversität TübingenTübingenGermany

Personalised recommendations