Skip to main content

Advertisement

Log in

On a Generalization of Local Independence in Item Response Theory Based on Knowledge Space Theory

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

Knowledge space theory (KST) structures are introduced within item response theory (IRT) as a possible way to model local dependence between items. The aim of this paper is threefold: firstly, to generalize the usual characterization of local independence without introducing new parameters; secondly, to merge the information provided by the IRT and KST perspectives; and thirdly, to contribute to the literature that bridges continuous and discrete theories of assessment. In detail, connections are established between the KST simple learning model (SLM) and the IRT General Graded Response Model, and between the KST Basic Local Independence Model and IRT models in general. As a consequence, local independence is generalized to account for the existence of prerequisite relations between the items, IRT models become a subset of KST models, IRT likelihood functions can be generalized to broader families, and the issues of local dependence and dimensionality are partially disentangled. Models are discussed for both dichotomous and polytomous items and conclusions are drawn on their interpretation. Considerations on possible consequences in terms of model identifiability and estimation procedures are also provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Similar content being viewed by others

Notes

  1. The three requirements given in Theorem 2 of Holland (1981, p. 86) can be shown to hold in the case of dependence between the items. If one considers for instance requirement (a), following Holland’s abbreviated notation, the set of responses \(X_{A}\) and \(X_{A'}\) associated with two subsets \(A,A'\) of items of a test T are such that \(P(X_{A+A'}=1)\ge P(X_A=1)P(X_{A'}=1)\). If now A is a set of prerequisites for \(A'\), we have that \(P(X_{A'}=1)=P(X_{A+A'}=1)\) so that it follows \(P(X_A=1)\le 1\) which is always satisfied by definition. The same rationale can be applied to the other requirements (b) and (c).

  2. The acronym SCC was also used by Holland (1981) to define a Subtest Characteristic Curve, which is a very similar concept although restricted to the power set of all possible items composing a test rather than defined for arbitrary structures.

Abbreviations

nPNO:

n Parameters normal ogive model, for the 4PNO see Eq. (3)

nPL:

n Parameters logistic model, for the 4PL see Eqs. (1) and (2)

BLIM:

Basic Local Independence Model, see Eq. (15)

CRF:

Category response function

CRMs:

Continuation ratio models

GRM:

Graded Response Model, see Eq. (5)

ICC:

Item characteristic curve

IRT:

Item response theory

KST:

Knowledge space theory

LD:

Local dependence

LKS:

Logistic knowledge structure, see Definition 1

PCM:

Partial credit model

PKS:

Probabilistic knowledge structure, see Eq. (14)

SLM:

Simple learning model, see Eq. (19)

SCC:

State characteristic curve (or subtest characteristic curve).

References

  • Albert, D. (Ed.). (1994). Knowledge structures. New York: Springer.

    Google Scholar 

  • Barton, M. A., & Lord, F. M. (1981). An upper asymptote for the three-parameter logistic item-response model. Princeton: Educational Testing Service.

    Book  Google Scholar 

  • Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37(1), 29–51.

    Article  Google Scholar 

  • Bradlow, E., Wainer, H., & Wang, X. (1999). A Bayesian random effects model for testlets. Psychometrika, 64(2), 153–168.

    Article  Google Scholar 

  • Braeken, J., Tuerlinckx, F., & De Boeck, P. (2007). Copulas for residual dependencies. Psychometrika, 72(3), 393–411.

    Article  Google Scholar 

  • Chen, W.-H., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22(3), 265–289.

    Article  Google Scholar 

  • Doignon, J.-P., & Falmagne, J.-C. (1985). Spaces for the assessment of knowledge. International Journal of Man-Machine Studies, 23(2), 175–196.

    Article  Google Scholar 

  • Falmagne, J.-C., & Doignon, J.-P. (2011). Learning spaces. Berlin: Springer.

    Book  Google Scholar 

  • Falmagne, J.-C., Koppen, M., Villano, M., Doignon, J.-P., & Johanneses, L. (1990). Introduction to knowledge spaces: How to build, test, and search them. Psychological Review, 97(2), 201–224.

    Article  Google Scholar 

  • Fischer, G. H., & Molenaar, I. W. (1995). Rasch models: foundations, recent developments, and applications. New York: Springer.

    Book  Google Scholar 

  • Guttman, L. (1950). The basis for scalogram analysis. In S. A. Stouffer, L. Guttman, E. A. Suchman, P. F. Lazarsfeld, S. A. Star, & J. A. Clausen (Eds.), Measurement and prediction [studies in social psychology in World War II] (Vol. IV). Princeton, NJ: Princeton University Press.

    Google Scholar 

  • Heller, J. (2017). Identifiability in probabilistic knowledge structures. Journal of Mathematical Psychology, 77, 46–57.

    Article  Google Scholar 

  • Heller, J., Stefanutti, L., Anselmi, P., & Robusto, E. (2015). On the link between cognitive diagnostic models and knowledge space theory. Psychometrika, 80(4), 995–1019.

    Article  Google Scholar 

  • Hemker, B. T., van der Ark, L. A., & Sijtsma, K. (2001). On Measurement properties of continuation ratio models. Psychometrika, 66(4), 487–506.

    Article  Google Scholar 

  • Holland, P. W. (1981). When are item response models consistent with observed data? Psychometrika, 46(1), 79–92.

    Article  Google Scholar 

  • Ip, E. H. (2002). Locally dependent latent trait model and the Dutch identity revisited. Psychometrika, 67(3), 367–386.

    Article  Google Scholar 

  • Ip, E. H. (2010). Empirically indistinguishable multidimensional IRT and locally dependent unidimensional item response models. British Journal of Mathematical and Statistical Psychology, 63, 395–416.

    Article  Google Scholar 

  • Irtel, H. (1995). An extension of the concept of specific objectivity. Psychometrika, 60(1), 115–118.

    Article  Google Scholar 

  • Irtel, H. (1996). Entscheidungs- und testtheoretische Grundlagen der Psychologischen Diagnostik. Frankfurt am Main: Verlag Peter Lang.

    Google Scholar 

  • Jannarone, R. J. (1986). Conjunctive item response theory kernels. Psychometrika, 51(3), 357–373.

    Article  Google Scholar 

  • Jannarone, R. J. (1997). Models for locally dependent responses: Conjunctive item response theory. In W. J. van der Linden, J. Updike, & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 465–479). New York: Springer.

    Chapter  Google Scholar 

  • Johnson, M. S., & Sinharay, S. (2016). Bayesian estimation. In W. J. van der Linden (Ed.), Handbook of item response theory. Vol. II: Statistical tools (pp. 237–257). Boca Raton, FL: CRC Press.

    Google Scholar 

  • Liu, Y., & Maydeu-Olivares, A. (2012). Local dependence diagnostics in IRT modeling of binary data. Educational and Psychological Measurement, 73(2), 254–274.

    Article  Google Scholar 

  • Loken, E., & Rulison, K. L. (2010). Estimation of a 4-parameter item response theory model. The British Journal of Mathematical and Statistical Psychology, 63, 509–525.

    Article  Google Scholar 

  • Lord, F. M., & Novik, M. R. (1968). Statistical theories of mental test scores. London: Addison-Wesley Publishing Company.

    Google Scholar 

  • Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47(2), 149–174.

    Article  Google Scholar 

  • Mokken, R. J. (1971). A theory and procedure of scale analysis with applications in political research. New York: Walter de Gruyter, Mouton.

    Book  Google Scholar 

  • Noventa, S., & Stefanutti, L. (2016). Some considerations on the factorization of knowledge state probabilities in knowledge structures. Manuscript submitted for publication.

  • Noventa, S., Stefanutti, L., & Vidotto, G. (2014). An analysis of item response theory and Rasch models based on the most probable distribution method. Psychometrika, 79(3), 377–402.

    Article  Google Scholar 

  • Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenaghen: Nielsen & Lydiche.

    Google Scholar 

  • Rusch, A., & Wille, R. (1996). Knowledge spaces and formal concept analysis. In H.-H. Bock & W. Polasek (Eds.), Data analysis and information systems (pp. 427–436). Berlin: Springer.

    Chapter  Google Scholar 

  • Rosenbaum, P. R. (1984). testing conditional independence and monotonicity assumptions of item response theory. Psychometrika, 49(3), 425–435.

    Article  Google Scholar 

  • Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores (psychometric monograph No. 17). Richmond, VA: Psychometric Society.

    Google Scholar 

  • Samejima, F. (1972). A general model for free-response data (psychometric monograph no. 18). Richmond, VA: Psychometric Society.

    Google Scholar 

  • Samejima, F. (1973). A comment of Birnbaum’s three parameter logistic model in the latent trait theory. Psychometrika, 38(2), 221–233.

    Article  Google Scholar 

  • Samejima, F. (1995). Acceleration model in the heterogeneous case of the general graded response model. Psychometrika, 60(4), 549–572.

    Article  Google Scholar 

  • Scheiblechner, H. (1995). Isotonic ordinal probabilistic models. Psychometrika, 60(2), 281–304.

    Article  Google Scholar 

  • Schrepp, M. (2003). A method for the analysis of hierarchical dependencies between items of a questionnaire. Methods of Psychological Research, 19, 43–79.

    Google Scholar 

  • Schrepp, M. (2005). About the connection between knowledge structures and latent class models. Methodology, 1(3), 92–102.

    Article  Google Scholar 

  • Sijtsma, K. (1998). Methodology review: Nonparametric IRT approaches to the analysis of dichotomous item scores. Applied Psychological Measurement, 22, 3–31.

    Article  Google Scholar 

  • Spoto, A., Stefanutti, L., & Vidotto, G. (2012). On the unidentifiability of a certain class of skill multi map based probabilistic knowledge structures. Journal of Mathematical Psychology, 56, 248–255.

    Article  Google Scholar 

  • Spoto, A., Stefanutti, L., & Vidotto, G. (2013). Considerations about the identification of forward- and backward-graded knowledge structures. Journal of Mathematical Psychology, 57, 249–254.

    Article  Google Scholar 

  • Spoto, A., Stefanutti, L., & Vidotto, G. (2016). An iterative procedure for extracting skillmaps from data. Behaviour Research Methods, 48, 729–741.

    Article  Google Scholar 

  • Stefanutti, L. (2006). A logistic approach to knowledge structures. Journal of Mathematical Psychology, 50, 545–561.

    Article  Google Scholar 

  • Stefanutti, L., Spoto, A., & Vidotto, G. (2018). Detecting and explaining BLIM’s unidentifiability: Forward and backward parameter transformation groups. Journal of Mathematical Psychology, 82, 38–51.

    Article  Google Scholar 

  • Thissen, D., & Steinberg, L. (1986). A taxonomy of item response models. Psychometrika, 51(4), 567–577.

    Article  Google Scholar 

  • Tutz, G. (1990). Sequential item response models with an ordered response. British Journal of Mathematical and Statistical psychology, 43, 39–55.

    Article  Google Scholar 

  • Tutz, G. (2016). Sequential models for ordered responses. In W. J. van der Linden (Ed.), Handbook of item response theory. Vol. I: Models (pp. 139–151). Boca Raton, FL: CRC Press.

    Google Scholar 

  • Ünlü, A. (2006). Estimation of careless error and lucky guess probabilities for dichotomous test items: A psychometric application of a biometric latent class model with random effects. Journal of Mathematical Psychology, 50, 309–328.

    Article  Google Scholar 

  • Ünlü, A. (2007). Nonparametric item response theory axioms and properties under nonlinearity and their exemplification with knowledge space theory. Journal of Mathematical Psychology, 51, 383–400.

    Article  Google Scholar 

  • Ünlü, A. (2011). A Note on the connection between knowledge structures and latent class models. Methodology, 7(2), 63–67.

    Article  Google Scholar 

  • Van der Linden, W. J., & Barrett, M. D. (2016). Linking item response model parameters. Psychometrika, 81(3), 650–673.

    Article  Google Scholar 

  • Van leeuwe, J. F. J. (1974). Item tree analysis. Nederlands Tijdschrift voor de Psychologie, 29, 475–484.

    Google Scholar 

  • Verhelst, N. D., Glas, C. A. W., & de Vries, H. H. (1997). A steps model to analyze partial credit. In W. J. van der Linden, J. Updike, & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 123–138). New York: Springer.

    Chapter  Google Scholar 

  • Wilson, M., & Adams, R. J. (1995). Rasch models for item bundles. Psychometrika, 60(2), 181–198.

    Article  Google Scholar 

  • Wood, R. (1978). Fitting the Rasch model—A heady tale. British Journal of Mathematical and Statistical Psychology, 31, 27–32.

    Article  Google Scholar 

  • Yen, W. M. (1993). Scaling performance assessments: Strategies for managing local item independence. Journal of Educational Measurement, 30(3), 187–213.

    Article  Google Scholar 

Download references

Acknowledgements

We wish to thank the two anonymous reviewers of the journal for their insight into the work and their helpful comments and suggestions. In particular, we would like to thank the second reviewer for pointing out the connection with Holland (1981).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Stefano Noventa.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Noventa, S., Spoto, A., Heller, J. et al. On a Generalization of Local Independence in Item Response Theory Based on Knowledge Space Theory. Psychometrika 84, 395–421 (2019). https://doi.org/10.1007/s11336-018-9645-6

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11336-018-9645-6

Keywords

Navigation