Abstract
When multiple items are clustered around a reading passage, the local independence assumption in item response theory is often violated. The amount of information contained in an item cluster is usually overestimated if violation of local independence is ignored and items are treated as locally independent when in fact they are not. In this article we provide a general method that adjusts for the inflation of information associated with a test containing item clusters. A computational scheme was presented for the evaluation of the factor of adjustment for clusters in the restrictive case of two items per cluster, and the general case of more than two items per cluster. The methodology was motivated by a study of the NAEP Reading Assessment. We present a simulated study along with an analysis of a NAEP data set.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ackerman, T., & Spray, J. (1987).A general model for item dependency (Report No. RR-87-9). Iowa City, IA: ACT.
Allen, N., Kline, D. L., & Zelenak, C. A. (1997).The NAEP 1994 Technical Report. Washington, D.C.: Department of Education.
Bahadur, R. (1961). A representation of the joint distribution of responses to n dichotomous items. In. H. Solomon (Ed.),Studies in item analysis and prediction (pp. 158–168). Palo Alto, CA: Stanford University Press.
Becker, R. A., Chambers, J. M., & Wilks, A. R. (1988).The new S language. New York: Chapman & Hall.
Bishop, Y., Fienberg, S., & Holland, P. (1975).Discrete multivariate analysis. Boston, MA: MIT Press.
Bradlow, E. (1996). Negative information and the three-parameter logistic model.Journal of Educational and Behavioral Statistics, 21, 179–185.
Cox, D. (1972). The analysis of multivariate binary data.Applied Statistics, 21, 113–120.
Dale, R. (1986). Global cross-ratio models for bivariate, discrete ordered responses.Biometrics, 42, 909–917.
Diggle, P. J., Liang, K. L., & Zeger, S. L. (1994).Analysis of longitudinal data. New York: Oxford University Press.
Efron, B., & Hinkley, D. (1978). Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information.Biometrika, 65, 457–481.
Efron, B., & Tibshirani, R. (1993).An introduction to the bootstrap. New York: Chapman & Hall.
Fahrmeir, L., & Tutz, G. (1994).Multivariate statistical modelling based on generalized linear models. New York: Springer-Verlag.
Fitzmaurice, G. M., Laird, N. M., & Rotnitsky, A. G. (1993). Regression model for discrete longitudinal responses (with discussion).Statistical Science, 8, 284–309.
Greenacre, M. J. (1984).Theory and applications of correspondence analysis. New York: Academic Press.
Holland, P., & Rosenbaum, P. (1986). Conditional association and unidimensionality in montone latent variable models.Annals of Statistics, 14, 1523–1543.
Jannarone, R. J. (1992). Local dependence: Objectively measurable or objectionably abominable? In M. Wilson (Ed.),Objective measurement: Theory into practice, Volume 2. Norwood, NJ: Ablex Publishing.
Junker, B. W. (1991). Essential independence and likelihood-based ability estimation for polytomous items.Psychometrika, 56, 255–278.
Junker, B. W. (1993). Conditional association, essential independence and monotone unidimensional item response model.Annals of Statistics, 21, 1359–1378.
Liang, K. L., Zeger, S. L., & Qaqish, B. (1992). Multivariate regression analyses for categorical data (with discussion).Journal of the Royal Statistical Society, Series B, 54, 3–40.
Prentice, R. L. (1986). Binary regression using an extended beta-binomial distribution, with discussion of correlation induced by covariate measurement errors.Journal of American Statistical Association, 81, 321–327.
Prentice, R. L. (1988). Correlated binary regression with covariates specific to each binary observation.Biometrics, 44, 1033–1048.
Press, W., Teukolsky, S., Vetterling, W., & Flannery, B. (1992).Numerical recipes in C. Cambridge University Press.
Sireci, S. G., Thissen, D., & Wainer, H. (1991). On the reliability of testlet-based tests.Journal of Educational Measurement, 28, 237–247.
Stout, W. (1987). A nonparametric approach for assessing latent trait dimensionality.Psychometrika, 52, 589–617.
Stout, W. (1990). A new item response theory modeling approach with application to unidimensionality assessment and ability estimation.Psychometrika, 55, 293–325.
Stout, W., Habing, B., Douglas, J., Kim, H., Roussos, L., & Zhang, J. (1996). Conditional covariance based nonparametric multidimensionality assessment.Applied Psychological Measurement, 20, 331–354.
Stout, W. F., Nandakumar, R., Junker, B., Chang, H. H., & Steidinger, D. (1991). DIMTEST and TESTSIM [Computer program]. Champaign: University of Illinois, Department of Statistics.
Suppes, P., & Zanotti, M. (1981). When are probabilistic explanations possible?Synthese, 48, 191–199.
Wainer, H. (1995). Precision and differential item functioning on a testlet-based test: The 1991 Law School Admissions Test as an example.Applied Measurement in Education, 8, 157–186.
Wainer, H., & Kiely, G.L. (1987). Item clusters and computerized adaptive testing: A case for testlets.Journal of Educational Measurement, 24, 185–201.
Wainer, H., & Thissen, D. (1996). How reliable should a test be? What is the effect of local dependence on reliability?Educational Measurement: Issues and Practice, 15, 22–29.
Woodruff, H., Ritter, G., Lowry, S., & Isenhour, T. (1975). Density estimations and the characterization of binary infrared spectra.Technometrics, 17, 455–462.
Wu, H. (1997).Some issues in item response theory. Unpublished doctoral dissertation, University of Illinois, Urbana, IL.
Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model.Applied Psychological Measurement, 8, 125–145.
Yen, W. M. (1993). Scaling performance assessments: Strategies for managing local item dependence.Journal of Educational Measurement, 30, 187–213.
Zhao, L. P., & Prenctice, R. L. (1990). Correlated binary regression using a generalized quadratic model.Biometrika, 77, 642–648.
Zwick, R. (1987). Assessing the dimensionality of NAEP reading data.Journal of Educational Measurement, 24, 293–308.
Author information
Authors and Affiliations
Additional information
The research was supported under the National Assessment of Educational Progress (Grant No. R999G30002) as administered by the Office of Educational Research and Improvement, U.S. Department of Education. This work was started when the author was at the Division of Statistics and Psychometrics at the Educational Testing Service. The author thanks Juliet Shaffer, Bob Mislevy, Eric Bradlow, three reviewers and an associate editor for their helpful comments on the paper.
Rights and permissions
About this article
Cite this article
Ip, E.Hs. Adjusting for information inflation due to local dependency in moderately large item clusters. Psychometrika 65, 73–91 (2000). https://doi.org/10.1007/BF02294187
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02294187