Skip to main content
SpringerLink
Log in

Exact tests for the rasch model via sequential importance sampling

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

Rasch proposed an exact conditional inference approach to testing his model but never implemented it because it involves the calculation of a complicated probability. This paper furthers Rasch’s approach by (1) providing an efficient Monte Carlo methodology for accurately approximating the required probability and (2) illustrating the usefulness of Rasch’s approach for several important testing problems through simulation studies. Our Monte Carlo methodology is shown to compare favorably to other Monte Carlo methods proposed for this problem in two respects: it is considerably faster and it provides more reliable estimates of the Monte Carlo standard error.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Andersen, E.B. (1973). Conditional inference for multiple-choice questionnaires. British Journal of Mathematical and Statistical Psychology 26, 31–44.

    Google Scholar 

  • Andersen, E.B. (1980). Discrete Statistical Models with Social Science Applications. Amsterdam: North-Holland.

    Google Scholar 

  • Andersen, E.B. (1995). What Georg Rasch would have thought about this book. In: G.H. Fischer, I.W. Molenaar (Eds.), Rasch Models: Their Foundations, Recent Developments and Applications. New York: Springer-Verlag.

    Google Scholar 

  • Békéssy, A., Békéssy, P., Komlos, J. (1972). Asymptotic enumeration of regular matrices. Studia Scientiarum Mathematicarum Hungarica 7, 343–353.

    Google Scholar 

  • Bender, E.A. (1974). The asymptotic number of nonnegative integer matrices with given row and column sums. Discrete Mathematics 10, 217–223.

    Article  Google Scholar 

  • Besag, J., Clifford, P. (1989). Generalized Monte-Carlo significance tests. Biometrika 76, 633–642.

    Google Scholar 

  • Birch, M.W. (1964). The detection of partial association I: The 2×2 case. Journal of the Royal Statistical Society Series B 26, 313–324.

    Google Scholar 

  • Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In: Lord FM, Novick MR (Eds) Statistical Theories of Mental Test Scores. Reading: Addison-Wesley.

    Google Scholar 

  • Bishop, Y.M.M., Fienberg, S.E., Holland, P.W. (1975). Discrete Multivariate Analysis. Cambridge: MIT Press.

    Google Scholar 

  • Chen, X.H., Dempster, A.P., Liu, J.S. (1994). Weighted finite population sampling to maximize entropy. Biometrika 81, 457–469.

    Google Scholar 

  • Chen, Y. (2001). Sequential importance sampling with resampling: theory and applications. PhD dissertation, Department of Statistics, Stanford University.

  • Chen, Y., Diaconis, P., Holmes, S., Liu, J.S. (2005). Sequential Monte Carlo methods for statistical analysis of tables. Journal of the American Statistical Association 100, 109–120.

    Article  Google Scholar 

  • Coleman, J.S. (1957). Multidimensional scale analysis. American Journal of Sociology 63, 253–263.

    Article  Google Scholar 

  • Connor, E.F., Simberloff, D. (1979). The assembly of species communities: chance or competition. Ecology 60, 1132–1140.

    Google Scholar 

  • Duncan, O.D. (1984). Rasch measurement: further examples and discussion. In: Turner, C.F., Martih, E. (Eds.), Surveying Subjective Phenomena (Volume 2). New York: Russell Sage Foundation.

    Google Scholar 

  • Efron, B., Tibshirani, R.J. (1993). An Introduction to the Bootstrap. Boca Raton: Chapman & Hall.

    Google Scholar 

  • Fischer, G.H. (1993). Notes on the Mantel–Haenszel procedure and another chi-squared test for the assessment of DIF. Methodika 7, 88–100.

    Google Scholar 

  • Fischer, G.H. (1995). Some neglected problems in IRT. Psychometrika 60:449–487.

    Google Scholar 

  • Fischer, G.H., Molenaar, I.W. (1995). Rasch Models: Foundations, Recent Developments, and Applications. New York, Springer-Verlag.

    Google Scholar 

  • Glas, C.A.W. (1988). The derivation of some tests for the Rasch model from the multinomial distribution. Psychometrika 63, 525–546.

    Google Scholar 

  • Glas, C.A.W., Verhelst, N.D. (1995). Testing the Rasch model. In: Fischer, G.H., Molenaar, I.W. (Eds.), Rasch Models: Their Foundations, Recent Developments and Applications. New York: Springer-Verlag.

    Google Scholar 

  • Good, I.J., Crook, J. (1977). The enumeration of arrays and a generalization related to contingency tables. Discrete Mathematics 19, 23–65.

    Article  Google Scholar 

  • Gustafsson, J. (1980). Testing and obtaining fit of data to the Rasch model. British Journal of Mathematical and Statistical Psychology 33, 205–233.

    Google Scholar 

  • Haberman, S.J. (1974). The Analysis of Frequency Data. Chicago: The University of Chicago Press.

    Google Scholar 

  • Holland, P.W., Thayer, D.T. (1988). Differential item performance and the Mantel–Haenszel procedure. In: Wainer, H., Braun, H.I. (Eds.), Test Validity. Hillsdale: Erlbaum.

    Google Scholar 

  • Kelderman, H. (1984). Loglinear Rasch model tests. Psychometrika 49, 223–245.

    Google Scholar 

  • Kelderman, H. (1989). Item bias detection using loglinear IRT. Psychometrika 54, 681–697.

    MathSciNet  Google Scholar 

  • Kong, A., Liu, J.S., Wong, W.H. (1994). Sequential imputations and Bayesian missing data problems. Journal of the American Statistical Association 89, 278–288.

    Google Scholar 

  • Lancaster, H.O. (1961). Significance tests in discrete distributions. Journal of the American Statistical Association 56, 223–234.

    Google Scholar 

  • Lehmann, E.L. (1986). Testing Statistical Hypotheses, 2nd ed. New York: Wiley.

    Google Scholar 

  • Liu, J.S. (2001). Monte Carlo Strategies in Scientific Computing. New York: Springer-Verlag

    Google Scholar 

  • Mantel, N., Fleiss, J.L. (1980). Minimum expected cell size requirements for the Mantel–Haenszel one-degree-of-freedom chi square test and a related rapid procedure. American Journal of Epidemiology 112, 129–134.

    PubMed  Google Scholar 

  • Martin-Löf, P. (1973). Statistika Modeller. Anteckningar från seminarier läsåret 1969-1970 utarbetade av Rolf Sundberg. 2: a uppl.) (Statistical models. Notes from seminars 1969-1970 by Rolf Sundberg, 2nd ed). Institutet för Försä kringsmatematik och Matematisk Statistik vid Stockholms Universitet.

  • Mazor, K.M., Clauser, B.E., Hambleton, R.K. (1992). The effect of sample size on the functioning of the Mantel–Haenszel statistic. Educational and Psychological Measurement 52, 443–451.

    Google Scholar 

  • Mengersen, K.L., Robert, C.P., Cuihenneuc-Joyaux, C. (1999). MCMC convergence diagnostics: A review. In: Bernando, J.M., Berger, J.O., Dawid, A.P., Smith, A.F.M. (Eds.), Bayesian Statistics (Volume 6). Oxford: Clarendon Press.

    Google Scholar 

  • Molenaar, I.W. (1983). Some improved diagnostics for failure in the Rasch model. Psychometrika 48, 49–72.

    Google Scholar 

  • Molenaar, I.W. (1995). Estimation of Item Parameters. In: Fischer, G.H., Molenaar, I.W. (Eds.), Rasch Models: Their Foundations, Recent Developments and Applications. New York: Springer-Verlag.

    Google Scholar 

  • Neyman, J., Pearson, E.S. (1933). On the problem of the most efficient tests of statistical hypotheses. Philosophical Transcations of the Royal Society A 231, 289–337.

    Google Scholar 

  • Orlando, M., Thissen, D. (2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement 24, 50–64.

    Google Scholar 

  • Pashall, C.G., Miller, T.R. (1995). Exact versus asymptotic Mantel–Haenszel DIF statistics: a comparison of performance under small-sample conditions. Journal of Educational Measurement. 32, 302–316.

    Google Scholar 

  • Ponocny, I. (2001). Nonparametric goodness-of-fit tests for the Rasch model. Psychometrika 66, 437–460.

    MathSciNet  Google Scholar 

  • Rao, A.R., Jana, R., Bandyopadhyay, S. (1996). A Markov chain Monte Carlo method for generating random (0,1) matrices with given marginals. Sankhya series A 58, 225–242.

    Google Scholar 

  • Rasch, G. (1960). Probabilistic Models for Some Intelligence and Attainment Tests. Copenhagen: Danish Institute for Educational Research.

    Google Scholar 

  • Rasch, G. (1966). An individualistic approach to item analysis. In: P.F. Lazarsfeld and N.W. Henry (Eds.), Readings Mathematical Social Science. Chicago: Science Research Associates.

    Google Scholar 

  • Roberts, A., Stone, L. (1990). Island-sharing by archipelago species. Oecologia 83, 560–567.

    Article  Google Scholar 

  • Shepard, L.A., Camilli, G., Williams, D.M. (1985). Validity of approximation techniques for detecting item bias. Journal of Educational Measurement 22, 77–105.

    Google Scholar 

  • Snijders, T.A.B. (1991). Enumeration and simulation methods for 0-1 matrices with given marginals. Psychometrika 56 (3), 397–417.

    MathSciNet  Google Scholar 

  • Stouffer, S.A., Toby, J. (1951). Role conflict and personality. American Journal of Sociology 56, 395–406.

    Article  Google Scholar 

  • Tatsuoka, K.K., Linn, R.L. (1983). Indices for detecting unusual patterns: links between two general approaches and potential applications. Applied Psychological Measurement 7, 81–96.

    Google Scholar 

  • van den Wollenberg, A.L. (1982). Two new test statistics for the Rasch model. Psychometrika 47, 123–139.

    Google Scholar 

  • Wright, B.D. (1977). Solving measurement problems with the Rasch model. Journal of Educational Measurement 14, 97–116.

    Google Scholar 

  • Wright, B.D., Stone, M.H. (1979). Best Test Design. Chicago: Mesa Press.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Duke University, USA

    Yuguo Chen

  2. University of Pennsylvania, USA

    Dylan Small

  3. Department of Statistics, The Wharton School, University of Pennsylvania, 400 Jon M. Huntsman Hall, 3730 Locust Walk, Philadelphia, PA, 19104, USA

    Dylan Small

Authors
  1. Yuguo Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dylan Small
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dylan Small.

Additional information

This Research was supported in part by National Science Foundation grant DMS-0203762 and a University of Pennsylvania Research Foundation grant.

The authors are grateful to Don Burdick for helpful comments. In addition, the authors wish to thank the editor, the associate editor, and the referees for their helpful suggestions.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chen, Y., Small, D. Exact tests for the rasch model via sequential importance sampling. Psychometrika 70, 11–30 (2005). https://doi.org/10.1007/s11336-003-1069-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11336-003-1069-1

Keywords

Search

Navigation