Skip to main content
SpringerLink
Log in
Book cover

The Annual Meeting of the Psychometric Society

IMPS 2016: Quantitative Psychology pp 71–79Cite as

A Comparison of Two MCMC Algorithms for the 2PL IRT Model

  • Conference paper

Part of the book series: Springer Proceedings in Mathematics & Statistics ((PROMS,volume 196))

Abstract

Markov chain Monte Carlo (MCMC) techniques have become popular for estimating item response theory (IRT) models. The current development of MCMC includes two major algorithms: Gibbs sampling and the No-U-Turn sampler (NUTS), which can be implemented in two specialized software packages JAGS and Stan, respectively. This study focused on comparing these two algorithms in estimating the two-parameter logistic (2PL) IRT model where different prior specifications for the discrimination parameter were considered. Results suggest that Gibbs sampling performed similarly to the NUTS under most of the conditions considered. In addition, both algorithms recovered model parameters with a similar precision except for small sample size situations. Findings from this study also shed light on the use of the two MCMC algorithms with more complicated IRT models.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  • J.H. Albert, Bayesian estimation of normal ogive item response curves using Gibbs sampling. J. Educ. Behav. Stat. 17(3), 251–269 (1992). doi:10.3102/10769986017003251

    Article  Google Scholar 

  • F.B. Baker, An investigation of the item parameter recovery characteristics of a Gibbs sampling procedure. Appl. Psychol. Meas. 22(2), 153–169 (1998). doi:10.1177/01466216980222005

    Article  Google Scholar 

  • A.A. Béguin, C.A.W. Glas, MCMC estimation and some model-fit analysis of multidimensional IRT models. Psychometrika 66(4), 541–561 (2001). doi:10.1007/BF02296195

    Article  MathSciNet  MATH  Google Scholar 

  • A. Birnbaum, in Statistical Theories of Mental Test Scores, ed. by F. M. Lord, M. R. Novick. Some latent trait models and their use in inferring an examinee’s ability (Addison-Wesley, Reading, MA, 1968), pp. 453–479

    Google Scholar 

  • D.M. Bolt, V.F. Lall, Estimation of compensatory and noncompensatory multidimensional item response models using Markov chain Monte Carlo. Appl. Psychol. Meas. 27(6), 395–414 (2003). doi:10.1177/0146621603258350

    Article  MathSciNet  Google Scholar 

  • E.T. Bradlow, H. Wainer, X.H. Wang, A Bayesian random effects model for testlets. Psychometrika 64(2), 153–168 (1999). doi:10.1007/BF02294533

    Article  MATH  Google Scholar 

  • D. Caughey, C. Warshaw, in Dynamic Representation in the American States, 1960–2012. American Political Science Association 2014 Annual Meeting Paper

    Google Scholar 

  • S. Chib, E. Greenberg, Understanding the Metropolis-Hastings algorithm. Am. Stat. 49(4), 327–335 (1995)

    Google Scholar 

  • M. Copelovitch, C. Gandrud, M. Hallerberg, in Financial Regulatory Transparency, International Institutions, and Borrowing Costs. The Political Economy of International Organizations Annual Conference, University of Utah, Salt Lake City, Utah, vol. 3, (2015), p. 2015, http://wp.peio.me/wp-content/uploads/PEIO8/ Copelovitch, Gandrud, Hallerberg

  • J. de la Torre, S. Stark, O.S. Chernyshenko, Markov chain Monte Carlo estimation of item parameters for the generalized graded unfolding model. Appl. Psychol. Meas. 30(3), 216–232 (2006). doi:10.1177/0146621605282772

    Article  MathSciNet  Google Scholar 

  • J.P. Fox, C.A. Glas, Bayesian estimation of a multilevel IRT model using Gibbs sampling. Psychometrika 66(2), 271–288 (2001). doi:10.1007/BF02294839

    Article  MathSciNet  MATH  Google Scholar 

  • A. Gelman, D.B. Rubin, Inference from iterative simulation using multiple sequences. Stat. Sci. 7, 457–472 (1992)

    Article  Google Scholar 

  • S. Geman, D. Geman, Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–741 (1984). doi:10.1109/TPAMI.1984.4767596

    Article  MATH  Google Scholar 

  • M. Ghosh, A. Ghosh, M.H. Chen, A. Agresti, Noninformative priors for one-parameter item response models. J. Stat. Plan. Inference 88(1), 99–115 (2000). doi:10.1016/S0378-3758(99)00201-3

    Article  MathSciNet  MATH  Google Scholar 

  • R.L. Grant, D.C. Furr, B. Carpenter, A. Gelman, Fitting Bayesian item response models in Stata and Stan. Preprint arXiv:1601.03443 (2016), https://arxiv.org/abs/1601.03443

  • M. Harwell, C.A. Stone, T.C. Hsu, L. Kirisci, Monte Carlo studies in item response theory. Appl. Psychol. Meas. 20(2), 101–125 (1996). doi:10.1177/014662169602000201

    Article  Google Scholar 

  • B.T. Hemker, K. Sijtsma, I.W. Molenaar, Selection of unidimensional scales from a multidimensional item bank in the polytomous Mokken IRT model. Appl. Psychol. Meas. 19(4), 337–352 (1995). doi:10.1177/014662169501900404

    Article  Google Scholar 

  • M.D. Hoffman, A. Gelman, The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. J. Mach. Learn. Res. 15(1), 1593–1623 (2014). https://arxiv.org/abs/1111.4246

    MathSciNet  MATH  Google Scholar 

  • M.S. Johnson, S. Sinharay, Calibration of polytomous item families using Bayesian hierarchical modeling. Appl. Psychol. Meas. 29(5), 369–400 (2005). doi:10.1177/0146621605276675

    Article  MathSciNet  Google Scholar 

  • D. Jurich, J. Goodman, in A Comparison of IRT Parameter Recovery in Mixed Format Examinations Using PARSCALE and ICL. Annual Meeting of Northeastern Educational Research Association: 21–23 October 2009

    Google Scholar 

  • F.M. Lord, Applications of Item Response Theory to Practical Testing Problems (L. Erlbaum Associates, Hillsdale, NJ, 1980), p. 1980

    Google Scholar 

  • N. Metropolis, A.W. Rosenbluth, M.N. Rosenbluth, A.H. Teller, E. Teller, Equation of state calculations by fast computing machines. J. Chem. Phys. 21(6), 1087–1092 (1953). doi:10.1063/1.1699114

    Article  Google Scholar 

  • R. Neal, Probabilistic Inference Using Markov Chain Monte Carlo Methods. Technical Report CRG-TR-93-1, Department of Computer Science, University of Toronto, Toronto, 1993

    Google Scholar 

  • R.J. Patz, B.W. Junker, Applications and extensions of MCMC in IRT: multiple item types, missing data, and rated responses. J. Educ. Behav. Stat. 24(4), 342–366 (1999). doi:10.3102/10769986024004342

    Article  Google Scholar 

  • M. Plummer, in JAGS: A Program for Analysis of Bayesian Graphical Models Using Gibbs Sampling. Proceedings of the 3rd International Workshop on Distributed Statistical Computing, vol. 124 (2003), p. 125. http://mcmc-jags.sourceforge.net

  • R Core Team, R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, Vienna, 2016.) https://www.R-project.org/

    Google Scholar 

  • G. Rasch, Probabilistic Models for Some Intelligence and Attainment Tests (University of Chicago Press, Chicago, IL, 1960)

    Google Scholar 

  • S.K. Sahu, Bayesian estimation and model choice in item response models. J. Stat. Comput. Simul. 72(3), 217–232 (2002). doi:10.1080/00949650212387

    Article  MathSciNet  MATH  Google Scholar 

  • Y. Sheng, A MATLAB package for Markov chain Monte Carlo with a multi-unidimensional IRT model. J. Stat. Softw. 28, 1–20 (2008)

    Article  Google Scholar 

  • Y. Sheng, A sensitivity analysis of Gibbs sampling for 3PNO IRT models: effects of prior specifications on parameter estimates. Behaviormetrika 37(2), 87–110 (2010). doi:10.2333/bhmk.37.87

    Article  MATH  Google Scholar 

  • Y. Sheng, C.K. Wikle, Comparing multiunidimensional and unidimensional item response theory models. Educ. Psychol. Meas. 67(6), 899–919 (2007). doi:10.1177/0013164406296977

    Article  MathSciNet  Google Scholar 

  • D. Spiegelhalter, A. Thomas, N. Best, D. Lunn, WinBUGS version 1.4 user manual. MRC Biostatistics Unit (2003), http://www.mrc-bsu.cam.ac.uk/bugs/

  • Stan Development Team, Stan modeling language users guide and reference manual, version 2.12.0 (2016), http://mc-stan.org/

  • M. Zimowski, E. Muraki, R.J. Mislevy, R.D. Bock, BILOG-MG 3: Item Analysis and Test Scoring with Binary Logistic Models [Computer Software] (Scientific Software, Chicago, IL, 2003)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Counseling, Quantitative Methods, and Special Education, Southern Illinois University, Carbondale, IL, 62901, USA

    Meng-I Chang & Yanyan Sheng

Authors
  1. Meng-I Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanyan Sheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meng-I Chang .

Editor information

Editors and Affiliations

  1. Research Institute for Child Development and Education, University of Amsterdam, Amsterdam, The Netherlands

    L. Andries van der Ark

  2. Department of Statistics, USBE, Umeå University, Umeå, Sweden

    Marie Wiberg

  3. Department of Statistics, University of Illinois at Urbana-Champaign, Champaign, Illinois, USA

    Steven A. Culpepper  & Jeffrey A. Douglas  & 

  4. Department of Psychology, The Educational University of Hong Kong, Hong Kong, China

    Wen-Chung Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Chang, MI., Sheng, Y. (2017). A Comparison of Two MCMC Algorithms for the 2PL IRT Model. In: van der Ark, L.A., Wiberg, M., Culpepper, S.A., Douglas, J.A., Wang, WC. (eds) Quantitative Psychology. IMPS 2016. Springer Proceedings in Mathematics & Statistics, vol 196. Springer, Cham. https://doi.org/10.1007/978-3-319-56294-0_7

Download citation

Publish with us

Policies and ethics

Search

Navigation