Model Selection for Monotonic Polynomial Item Response Models

  • Carl F. FalkEmail author
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 265)


One flexible approach for item response modeling involves use of a monotonic polynomial in place of the linear predictor for commonly used parametric item response models. Since polynomial order may vary across items, model selection can be difficult. For polynomial orders greater than one, the number of possible order combinations increases exponentially with test length. I reframe this issue as a combinatorial optimization problem and apply an algorithm known as simulated annealing to aid in finding a suitable model. Simulated annealing resembles Metropolis-Hastings: A random perturbation of polynomial order for some item is generated and acceptance depends on the change in model fit and the current algorithm state. Simulations suggest that this approach is often a feasible way to select a better fitting model.


Combinatorial optimization Nonparametric item response theory Monotonic polynomial Balanced incomplete block design 



We acknowledge the support of the Natural Sciences and Engineering Research Council of Canada (NSERC), [funding reference number RGPIN-2018-05357]. Cette recherche a été financée par le Conseil de recherches en sciences naturelles et en génie du Canada (CRSNG), [numéro de référence RGPIN-2018-05357].


  1. Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability (pp. 395–479). Reading, MA: Addison-Wesley.Google Scholar
  2. Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443–459.MathSciNetCrossRefGoogle Scholar
  3. Černý, V. (1985). Thermodynamical approach to the traveling salesman problem: An efficient simulation algorithm. Journal of Optimization Theory and Applications, 45, 41–51.MathSciNetCrossRefGoogle Scholar
  4. Drezner, Z., & Marcoulides, G. A. (1999). Using simulated annealing for selection in multiple regression. Multiple Linear Regression Viewpoints, 25, 1–4.Google Scholar
  5. Duncan, K. A., & MacEachern, S. N. (2013). Nonparametric Bayesian modeling of item response curves with a three-parameter logistic prior mean. In M. C. Edwards & R. C. MacCallum (Eds.), Current topics in the theory and application of latent variable models (pp. 108–125). New York, NY: Routledge.Google Scholar
  6. Edwards, M. C., Flora, D. B., & Thissen, D. (2012). Multistage computerized adaptive testing with uniform item exposure. Applied Measurement in Education, 25, 118–114.CrossRefGoogle Scholar
  7. Falk, C. F. (2018). A monotonic polynomial graded response model. Presentation at the International Test Commission Conference, Montreal, Canada.Google Scholar
  8. Falk, C. F., & Cai, L. (2016a). Maximum marginal likelihood estimation of a monotonic polynomial generalized partial credit model with applications to multiple group analysis. Psychometrika, 81, 434–460.MathSciNetCrossRefGoogle Scholar
  9. Falk, C. F., & Cai, L. (2016b). Semi-parametric item response functions in the context of guessing. Journal of Educational Measurement, 53, 229–247.CrossRefGoogle Scholar
  10. Feuerstahler, L. (2016). Exploring alternate latent trait metrics with the filtered monotonic polynomial IRT model. Ph.D. thesis, Department of Psychology, University of Minnesota.Google Scholar
  11. Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220, 671–680.MathSciNetCrossRefGoogle Scholar
  12. Lee, S., & Bolt, D. M. (2018). Asymmetric item characteristic curves and item complexity: Insights from simulation and real data analyses. Psychometrika, 83, 453–475.MathSciNetCrossRefGoogle Scholar
  13. Lee, Y. S. (2007). A comparison of methods for nonparametric estimation of item characteristic curves for binary items. Applied Psychological Measurement, 31, 121–134.MathSciNetCrossRefGoogle Scholar
  14. Liang, L., & Browne, M. W. (2015). A quasi-parametric method for fitting flexible item response functions. Journal of Educational and Behavioral Statistics, 40, 5–34.CrossRefGoogle Scholar
  15. Meijer, R. R., & Baneke, J. J. (2004). Analyzing psychopathology items: A case for nonparametric item response theory modeling. Psychological Methods, 9(3), 354–368.CrossRefGoogle Scholar
  16. Molenaar, D. (2015). Heteroscedastic latent trait models for dichotomous data. Psychometrika, 80(3), 625–644.MathSciNetCrossRefGoogle Scholar
  17. Neale, M. C., Hunter, M. D., Pritikin, J. N., Zahery, M., Brick, T. R., Kickpatrick, R. M., et al. (2016). OpenMx 2.0: Extended structural equation and statistical modeling. Psychometrika, 81, 535–549.MathSciNetCrossRefGoogle Scholar
  18. Orlando, M., & Thissen, D. (2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24, 50–64.CrossRefGoogle Scholar
  19. Pritikin, J. N. (2016). rpf: Response probability functions., r package version 0.53.
  20. R Core Team. (2015). R: A language and environment for statistical computing., ISBN 3-900051-07-0.
  21. Ramsay, J. O. (1991). Kernel smoothing approaches to nonparametric item characteristic curve estimation. Psychometrika, 56(4), 611–630.CrossRefGoogle Scholar
  22. Ramsay, J. O., & Wiberg, M. (2017). A strategy for replacing sum scoring. Journal of Educational and Behavioral Statistics, 42(3), 282–307.CrossRefGoogle Scholar
  23. Stander, J., & Silverman, B. W. (1994). Temperature schedules for simulated annealing. Statistics and Computing, 4, 21–32.CrossRefGoogle Scholar
  24. Woods, C. M. (2007). Empirical histograms in item response theory with ordinal data. Educational and Psychological Measurement, 67, 73–87.MathSciNetCrossRefGoogle Scholar
  25. Xu, X., & Douglas, J. A. (2006). Computerized adaptive testing under nonparametric IRT models. Psychometrika, 71(1), 121–137.MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of PsychologyMcGill College, McGill UniversityMontrealCanada

Personalised recommendations