Skip to main content

A Hierarchical Framework for Modeling Speed and Accuracy on Test Items


Current modeling of response times on test items has been strongly influenced by the paradigm of experimental reaction-time research in psychology. For instance, some of the models have a parameter structure that was chosen to represent a speed-accuracy tradeoff, while others equate speed directly with response time. Also, several response-time models seem to be unclear as to the level of parametrization they represent. A hierarchical framework for modeling speed and accuracy on test items is presented as an alternative to these models. The framework allows a “plug-and-play approach” with alternative choices of models for the response and response-time distributions as well as the distributions of their parameters. Bayesian treatment of the framework with Markov chain Monte Carlo (MCMC) computation facilitates the approach. Use of the framework is illustrated for the choice of a normal-ogive response model, a lognormal model for the response times, and multivariate normal models for their parameters with Gibbs sampling from the joint posterior distribution.

This is a preview of subscription content, access via your institution.


  • Albert, J.H. (1992). Bayesian estimation of normal-ogive item response curves using Gibbs sampling. Journal of Educational and Behavioral Statistics, 17, 261–269.

    Article  Google Scholar 

  • Béguin, A.A., & Glas, C.A.W. (2001). MCMC estimation and some fit analysis of multidimensional IRT models. Psychometrika, 66, 541–562.

    Article  Google Scholar 

  • Carlin, B.P., & Louis, T.A. (2000). Bayes and Empirical Bayes Methods for Data Analysis. Boca Raton, FL: Chapman & Hall.

  • Douglas, J., Kosorok, M., & Chewning, B. (1999). A latent variable model for multivariate psychometric response times. Psychometrika, 64, 69–82.

    Article  Google Scholar 

  • Dubey, S.D. (1969). A new derivation of the logistic distribution. Naval Research Logistics Quarterly, 16, 37–40.

    Google Scholar 

  • Fox, J.P., & Glas, C.A.W. (2001). Bayesian estimation of a multilevel IRT model using Gibbs sampling. Psychometrika, 66, 271–288.

    Article  Google Scholar 

  • Glas, C.A.W., & van der Linden, W.J. (2006). Modeling variability in item parameters in item response models. Psychometrika. Submitted.

  • Jansen, M.G.H. (1986). A Bayesian version of Rasch’s multiplicative Poisson model for the number of errors on achievement tests. Journal of Educational Statistics, 11, 51–65.

    Article  Google Scholar 

  • Jansen, M.G.H. (1997a). Rasch model for speed tests and some extensions with applications to incomplete designs. Journal of Educational and Behavioral Statistics, 22, 125–140.

    Google Scholar 

  • Jansen, M.G.H. (1997b). Rasch’s model for reading speed with manifest exploratory variables. Psychometrika, 62, 393–409.

    Article  Google Scholar 

  • Jansen, M.G.H., & Duijn, M.A.J. (1992). Extensions of Rasch’s multiplicative Poisson model. Psychometrika, 57, 405–414.

    Article  Google Scholar 

  • Johnson, V.E., & Albert, J.H. (1999). Ordinal Data Modeling. New York: Springer-Verlag.

    Google Scholar 

  • Luce, R.D. (1986). Response times: Their Roles in Inferring Elementary Mental Organization. Oxford, UK: Oxford University Press.

  • Maris, E. (1993). Additive and multiplicative models for gamma distributed variables, and their application as psychometric models for response times. Psychometrika, 58, 445–469.

    Article  Google Scholar 

  • Oosterloo, S.J. (1975). Modellen voor Reaktie-tijden [Models for Reaction Times]. Unpublished master’s thesis, Faculty of Psychology, University of Groningen, The Netherlands.

  • Patz, R.J., & Junker, B.W. (1999). Applications and extensions of MCMC in IRT: Mulitple item types, missing data, and rated responses. Journal of Educational and Behavioral Statistics, 34, 342–366.

    Google Scholar 

  • Rasch, G. (1980). Probabilistic models for some intelligence and attainment tests. Chicago: The University of Chicago Press. (Original published in 1960).

  • Roskam, E.E. (1987). Toward a psychometric theory of intelligence. In E.E. Roskam & R. Suck (Eds.), Progress in Mathematical Psychology (pp. 151–171). Amsterdam: North-Holland.

  • Roskam, E.E. (1997). Models for speed and time-limit tests. In W.J. van der Linden & R.K. Hambleton (Eds.), Handbook of Modern Item Response Theory (pp. 187–208). New York: Springer.

  • Rouder, J.N., Sun, D., Speckman, P.L., Lu, J., & Zhou, D. (2003). A hierarchical Bayesian statistical framework for response time distributions. Psychometrika, 68, 589–606.

    Article  Google Scholar 

  • Scheiblechner, H. (1979). Specific objective stochastic latency mechanisms. Journal of Mathematical Psychology, 19, 18–38.

    Article  Google Scholar 

  • Scheiblechner, H. (1985). Psychometric models for speed-test construction: The linear exponential model. In S.E. Embretson (Ed.), Test design: Developments in psychology and education (pp. 219–244). New York: Academic Press.

  • Schnipke, D.L., & Scrams, D.J. (1997). Modeling response times with a two-state mixture model: A new method of measuring speededness. Journal of Educational Measurement, 34, 213–232.

    Article  Google Scholar 

  • Schnipke, D.L., & Scrams, D.J. (1999). Representing response time information in item banks (LSAC Computerized Testing Report No. 97-09). Newtown, PA: Law School Admission Council.

  • Schnipke, D.L., & Scrams, D.J. (2002). Exploring issues of examinee behavior: Insights gained from response-time analyses. In C.N. Mills, M. Potenza, J.J. Fremer & W. Ward (Eds.), Computer-Based Testing: Building the Foundation for Future Assessments (pp. 237–266). Hillsdale, NJ: Lawrence Erlbaum Associates.

  • Swanson, D.B., Featherman, C.M., Case, S.M., Luecht, R.M., & Nungester, R. (1999, March). Relationship of response latency to test design, examinee proficiency and item difficulty in computer-based test administration. Paper presented at the Annual Meeting of the National Council on Measurement in Education, Chicago, IL.

  • Swanson, D.B., Case, S.E., Ripkey, D.R., Clauser, B.E., & Holtman, M.C. (2001). Relationships among item characteristics, examinee characteristics, and response times on USMLE, Step 1. Academic Medicine, 76, 114–116.

    Article  Google Scholar 

  • Tatsuoka, K.K., & Tatsuoka, M.M. (1980). A model for incorporating response-time data in scoring achievement tests. In D.J. Weiss (Ed.), Proceedings of the 1979 Computerized Adaptive Testing Conference (pp. 236–256). Minneapolis, MN: University of Minnesota, Department of Psychology, Psychometric Methods Program.

  • Thissen, D. (1983). Timed testing: An approach using item response theory. In D.J. Weiss (Ed.), New Horizons in Testing: Latent Trait Test Theory and Computerized Adaptive Testing. New York: Academic Press.

  • Townsend, J.T., & Ashby, F.G. (1983). Stochastic Modeling of Elementary Psychological Processes. Cambridge, UK: Cambridge University Press.

  • van Breukelen, G.J.P. (2005). Psychometric modeling of response speed and accuracy with mixed and conditional regression. Psychometrika, 70, 359–376.

    Article  Google Scholar 

  • van der Linden, W.J. (2005). Linear Models for Optimal Test Design. New York: Springer-Verlag.

    Google Scholar 

  • van der Linden, W.J. (2006). A lognormal model for response times on test items. Journal of Educational and Behavioral Statistics, 31, 181–204.

    Article  Google Scholar 

  • van der Linden, W.J. (2007a). Conceptual Issues in Response-Time Modeling. Submitted.

  • van der Linden, W.J. (2007b). Using response times for item selection in adaptive tests. Journal of Educational and Behavioral Statistics, 32.

  • van der Linden, W.J., & Guo, F. (2006). Two Bayesian Procedures for Identifying Aberrant Response-Time Patterns in Adaptive Testing. Manuscript submitted for publication.

  • van der Linden, W.J., & Hambleton, R.K. (1997). Handbook of Modern Item Response Theory. New York: Springer-Verlag.

    Google Scholar 

  • van der Linden, W.J., Breithaupt, K., Chuah, S.C., & Zhang, Y. (2007). Detecting differential speededness in multistage testing. Journal of Educational Measurement, 44, in press.

  • van der Linden, W.J., Klein Entink, R.H., & Fox, J.-P. (2006). IRT Parameter Estimation with Response Times as Collateral Information. Manuscript submitted for publication.

  • van der Linden, W.J., Scrams, D.J., & Schnipke, D.L. (1999). Using response-time constraints to control for speededness in computerized adaptive testing. Applied Psychological Measurement, 23, 195–210.

    Article  Google Scholar 

  • Verhelst, N.D., Verstralen, H.H.F.M., & Jansen, M.G. (1997). A logistic model for time-limit tests. In W.J. van der Linden & R.K. Hambleton (Eds.), Handbook of Modern Item Response Theory (pp. 169–185). New York: Springer-Verlag.

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Wim J. van der Linden.

Additional information

This study received funding from the Law School Admission Council (LSAC). The opinions and conclusions contained in this paper are those of the author and do not necessarily reflect the policy and position of LSAC. The author is indebted to the American Institute of Certified Public Accountants for the data set in the empirical example and to Rinke H. Klein Entink for his computational assistance

Rights and permissions

Reprints and Permissions

About this article

Cite this article

van der Linden, W.J. A Hierarchical Framework for Modeling Speed and Accuracy on Test Items. Psychometrika 72, 287–308 (2007).

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI:

Key words