Skip to main content
SpringerLink
Log in

Psychometric Modeling of response speed and accuracy with mixed and conditional regression

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

Human performance in cognitive testing and experimental psychology is expressed in terms of response speed and accuracy. Data analysis is often limited to either speed or accuracy, and/or to crude summary measures like mean response time (RT) or the percentage correct responses. This paper proposes the use of mixed regression for the psychometric modeling of response speed and accuracy in testing and experiments. Mixed logistic regression of response accuracy extends logistic item response theory modeling to multidimensional models with covariates and interactions. Mixed linear regression of response time extends mixed ANOVA to unbalanced designs with covariates and heterogeneity of variance. Related to mixed regression is conditional regression, which requires no normality assumption, but is limited to unidimensional models. Mixed and conditional methods are both applied to an experimental study of mental rotation. Univariate and bivariate analyzes show how within-subject correlation between response and RT can be distinguished from between-subject correlation, and how latent traits can be detected, given careful item design or content analysis. It is concluded that both response and RT must be recorded in cognitive testing, and that mixed regression is a versatile method for analyzing test data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Bloxom B. (1985). Considerations in psychometric modeling of response time. Psychometrika, 50:383–397

    Google Scholar 

  • Cox D.R. (1972). Regression models and life tables (with discussion). Journal of the Royal Statistical Society B, 34:187–220

    Google Scholar 

  • Cox D.R., Oakes D. (1984). Analysis of Survival Data. London: Chapman and Hall

    Google Scholar 

  • Donders R. (1997). The Validity of Basic Assumptions Underlying Models for Time Limit Tests. PhD thesis, Nijmegen University, The Netherlands

  • Fischer G.H. (1974). Einführung in die theorie psychologischer tests [Introduction to the theory of psychological tests]. Huber, Bern

    Google Scholar 

  • Gao S. (2004). A shared random effect parameter approach for longitudinal dementia data with non-ignorable missing data. Statistics in Medicine 23:211–219

    Article  PubMed  Google Scholar 

  • Goldstein H. (1995). Multilevel Statistical Models, (2nd ed). Edward Arnold, London

    Google Scholar 

  • Hambleton R.K., Swaminathan H. (1985). Item Response Theory: Principles and Applications. Kluwer Academic Publishers, Boston (MA)

    Google Scholar 

  • Hedeker D., Gibbons R.D. (1996a). MIXOR: a computer program for mixed-effects ordinal regression. Computer Methods and Programs in Biomedicine, 49:157–176

    Article  PubMed  Google Scholar 

  • Hedeker D., Gibbons R.D. (1996b). MIXREG: a computer program for mixed-effects regression with autocorrelated errors. Computer Methods and Programs in Biomedicine, 49:229–252

    Article  PubMed  Google Scholar 

  • Hosmer D.W., Lemeshow S. (1989). Applied Logistic Regression. Wiley, New York

    Google Scholar 

  • Kahane M., Loftus G. (1999). Response time versus accuracy in human memory. In: Sternberg R.J. (ed) The Nature of Cognition. MIT, Cambridge (MA), pp 323–384

    Google Scholar 

  • Lord F.M., Novick M.R. (1968). Statistical Theories of Mental Test Scores. Addison-Wesley, Reading (MA)

    Google Scholar 

  • Luce R.D. (1986). Response Times: Their Role in Inferring Elementary Mental Organization. Oxford University Press, New York

    Google Scholar 

  • Maris E. (1993). Additive and multiplicative models for gamma distributed variables, and their application as models for response times. Psychometrika 58:445–469

    Google Scholar 

  • Marley A.A.J., Colonius H. (1992). The “horse race” random utility model for choice probabilities and reaction times, and its competing risks interpretation. Journal of Mathematical Psychology 36:1–20

    Article  Google Scholar 

  • Metzler J., Shepard R.N. (1974). Tranformational studies of the internal representation of three-dimensional objects. In: Solso R.L. (ed) Theories in Cognitive Psychology: The Loyola Symposium. Erlbaum, Potomac (MD), pp 147–201

    Google Scholar 

  • Moerbeek M., Van Breukelen G., Berger M. (2001). Optimal experimental design for multilevel logistic models. The Statistician 50:17–30

    MathSciNet  Google Scholar 

  • Moerbeek M., Van Breukelen G., Berger M. (2003). A comparison of estimation methods for multilevel logistic models. Computational Statistics 18:19–38

    MathSciNet  Google Scholar 

  • Pachella R.G. (1974). The interpretation of reaction time in information-processing research. In: Kantowitz B.H. (ed) Human Information Processing: Tutorials in Performance and Cognition. Erlbaum, Hillsdale (NJ), pp 41–82

    Google Scholar 

  • Rasbash J., Browne W., Goldstein H., Yang M., Plewis I,. Healy M,. Woodhouse G., Draper D., Langford I., Lewis T. (2000). A User’s Guide to MLwiN. Multilevel Models Project, Institute of Education, University of London, Version 2.1

  • Ratcliff R. (1988). Continuous versus discrete information processing: modeling accumulation of partial information. Psychological Review 95:238–255

    Article  PubMed  Google Scholar 

  • Ratcliff R., Smith P.L. (2004). A comparison of sequential sampling models for two-choice reaction time. Psychological Review 111:333–367

    Article  PubMed  Google Scholar 

  • Rijmen F., DeBoeck P. (2002). The random weights linear logistic test model. Applied Psychological Measurement 26:271–285

    Article  MathSciNet  Google Scholar 

  • Shepard R.N., Metzler J. (1971). Mental rotation of three-dimensional objects. Science 171:701–703

    PubMed  Google Scholar 

  • Snijders T.A.B., Bosker R.J. (1999). Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling. Sage Publications, London

    Google Scholar 

  • Sternberg S. (1969). The discovery of processing stages: extensions of Donders’ method. Acta Psychologica 30: 276–315

    Article  Google Scholar 

  • Storms G., Delbeke L. (1992). The irrelevance of distributional assumptions on RTs in in multidimensional scaling of same/different tasks. Psychometrika 57:599–614

    Google Scholar 

  • Therneau T.M., Grambsch P.M. (2000) Modeling Survival Data: Extending the Cox Model. Springer, New York

    Google Scholar 

  • Thissen D. (1983). Timed testing: an approach using item response theory. In: Weiss D.J. (ed) New Horizons in Testing: Latent Trait Theory and Computerized Adaptive Testing. Academic Press, New York, pp 179–203

    Google Scholar 

  • Thurstone L.L. (1937). Ability, motivation and speed. Psychometrika 2:249–254

    Google Scholar 

  • Townsend J.T., Ashby F.G. (1983). The Stochastic Modeling of Elementary Psychological Processes.University Press, Cambridge

    Google Scholar 

  • Townsend J.T., Nozawa G. (1995). Spatio-temporal properties of elementary perception: an investigation of parallel, serial, and coactive theories. Journal of Mathematical psychology 39:321–359

    Article  Google Scholar 

  • Ulrich R., Miller J (1993). Information processing models generating lognormally distributed reaction times. Journal of Mathematical Psychology 37:513–525

    Article  Google Scholar 

  • Van Breukelen G.J.P. (1989). Concentration, Speed and Precision in Mental Tests: a Psychonometric Approach. PhD thesis, The Netherlands, Nijmegen University

  • Van Breukelen G.J.P. (1995a). Psychometric and information processing properties of selected response time models. Psychometrika, 60:95–113

    Google Scholar 

  • Van Breukelen G.J.P. (1995b). Parallel processing models compatible with lognormally distributed response times. Journal of Mathematical Psychology 39:396–399

    Article  Google Scholar 

  • Van Breukelen G.J.P. (1997). Separability of item and person parameters in response time models. Psychometrika, 62:525–544

    Google Scholar 

  • Van Breukelen G.J.P., Roskam E.E.Ch.I.,(1991). A Rasch model for the speed-accuracy tradeoff in time limit tests. In: Doignon J.P., Falmagne J.C., (eds.) Mathematical Psychology: Current Developments. Springer, New York, pp 251–271

    Google Scholar 

  • Van der Linden, W.J., Hambleton, R.K. (1997). Handbook of Modern Item Response Theory. New York: Springer.

    Google Scholar 

  • Van der Linden W.J., Scrams D.J., Schnipke D.L. (1999). Using response-time constraints to control for differential speededness in computerized adaptive testing. Applied Psychological Measurement, 23:195–210

    Article  Google Scholar 

  • Verbeke G., Molenberghs G. (2000). Linear Mixed Models for Longitudinal Data. Springer, New York

    Google Scholar 

  • Verhelst N.D., Verstralen H.H.F.M., Jansen M.G.H. (1997). A logistic model for time limit tests. In: Van Der Linden W.J., Hambleton R.K. (eds) Handbook of Modern Item Response Theory Springer, New York, pp 169–186

    Google Scholar 

  • Vorberg D., Ulrich R. (1987). Random search with unequal rates: serial and parallel generalizations of McGill’s model. Journal of Mathematical Psychology 31:1–23

    Article  MathSciNet  Google Scholar 

  • Wenger M.J., Gibson B.S. (2004). Using hazard functions to assess changes in processing capacity in an attentional cueing paradigm. Journal of Experimental Psychology: Human Perception and Performance, 30:708–719

    Article  Google Scholar 

  • Zwinderman A.H. (1991). A generalized Rasch model for manifest predictors. Psychometrika, 56:589–600

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Methodology and Statistics, Maastricht University, P.O. Box 616, 6200, MD, Maastricht, The Netherlands

    Gerard J. P. Van Breukelen

Authors
  1. Gerard J. P. Van Breukelen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gerard J. P. Van Breukelen.

Additional information

I am grateful to Rogier Donders for putting his data at my disposal.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Van Breukelen, G.J.P. Psychometric Modeling of response speed and accuracy with mixed and conditional regression. Psychometrika 70, 359–376 (2005). https://doi.org/10.1007/s11336-003-1078-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11336-003-1078-0

Keywords

Search

Navigation