Advertisement

Psychometric Modeling of response speed and accuracy with mixed and conditional regression

  • 517 Accesses

  • 28 Citations

Abstract

Human performance in cognitive testing and experimental psychology is expressed in terms of response speed and accuracy. Data analysis is often limited to either speed or accuracy, and/or to crude summary measures like mean response time (RT) or the percentage correct responses. This paper proposes the use of mixed regression for the psychometric modeling of response speed and accuracy in testing and experiments. Mixed logistic regression of response accuracy extends logistic item response theory modeling to multidimensional models with covariates and interactions. Mixed linear regression of response time extends mixed ANOVA to unbalanced designs with covariates and heterogeneity of variance. Related to mixed regression is conditional regression, which requires no normality assumption, but is limited to unidimensional models. Mixed and conditional methods are both applied to an experimental study of mental rotation. Univariate and bivariate analyzes show how within-subject correlation between response and RT can be distinguished from between-subject correlation, and how latent traits can be detected, given careful item design or content analysis. It is concluded that both response and RT must be recorded in cognitive testing, and that mixed regression is a versatile method for analyzing test data.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 99

This is the net price. Taxes to be calculated in checkout.

References

  1. Bloxom B. (1985). Considerations in psychometric modeling of response time. Psychometrika, 50:383–397

  2. Cox D.R. (1972). Regression models and life tables (with discussion). Journal of the Royal Statistical Society B, 34:187–220

  3. Cox D.R., Oakes D. (1984). Analysis of Survival Data. London: Chapman and Hall

  4. Donders R. (1997). The Validity of Basic Assumptions Underlying Models for Time Limit Tests. PhD thesis, Nijmegen University, The Netherlands

  5. Fischer G.H. (1974). Einführung in die theorie psychologischer tests [Introduction to the theory of psychological tests]. Huber, Bern

  6. Gao S. (2004). A shared random effect parameter approach for longitudinal dementia data with non-ignorable missing data. Statistics in Medicine 23:211–219

  7. Goldstein H. (1995). Multilevel Statistical Models, (2nd ed). Edward Arnold, London

  8. Hambleton R.K., Swaminathan H. (1985). Item Response Theory: Principles and Applications. Kluwer Academic Publishers, Boston (MA)

  9. Hedeker D., Gibbons R.D. (1996a). MIXOR: a computer program for mixed-effects ordinal regression. Computer Methods and Programs in Biomedicine, 49:157–176

  10. Hedeker D., Gibbons R.D. (1996b). MIXREG: a computer program for mixed-effects regression with autocorrelated errors. Computer Methods and Programs in Biomedicine, 49:229–252

  11. Hosmer D.W., Lemeshow S. (1989). Applied Logistic Regression. Wiley, New York

  12. Kahane M., Loftus G. (1999). Response time versus accuracy in human memory. In: Sternberg R.J. (ed) The Nature of Cognition. MIT, Cambridge (MA), pp 323–384

  13. Lord F.M., Novick M.R. (1968). Statistical Theories of Mental Test Scores. Addison-Wesley, Reading (MA)

  14. Luce R.D. (1986). Response Times: Their Role in Inferring Elementary Mental Organization. Oxford University Press, New York

  15. Maris E. (1993). Additive and multiplicative models for gamma distributed variables, and their application as models for response times. Psychometrika 58:445–469

  16. Marley A.A.J., Colonius H. (1992). The “horse race” random utility model for choice probabilities and reaction times, and its competing risks interpretation. Journal of Mathematical Psychology 36:1–20

  17. Metzler J., Shepard R.N. (1974). Tranformational studies of the internal representation of three-dimensional objects. In: Solso R.L. (ed) Theories in Cognitive Psychology: The Loyola Symposium. Erlbaum, Potomac (MD), pp 147–201

  18. Moerbeek M., Van Breukelen G., Berger M. (2001). Optimal experimental design for multilevel logistic models. The Statistician 50:17–30

  19. Moerbeek M., Van Breukelen G., Berger M. (2003). A comparison of estimation methods for multilevel logistic models. Computational Statistics 18:19–38

  20. Pachella R.G. (1974). The interpretation of reaction time in information-processing research. In: Kantowitz B.H. (ed) Human Information Processing: Tutorials in Performance and Cognition. Erlbaum, Hillsdale (NJ), pp 41–82

  21. Rasbash J., Browne W., Goldstein H., Yang M., Plewis I,. Healy M,. Woodhouse G., Draper D., Langford I., Lewis T. (2000). A User’s Guide to MLwiN. Multilevel Models Project, Institute of Education, University of London, Version 2.1

  22. Ratcliff R. (1988). Continuous versus discrete information processing: modeling accumulation of partial information. Psychological Review 95:238–255

  23. Ratcliff R., Smith P.L. (2004). A comparison of sequential sampling models for two-choice reaction time. Psychological Review 111:333–367

  24. Rijmen F., DeBoeck P. (2002). The random weights linear logistic test model. Applied Psychological Measurement 26:271–285

  25. Shepard R.N., Metzler J. (1971). Mental rotation of three-dimensional objects. Science 171:701–703

  26. Snijders T.A.B., Bosker R.J. (1999). Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling. Sage Publications, London

  27. Sternberg S. (1969). The discovery of processing stages: extensions of Donders’ method. Acta Psychologica 30: 276–315

  28. Storms G., Delbeke L. (1992). The irrelevance of distributional assumptions on RTs in in multidimensional scaling of same/different tasks. Psychometrika 57:599–614

  29. Therneau T.M., Grambsch P.M. (2000) Modeling Survival Data: Extending the Cox Model. Springer, New York

  30. Thissen D. (1983). Timed testing: an approach using item response theory. In: Weiss D.J. (ed) New Horizons in Testing: Latent Trait Theory and Computerized Adaptive Testing. Academic Press, New York, pp 179–203

  31. Thurstone L.L. (1937). Ability, motivation and speed. Psychometrika 2:249–254

  32. Townsend J.T., Ashby F.G. (1983). The Stochastic Modeling of Elementary Psychological Processes.University Press, Cambridge

  33. Townsend J.T., Nozawa G. (1995). Spatio-temporal properties of elementary perception: an investigation of parallel, serial, and coactive theories. Journal of Mathematical psychology 39:321–359

  34. Ulrich R., Miller J (1993). Information processing models generating lognormally distributed reaction times. Journal of Mathematical Psychology 37:513–525

  35. Van Breukelen G.J.P. (1989). Concentration, Speed and Precision in Mental Tests: a Psychonometric Approach. PhD thesis, The Netherlands, Nijmegen University

  36. Van Breukelen G.J.P. (1995a). Psychometric and information processing properties of selected response time models. Psychometrika, 60:95–113

  37. Van Breukelen G.J.P. (1995b). Parallel processing models compatible with lognormally distributed response times. Journal of Mathematical Psychology 39:396–399

  38. Van Breukelen G.J.P. (1997). Separability of item and person parameters in response time models. Psychometrika, 62:525–544

  39. Van Breukelen G.J.P., Roskam E.E.Ch.I.,(1991). A Rasch model for the speed-accuracy tradeoff in time limit tests. In: Doignon J.P., Falmagne J.C., (eds.) Mathematical Psychology: Current Developments. Springer, New York, pp 251–271

  40. Van der Linden, W.J., Hambleton, R.K. (1997). Handbook of Modern Item Response Theory. New York: Springer.

  41. Van der Linden W.J., Scrams D.J., Schnipke D.L. (1999). Using response-time constraints to control for differential speededness in computerized adaptive testing. Applied Psychological Measurement, 23:195–210

  42. Verbeke G., Molenberghs G. (2000). Linear Mixed Models for Longitudinal Data. Springer, New York

  43. Verhelst N.D., Verstralen H.H.F.M., Jansen M.G.H. (1997). A logistic model for time limit tests. In: Van Der Linden W.J., Hambleton R.K. (eds) Handbook of Modern Item Response Theory Springer, New York, pp 169–186

  44. Vorberg D., Ulrich R. (1987). Random search with unequal rates: serial and parallel generalizations of McGill’s model. Journal of Mathematical Psychology 31:1–23

  45. Wenger M.J., Gibson B.S. (2004). Using hazard functions to assess changes in processing capacity in an attentional cueing paradigm. Journal of Experimental Psychology: Human Perception and Performance, 30:708–719

  46. Zwinderman A.H. (1991). A generalized Rasch model for manifest predictors. Psychometrika, 56:589–600

Download references

Author information

Correspondence to Gerard J. P. Van Breukelen.

Additional information

I am grateful to Rogier Donders for putting his data at my disposal.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Van Breukelen, G.J.P. Psychometric Modeling of response speed and accuracy with mixed and conditional regression. Psychometrika 70, 359–376 (2005) doi:10.1007/s11336-003-1078-0

Download citation

Keywords

  • time limit tests
  • conditional accuracy function
  • speed-accuracy tradeoff
  • conditional logistic regression
  • Cox regression
  • mixed regression
  • multilevel analysis
  • latent trait
  • mental rotation