Psychometrika

, Volume 70, Issue 2, pp 359–376 | Cite as

Psychometric Modeling of response speed and accuracy with mixed and conditional regression

Article

Abstract

Human performance in cognitive testing and experimental psychology is expressed in terms of response speed and accuracy. Data analysis is often limited to either speed or accuracy, and/or to crude summary measures like mean response time (RT) or the percentage correct responses. This paper proposes the use of mixed regression for the psychometric modeling of response speed and accuracy in testing and experiments. Mixed logistic regression of response accuracy extends logistic item response theory modeling to multidimensional models with covariates and interactions. Mixed linear regression of response time extends mixed ANOVA to unbalanced designs with covariates and heterogeneity of variance. Related to mixed regression is conditional regression, which requires no normality assumption, but is limited to unidimensional models. Mixed and conditional methods are both applied to an experimental study of mental rotation. Univariate and bivariate analyzes show how within-subject correlation between response and RT can be distinguished from between-subject correlation, and how latent traits can be detected, given careful item design or content analysis. It is concluded that both response and RT must be recorded in cognitive testing, and that mixed regression is a versatile method for analyzing test data.

Keywords

time limit tests conditional accuracy function speed-accuracy tradeoff conditional logistic regression Cox regression mixed regression multilevel analysis latent trait mental rotation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bloxom B. (1985). Considerations in psychometric modeling of response time. Psychometrika, 50:383–397Google Scholar
  2. Cox D.R. (1972). Regression models and life tables (with discussion). Journal of the Royal Statistical Society B, 34:187–220Google Scholar
  3. Cox D.R., Oakes D. (1984). Analysis of Survival Data. London: Chapman and HallGoogle Scholar
  4. Donders R. (1997). The Validity of Basic Assumptions Underlying Models for Time Limit Tests. PhD thesis, Nijmegen University, The NetherlandsGoogle Scholar
  5. Fischer G.H. (1974). Einführung in die theorie psychologischer tests [Introduction to the theory of psychological tests]. Huber, BernGoogle Scholar
  6. Gao S. (2004). A shared random effect parameter approach for longitudinal dementia data with non-ignorable missing data. Statistics in Medicine 23:211–219CrossRefPubMedGoogle Scholar
  7. Goldstein H. (1995). Multilevel Statistical Models, (2nd ed). Edward Arnold, LondonGoogle Scholar
  8. Hambleton R.K., Swaminathan H. (1985). Item Response Theory: Principles and Applications. Kluwer Academic Publishers, Boston (MA)Google Scholar
  9. Hedeker D., Gibbons R.D. (1996a). MIXOR: a computer program for mixed-effects ordinal regression. Computer Methods and Programs in Biomedicine, 49:157–176CrossRefPubMedGoogle Scholar
  10. Hedeker D., Gibbons R.D. (1996b). MIXREG: a computer program for mixed-effects regression with autocorrelated errors. Computer Methods and Programs in Biomedicine, 49:229–252CrossRefPubMedGoogle Scholar
  11. Hosmer D.W., Lemeshow S. (1989). Applied Logistic Regression. Wiley, New YorkGoogle Scholar
  12. Kahane M., Loftus G. (1999). Response time versus accuracy in human memory. In: Sternberg R.J. (ed) The Nature of Cognition. MIT, Cambridge (MA), pp 323–384Google Scholar
  13. Lord F.M., Novick M.R. (1968). Statistical Theories of Mental Test Scores. Addison-Wesley, Reading (MA)Google Scholar
  14. Luce R.D. (1986). Response Times: Their Role in Inferring Elementary Mental Organization. Oxford University Press, New YorkGoogle Scholar
  15. Maris E. (1993). Additive and multiplicative models for gamma distributed variables, and their application as models for response times. Psychometrika 58:445–469Google Scholar
  16. Marley A.A.J., Colonius H. (1992). The “horse race” random utility model for choice probabilities and reaction times, and its competing risks interpretation. Journal of Mathematical Psychology 36:1–20CrossRefGoogle Scholar
  17. Metzler J., Shepard R.N. (1974). Tranformational studies of the internal representation of three-dimensional objects. In: Solso R.L. (ed) Theories in Cognitive Psychology: The Loyola Symposium. Erlbaum, Potomac (MD), pp 147–201Google Scholar
  18. Moerbeek M., Van Breukelen G., Berger M. (2001). Optimal experimental design for multilevel logistic models. The Statistician 50:17–30MathSciNetGoogle Scholar
  19. Moerbeek M., Van Breukelen G., Berger M. (2003). A comparison of estimation methods for multilevel logistic models. Computational Statistics 18:19–38MathSciNetGoogle Scholar
  20. Pachella R.G. (1974). The interpretation of reaction time in information-processing research. In: Kantowitz B.H. (ed) Human Information Processing: Tutorials in Performance and Cognition. Erlbaum, Hillsdale (NJ), pp 41–82Google Scholar
  21. Rasbash J., Browne W., Goldstein H., Yang M., Plewis I,. Healy M,. Woodhouse G., Draper D., Langford I., Lewis T. (2000). A User’s Guide to MLwiN. Multilevel Models Project, Institute of Education, University of London, Version 2.1Google Scholar
  22. Ratcliff R. (1988). Continuous versus discrete information processing: modeling accumulation of partial information. Psychological Review 95:238–255CrossRefPubMedGoogle Scholar
  23. Ratcliff R., Smith P.L. (2004). A comparison of sequential sampling models for two-choice reaction time. Psychological Review 111:333–367CrossRefPubMedGoogle Scholar
  24. Rijmen F., DeBoeck P. (2002). The random weights linear logistic test model. Applied Psychological Measurement 26:271–285CrossRefMathSciNetGoogle Scholar
  25. Shepard R.N., Metzler J. (1971). Mental rotation of three-dimensional objects. Science 171:701–703PubMedGoogle Scholar
  26. Snijders T.A.B., Bosker R.J. (1999). Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling. Sage Publications, LondonGoogle Scholar
  27. Sternberg S. (1969). The discovery of processing stages: extensions of Donders’ method. Acta Psychologica 30: 276–315CrossRefGoogle Scholar
  28. Storms G., Delbeke L. (1992). The irrelevance of distributional assumptions on RTs in in multidimensional scaling of same/different tasks. Psychometrika 57:599–614Google Scholar
  29. Therneau T.M., Grambsch P.M. (2000) Modeling Survival Data: Extending the Cox Model. Springer, New YorkGoogle Scholar
  30. Thissen D. (1983). Timed testing: an approach using item response theory. In: Weiss D.J. (ed) New Horizons in Testing: Latent Trait Theory and Computerized Adaptive Testing. Academic Press, New York, pp 179–203Google Scholar
  31. Thurstone L.L. (1937). Ability, motivation and speed. Psychometrika 2:249–254Google Scholar
  32. Townsend J.T., Ashby F.G. (1983). The Stochastic Modeling of Elementary Psychological Processes.University Press, CambridgeGoogle Scholar
  33. Townsend J.T., Nozawa G. (1995). Spatio-temporal properties of elementary perception: an investigation of parallel, serial, and coactive theories. Journal of Mathematical psychology 39:321–359CrossRefGoogle Scholar
  34. Ulrich R., Miller J (1993). Information processing models generating lognormally distributed reaction times. Journal of Mathematical Psychology 37:513–525CrossRefGoogle Scholar
  35. Van Breukelen G.J.P. (1989). Concentration, Speed and Precision in Mental Tests: a Psychonometric Approach. PhD thesis, The Netherlands, Nijmegen UniversityGoogle Scholar
  36. Van Breukelen G.J.P. (1995a). Psychometric and information processing properties of selected response time models. Psychometrika, 60:95–113Google Scholar
  37. Van Breukelen G.J.P. (1995b). Parallel processing models compatible with lognormally distributed response times. Journal of Mathematical Psychology 39:396–399CrossRefGoogle Scholar
  38. Van Breukelen G.J.P. (1997). Separability of item and person parameters in response time models. Psychometrika, 62:525–544Google Scholar
  39. Van Breukelen G.J.P., Roskam E.E.Ch.I.,(1991). A Rasch model for the speed-accuracy tradeoff in time limit tests. In: Doignon J.P., Falmagne J.C., (eds.) Mathematical Psychology: Current Developments. Springer, New York, pp 251–271Google Scholar
  40. Van der Linden, W.J., Hambleton, R.K. (1997). Handbook of Modern Item Response Theory. New York: Springer.Google Scholar
  41. Van der Linden W.J., Scrams D.J., Schnipke D.L. (1999). Using response-time constraints to control for differential speededness in computerized adaptive testing. Applied Psychological Measurement, 23:195–210CrossRefGoogle Scholar
  42. Verbeke G., Molenberghs G. (2000). Linear Mixed Models for Longitudinal Data. Springer, New YorkGoogle Scholar
  43. Verhelst N.D., Verstralen H.H.F.M., Jansen M.G.H. (1997). A logistic model for time limit tests. In: Van Der Linden W.J., Hambleton R.K. (eds) Handbook of Modern Item Response Theory Springer, New York, pp 169–186Google Scholar
  44. Vorberg D., Ulrich R. (1987). Random search with unequal rates: serial and parallel generalizations of McGill’s model. Journal of Mathematical Psychology 31:1–23CrossRefMathSciNetGoogle Scholar
  45. Wenger M.J., Gibson B.S. (2004). Using hazard functions to assess changes in processing capacity in an attentional cueing paradigm. Journal of Experimental Psychology: Human Perception and Performance, 30:708–719CrossRefGoogle Scholar
  46. Zwinderman A.H. (1991). A generalized Rasch model for manifest predictors. Psychometrika, 56:589–600Google Scholar

Copyright information

© The Psychometric Society 2005

Authors and Affiliations

  1. 1.Department of Methodology and StatisticsMaastricht UniversityMaastrichtThe Netherlands

Personalised recommendations