Advertisement

Speed–accuracy manipulations and diffusion modeling: Lack of discriminant validity of the manipulation or of the parameter estimates?

  • Veronika Lerche
  • Andreas Voss
Article

Abstract

The diffusion model (Ratcliff, 1978) is a mathematical model theorized to untangle different cognitive processes involved in binary decision tasks. To test the validity of the diffusion model parameters, several experimental validation studies have been conducted. In these studies, the validity of the threshold separation parameter was tested with speed–accuracy manipulations. Typically, this manipulation not only results in the expected effect on the threshold separation parameter but it also impacts nondecision time: Nondecision time is longer in the accuracy than in the speed condition. There are two possible interpretations of the finding: On the one hand, it could indicate that speed versus accuracy instructions really have an impact on the duration of extradecisional processes. On the other hand, the effect on the measure for nondecision time could be spurious—that is, based on a problem in the parameter estimation procedures. In simulation studies—with the parameter sets based on typical values from experimental validation studies—we checked for possible biases in the parameter estimation. Our analyses strongly suggest that the observed pattern (i.e., slower nondecision processes under accuracy instructions) is attributable to a lack of discriminant validity of the manipulation rather than to trade-offs in the parameter estimations.

Keywords

Diffusion model Mathematical models Reaction time methods Fast-dm 

References

  1. Arnold, N. R., Bröder, A., & Bayen, U. J. (2015). Empirical validation of the diffusion model for recognition memory and a comparison of parameter-estimation methods. Psychological Research, 79, 882–898.  https://doi.org/10.1007/s00426-014-0608-y CrossRefPubMedGoogle Scholar
  2. Brown, S. D., & Heathcote, A. (2008). The simplest complete model of choice response time: Linear ballistic accumulation. Cognitive Psychology, 57, 153–178.  https://doi.org/10.1016/j.cogpsych.2007.12.002 CrossRefPubMedGoogle Scholar
  3. Bruyer, R., & Brysbaert, M. (2011). Combining speed and accuracy in cognitive psychology: Is the inverse efficiency score (IES) a better dependent variable than the mean reaction time (RT) and the percentage of errors (PE)? Psychologica Belgica, 51, 5–13.  https://doi.org/10.5334/pb-51-1-5 CrossRefGoogle Scholar
  4. Donkin, C., Brown, S., Heathcote, A., & Wagenmakers, E.-J. (2011). Diffusion versus linear ballistic accumulation: Different models but the same conclusions about psychological processes? Psychonomic Bulletin & Review, 18, 61–69.  https://doi.org/10.3758/s13423-010-0022-4 CrossRefGoogle Scholar
  5. Dutilh, G., Annis, J., Brown, S. D., Cassey, P., Evans, N. J., Grasman, R. P. P. P., … Donkin, C. (2018). The quality of response time data inference: A blinded, collaborative assessment of the validity of cognitive models. Psychonomic Bulletin & Review.  https://doi.org/10.3758/s13423-017-1417-2
  6. Dutilh, G., Vandekerckhove, J., Tuerlinckx, F., & Wagenmakers, E.-J. (2009). A diffusion model decomposition of the practice effect. Psychonomic Bulletin & Review, 16, 1026–1036.  https://doi.org/10.3758/16.6.1026 CrossRefGoogle Scholar
  7. Forstmann, B. U., Tittgemeyer, M., Wagenmakers, E.-J., Derrfuss, J., Imperati, D., & Brown, S. (2011). The speed–accuracy tradeoff in the elderly brain: A structural model-based approach. Journal of Neuroscience, 31, 17242–17249.  https://doi.org/10.1523/JNEUROSCI.0309-11.2011 CrossRefPubMedGoogle Scholar
  8. Germar, M., Schlemmer, A., Krug, K., Voss, A., & Mojzisch, A. (2014). Social influence and perceptual decision making: A diffusion model analysis. Personality and Social Psychology Bulletin, 40, 217–231.  https://doi.org/10.1177/0146167213508985 CrossRefPubMedGoogle Scholar
  9. Grasman, R. P. P. P., Wagenmakers, E.-J., & van der Maas, H. L. J. (2009). On the mean and variance of response times under the diffusion model with an application to parameter estimation. Journal of Mathematical Psychology, 53, 55–68.  https://doi.org/10.1016/j.jmp.2009.01.006 CrossRefGoogle Scholar
  10. Heitz, R. P. (2014). The speed–accuracy tradeoff: History, physiology, methodology, and behavior. Frontiers in Neuroscience, 8, 150.  https://doi.org/10.3389/fnins.2014.00150 CrossRefPubMedPubMedCentralGoogle Scholar
  11. Ho, T., Brown, S., van Maanen, L., Forstmann, B. U., Wagenmakers, E.-J., & Serences, J. T. (2012). The optimality of sensory processing during the speed–accuracy tradeoff. Journal of Neuroscience, 32, 7992–8003.  https://doi.org/10.1523/JNEUROSCI.0340-12.2012 CrossRefPubMedPubMedCentralGoogle Scholar
  12. Hughes, M. M., Linck, J. A., Bowles, A. R., Koeth, J. T., & Bunting, M. F. (2014). Alternatives to switch-cost scoring in the task-switching paradigm: Their reliability and increased validity. Behavior Research Methods, 46, 702–721.CrossRefPubMedGoogle Scholar
  13. Lerche, V., & Voss, A. (2016). Model complexity in diffusion modeling: Benefits of making the model more parsimonious. Frontiers in Psychology, 7.  https://doi.org/10.3389/fpsyg.2016.01324
  14. Lerche, V., & Voss, A. (2017a). Experimental validation of the diffusion model based on a slow response time paradigm. Psychological Research.  https://doi.org/10.1007/s00426-017-0945-8
  15. Lerche, V., & Voss, A. (2017b). Retest reliability of the parameters of the Ratcliff diffusion model. Psychological Research, 81, 629–652.  https://doi.org/10.1007/s00426-016-0770-5 CrossRefPubMedGoogle Scholar
  16. Lerche, V., Voss, A., & Nagler, M. (2017). How many trials are required for parameter estimation in diffusion modeling? A comparison of different optimization criteria. Behavior Research Methods, 49, 513–537.  https://doi.org/10.3758/s13428-016-0740-2 CrossRefPubMedGoogle Scholar
  17. Mulder, M. J., Bos, D., Weusten, J. M. H., van Belle, J., van Dijk, S. C., Simen, P., … Durston, S. (2010). Basic impairments in regulating the speed–accuracy tradeoff predict symptoms of attention-deficit/hyperactivity disorder. Biological Psychiatry, 68, 1114–1119.  https://doi.org/10.1016/j.biopsych.2010.07.031
  18. Näätänen, R. (1972). Time uncertainty and occurrence uncertainty of the stimulus in a simple reaction time task. Acta Psychologica, 36, 492–503.  https://doi.org/10.1016/0001-6918(72)90029-7 CrossRefGoogle Scholar
  19. Naefgen, C., Dambacher, M., & Janczyk, M. (2017). Why free choices take longer than forced choices: Evidence from response threshold manipulations. Psychological Research.  https://doi.org/10.1007/s00426-017-0887-1
  20. Osman, A., Lou, L., Muller-Gethmann, H., Rinkenauer, G., Mattes, S., & Ulrich, R. (2000). Mechanisms of speed–accuracy tradeoff: Evidence from covert motor processes. Biological Psychology, 51, 173–199.  https://doi.org/10.1016/S0301-0511(99)00045-9 CrossRefPubMedGoogle Scholar
  21. Rae, B., Heathcote, A., Donkin, C., Averell, L., & Brown, S. (2014). The hare and the tortoise: Emphasizing speed can change the evidence used to make decisions. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40, 1226–1243.  https://doi.org/10.1037/a0036801 PubMedGoogle Scholar
  22. Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85, 59–108.  https://doi.org/10.1037/0033-295x.85.2.59 CrossRefGoogle Scholar
  23. Ratcliff, R. (2014). Measuring psychometric functions with the diffusion model. Journal of Experimental Psychology: Human Perception and Performance, 40, 870–888.  https://doi.org/10.1037/a0034954 PubMedPubMedCentralGoogle Scholar
  24. Ratcliff, R., & Childers, R. (2015). Individual differences and fitting methods for the two-choice diffusion model of decision making. Decision, 2, 237–279.  https://doi.org/10.1037/dec0000030 CrossRefGoogle Scholar
  25. Ratcliff, R., & Rouder, J. N. (1998). Modeling response times for two-choice decisions. Psychological Science, 9, 347–356.  https://doi.org/10.1111/1467-9280.00067 CrossRefGoogle Scholar
  26. Ratcliff, R., Thapar, A., & McKoon, G. (2001). The effects of aging on reaction time in a signal detection task. Psychology and Aging, 16, 323.  https://doi.org/10.1037/0882-7974.16.2.323 CrossRefPubMedGoogle Scholar
  27. Ratcliff, R., Thapar, A., & McKoon, G. (2003). A diffusion model analysis of the effects of aging on brightness discrimination. Perception & Psychophysics, 65, 523–535.  https://doi.org/10.3758/BF03194580 CrossRefGoogle Scholar
  28. Ratcliff, R., Thapar, A., & McKoon, G. (2004). A diffusion model analysis of the effects of aging on recognition memory. Journal of Memory and Language, 50, 408–424.  https://doi.org/10.1016/j.jml.2003.11.002 CrossRefGoogle Scholar
  29. Ratcliff, R., Thapar, A., & McKoon, G. (2010). Individual differences, aging, and IQ in two-choice tasks. Cognitive Psychology, 60, 127–157.  https://doi.org/10.1016/j.cogpsych.2009.09.001 CrossRefPubMedGoogle Scholar
  30. Ratcliff, R., & Tuerlinckx, F. (2002). Estimating parameters of the diffusion model: Approaches to dealing with contaminant reaction times and parameter variability. Psychonomic Bulletin & Review, 9, 438–481.  https://doi.org/10.3758/bf03196302 CrossRefGoogle Scholar
  31. Rinkenauer, G., Osman, A., Ulrich, R., Müller-Gethmann, H., & Mattes, S. (2004). On the Locus of Speed–accuracy Trade-Off in Reaction Time: Inferences From the Lateralized Readiness Potential. Journal of Experimental Psychology. General, 133, 261–282.  https://doi.org/10.1037/0096-3445.133.2.261 Google Scholar
  32. Schmiedek, F., Oberauer, K., Wilhelm, O., Süß, H.-M., & Wittmann, W. W. (2007). Individual differences in components of reaction time distributions and their relations to working memory and intelligence. Journal of Experimental Psychology: General, 136, 414–429.  https://doi.org/10.1037/0096-3445.136.3.414 CrossRefGoogle Scholar
  33. Schubert, A.-L., Hagemann, D., Voss, A., Schankin, A., & Bergmann, K. (2015). Decomposing the relationship between mental speed and mental abilities. Intelligence, 51, 28–46.  https://doi.org/10.1016/j.intell.2015.05.002 CrossRefGoogle Scholar
  34. Schulz-Zhecheva, Y., Voelkle, M., Beauducel, A., Biscaldi, M., & Klein, C. (2016). Predicting fluid intelligence by components of reaction time distributions from simple choice reaction time tasks. Journal of Intelligence, 4, 8.  https://doi.org/10.3390/jintelligence4030008 CrossRefGoogle Scholar
  35. Seibold, V. C., Bausenhart, K. M., Rolke, B., & Ulrich, R. (2011). Does temporal preparation increase the rate of sensory information accumulation? Acta Psychologica, 137, 56–64.  https://doi.org/10.1016/j.actpsy.2011.02.006 CrossRefPubMedGoogle Scholar
  36. Spaniol, J., Voss, A., & Grady, C. L. (2008). Aging and emotional memory: Cognitive mechanisms underlying the positivity effect. Psychology and Aging, 23, 859–872.  https://doi.org/10.1037/a0014218 CrossRefPubMedGoogle Scholar
  37. Starns, J. J., Ratcliff, R., & McKoon, G. (2012). Evaluating the unequal-variance and dual-process explanations of zROC slopes with response time data and the diffusion model. Cognitive Psychology, 64, 1–34.  https://doi.org/10.1016/j.cogpsych.2011.10.002 CrossRefPubMedGoogle Scholar
  38. Thapar, A., Ratcliff, R., & McKoon, G. (2003). A diffusion model analysis of the effects of aging on letter discrimination. Psychology and Aging, 18, 415–429.  https://doi.org/10.1037/0882-7974.18.3.415 CrossRefPubMedPubMedCentralGoogle Scholar
  39. van der Lubbe, R. H. J., Jaśkowski, P., Wauschkuhn, B., & Verleger, R. (2001). Influence of time pressure in a simple response task, a choice-by-location task, and the Simon task. Journal of Psychophysiology, 15, 241–255.  https://doi.org/10.1027/0269-8803.15.4.241 CrossRefGoogle Scholar
  40. van Ravenzwaaij, D., & Oberauer, K. (2009). How to use the diffusion model: Parameter recovery of three methods: Ez, fast-dm, and DMAT. Journal of Mathematical Psychology, 53, 463–473.  https://doi.org/10.1016/j.jmp.2009.09.004 CrossRefGoogle Scholar
  41. Vandekerckhove, J., & Tuerlinckx, F. (2007). Fitting the Ratcliff diffusion model to experimental data. Psychonomic Bulletin & Review, 14, 1011–1026.  https://doi.org/10.3758/bf03193087 CrossRefGoogle Scholar
  42. Vandekerckhove, J., & Tuerlinckx, F. (2008). Diffusion model analysis with MATLAB: A DMAT primer. Behavior Research Methods, 40, 61–72.  https://doi.org/10.3758/brm.40.1.61 CrossRefPubMedGoogle Scholar
  43. Vandierendonck, A. (2017). A comparison of methods to combine speed and accuracy measures of performance: A rejoinder on the binning procedure. Behavior Research Methods, 49, 653–673.  https://doi.org/10.3758/s13428-016-0721-5 CrossRefPubMedGoogle Scholar
  44. Voss, A., Rothermund, K., & Brandtstädter, J. (2008). Interpreting ambiguous stimuli: Separating perceptual and judgmental biases. Journal of Experimental Social Psychology, 44, 1048–1056.  https://doi.org/10.1016/j.jesp.2007.10.009 CrossRefGoogle Scholar
  45. Voss, A., Rothermund, K., & Voss, J. (2004). Interpreting the parameters of the diffusion model: An empirical validation. Memory & Cognition, 32, 1206–1220.  https://doi.org/10.3758/BF03196893 CrossRefGoogle Scholar
  46. Voss, A., & Voss, J. (2007). Fast-dm: A free program for efficient diffusion model analysis. Behavior Research Methods, 39, 767–775.  https://doi.org/10.3758/bf03192967 CrossRefPubMedGoogle Scholar
  47. Voss, A., & Voss, J. (2008). A fast numerical algorithm for the estimation of diffusion model parameters. Journal of Mathematical Psychology, 52, 1–9.  https://doi.org/10.1016/j.jmp.2007.09.005
  48. Voss, A., Voss, J., & Lerche, V. (2015). Assessing cognitive processes with diffusion model analyses: A tutorial based on fast-dm-30. Frontiers in Psychology, 6.  https://doi.org/10.3389/fpsyg.2015.00336
  49. Wagenmakers, E.-J., Ratcliff, R., Gomez, P., & McKoon, G. (2008). A diffusion model account of criterion shifts in the lexical decision task. Journal of Memory and Language, 58, 140–159.  https://doi.org/10.1016/j.jml.2007.04.006 CrossRefPubMedPubMedCentralGoogle Scholar
  50. Wagenmakers, E.-J., van der Maas, H. L. J., & Grasman, R. P. P. P. (2007). An EZ-diffusion model for response time and accuracy. Psychonomic Bulletin & Review, 14, 3–22.  https://doi.org/10.3758/bf03194023 CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2018

Authors and Affiliations

  1. 1.Psychologisches InstitutRuprecht-Karls-Universität HeidelbergHeidelbergGermany

Personalised recommendations