Advertisement

Behavior Research Methods

, Volume 51, Issue 1, pp 40–60 | Cite as

Combining speed and accuracy to control for speed-accuracy trade-offs(?)

  • Heinrich René LiesefeldEmail author
  • Markus Janczyk
Article

Abstract

In psychological experiments, participants are typically instructed to respond as fast as possible without sacrificing accuracy. How they interpret this instruction and, consequently, which speed–accuracy trade-off they choose might vary between experiments, between participants, and between conditions. Consequently, experimental effects can appear unpredictably in either RTs or error rates (i.e., accuracy). Even more problematic, spurious effects might emerge that are actually due only to differential speed–accuracy trade-offs. An often-suggested solution is the inverse efficiency score (IES; Townsend & Ashby, 1983), which combines speed and accuracy into a single score. Alternatives are the rate-correct score (RCS; Woltz & Was, 2006) and the linear-integrated speed–accuracy score (LISAS; Vandierendonck, 2017, 2018). We report analyses on simulated data generated with the standard diffusion model (Ratcliff, 1978) showing that IES, RCS, and LISAS put unequal weights on speed and accuracy, depending on the accuracy level, and that these measures are actually very sensitive to speed–accuracy trade-offs. These findings stand in contrast to a fourth alternative, the balanced integration score (BIS; Liesefeld, Fu, & Zimmer, 2015), which was devised to integrate speed and accuracy with equal weights. Although all of the measures maintain “real” effects, only BIS is relatively insensitive to speed–accuracy trade-offs.

Keywords

Speed–accuracy trade-off Integration of errors and RTs Integrated scoring Task instructions Performance strategies Methods in experimental psychology 

References

  1. Alvarez, G. A., & Cavanagh, P. (2004). The capacity of visual short term memory is set both by visual information load and by number of objects. Psychological Science, 15, 106–111. doi: https://doi.org/10.1111/j.0963-7214.2004.01502006.x CrossRefPubMedGoogle Scholar
  2. Akhtar, N., & Enns, J. T. (1989). Relations between covert orienting and filtering in the development of visual attention. Journal of Experimental Child Psychology, 48, 315–334. doi: https://doi.org/10.1016/0022-0965(89)90008-8 CrossRefPubMedGoogle Scholar
  3. Balci, F., Simen, P., Niyogi, R., Saxe, A., Hughes, J. A., Holmes, P., & Cohen, J. D. (2011). Acquisition of decision making criteria: Reward rate ultimately beats accuracy. Attention, Perception, & Psychophysics, 73, 640–657. doi: https://doi.org/10.3758/s13414-010-0049-7 CrossRefGoogle Scholar
  4. Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., & Cohen, J. D. (2001). Conflict monitoring and cognitive control. Psychological Review, 108, 624–652. doi: https://doi.org/10.1037/0033-295X.108.3.624 CrossRefPubMedGoogle Scholar
  5. Brown, S. D., & Heathcote, A. (2008). The simplest complete model of choice response time: Linear ballistic accumulation. Cognitive Psychology, 57, 153–178. doi: https://doi.org/10.1016/j.cogpsych.2007.12.002 CrossRefPubMedGoogle Scholar
  6. Bruyer, R., & Brysbaert, M. (2011). Combining speed and accuracy in cognitive psychology: Is the inverse efficiency score (IES) a better dependent variable than the mean reaction time (RT) and the percentage of errors (PE)? Psychologica Belgica, 51, 5–13. doi: https://doi.org/10.5334/pb-51-1-5 CrossRefGoogle Scholar
  7. Bush, L. K., Hess, U., & Wolford, G. (1993). Transformations for within-subject designs: A Monte Carlo investigation. Psychological Bulletin, 113, 566–579. doi: https://doi.org/10.1037/0033-2909.113.3.566 CrossRefPubMedGoogle Scholar
  8. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.Google Scholar
  9. Collignon, O., Girard, S., Gosselin, F., Roy, S., Saint-Amour, D., Lassonde, M., & Lepore, F. (2008). Audio-visual integration of emotion expression. Brain Research, 1242, 126–135. doi: https://doi.org/10.1016/j.brainres.2008.04.023 CrossRefPubMedGoogle Scholar
  10. Donkin, C., Brown, S., & Heathcote, A. (2011). Drawing conclusions from choice response time models: A tutorial using the linear ballistic accumulator. Journal of Mathematical Psychology, 55, 140–151. doi: https://doi.org/10.1016/j.jmp.2010.10.001 CrossRefGoogle Scholar
  11. Draheim, C., Hicks, K. L., & Engle, R. W. (2016). Combining reaction time and accuracy: The relationship between working memory capacity and task switching as a case example. Perspectives on Psychological Science, 11, 133–155. doi: https://doi.org/10.1177/1745691615596990 CrossRefPubMedGoogle Scholar
  12. Dutilh, G., van Ravenzwaaij, D., Nieuwenhuis, S., van der Maas, H. J., Forstmann, B. U., & Wagenmakers, E.-J. (2012). How to measure post-error slowing: A confound and a simple solution. Journal of Mathematical Psychology, 56, 208–216. doi: https://doi.org/10.1016/j.jmp.2012.04.001 CrossRefGoogle Scholar
  13. Faust, M. E., Balota, D. A., Spieler, D. H., & Ferraro, F. R. (1999). Individual differences in information-processing rate and amount: Implications for group differences in response latency. Psychological Bulletin, 125, 777–799. doi: https://doi.org/10.1037/0033-2909.125.6.777 CrossRefPubMedGoogle Scholar
  14. Forstmannn, B. U., Ratcliff, R., & Wagenmakers, E.-J. (2016). Sequential sampling models in cognitive neuroscience: Advantages, applications, and extensions. Annual Review of Psychology, 67, 641–666. doi: https://doi.org/10.1146/annurev-psych-122414-033645 CrossRefGoogle Scholar
  15. Gabay, S., Nestor, A., Dundas, E., & Behrmann, M. (2014). Monocular advantage for face perception implicates subcortical mechanisms in adult humans. Journal of Cognitive Neuroscience, 26, 927–937. doi: https://doi.org/10.1162/jocn_a_00528 CrossRefPubMedGoogle Scholar
  16. Germar, M., Schlemmer, A., Krug, K, Voss, A., & Mojzisch, A. (2014). Social influence and perceptual decision-making: A diffusion model analysis. Personality and Social Psychology Bulletin, 40, 217–231. doi: https://doi.org/10.1177/0146167213508985 CrossRefPubMedGoogle Scholar
  17. Gilchrist, A. L., & Cowan, N. (2014). A two-stage search of visual working memory: Investigating speed in the change-detection paradigm. Attention, Perception, & Psychophysics, 76, 2031–2050. doi: https://doi.org/10.3758/s13414-014-0704-5 CrossRefGoogle Scholar
  18. Gold, J. I., & Shadlen, M. N. (2002). Banburismus and the brain: Decoding the relationship between sensory stimuli, decisions, and reward. Neuron, 36, 299–308. doi: https://doi.org/10.1016/S0896-6273(02)00971-6
  19. Gueugneau, N., Pozzo, T., Darlot, C., & Papaxanthis, C. (2017). Daily modulation of the speed–accuracy trade-off. Neuroscience, 356, 142–150. doi: https://doi.org/10.1016/j.neuroscience.2017.04.043 CrossRefPubMedGoogle Scholar
  20. Heitz, R. P. (2014). The speed–accuracy tradeoff: History, physiology, methodology, and behavior. Frontiers in Neuroscience, 8, 150. doi: https://doi.org/10.3389/fnins.2014.00150 CrossRefPubMedPubMedCentralGoogle Scholar
  21. Hughes, M. M., Linck, J. A., Bowles, A. R., Koeth, J. T., & Bunting, M. F. (2014). Alternatives to switch-cost scoring in the task-switching paradigm: Their reliability and increased validity. Behavior Research Methods, 46, 702–721. doi: https://doi.org/10.3758/s13428-013-0411-5 CrossRefPubMedGoogle Scholar
  22. Hyun, J., Woodman, G. F., Vogel, E. K., Hollingworth, A., & Luck, S. J. (2009). The comparison of visual working memory representations with perceptual inputs. Journal of Experimental Psychology: Human Perception and Performance, 35, 1140–1160. doi: https://doi.org/10.1037/a0015019 PubMedGoogle Scholar
  23. Janczyk, M., & Lerche, V. (in press). A diffusion model analysis of the response–effect compatibility effect. Journal of Experimental Psychology: General. doi:10.1037/xge0000430Google Scholar
  24. Janczyk, M., Mittelstädt, P., & Wienrich, C. (2018). Parallel dual-task processing and task-shielding in older and younger adults: Behavioral and diffusion model results. Experimental Aging Research, 44, 95–116. doi: https://doi.org/10.1080/0361073X.2017.1422459 CrossRefPubMedGoogle Scholar
  25. Kiss, M., Driver, J., & Eimer, M. (2009). Reward priority of visual target singletons modulates event-related potential signatures of attentional selection. Psychological Science, 20, 245–251. doi: https://doi.org/10.1111/j.1467-9280.2009.02281.x CrossRefPubMedPubMedCentralGoogle Scholar
  26. Kristjánsson, Á. (2016). The slopes remain the same: Reply to Wolfe (2016). i-Perception, 7, 1–4.CrossRefGoogle Scholar
  27. Kunde, W., Pfister, R., & Janczyk, M. (2012). The locus of tool-transformation costs. Journal of Experimental Psychology: Human Perception and Performance, 38, 703–714. doi: https://doi.org/10.1037/a0026315 PubMedGoogle Scholar
  28. Küper, K., Gajewski, P. D., Frieg, C., & Falkenstein, M. (2017). A randomized controlled ERP study on the effects of multi-domain cognitive training and task difficulty on task switching performance in older adults. Frontiers in Human Neuroscience, 11, 184. doi: https://doi.org/10.3389/fnhum.2017.00184 CrossRefPubMedPubMedCentralGoogle Scholar
  29. Laming, D. R. J. (1968). Information theory of choice-reaction times. London, UK: Academic Press.Google Scholar
  30. Lerche, V., & Voss, A. (in press). Speed–accuracy manipulation in diffusion modeling: Lack of discriminant validity of the manipulation or of the parameter estimates? Behavior Research Methods. doi: https://doi.org/10.3758/s13428-018-1034-7
  31. Liesefeld, H. R., Fu, X., & Zimmer, H. D. (2015). Fast and careless or careful and slow? Apparent holistic processing in mental rotation is explained by speed–accuracy trade-offs. Journal of Experimental Psychology: Learning, Memory, and Cognition, 41, 1140–1151. doi: https://doi.org/10.1037/xlm0000081 PubMedGoogle Scholar
  32. Liesefeld, H. R., Liesefeld, A. M., Müller, H. J., & Rangelov, D. (2017). Saliency maps for finding changes in visual scenes?. Attention, Perception, & Psychophysics, 79, 2190–2201. doi: https://doi.org/10.3758/s13414-017-1383-9 CrossRefGoogle Scholar
  33. Lo, S., & Andrews, S. (2015). To transform or not to transform: Using generalized linear mixed models to analyse reaction time data. Frontiers in Psychology, 6, 1171. doi: https://doi.org/10.3389/fpsyg.2015.01171 CrossRefPubMedPubMedCentralGoogle Scholar
  34. Luce, R. D. (1986). Response times: Their role in inferring elementary mental organisation (Oxford Psychology Series, Vol. 8). New York, NY: Oxford University Press.Google Scholar
  35. Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working memory for features and conjunctions. Nature, 390, 279–281. doi: https://doi.org/10.1038/36846 CrossRefPubMedGoogle Scholar
  36. Luck, S. J., & Vogel, E. K. (2013). Visual working memory capacity: From psychophysics and neurobiology to individual differences. Trends in Cognitive Sciences, 17, 391–400. doi: https://doi.org/10.1016/j.tics.2013.06.006 CrossRefPubMedPubMedCentralGoogle Scholar
  37. Mevorach, C., Humphreys, G. W., & Shalev, L. (2006). Opposite biases in salience-based selection for the left and right posterior parietal cortex. Nature Neuroscience, 9, 740–742. doi: https://doi.org/10.1038/nn1709 CrossRefPubMedGoogle Scholar
  38. Ollman, R. (1966). Fast guesses in choice reaction time. Psychonomic Science, 6, 155–156. doi: https://doi.org/10.3758/BF03328004 CrossRefGoogle Scholar
  39. Paas, F. G. W. C., & Van Merriënboer, J. J. G. (1993). The efficiency of instructional conditions: An approach to combine mental effort and performance measures. Human Factors, 35, 737–743. doi: https://doi.org/10.1177/001872089303500412 CrossRefGoogle Scholar
  40. Pachella, R. G. (1974). The interpretation of reaction time in information processing research. In B. H. Kantowitz (Ed.), Human information processing: Tutorials in performance and cognition (pp. 41–82). Hillsdale, NJ: Erlbaum.Google Scholar
  41. Paoletti, D., Weaver, M. D., Braun, C., & van Zoest, W. (2015). Trading off stimulus salience for identity: A cueing approach to disentangle visual selection strategies. Vision Research, 113(Pt. B), 116–124. doi: https://doi.org/10.1016/j.visres.2014.08.003 CrossRefPubMedGoogle Scholar
  42. Petrini, K., McAleer, P., & Pollick, F. (2010). Audiovisual integration of emotional signals from music improvisation does not depend on temporal correspondence. Brain Research, 1323, 139–148. doi: https://doi.org/10.1016/j.brainres.2010.02.012 CrossRefPubMedGoogle Scholar
  43. Rae, B., Heathcote, A., Donkin, C., Averell, L., & Brown, S. (2014). The hare and the tortoise: Emphasizing speed can change the evidence used to make decisions. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40, 1226–1243. doi: https://doi.org/10.1037/a0036801 PubMedGoogle Scholar
  44. Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85, 59–108. doi: https://doi.org/10.1037/0033-295X.85.2.59 CrossRefGoogle Scholar
  45. Ratcliff, R. (2008). The EZ diffusion method: Too EZ? Psychonomic Bulletin & Review, 15, 1218–1228. doi: https://doi.org/10.3758/PBR.15.6.1218 CrossRefGoogle Scholar
  46. Ratcliff, R., Smith, P. L., Brown, S. D., & McKoon, G. (2016). Diffusion decision model: Current issues and history. Trends in Cognitive Sciences, 20, 260–281. doi: https://doi.org/10.1016/j.tics.2016.01.007 CrossRefPubMedPubMedCentralGoogle Scholar
  47. Ratcliff, R., Thapar, A., & McKoon, G. (2011). Effects of aging and IQ on item and associative memory. Journal of Experimental Psychology: General, 140, 464–487. doi: https://doi.org/10.1037/a0023810 CrossRefGoogle Scholar
  48. Reeder, R. R., Hanke, M., & Pollmann, S. (2017). Task relevance modulates the representation of feature conjunctions in the target template. Scientific Reports, 7, 4514. doi: https://doi.org/10.1038/s41598-017-04123-8
  49. Reuss, H., Kiesel, A., & Kunde, W. (2015). Adjustments of response speed and accuracy to unconscious cues. Cognition, 134, 57–62. doi: https://doi.org/10.1016/j.cognition.2014.09.005 CrossRefPubMedGoogle Scholar
  50. Rinkenauer, G., Osman, A., Ulrich, R., Müller-Gethmann, H., & Mattes, S. (2004). On the locus of speed–accuracy trade-off in reaction time: inferences from the lateralized readiness potential. Journal of Experimental Psychology: General, 133, 261–282. doi: https://doi.org/10.1037/0096-3445.133.2.261 CrossRefGoogle Scholar
  51. Röder, B., Kusmierek, A., Spence, C., & Schicke, T. (2007). Developmental vision determines the reference frame for the multisensory control of action. Proceedings of the National Academy of Sciences, 104, 4753–4758. doi: https://doi.org/10.1073/pnas.0607158104 CrossRefGoogle Scholar
  52. Sanders, A. F. (1998). Elements of human performance: Reaction processes and attention in human skill. Mahwah, NJ: Erlbaum.Google Scholar
  53. Schmitz, F., & Voss, A. (2012). Decomposing task-switching costs with the diffusion model. Journal of Experimental Psychology: Human Perception and Performance, 38, 222–250. doi: https://doi.org/10.1037/a0026003 PubMedGoogle Scholar
  54. Schubert, A.-L., Hagemann, D., Voss, A., Schankin, A., & Bergmann, K. (2015). Decomposing the relationship between mental speed and mental abilities. Intelligence, 51, 28–46. doi: https://doi.org/10.1016/j.intell.2015.05.002 CrossRefGoogle Scholar
  55. Spence, C., Kingstone, A., Shore, D. I., & Gazzaniga, M. S. (2001a). Representation of visuotactile space in the split brain. Psychological Science, 12, 90–93. doi: https://doi.org/10.1111/1467-9280.00316 CrossRefPubMedGoogle Scholar
  56. Spence, C., Shore, D. I., Gazzaniga, M. S., Soto-Faraco, S., & Kingstone, A. (2001b). Failure to remap visuotactile space across the midline in the split-brain. Canadian Journal of Experimental Psychology, 55, 133–140. doi: https://doi.org/10.1037/h0087360 CrossRefPubMedGoogle Scholar
  57. Starns, J. J., Ratcliff, R., & McKoon, G. (2012). Evaluating the unequal-variance and dual-process explanations of zROC slopes with response time data and the diffusion model. Cognitive Psychology, 64, 1–34. doi: https://doi.org/10.1016/j.cogpsych.2011.10.002 CrossRefPubMedGoogle Scholar
  58. Thura, D., Guberman, G., & Cisek, P. (2017). Trial-to-trial adjustments of speed–accuracy trade-offs in premotor and primary motor cortex. Journal of Neurophysiology, 117, 665–683. doi: https://doi.org/10.1152/jn.00726.2016 CrossRefPubMedGoogle Scholar
  59. Townsend, J. T., & Ashby, F. G. (1983). Stochastic modelling of elementary psychological processes. New York, NY: Cambridge University Press.Google Scholar
  60. Ulrich, R., Schröter, H., Leuthold, H., & Birngruber, T. (2015). Automatic and controlled stimulus processing in conflict tasks: Superimposed diffusion processes and delta functions. Cognitive Psychology, 78, 148–174. doi: https://doi.org/10.1016/j.cogpsych.2015.02.005 CrossRefPubMedGoogle Scholar
  61. Unsworth, N., Redick, T. S., Heitz, R. P., Broadway, J. M., & Engle, R. W. (2009). Complex working memory span tasks and higher-order cognition: A latent-variable analysis of the relationship between processing and storage. Memory, 17, 635–654. doi: https://doi.org/10.1080/09658210902998047 CrossRefPubMedGoogle Scholar
  62. Usher, M., & McClelland, J. L. (2001). On the time course of perceptual choice: The leaky competing accumulator model. Psychological Review, 108, 550–592. doi: https://doi.org/10.1037/0033-295X.108.3.550 CrossRefPubMedGoogle Scholar
  63. Vandierendonck, A. (2017). A comparison of methods to combine speed and accuracy measures of performance: A rejoinder on the binning procedure. Behavior Research Methods, 49, 653–673. doi: https://doi.org/10.3758/s13428-016-0721-5 CrossRefPubMedGoogle Scholar
  64. Vandierendonck, A. (2018). Further tests of the utility of integrated speed–accuracy measures in task switching. Journal of Cognition, 1, 8. doi: https://doi.org/10.5334/joc.6 CrossRefGoogle Scholar
  65. Vandekerckhove, J., & Tuerlinckx, F. (2007). Fitting the Ratcliff diffusion model to experimental data. Psychonomic Bulletin & Review, 14, 1011–1026. doi: https://doi.org/10.3758/BF03193087 CrossRefGoogle Scholar
  66. Van Ravenzwaaij, D., & Oberauer, K. (2009). How to use the diffusion model: Parameter recovery of three methods: EZ, fast-dm, and DMAT. Journal of Mathematical Psychology, 53, 463–473. doi: https://doi.org/10.1016/j.jmp.2009.09.004 CrossRefGoogle Scholar
  67. Voss, A., Rothermund, K., & Brandtstädter, J. (2008). Interpreting ambiguous stimuli: Separating perceptual and judgmental biases. Journal of Experimental Social Psychology, 44, 1048–1056. doi: https://doi.org/10.1016/j.jesp.2007.10.009 CrossRefGoogle Scholar
  68. Voss, A., Rothermund, K., & Voss, J. (2004). Interpreting the parameters of the diffusion model: An empirical validation. Memory & Cognition, 32, 1206–1220. doi: https://doi.org/10.3758/BF03196893 CrossRefGoogle Scholar
  69. Voss, A., & Voss, J. (2007). Fast-dm: A free program for efficient diffusion model analysis. Behavior Research Methods, 39, 767–775. doi: https://doi.org/10.3758/BF03192967 CrossRefPubMedGoogle Scholar
  70. Voss, A., Voss, J., & Lerche, V. (2015). Assessing cognitive processes with diffusion model analyses: A tutorial based on fast-dm-30. Frontiers in Psychology, 6, 336. doi: https://doi.org/10.3389/fpsyg.2015.00336 CrossRefPubMedPubMedCentralGoogle Scholar
  71. Wagenmakers, E.-J. (2009). Methodological and empirical developments for the Ratcliff diffusion model of response times and accuracy. European Journal of Cognitive Psychology, 21, 641–671. doi: https://doi.org/10.1080/09541440802205067 CrossRefGoogle Scholar
  72. Wagenmakers, E.-J., van der Maas, H. L. J., Dolan, C. V., & Grasman, R. P. P. P. (2008). EZ does it! Extensions of the EZ-diffusion model. Psychonomic Bulletin & Review, 15, 1229–1235. doi: https://doi.org/10.3758/PBR.15.6.1229 CrossRefGoogle Scholar
  73. Wagenmakers, E.-J., van der Maas, H. J., & Grasman, R. P. (2007). An EZ-diffusion model for response time and accuracy. Psychonomic Bulletin & Review, 14, 3–22. doi: https://doi.org/10.3758/BF03194023 CrossRefGoogle Scholar
  74. Wickelgren, W. A. (1977). Speed–accuracy tradeoff and information processing dynamics. Acta Psychologica, 41, 67–85. doi: https://doi.org/10.1016/0001-6918(77)90012-9 CrossRefGoogle Scholar
  75. Woltz, D. J., & Was, C. A. (2006). Availability of related long-term memory during and after attention focus in working memory. Memory & Cognition, 34, 668–684. doi: https://doi.org/10.3758/BF03193587 CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2018

Authors and Affiliations

  1. 1.Department Psychologie, and Graduate School of Systemic NeurosciencesLudwig-Maximilians-Universität MünchenMunichGermany
  2. 2.Department of PsychologyEberhard Karls University of TübingenTübingenGermany

Personalised recommendations