Response-time data provide critical constraints on dynamic models of multi-alternative, multi-attribute choice

  • Nathan J. Evans
  • William R. HolmesEmail author
  • Jennifer S. TruebloodEmail author
Theoretical Review


Understanding the cognitive processes involved in multi-alternative, multi-attribute choice is of interest to a wide range of fields including psychology, neuroscience, and economics. Prior investigations in this domain have relied primarily on choice data to compare different theories. Despite numerous such studies, results have largely been inconclusive. Our study uses state-of-the-art response-time modeling and data from 12 different experiments appearing in six different published studies to compare four previously proposed theories/models of these effects: multi-alternative decision field theory (MDFT), the leaky-competing accumulator (LCA), the multi-attribute linear ballistic accumulator (MLBA), and the associative accumulation model (AAM). All four models are, by design, dynamic process models and thus a comprehensive evaluation of their theoretical properties requires quantitative evaluation with both choice and response-time data. Our results show that response-time data is critical at distinguishing among these models and that using choice data alone can lead to inconclusive results for some datasets. In conclusion, we encourage future research to include response-time data in the evaluation of these models.


Decision-making Multi-attribute choice Context effects Bayesian methods 



The authors would like to thank Audrey Parrish, Michael Beran, and George Farmer for sharing their data. The authors would also like to thank Jerome Busemeyer for his comments on extending MDFT to SDE formalism. All authors were supported by NSF grant SES-1556415. The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of the funding agency.

Supplementary material

13423_2018_1557_MOESM1_ESM.pdf (904 kb)
(PDF 903 KB)


  1. Berkowitsch, N. A., Scheibehenne, B., & Rieskamp, J. (2014). Rigorously testing multialternative decision field theory against random utility models. Journal of Experimental Psychology: General, 143(3), 1331.Google Scholar
  2. Berkowitsch, N. A., Scheibehenne, B., Rieskamp, J., & Matthäus, M. (2015). A generalized distance function for preferential choices. British Journal of Mathematical and Statistical Psychology, 68(2), 310–325.PubMedGoogle Scholar
  3. Bhatia, S. (2013). Associations and the accumulation of preference. Psychological Review, 120(3), 522.PubMedGoogle Scholar
  4. Brown, S. D., & Heathcote, A. (2008). The simplest complete model of choice response time: Linear ballistic accumulation. Cognitive Psychology, 57, 153–178.PubMedGoogle Scholar
  5. Busemeyer, J. R., & Diederich, A. (2002). Survey of decision field theory. Mathematical Social Sciences, 43 (3), 345–370.Google Scholar
  6. Busemeyer, J. R., & Townsend, J. T. (1992). Fundamental derivations from decision field theory. Mathematical Social Sciences, 23(3), 255–282.Google Scholar
  7. Busemeyer, J. R., & Townsend, J. T. (1993). Decision field theory: A dynamic-cognitive approach to decision-making in an uncertain environment. Psychological Review, 100(3), 432.PubMedGoogle Scholar
  8. Busemeyer, J.R., & Wang, Y.-M. (2000). Model comparisons and model selections based on generalization criterion methodology. Journal of Mathematical Psychology, 44(1), 171–189.PubMedGoogle Scholar
  9. Cataldo, A.M., & Cohen, A.L. (2018). Reversing the similarity effect: The effect of presentation format. Cognition, 175, 141–156.PubMedGoogle Scholar
  10. Cohen, A. L., Kang, N., & Leise, T.L. (2017). Multi-attribute, multi-alternative models of choice: Choice, reaction time, and process tracing. Cognitive Psychology, 98, 45–72.PubMedGoogle Scholar
  11. Donkin, C., Brown, S., Heathcote, A. J., & Wagenmakers, E. -J. (2011). Diffusion versus linear ballistic accumulation: Different models for response time, same conclusions about psychological mechanisms? Psychonomic Bulletin & Review, 55, 140–151.Google Scholar
  12. Dutilh, G., Annis, J., Brown, S.D., Cassey, P., Evans, N.J., Grasman, R.P.P.P., & Donkin, C. (2018). The quality of response time data inference: A blinded, collaborative assessment of the validity of cognitive models. Psychonomic Bulletin & Review.
  13. Estes, W. K. (1956). The problem of inference from curves based on group data. Psychological Bulletin, 53 (2), 134.PubMedGoogle Scholar
  14. Evans, N.J., & Brown, S.D. (2017). People adopt optimal policies in simple decision-making, after practice and guidance. Psychonomic Bulletin & Review, 24(2), 597–606.Google Scholar
  15. Evans, N.J., & Brown, S.D. (2018). Bayes factors for the linear ballistic accumulator model of decision-making. Behavior Research Methods, 50(2), 589–603.PubMedGoogle Scholar
  16. Evans, N. J., Hawkins, G. E., Boehm, U., Wagenmakers, E. -J., & Brown, S. D. (2017a). The computations that support simple decision-making: A comparison between the diffusion and urgency-gating models. Scientific Reports, 7, 16433.Google Scholar
  17. Evans, N. J., Howard, Z. L., Heathcote, A., & Brown, S. D. (2017b). Model flexibility analysis does not measure the persuasiveness of a fit. Psychological Review, 124(3), 339.Google Scholar
  18. Evans, N.J., Rae, B., Bushmakin, M., Rubin, M., & Brown, S.D. (2017c). Need for closure is associated with urgency in perceptual decision-making. Memory & Cognition, 45(7), 1193–1205.Google Scholar
  19. Evans, N. J., Brown, S. D., Mewhort, D. J., & Heathcote, A. (2018). Refining the law of practice. Psychological Review, 125(4), 592.PubMedGoogle Scholar
  20. Evans, N. J., Steyvers, M., & Brown, S.D (2018). Modeling the covariance structure of complex datasets using cognitive models: An application to individual differences and the heritability of cognitive ability. Cognitive Science.Google Scholar
  21. Farmer, G. D., Warren, P. A., El-Deredy, W., & Howes, A. (2016). The effect of expected value on attraction effect preference reversals. Journal of Behavioral Decision Making.Google Scholar
  22. Gutenkunst, R. N., Waterfall, J. J., Casey, F. P., Brown, K. S., Myers, C. R., & Sethna, J. P. (2007). Universally sloppy parameter sensitivities in systems biology models. PLoS Computational Biology, 3(10), e189.PubMedCentralGoogle Scholar
  23. Heathcote, A., Brown, S., & Mewhort, D.J. (2000). The power law repealed: The case for an exponential law of practice. Psychonomic Bulletin & Review, 7(2), 185–207.Google Scholar
  24. Ho, T. C., Yang, G., Wu, J., Cassey, P., Brown, S.D., Hoang, N., & Yang, T. T. (2014). Functional connectivity of negative emotional processing in adolescent depression. Journal of Affective Disorders, 155, 65–74. PubMedGoogle Scholar
  25. Holmes, W.R. (2015). A practical guide to the probability density approximation (PDA) with improved implementation and error characterization. Journal of Mathematical Psychology, 68, 13–24.Google Scholar
  26. Holmes, W.R., & Trueblood, J.S. (2018). Bayesian analysis of the piecewise diffusion decision model. Behavior Research Methods, 50(2), 730–743.PubMedGoogle Scholar
  27. Holmes, W. R., Trueblood, J.S., & Heathcote, A. (2016). A new framework for modeling decisions about changing information: The piecewise linear ballistic accumulator model. Cognitive Psychology, 85, 1–29.PubMedPubMedCentralGoogle Scholar
  28. Hotaling, J. M., Busemeyer, J. R., & Li, J. (2010). Theoretical developments in decision field theory: A comment on K. Tsetsos, N. Chater, and M. Usher. Psychological Review, 117, 1294– 1298.PubMedGoogle Scholar
  29. Howes, A., Warren, P. A., Farmer, G., El-Deredy, W., & Lewis, R. L. (2016). Why contextual preference reversals maximize expected value. Psychological Review, 123(4), 368.PubMedPubMedCentralGoogle Scholar
  30. Huang, K., Sen, S., & Szidarovszky, F. (2012). Connections among decision field theory models of cognition. Journal of Mathematical Psychology, 56(5), 287–296.Google Scholar
  31. Huber, J., Payne, J. W., & Puto, C. (1982). Adding asymmetrically dominated alternatives: Violations of regularity and the similarity hypothesis. Journal of Consumer Research, 9, 90–98.Google Scholar
  32. Lerche, V., Voss, A., & Nagler, M. (2017). How many trials are required for parameter estimation in diffusion modeling? A comparison of different optimization criteria. Behavior Research Methods, 49(2), 513–537.PubMedGoogle Scholar
  33. Liew, S.X., Howe, P.D., & Little, D.R. (2016). The appropriacy of averaging in the study of context effects. Psychonomic Bulletin & Review, 23(5), 1639–1646.Google Scholar
  34. Miletic̀, S., Turner, B. M., Forstmann, B. U., & van Maanen, L. (2017). Parameter recovery for the leaky competing accumulator model. Journal of Mathematical Psychology, 76, 25–50.Google Scholar
  35. Myung, I.J. (2000). The importance of complexity in model selection. Journal of Mathematical Psychology, 44 (1), 190–204.PubMedGoogle Scholar
  36. Myung, I.J., & Pitt, M.A. (1997). Applying Occam’s razor in modeling cognition: A Bayesian approach. Psychonomic Bulletin & Review, 4(1), 79–95.Google Scholar
  37. Nosofsky, R.M., & Palmeri, T.J. (2015). An exemplar-based random-walk model of categorization and recognition. In The Oxford handbook of computational and mathematical psychology (p. 142). Oxford University Press, USA.Google Scholar
  38. Parrish, A.E., Evans, T.A., & Beran, M.J. (2015). Rhesus macaques (Macaca mulatta) exhibit the decoy effect in a perceptual discrimination task. Attention, Perception, & Psychophysics, 77(5), 1715–1725.Google Scholar
  39. Pettibone, J. C. (2012). Testing the effect of time pressure on asymmetric dominance and compromise decoys in choice. Judgment and Decision Making, 7(4), 513.Google Scholar
  40. Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85, 59–108.Google Scholar
  41. Ratcliff, R., Smith, P. L., Brown, S.D., & McKoon, G. (2016). Diffusion decision model: Current issues and history. Trends in Cognitive Sciences, 20(4), 260–281.PubMedPubMedCentralGoogle Scholar
  42. Roe, R. M., Busemeyer, J. R., & Townsend, J. T. (2001). Multialternative decision field theory: A dynamic connectionist model of decision making. Psychological Review, 108, 370–392.PubMedGoogle Scholar
  43. Simonson, I. (1989). Choice based on reasons: The case of attraction and compromise effects. Journal of Consumer Research, 16, 158–174.Google Scholar
  44. Soltani, A., De Martino, B., & Camerer, C. (2012). A range-normalization model of context-dependent choice: A new model and evidence. PLoS Computational Biology, 8(7), 1–15.Google Scholar
  45. Spiegelhalter, D.J., Best, N.G., Carlin, B.P., & Van Der Linde, A. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 64(4), 583–639.Google Scholar
  46. Ter Braak, C.J. (2006). A Markov chain Monte Carlo version of the genetic algorithm differential evolution: Easy Bayesian computing for real parameter spaces. Statistics and Computing, 16(3), 239–249.Google Scholar
  47. Trueblood, J. S. (2012). Multi-alternative context effects obtained using an inference task. Psychonomic Bulletin & Review, 19(5), 962–968.Google Scholar
  48. Trueblood, J. S., Brown, S. D., & Heathcote, A. (2014). The multiattribute linear ballistic accumulator model of context effects in multialternative choice. Psychological Review, 121(2), 179.PubMedGoogle Scholar
  49. Trueblood, J.S., Brown, S.D., & Heathcote, A. (2015). The fragile nature of contextual preference reversals: Reply to Tsetsos, Chater, and Usher (2015). Psychological Review, 122(4), 848–853.PubMedGoogle Scholar
  50. Trueblood, J. S., Brown, S. D., Heathcote, A., & Busemeyer, J. R. (2013). Not just for consumers: Context effects are fundamental to decision-making. Psychological Science, 24, 901–908.PubMedGoogle Scholar
  51. Trueblood, J. S., Holmes, W. R., Seegmiller, A. C., Douds, J., Compton, M., Szentirmai, E., & Eichbaum, Q. (2018). The impact of speed and bias on the cognitive processes of experts and novices in medical image decision-making. Cognitive Research: Principles and Implications, 3(1), 28.Google Scholar
  52. Trueblood, J. S., & Pettibone, J. C. (2017). The phantom decoy effect in perceptual decision making. Journal of Behavioral Decision Making, 30(2), 157–167.Google Scholar
  53. Tsetsos, K., Chater, N., & Usher, M. (2015). Examining the mechanisms underlying contextual preference reversal: Comment on Trueblood, Brown, and Heathcote (2014). Psychological Review, 122(4), 838–847.PubMedGoogle Scholar
  54. Tsetsos, K., Usher, M., & Chater, N. (2010). Preference reversal in multi-attribute choice. Psychological Review, 117, 1275–1291.PubMedGoogle Scholar
  55. Turner, B. M., Schley, D. R., Muller, C., & Tsetsos, K. (2018). Competing models of multi-attribute, multi-alternative preferential choice. Psychological Review, 125, 329–362.PubMedGoogle Scholar
  56. Turner, B. M., & Sederberg, P. B. (2014). A generalized, likelihood-free method for posterior estimation. Psychonomic Bulletin & Review, 21(2), 227–250.Google Scholar
  57. Turner, B. M., Sederberg, P. B., Brown, S. D., & Steyvers, M. (2013). A method for efficiently sampling from distributions with correlated dimensions. Psychological Methods, 18(3), 368.PubMedPubMedCentralGoogle Scholar
  58. Tversky, A. (1972). Elimination by aspects: A theory of choice. Psychological Review, 79, 281–299.Google Scholar
  59. Usher, M., Elhalal, A., & McClelland, J. L. (2008). The neurodynamics of choice, value-based decisions, and preference reversal. In N. Chater, & M. Oaksford (Eds.) The probabilistic mind: Prospects for Bayesian cognitive science (pp. 277–300). Oxford: Oxford University Press.Google Scholar
  60. Usher, M., & McClelland, J. L. (2001). The time course of perceptual choice: The leaky, competing accumulator model. Psychological Review, 108(3), 550.PubMedGoogle Scholar
  61. Usher, M., & McClelland, J. L. (2004). Loss aversion and inhibition in dynamical models of multialternative choice. Psychological Review, 111, 757–769.PubMedGoogle Scholar
  62. van Ravenzwaaij, D., Dutilh, G., & Wagenmakers, E.-J. (2012). A diffusion model decomposition of the effects of alcohol on perceptual decision making. Psychopharmacology, 219(4), 1017–1025. PubMedGoogle Scholar
  63. Wollschlager, L. M., & Diederich, A. (2012). The 2n-ary choice tree model for n-alternative preferential choice. Frontiers in Cognitive Science, 3, 1–11.Google Scholar

Copyright information

© The Psychonomic Society, Inc. 2019

Authors and Affiliations

  1. 1.Department of PsychologyVanderbilt UniversityNashvilleUSA
  2. 2.Department of PsychologyUniversity of AmsterdamAmsterdamThe Netherlands
  3. 3.Department of Physics and Astronomy, Department of MathematicsVanderbilt UniversityNashvilleUSA

Personalised recommendations