Skip to main content

What Do the Rules for the Wrong Game Tell us About How to Play the Right Game?


Psychological science has rightly become worried about questionable practices in experimental research, with a range of recent suggestions being made about remedies for this “replication crisis”. To avoid similar problems in psychological-process modelling, Lee et al. (in review) propose ingenious adaptions of these remedies along with insightful new suggestions. Although in the main applauding of these developments, I question whether some of the lessons drawn from the replication crisis are applicable, particularly with respect to the confirmatory vs. exploratory dichotomy given the intrinsically explanatory nature of most psychological-process models.

This is a preview of subscription content, access via your institution.


  1. I acknowledge borrowing in my commentary from the style and substance of Alan Newell’s wonderful “You cannot play 20 questions with nature and win”. I dedicate this commentary to the memory of Doug Mewhort, who brought Newell’s commentary to my attention, and whose work and teaching inspired in me a life-long interest in psychological modelling. Doug leavened this interest with a dollop of caution borne of another idea he made me aware of, that participants should not be viewed as always the same, but instead as deploying highly flexible virtual machines that adapt to different tasks. Thanks also to Dora Matzke for discussions related to this commentary.


  • Box, G. E. P. (1979). Robustness in the strategy of scientific model building. In R. L. Launer & G. N. Wilkinson (Eds.), Robustness in statistics (pp. 201–236). Academic Press.

  • Dennis, S. (2005). A memory-based theory of verbal cognition. Cognitive Science, 29(2), 145–193.

    Article  Google Scholar 

  • Heathcote, A., Wagenmakers, E. J., & Brown, S. D. (2014). The falsifiability of actual decision-making models. Psychological Review, 121, 676–678.

    Article  Google Scholar 

  • Hübner, R., Steinhauser, M., & Lehle, C. (2010). A dual-stage two-phase model of selective attention. Psychological Review, 117, 759–784.

    Article  Google Scholar 

  • Jones, M., & Dzhafarov, E. N. (2014). Unfalsifiability and mutual translatability of major modelling schemes for choice reaction time. Psychological Review, 121, 1–32.

    Article  Google Scholar 

  • Newell, A. (1973). You can’t play 20 questions with nature and win: projective comments on the papers of this symposium. In W. G. Chase (Ed.), Visual information processing. New York: Academic Press.

    Google Scholar 

  • Newell, A. (1990). Unified theories of cognition. Harvard University Press.

  • Rae, B., Heathcote, A., Donkin, C., Averell, L., & Brown, S. (2014). The hare and the tortoise: emphasizing speed can change the evidence used to make decisions. Journal of Experimental Psychology. Learning, Memory, and Cognition, 40, 1226–1243.

    Article  Google Scholar 

  • Smith, P. L., Ratcliff, R., & McKoon, G. (2014). The diffusion model is not a deterministic growth model: comment on Jones and Dzhafarov (2013). Psychological Review, 121, 679–688.

    Article  Google Scholar 

  • Walsh, M. M., Gunzelmann, G., & Van Dongen, H. P. (2017). Computational cognitive modeling of the temporal dynamics of fatigue from sleep loss. Psychonomic Bulletin & Review, 24, 1785–1807.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Andrew Heathcote.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Heathcote, A. What Do the Rules for the Wrong Game Tell us About How to Play the Right Game?. Comput Brain Behav 2, 187–189 (2019).

Download citation

  • Published:

  • Issue Date:

  • DOI: