Skip to main content
SpringerLink
Log in

A rational model of the Dunning–Kruger effect supports insensitivity to evidence in low performers

  • Article
  • Published:

From Nature Human Behaviour

View current issue Submit your manuscript

Abstract

Evaluating one’s own performance on a task, typically known as ‘self-assessment’, is perceived as a fundamental skill, but people appear poorly calibrated to their abilities. Studies seem to show poorer calibration for low performers than for high performers, which could indicate worse metacognitive ability among low performers relative to others (the Dunning–Kruger effect). By developing a rational model of self-assessment, we show that such an effect could be produced by two psychological mechanisms, in either isolation or conjunction: influence of prior beliefs about ability or a relation between performance and skill at determining correctness on each problem. To disentangle these explanations, we conducted a large-scale replication of a seminal paper with approximately 4,000 participants in each of two studies. Comparing the predictions of two variants of our rational model provides support for low performers being less able to estimate whether they are correct in the domains of grammar and logical reasoning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1: Graphical representation of the model.
Fig. 2: Model predictions in a toy example where participants solve ten problems in the baseline model (μθ, μβ = 0, σθ, σβ = 1 and ϵ = 0).
Fig. 3: Model predictions in a toy example.
Fig. 4: Histograms of results in the two studies.
Fig. 5: Findings of the grammar study.
Fig. 6: Findings of the logical reasoning study.

Similar content being viewed by others

Data availability

The anonymized data that support the findings of this study are available on the Open Science Framework (https://osf.io/er9ms/).

Code availability

The Qualtrics code that generates the surveys is available in the same repository (https://osf.io/er9ms/) on the Open Science Framework. All code used for analyses and the model are available on GitHub (https://github.com/racheljansen/self-assessment).

References

  1. Dunning, D., Heath, C. & Suls, J. Flawed self-assessment: implications for health, education, and the workplace. Psychol. Sci. Public Interest 5, 69–106 (2004).

    Article  Google Scholar 

  2. Ehrlinger, J., Johnson, K., Banner, M., Dunning, D. & Kruger, J. Why the unskilled are unaware: further explorations of (absent) self-insight among the incompetent. Organ. Behav. Hum. Decis. Process. 105, 98–121 (2008).

    Article  Google Scholar 

  3. Zell, E. & Krizan, Z. Do people have insight into their abilities? A metasynthesis. Perspect. Psychol. Sci. 9, 111–125 (2014).

    Article  Google Scholar 

  4. Bjorklund, D. F. & Green, B. L. The adaptive nature of cognitive immaturity. Am. Psychol. 47, 46–54 (1992).

    Article  Google Scholar 

  5. Tyszka, T. & Zielonka, P. Expert judgments: financial analysts versus weather forecasters. J. Psychol. Financial Mark. 3, 152–160 (2002).

    Article  Google Scholar 

  6. Jansen, R.A., Rafferty, A.N. and Griffiths, T.L. Algebra is not like trivia: evaluating self-assessment in an online math tutor. in Proceedings of the 39th Annual Conference of the Cognitive Science Society (Cognitive Science Society, 2017).

  7. Nelson, T. O. & Dunlosky, J. When people’s judgments of learning (JOLs) are extremely accurate at predicting subsequent recall: the ‘delayed-JOL effect’. Psychol. Sci. 2, 267–271 (1991).

    Article  Google Scholar 

  8. Kruger, J. & Dunning, D. Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. J. Personal. Soc. Psychol. 77, 1121–1134 (1999).

    Article  CAS  Google Scholar 

  9. Lopez, G. Why incompetent people often think they’re actually the best. Vox https://www.vox.com/science-and-health/2017/11/18/16670576/dunning-kruger-effect-video (18 November 2017).

  10. Andrews, R. This psychological effect explains why anti-vaxxers believe what they velieve. IFLScience http://www.iflscience.com/health-and-medicine/antivaxxers-suffer-from-a-wellknown-cognitive-effect-according-to-study/ (2018).

  11. Purtill, C. This psychological quirk could explain why Trump’s least experienced lawyer feels so confident. Quartz https://work.qz.com/1240245/the-dunning-kruger-effect-what-trumps-legal-team-and-the-russia-probe-have-to-do-with-it/ (29 March 2018).

  12. Healy, P.J. & Moore, D.A. Bayesian overconfidence. SSRN https://doi.org/10.2139/ssrn.1001820 (2007).

  13. Krueger, J. & Mueller, R. A. Unskilled, unaware, or both? The better-than-average heuristic and statistical regression predict errors in estimates of own performance. J. Personal. Soc. Psychol. 82, 180–188 (2002).

    Article  Google Scholar 

  14. Kruger, J. & Dunning, D. Unskilled and unaware-but why? A reply to Krueger and Mueller. J. Personal. Soc. Psychol. 82, 189–192 (2002).

    Article  Google Scholar 

  15. Burson, K., Larrick, R. P. & Klayman, J. Skilled or unskilled, but still unaware of it: how perceptions of difficulty drive miscalibration in relative comparisons. J. Personal. Soc. Psychol. 90, 60–77 (2006).

    Article  Google Scholar 

  16. Fleming, S. M. & Daw, N. D. Self-evaluation of decision-making: a general Bayesian framework for metacognitive computation. Psychol. Rev. 124, 91–114 (2017).

    Article  Google Scholar 

  17. Feld, J., Sauermann, J. & De Grip, A. Estimating the relationship between skill and overconfidence. J. Behav. Exp. Econ. 68, 18–24 (2017).

    Article  Google Scholar 

  18. Krajč, M. & Ortmann, A. Are the unskilled really that unaware? Alternative explanation. J. Econ. Psychol. 29, 724–738 (2008).

    Article  Google Scholar 

  19. Schlösser, T., Dunning, D., Johnson, K. L. & Kruger, J. How unaware are the unskilled? Empirical tests of the ‘signal extraction’ counterexplanation for the Dunning–Kruger effect in self-evaluation of performance. J. Econ. Psychol. 39, 85–100 (2013).

    Article  Google Scholar 

  20. Ehrlinger, J. & Dunning, D. How chronic self-views influence (and potentially mislead) estimates of performance. J. Personal. Soc. Psychol. 84, 5–17 (2003).

    Article  Google Scholar 

  21. Dunning, D. & Helzer, E. G. Beyond the correlation coefficient in studies of self-assessment accuracy: commentary on Zell & Krizan (2014). Perspect. Psychol. Sci. 9, 126–130 (2014).

    Article  Google Scholar 

  22. Anderson, J. R. The Adaptive Character of Thought (Earlbaum, 1990).

  23. Oaksford, M. & Chater, N. A rational analysis of the selection task as optimal data selection. Psychol. Rev. 101, 608 (1994).

    Article  Google Scholar 

  24. Embretson, S. E. & Reise, S. P. Item Response Theory (Psychology Press, 2013).

  25. Gilks, W. R, Richardson, S. & Spiegelhalter, D. Markov Chain Monte Carlo in Practice. (Chapman and Hall/CRC, 1995).

  26. Pronin, E., Lin, D. Y. & Ross, L. The bias blind spot: perceptions of bias in self versus others. Personal. Soc. Psychol. Bull. 28, 369–381 (2002).

    Article  Google Scholar 

  27. Chandler, J., Rosenzweig, C., Moss, A. J., Robinson, J. & Litman, L. Online panels in social science research: expanding sampling methods beyond Mechanical Turk. Behav. Res. Methods 51, 2022–2038 (2019).

    Article  Google Scholar 

  28. Mason, W. & Suri, S. Conducting behavioral research on Amazon’s Mechanical Turk. Behav. Res. Methods 44, 1–23 (2012).

    Article  Google Scholar 

  29. Sanchez, C. & Dunning, D. Overconfidence among beginners: is a little learning a dangerous thing? J. Personal. Soc. Psychol. 114, 10 (2018).

    Article  Google Scholar 

Download references

Acknowledgements

The authors received no specific funding for this work.

Author information

Authors and Affiliations

  1. Department of Psychology, University of California, Berkeley, Berkeley, CA, USA

    Rachel A. Jansen

  2. Department of Computer Science, Carleton College, Northfield, MN, USA

    Anna N. Rafferty

  3. Department of Psychology, Princeton University, Princeton, NJ, USA

    Thomas L. Griffiths

Authors
  1. Rachel A. Jansen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anna N. Rafferty
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas L. Griffiths
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

R.A.J. designed the studies, collected data, developed the model and performed the simulations. A.N.R. and T.L.G. supervised the study design and model development. All authors discussed the results and contributed to the final manuscript.

Corresponding author

Correspondence to Rachel A. Jansen.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Nature Human Behaviour thanks Stephen Fleming, Sam Gilbert and Matthew Rhodes for their contribution to the peer review of this work. Primary Handling Editor: Marike Schiffer.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Interpreting σθ.

Model predictions in a toy example where participants solve 10 problems (a) when the standard deviation on ability (σθ) is adjusted (σθ = 1 or 2) and (b) when both this and the parameter ϵ are adjusted (σθ= 1, ϵ = 0.35 or ϵ = 0, σθ= 0.5) to reveal comparable results. In the main paper, we consider a single value for the standard deviation of the prior on ability (σθ). As shown in Fig. 1a, increasing the standard deviation of the prior implies more accurate estimation of scores, although some under and over estimation is still present. The pattern of Fig. 1a is similar to the pattern of predictions when changing ϵ. As shown in Fig. 1b, adjustments to either of these parameters can lead to very similar predictions for the relationship between true scores and estimated scores. This is not surprising given that both of these parameters represent uncertainty. Choosing to focus on fitting participants' values of ϵ allows us to capture variation in estimates of correctness on each question. On the other hand, if we were to focus on fitting participants' σθ values, we would be assuming variation in prior beliefs about ability. Given the framing of the Dunning–Kruger effect in terms of sensitivity to errors, we fixed σθ and focused on ϵ in our modeling approach. We have expressed our conclusions in terms consistent with variation in either ϵ or σθ, which affect the degree of updating of prior beliefs in light of evidence.

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jansen, R.A., Rafferty, A.N. & Griffiths, T.L. A rational model of the Dunning–Kruger effect supports insensitivity to evidence in low performers. Nat Hum Behav 5, 756–763 (2021). https://doi.org/10.1038/s41562-021-01057-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-021-01057-0

  • Springer Nature Limited

This article is cited by

Search

Navigation