Dunning–Kruger effects in reasoning: Theoretical implications of the failure to recognize incompetence

  • Gordon Pennycook
  • Robert M. Ross
  • Derek J. Koehler
  • Jonathan A. Fugelsang
Theoretical Review

Abstract

The DunningKruger effect refers to the observation that the incompetent are often ill-suited to recognize their incompetence. Here we investigated potential Dunning–Kruger effects in high-level reasoning and, in particular, focused on the relative effectiveness of metacognitive monitoring among particularly biased reasoners. Participants who made the greatest numbers of errors on the cognitive reflection test (CRT) overestimated their performance on this test by a factor of more than 3. Overestimation decreased as CRT performance increased, and those who scored particularly high underestimated their performance. Evidence for this type of systematic miscalibration was also found on a self-report measure of analytic-thinking disposition. Namely, genuinely nonanalytic participants (on the basis of CRT performance) overreported their “need for cognition” (NC), indicating that they were dispositionally analytic when their objective performance indicated otherwise. Furthermore, estimated CRT performance was just as strong a predictor of NC as was actual CRT performance. Our results provide evidence for Dunning–Kruger effects both in estimated performance on the CRT and in self-reported analytic-thinking disposition. These findings indicate that part of the reason why people are biased is that they are either unaware of or indifferent to their own bias.

Keywords

Decision making High-order cognition Judgment 

Supplementary material

13423_2017_1242_MOESM1_ESM.pdf (283 kb)
ESM 1(PDF 283 kb)

References

  1. Atir, S., Rosenzweig, E., & Dunning, D. (2015). When knowledge knows no bounds: Self-perceived expertise predicts claims of impossible knowledge. Psychological Science, 26, 1295–1303. doi:10.1177/0956797615588195 CrossRefPubMedGoogle Scholar
  2. Banks, A. P., & Hope, C. (2014). Heuristic and analytic processes in reasoning: An event-related potential study of belief bias. Psychophysiology, 51, 290–297. doi:10.1111/psyp.12169 CrossRefPubMedGoogle Scholar
  3. Bonner, C., & Newell, B. R. (2010). In conflict with ourselves? An investigation of heuristic and analytic processes in decision making. Memory & Cognition, 38, 186–196. doi:10.3758/MC.38.2.186 CrossRefGoogle Scholar
  4. Cacioppo, J. T., & Petty, R. E. (1982). The need for cognition. Journal of Personality and Social Psychology, 42, 116–131. doi:10.1037/0022-3514.42.1.116 CrossRefGoogle Scholar
  5. Cacioppo, J. T., Petty, R. E., Feinstein, J. A., & Jarvis, W. B. G. (1996). Dispositional differences in cognitive motivation: The life and times of individuals varying in need for cognition. Psychological Bulletin, 119, 197–253. doi:10.1037/0033-2909.119.2.197 CrossRefGoogle Scholar
  6. Cacioppo, J. T., Petty, R. E., & Kao, C. F. (1984). The efficient assessment of need for cognition. Journal of Personality Assessment, 48, 306–307. doi:10.1207/s15327752jpa4803_13 CrossRefPubMedGoogle Scholar
  7. Conway, P., & Gawronski, B. (2013). Deontological and utilitarian inclinations in moral decision making: A process dissociation approach. Journal of Personality and Social Psychology, 104, 216–235. doi:10.1037/a0031021 CrossRefPubMedGoogle Scholar
  8. Critcher, C. R., & Dunning, D. (2009). How chronic self-views influence (and mislead) self-assessments of task performance: Self-views shape bottom-up experiences with the task. Journal of Personality and Social Psychology, 97, 931–945. doi:10.1037/a0017452 CrossRefPubMedGoogle Scholar
  9. De Neys, W. (2012). Bias and conflict: A case for logical intuitions. Perspectives on Psychological Science, 7, 28–38. doi:10.1177/1745691611429354 CrossRefPubMedGoogle Scholar
  10. De Neys, W. (2014). Conflict detection, dual processes, and logical intuitions: Some clarifications. Thinking & Reasoning, 20, 169–187. doi:10.1080/13546783.2013.854725 CrossRefGoogle Scholar
  11. De Neys, W., Cromheeke, S., & Osman, M. (2011). Biased but in doubt: Conflict and decision confidence. PLoS ONE, 6, e15954. doi:10.1371/journal.pone.0015954 CrossRefPubMedPubMedCentralGoogle Scholar
  12. De Neys, W., & Franssens, S. (2009). Belief inhibition during thinking: Not always winning but at least taking part. Cognition, 113, 45–61. doi:10.1016/j.cognition.2009.07.009 CrossRefPubMedGoogle Scholar
  13. De Neys, W., & Glumicic, T. (2008). Conflict monitoring in dual process theories of thinking. Cognition, 106, 1248–1299. doi:10.1016/j.cognition.2007.06.002 CrossRefPubMedGoogle Scholar
  14. De Neys, W., Moyens, E., & Vansteenwegen, D. (2010). Feeling we’re biased: Autonomic arousal and reasoning conflict. Cognitive, Affective, & Behavioral Neuroscience, 10, 208–216. doi:10.3758/CABN.10.2.208 CrossRefGoogle Scholar
  15. De Neys, W., Rossi, S., & Houdé, O. (2013). Bats, balls, and substitution sensitivity: Cognitive misers are no happy fools. Psychonomic Bulletin & Review, 20, 269–273. doi:10.3758/s13423-013-0384-5 CrossRefGoogle Scholar
  16. De Neys, W., Vartanian, O., & Goel, V. (2008). Smarter than we think: When our brains detect that we are biased. Psychological Science, 19, 483–489. doi:10.1111/j.1467-9280.2008.02113.x CrossRefPubMedGoogle Scholar
  17. Dunning, D., Johnson, K., Ehrlinger, J., & Kruger, J. (2003). Why people fail to recognize their own incompetence. Current Directions in Psychological Science, 12, 83–87. doi:10.1111/1467-8721.01235 CrossRefGoogle Scholar
  18. Ehrlinger, J., & Dunning, D. (2003). How chronic self-views influence (and potentially mislead) estimates of performance. Journal of Personality and Social Psychology, 84, 5–17. doi:10.1037/0022-3514.84.1.5 CrossRefPubMedGoogle Scholar
  19. Epstein, S., Pacini, R., Denes-Raj, V., & Heier, H. (1996). Individual differences in intuitive–experiential and analytical–rational thinking styles. Journal of Personality and Social Psychology, 71, 390–405. doi:10.1037/0022-3514.71.2.390 CrossRefPubMedGoogle Scholar
  20. Evans, J., & Stanovich, K. E. (2013). Dual-process theories of higher cognition: Advancing the debate. Perspectives on Psychological Science, 8, 223–241. doi:10.1177/1745691612460685 CrossRefPubMedGoogle Scholar
  21. Fleischhauer, M., Enge, S., Brocke, B., Ullrich, J., Strobel, A., & Strobel, A. (2010). Same or different? Clarifying the relationship of need for cognition to personality and intelligence. Personality and Social Psychology Bulletin, 36, 82–96. doi:10.1177/0146167209351886 CrossRefPubMedGoogle Scholar
  22. Franssens, S., & De Neys, W. (2009). The effortless nature of conflict detection during thinking. Thinking & Reasoning, 15, 105–128. doi:10.1080/13546780802711185 CrossRefGoogle Scholar
  23. Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic Perspectives, 19, 25–42. doi:10.1257/089533005775196732 CrossRefGoogle Scholar
  24. Gauthier, K. J., Christopher, A. N., Walter, M. I., Mourad, R., & Marek, P. (2006). Religiosity, religious doubt, and the need for cognition: Their interactive relationship with life satisfaction. Journal of Happiness Studies, 7, 139–154. doi:10.1007/s10902-005-1916-0 CrossRefGoogle Scholar
  25. Handley, S. J., & Trippas, D. (2015). Dual processes and the interplay between knowledge and structure: A new parallel processing model. In B. H. Ross (Ed.), The psychology of learning and motivation (Vol. 62, pp. 33–58). San Diego: Elsevier Academic Press. doi:10.1016/bs.plm.2014.09.002 Google Scholar
  26. Johnson, E. D., Tubau, E., & De Neys, W. (2016). The doubting system 1: Evidence for automatic substitution sensitivity. Acta Psychologica, 164, 56–64. doi:10.1016/j.actpsy.2015.12.008 CrossRefPubMedGoogle Scholar
  27. Kahneman, D. (2011). Thinking, fast and slow. New York: Farrar, Straus and Giroux.Google Scholar
  28. Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77, 1121–1134. Retrieved from http://psycnet.apa.orgjournals/psp/77/6/1121.CrossRefPubMedGoogle Scholar
  29. Mata, A., Ferreira, M. B., & Sherman, S. J. (2013). The metacognitive advantage of deliberative thinkers: A dual-process perspective on overconfidence. Journal of Personality and Social Psychology, 105, 353–373. doi:10.1037/a0033640 CrossRefPubMedGoogle Scholar
  30. Mata, A., Fiedler, K., Ferreira, M. B., & Almeida, T. (2013). Reasoning about others’ reasoning. Journal of Experimental Social Psychology, 49, 486–491. doi:10.1016/j.jesp.2013.01.010 CrossRefGoogle Scholar
  31. Mevel., K., Poirel, N., Rossi, S., Cassotti, M., Simon, G., Houdé, O., & De Neys, W. (2015). Bias detection: Response confidence evidence for conflict sensitivity in the ratio bias task. Journal of Cognitive Psychology, 27, 227-237.Google Scholar
  32. Noori, M. (2016). Cognitive reflection as a predictor of susceptibility to behavioral anomalies. Judgment and Decision Making, 11, 114–120.Google Scholar
  33. Pacini, R., & Epstein, S. (1999). The relation of rational and experiential information processing styles to personality, basic beliefs, and the ratio-bias phenomenon. Journal of Personality and Social Psychology, 76, 972–987. doi:10.1037/0022-3514.76.6.972 CrossRefPubMedGoogle Scholar
  34. Pennycook, G., Cheyne, J. A., Barr, N., Koehler, D. J., & Fugelsang, J. A. (2014). Cognitive style and religiosity: The role of conflict detection. Memory & Cognition, 42, 1–10. doi:10.3758/s13421-013-0340-7 CrossRefGoogle Scholar
  35. Pennycook, G., Cheyne, J. A., Koehler, D. J., & Fugelsang, J. A. (2016). Is the cognitive reflection test a measure of both reflection and intuition? Behavior Research Methods, 48, 341–348. doi:10.3758/s13428-015-0576-1 CrossRefPubMedGoogle Scholar
  36. Pennycook, G., Fugelsang, J. A., & Koehler, D. J. (2012). Are we good at detecting conflict during reasoning? Cognition, 124, 101–106. doi:10.1016/j.cognition.2012.04.004 CrossRefPubMedGoogle Scholar
  37. Pennycook, G., Fugelsang, J. A., & Koehler, D. J. (2015a). Everyday consequences of analytic thinking. Current Directions in Psychological Science, 24, 425–432. doi:10.1177/0963721415604610 CrossRefGoogle Scholar
  38. Pennycook, G., Fugelsang, J. A., & Koehler, D. J. (2015b). What makes us think? A three-stage dual-process model of analytic engagement. Cognitive Psychology, 80, 34–72. doi:10.1016/j.cogpsych.2015.05.001 CrossRefPubMedGoogle Scholar
  39. Pennycook, G., & Ross, R. M. (2016). Commentary on: Cognitive reflection vs. calculation in decision making. Frontiers in Psychology, 7, 9. doi:10.3389/fpsyg.2015.00532 PubMedPubMedCentralGoogle Scholar
  40. Petty, R. E., Brinol, P., Loersch, C., & McCaslin, M. J. (2009). The need for cognition. In M. R. Leary & R. H. Hoyle (Eds.), Handbook of individual differences in social behavior (pp. 318–329). New York: Guilford.Google Scholar
  41. Sargent, M. J. (2004). Less thought, more punishment: Need for cognition predicts support for punitive responses to crime. Personality and Social Psychology Bulletin, 30, 1485–1493. doi:10.1177/0146167204264481 CrossRefPubMedGoogle Scholar
  42. Sloman, S. (2014). Two systems of reasoning: An update. In J. W. Sherman, B. Gawronski, & Y. Trope (Eds.), Dual-process theories of the social mind (pp. 69–79). New York: Guilford Press.Google Scholar
  43. Stanovich, K. E. (2012). On the distinction between rationality and intelligence: Implications for understanding individual diff erences in reasoning. In The Oxford handbook of thinking and reasoning (pp. 433–455). Oxford: Oxford University Press.Google Scholar
  44. Stanovich, K. E., & West, R. F. (1998). Individual differences in rational thought. Journal of Experimental Psychology: General, 127, 161–188. doi:10.1037/0096-3445.127.2.161 CrossRefGoogle Scholar
  45. Stanovich, K. E., & West, R. F. (2000). Individual differences in reasoning: Implications for the rationality debate? Behavioral and Brain Sciences, 23, 645–665. doi:10.1017/S0140525X00003435. disc. 665–726.CrossRefPubMedGoogle Scholar
  46. Svedholm, A. M., & Lindeman, M. (2013). The separate roles of the reflective mind and involuntary inhibitory control in gatekeeping paranormal beliefs and the underlying intuitive confusions. British Journal of Psychology, 104, 303–319. doi:10.1111/j.2044-8295.2012.02118.x CrossRefPubMedGoogle Scholar
  47. Thompson, V. A., & Johnson, S. C. (2014). Conflict, metacognition, and analytic thinking. Thinking & Reasoning, 20, 215–244. doi:10.1080/13546783.2013.869763 CrossRefGoogle Scholar
  48. Thompson, V. A., Prowse Turner, J. A., & Pennycook, G. (2011). Intuition, reason, and metacognition. Cognitive Psychology, 63, 107–140. doi:10.1016/j.cogpsych.2011.06.001 CrossRefPubMedGoogle Scholar
  49. Thompson, V. A., Turner, J. A. P., Pennycook, G., Ball, L. J., Brack, H., Ophir, Y., & Ackerman, R. (2013). The role of answer fluency and perceptual fluency as metacognitive cues for initiating analytic thinking. Cognition, 128, 237–251. doi:10.1016/j.cognition.2012.09.012 CrossRefPubMedGoogle Scholar
  50. Thomson, K. S., & Oppenheimer, D. M. (2016). Investigating an alternate form of the cognitive reflection test. Judgment and Decision Making, 11, 99–113.Google Scholar
  51. Toplak, M., West, R., & Stanovich, K. (2011). The cognitive reflection test as a predictor of performance on heuristics-and-biases tasks. Memory & Cognition, 39, 1275–1289. doi:10.3758/s13421-011-0104-1 CrossRefGoogle Scholar
  52. Toplak, M. E., West, R. F., & Stanovich, K. E. (2014). Assessing miserly information processing: An expansion of the cognitive reflection test. Thinking & Reasoning, 20, 147–168. doi:10.1080/13546783.2013.844729 CrossRefGoogle Scholar
  53. Travers, E., Rolison, J. J., & Feeney, A. (2016). The time course of conflict on the cognitive reflection test. Cognition, 150, 109–118. doi:10.1016/j.cognition.2016.01.015 CrossRefPubMedGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2017

Authors and Affiliations

  • Gordon Pennycook
    • 1
  • Robert M. Ross
    • 2
    • 3
  • Derek J. Koehler
    • 4
  • Jonathan A. Fugelsang
    • 4
  1. 1.Department of PsychologyYale UniversityNew HavenUSA
  2. 2.Department of Psychology, Royal HollowayUniversity of LondonLondonUK
  3. 3.ARC Centre of Excellence in Cognition and its DisordersMacquarie UniversitySydneyAustralia
  4. 4.Department of PsychologyUniversity of WaterlooWaterlooCanada

Personalised recommendations