Psychonomic Bulletin & Review

, Volume 21, Issue 5, pp 1157–1164

Robust misinterpretation of confidence intervals

  • Rink Hoekstra
  • Richard D. Morey
  • Jeffrey N. Rouder
  • Eric-Jan Wagenmakers
Brief Report

DOI: 10.3758/s13423-013-0572-3

Cite this article as:
Hoekstra, R., Morey, R.D., Rouder, J.N. et al. Psychon Bull Rev (2014) 21: 1157. doi:10.3758/s13423-013-0572-3

Abstract

Null hypothesis significance testing (NHST) is undoubtedly the most common inferential technique used to justify claims in the social sciences. However, even staunch defenders of NHST agree that its outcomes are often misinterpreted. Confidence intervals (CIs) have frequently been proposed as a more useful alternative to NHST, and their use is strongly encouraged in the APA Manual. Nevertheless, little is known about how researchers interpret CIs. In this study, 120 researchers and 442 students—all in the field of psychology—were asked to assess the truth value of six particular statements involving different interpretations of a CI. Although all six statements were false, both researchers and students endorsed, on average, more than three statements, indicating a gross misunderstanding of CIs. Self-declared experience with statistics was not related to researchers’ performance, and, even more surprisingly, researchers hardly outperformed the students, even though the students had not received any education on statistical inference whatsoever. Our findings suggest that many researchers do not know the correct interpretation of a CI. The misunderstandings surrounding p-values and CIs are particularly unfortunate because they constitute the main tools by which psychologists draw conclusions from data.

Keywords

Confidence intervalsSignificance testingInference

Copyright information

© Psychonomic Society, Inc. 2014

Authors and Affiliations

  • Rink Hoekstra
    • 1
  • Richard D. Morey
    • 1
  • Jeffrey N. Rouder
    • 2
  • Eric-Jan Wagenmakers
    • 1
  1. 1.University of GroningenGroningenThe Netherlands
  2. 2.University of MissouriColumbiaUSA