European Journal of Epidemiology

, Volume 31, Issue 5, pp 443–444 | Cite as

Disengaging from statistical significance

  • Kenneth J. RothmanEmail author

In the marketplace of scientific results, the preferred currency by which results have been valued has been statistical significance, expressed either as a dichotomous label or by the underlying p value, which may be given as a number or an inequality. Like other modern currencies, the value of this one is not inherent but derived from widely held assumptions and expectations. Indeed, reliance on statistical significance is as misplaced as faith in some dubious paper monies. At the risk of stretching the analogy, I suggest that a version of Gresham’s Law has been operating, allowing statistical significance to force out of circulation better ways to analyze data, and leaving us with results that are, all too often, astonishingly misleading.

As the recently published ASA statement [1, 2] indicates, a fundamental flaw of relying on statistical significance for inference is the need to dichotomize all results into those that are significant or not significant. This practice degrades vast...


Estimate Effect Size Statistical Significance Testing Measure Effect Size Fruitful Research False Dichotomy 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    American Statistical Association (ASA). ASA statement on statistical significance and P-values. Am Stat. 2016. doi: 10.1080/00031305.2016.1154108.Google Scholar
  2. 2.
    Greenland S, Senn SJ, Rothman KL, Carlin JB, Poole C, Goodman SN, Altman DG. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol 2016;31:337–50.CrossRefPubMedGoogle Scholar
  3. 3.
    Lang J, Rothman KJ, Cann CI. That confounded P-value. Epidemiology. 1998;9:7–8.CrossRefPubMedGoogle Scholar
  4. 4.
    Poole C. Low P-values or narrow confidence intervals: which are more durable? Epidemiology. 2001;12:291–4.CrossRefPubMedGoogle Scholar
  5. 5.
    Cumming G. Understanding the new statistics: effect sizes, confidence intervals, and meta-analysis. New York: Routledge; 2012.Google Scholar
  6. 6.
    Hauer E. The harm done by tests of significance. Accid Anal Prev. 2004;36:495–500.CrossRefPubMedGoogle Scholar
  7. 7.
    Schmidt M, Rothman KJ. Mistaken inference caused by reliance on and misinterpretation of a significance test. Int J Cardiol. 2014;177:1089–90.CrossRefPubMedGoogle Scholar

Copyright information

© The Author(s) 2016

Authors and Affiliations

  1. 1.Research Triangle InstituteRTI-Health SolutionsResearch Triangle ParkUSA

Personalised recommendations