Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Ending Reliance on Statistical Significance Will Improve Environmental Inference and Communication

Abstract

Recently, there has been much discussion about the role of the p-value in scientific research. The American Statistical Association has published an editorial that presents guidelines for the use and interpretation of p-values. Numerous authors have commented and criticized its use as a means to identify scientific importance of results and have called for an end to using the term “statistical significance.” Recent articles in Estuaries and Coasts were evaluated for reliance on the use of statistical significance and reporting errors were identified. Suggestions are made for improving what is reported related to statistical testing. Focus should be on scientific importance of estimates, estimation of the size of the effect and the certainty in the size of the effect instead of simply reporting a p-value and relying on hypothesis tests.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3

Notes

  1. 1.

    accessed August 20, 2019 articles from Estuaries and Coasts 42(6):1405–1694

References

  1. Amrhein, V., S. Greenland, and B. McShane. 2019. Retire statistical significance. Nature 567: 305–307.

  2. Bopp, J.J., M. Sclafani, D.R. Smith, K.A. McKown, R. Sysak, and R.M. Cerrato. 2019. Geographic-specific capture–recapture models reveal contrasting migration and survival rates of adult horseshoe crabs (Limulus polyphemus). Estuaries and Coasts 42 (6): 1570–1585. https://doi.org/10.1007/s12237-019-00595-1.

  3. Calin-Jageman, R.J., and G. Cumming. 2019. The new statistics for better science: ask how much, how uncertain, and what else is known. The American Statistician 73 (Suppl 1): 271–280. https://doi.org/10.1080/00031305.2018.1518266.

  4. Cohen, J. 1988. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates.

  5. Cumming, G. 2014. The new statistics: Why and how. Psychological Science 25 (1): 7–29. https://doi.org/10.1177/0956797613504966.

  6. Fidler, F., M.A. Burgman, G. Cumming, R. Buttrose, and N. Thomason. 2006. Impact of criticism of null-hypothesis significance testing on statistical reporting practices in Conservation Biology. Conservation Biology 20: 1539–1544.

  7. Fricker, R., K. Burke, X. Han, and W.H. Woodall. 2019. Assessing the statistical analyses used in Basic and Applied Social Psychology after their p-value ban. The American Statistician 73: 374–384.

  8. Goodman, S.N., and J.A. Berlin. 1994. The use of predicted confidence intervals when planning experiments and the misuse of power when interpreting results. Annals of Internal Medicine 121 (3): 200–206.

  9. Halsey, L.G. 2019. The reign of the p-value is over: What alternative analyses could we employ to fill the power vacuum? Biology Letters 15: 20190174.

  10. Harrington, D., R.B. D’Agostino Sr., C. Gatsonis, J.W. Hogan, D.J. Hunter, S.T. Normand, J.M. Drazen, and M.B. Hamel. 2019. New guidelines for statistical reporting in the journal. The New England Journal of Medicine 381 (3): 285–286. https://doi.org/10.1056/NEJMe1906559.

  11. Hurlbert, S.H., R.R. Levine, and J. Utts. 2019. Coup de grâce for a tough old bull: “Statistically significant” expires. The American Statistician 73: 352–357. https://doi.org/10.1080/00031305.2018.1543616.

  12. Ioannidis, J.P. 2005. Why most published research findings are false. PLoS Medicine 2 (8): e124. https://doi.org/10.1371/journal.pmed.0020124.

  13. Ioannidis, J.P. 2019. What have we (not) learnt from millions of scientific papers with p values? The American Statistician 73: 20–25. https://doi.org/10.1080/00031305.2018.1447512.

  14. Krausman, P.R., and A.S. Cox. 2019. Vexing vocabulary in submissions to the Journal of Wildlife Management. The Journal of Wildlife Management 83 (6): 1279–1280. https://doi.org/10.1002/jwmg.21726.

  15. Lenhard, W. and A. Lenhard. 2016. Calculation of effect sizes. Retrieved from: https://www.psychometrica.de/effect_size.html. Dettelbach (Germany): Psychometrica. https://doi.org/10.13140/RG.2.1.3478.4245. Accessed 19 Oct 2019.

  16. Lenth, R.V. 2001. Some practical guidelines for effective sample size determination. The American Statistician 55: 187–193. https://doi.org/10.1198/000313001317098149.

  17. Meehl, P.E. 1978. Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology 4: 806–843.

  18. Millette, N.C., C. Kelble, A. Linhoss, S. Ashby, and L. Visser. 2019. Using spatial variability in the rate of change of chlorophyll a to improve water quality management in a subtropical estuary. Estuaries and Coasts 42 (7): 1792–1803. https://doi.org/10.1007/s12237-019-00610-5.

  19. NISS, 2019. Digging Deeper from the radical to the reasoned: p-value alternatives offered by experts in NISS webinar. https://www.niss.org/news/digging-deeper-radical-reasoned-p-value-alternatives-offered-experts-niss-webinar (accessed 11/26/2019).

  20. Pocock, S.J., G. Bakris, D.L. Rhatt, S. Brar, M. Fahy, and B.J. Gersh. 2016. Regression to the mean in SYMPLICITY HTN-3: Implications for design and reporting of future trials. Journal of the American College of Cardiology 68: 2016–2025. https://doi.org/10.1016/j.jacc.2016.07.775.

  21. Pogrow, S. 2019. How effect size (practical significance) misleads clinical practice: The case for switching to practical benefit to assess applied research findings. The American Statistician 73: 223–234.

  22. R Core Team. 2019. R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing http://www.R-project.org/. Accessed 01 July 2019.

  23. Ritter, M. 2019. Sorry, wrong number: Statistical benchmark comes under fire. Miami Herald. https://www.miamiherald.com/news/article237281119.html. Accessed 11/20/2019.

  24. Rosenthal, R. 1979. The “file drawer problem” and tolerance for null results. Psychological Bulletin 86: 838–641.

  25. Wasserstein, R.L., and N.A. Lazar. 2016. Editorial: The ASA statement on p-values: context, process, and purpose. The American Statistician 70 (2): 129–133.

  26. Wasserstein, R.L., A.L. Schirm, and N.A. Lazar. 2019. Moving to a world beyond “p < 0.05”. The American Statistician 73 (sup1): 1–9. https://doi.org/10.1080/00031305.2019.1583913.

Download references

Acknowledgments

Comments on previous versions of the manuscript by K. Rose, W. Woodall and the reviewers were greatly appreciated. Thanks to V. Amrhein for some recent references on the p-value issue.

Author information

Correspondence to Eric P. Smith.

Additional information

Communicated by Paul A. Montagna

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Smith, E.P. Ending Reliance on Statistical Significance Will Improve Environmental Inference and Communication. Estuaries and Coasts 43, 1–6 (2020). https://doi.org/10.1007/s12237-019-00679-y

Download citation

Keywords

  • p-value
  • Statistical inference
  • Hypothesis testing