Environmental Science and Pollution Research

, Volume 20, Issue 10, pp 7341–7347 | Cite as

Misuse of null hypothesis significance testing: would estimation of positive and negative predictive values improve certainty of chemical risk assessment?

  • Mirco Bundschuh
  • Michael C. Newman
  • Jochen P. Zubrod
  • Frank Seitz
  • Ricki R. Rosenfeldt
  • Ralf Schulz
Research Article


Although generally misunderstood, the p value is the probability of the test results or more extreme results given H0 is true: it is not the probability of H0 being true given the results. To obtain directly useful insight about H0, the positive predictive value (PPV) and the negative predictive value (NPV) may be useful extensions of null hypothesis significance testing (NHST). They provide information about the probability of statistically significant and non-significant test outcomes being true based on an a priori defined biologically meaningful effect size. The present study explores the utility of PPV and NPV in an ecotoxicological context by using the frequently applied Daphnia magna reproduction test (OECD guideline 211) and the chemical stressor lindane as a model system. The results indicate that especially the NPV deviates meaningfully between a test design strictly following the guideline and an experimental procedure controlling for α and β at the level of 0.05. Consequently, PPV and NPV may be useful supplements to NHST that inform the researcher about the level of confidence warranted by both statistically significant and non-significant test results. This approach also reinforces the value of considering α, β, and a biologically meaningful effect size a priori.


Sample size Bayesian Power analysis Effect size Type I error rate Type II error rate 


  1. Altman DG, Bland JM (1994) Diagnostic tests 2: predictive values. British Med J 309:102CrossRefGoogle Scholar
  2. Altman DG, Machin D, Bryant TN, Gardner MJ (2000) Statistics with confidence, 2nd edn. BMJ Books, BristolGoogle Scholar
  3. Antunes SC, Castro BB, Goncalves F (2004) Effect of food level on the acute and chronic responses of daphnids to lindane. Environ Pollut 127:367–375CrossRefGoogle Scholar
  4. ASTM (2007) ASTM E729-96: Standard guide for conducting acute toxicity tests on test materials with fishes, macroinvertebrates, and amphibians. ASTM International, West Conshohocken, PA, 2007, doi:10.1520/E0729-96R07
  5. Bundschuh M, Zubrod JP, Seitz F, Newman MC, Schulz R (2011) Mercury-contaminated sediments affect amphipod feeding. Arch Environ Contam Toxicol 60:437–443CrossRefGoogle Scholar
  6. Crane M, Newman MC (2000) What level of effect is a no observed effect? Environ Toxicol Chem 19:516–519CrossRefGoogle Scholar
  7. DeCoen WM, Janssen CR (1997) The use of biomarkers in Daphnia magna toxicity testing. IV. cellular energy allocation: a new methodology to assess the energy budget of toxicant-stressed Daphnia populations. J Aquat Ecosyst Stress Recovery 6:43–55CrossRefGoogle Scholar
  8. Dunnett CW (1955) A multiple comparison procedure for comparing several treatments with a control. J Am Stat Assoc 50:1096–1121CrossRefGoogle Scholar
  9. Ferrando MD, Sancho E, Andreumoliner E (1995) Effects of lindane on Daphnia magna during chronic exposure. J Environ Sci Health Part B 30:815–825CrossRefGoogle Scholar
  10. Fox DR (2009) Is the ECx a legitimate surrogate for a NOEC? Integr Environ Assess Manag 5:351–353CrossRefGoogle Scholar
  11. Fox DR (2010) A Bayesian approach for determining the no effect concentration and hazardous concentration in ecotoxicology. Ecotoxicol Environ Saf 73:123–131CrossRefGoogle Scholar
  12. Gigerenzer G (2004) Mindless statistics. J Socio-Econom 33:587–606CrossRefGoogle Scholar
  13. Ioannidis JPA (2005) Why most published research findings are false. Plos Med 2:696–701Google Scholar
  14. Jager T (2012) Bad habits die hard: the NOEC's persistence reflects poorly on ecotoxicology. Environ Toxicol Chem 31:228–229CrossRefGoogle Scholar
  15. Kline RB (2004) Beyond significance testing: reforming data analysis methods in behavioral research. American Psychological Association, WashingtonCrossRefGoogle Scholar
  16. Landis WG, Chapman PM (2011). Well past time to stop using NOELs and LOELs. Integr Environ Assess Manag 7:vi-viiiGoogle Scholar
  17. Mudge JF, Baker LF, Edge CB, Houlahan JE (2012) Setting an optimal alpha that minimizes errors in null hypothesis significance tests. PlosOne 7:e32734CrossRefGoogle Scholar
  18. Nakagawa S, Cuthill IC (2007) Effect size, confidence interval and statistical significance: a practical guide for biologists. Biol Rev 82:591–605CrossRefGoogle Scholar
  19. Nakagawa S, Forster TM (2004) The case against retrospective statisical power analyses with an introduction to power analysis. Acta etholog 7:103–108CrossRefGoogle Scholar
  20. Newman MC (2008) "What exactly are you inferring?" A closer look at hypothesis testing. Environ Toxicol Chem 27:1013–1019CrossRefGoogle Scholar
  21. Newman MC (2013) Quantitative ecotoxicology. CRC/Taylor & Francis, Boca RatonGoogle Scholar
  22. OECD (2008) OECD 211: Daphnia magna reproduction test. OECD Publishing, ParisCrossRefGoogle Scholar
  23. Preuss TG, Hammers-Wirtz M, Ratte HT (2010) The potential of individual based population models to extrapolate effects measured at standardized test conditions to relevant environmental conditions-an example for 3,4-dichloroaniline on Daphnia magna. J Environ Monit 12:2070–2079CrossRefGoogle Scholar
  24. van der Hoeven N (1998) Power analysis for the NOEC: what is the probability of detecting small toxic effects on three different species using the appropriate standardized test protocols? Ecotoxicology 7:355–361CrossRefGoogle Scholar
  25. Wacholder S, Chanock S, Garcia-Closas M, El Ghormli L, Rothman N (2004) Assessing the probability that a positive report is false: an approach for molecular epidemiology studies. J Nation Cancer Inst 96:434–442CrossRefGoogle Scholar
  26. Zubrod JP, Bundschuh M, Feckler A, Englert D, Schulz R (2011) Ecotoxicological impact of the fungicide tebuconazole on an aquatic decomposer-detritivore system. Environ Toxicol Chem 30:2718–2724CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Mirco Bundschuh
    • 1
    • 2
  • Michael C. Newman
    • 3
  • Jochen P. Zubrod
    • 1
  • Frank Seitz
    • 1
  • Ricki R. Rosenfeldt
    • 1
  • Ralf Schulz
    • 1
  1. 1.Institute for Environmental SciencesUniversity of Koblenz-LandauLandauGermany
  2. 2.Department of Aquatic Sciences and AssessmentSwedish University of Agricultural SciencesUppsalaSweden
  3. 3.Virginia Institute of Marine ScienceCollege of William and MaryVirginiaUSA

Personalised recommendations