Abstract
Although generally misunderstood, the p value is the probability of the test results or more extreme results given H0 is true: it is not the probability of H0 being true given the results. To obtain directly useful insight about H0, the positive predictive value (PPV) and the negative predictive value (NPV) may be useful extensions of null hypothesis significance testing (NHST). They provide information about the probability of statistically significant and non-significant test outcomes being true based on an a priori defined biologically meaningful effect size. The present study explores the utility of PPV and NPV in an ecotoxicological context by using the frequently applied Daphnia magna reproduction test (OECD guideline 211) and the chemical stressor lindane as a model system. The results indicate that especially the NPV deviates meaningfully between a test design strictly following the guideline and an experimental procedure controlling for α and β at the level of 0.05. Consequently, PPV and NPV may be useful supplements to NHST that inform the researcher about the level of confidence warranted by both statistically significant and non-significant test results. This approach also reinforces the value of considering α, β, and a biologically meaningful effect size a priori.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Altman DG, Bland JM (1994) Diagnostic tests 2: predictive values. British Med J 309:102
Altman DG, Machin D, Bryant TN, Gardner MJ (2000) Statistics with confidence, 2nd edn. BMJ Books, Bristol
Antunes SC, Castro BB, Goncalves F (2004) Effect of food level on the acute and chronic responses of daphnids to lindane. Environ Pollut 127:367–375
ASTM (2007) ASTM E729-96: Standard guide for conducting acute toxicity tests on test materials with fishes, macroinvertebrates, and amphibians. ASTM International, West Conshohocken, PA, 2007, doi:10.1520/E0729-96R07
Bundschuh M, Zubrod JP, Seitz F, Newman MC, Schulz R (2011) Mercury-contaminated sediments affect amphipod feeding. Arch Environ Contam Toxicol 60:437–443
Crane M, Newman MC (2000) What level of effect is a no observed effect? Environ Toxicol Chem 19:516–519
DeCoen WM, Janssen CR (1997) The use of biomarkers in Daphnia magna toxicity testing. IV. cellular energy allocation: a new methodology to assess the energy budget of toxicant-stressed Daphnia populations. J Aquat Ecosyst Stress Recovery 6:43–55
Dunnett CW (1955) A multiple comparison procedure for comparing several treatments with a control. J Am Stat Assoc 50:1096–1121
Ferrando MD, Sancho E, Andreumoliner E (1995) Effects of lindane on Daphnia magna during chronic exposure. J Environ Sci Health Part B 30:815–825
Fox DR (2009) Is the ECx a legitimate surrogate for a NOEC? Integr Environ Assess Manag 5:351–353
Fox DR (2010) A Bayesian approach for determining the no effect concentration and hazardous concentration in ecotoxicology. Ecotoxicol Environ Saf 73:123–131
Gigerenzer G (2004) Mindless statistics. J Socio-Econom 33:587–606
Ioannidis JPA (2005) Why most published research findings are false. Plos Med 2:696–701
Jager T (2012) Bad habits die hard: the NOEC's persistence reflects poorly on ecotoxicology. Environ Toxicol Chem 31:228–229
Kline RB (2004) Beyond significance testing: reforming data analysis methods in behavioral research. American Psychological Association, Washington
Landis WG, Chapman PM (2011). Well past time to stop using NOELs and LOELs. Integr Environ Assess Manag 7:vi-viii
Mudge JF, Baker LF, Edge CB, Houlahan JE (2012) Setting an optimal alpha that minimizes errors in null hypothesis significance tests. PlosOne 7:e32734
Nakagawa S, Cuthill IC (2007) Effect size, confidence interval and statistical significance: a practical guide for biologists. Biol Rev 82:591–605
Nakagawa S, Forster TM (2004) The case against retrospective statisical power analyses with an introduction to power analysis. Acta etholog 7:103–108
Newman MC (2008) "What exactly are you inferring?" A closer look at hypothesis testing. Environ Toxicol Chem 27:1013–1019
Newman MC (2013) Quantitative ecotoxicology. CRC/Taylor & Francis, Boca Raton
OECD (2008) OECD 211: Daphnia magna reproduction test. OECD Publishing, Paris
Preuss TG, Hammers-Wirtz M, Ratte HT (2010) The potential of individual based population models to extrapolate effects measured at standardized test conditions to relevant environmental conditions-an example for 3,4-dichloroaniline on Daphnia magna. J Environ Monit 12:2070–2079
van der Hoeven N (1998) Power analysis for the NOEC: what is the probability of detecting small toxic effects on three different species using the appropriate standardized test protocols? Ecotoxicology 7:355–361
Wacholder S, Chanock S, Garcia-Closas M, El Ghormli L, Rothman N (2004) Assessing the probability that a positive report is false: an approach for molecular epidemiology studies. J Nation Cancer Inst 96:434–442
Zubrod JP, Bundschuh M, Feckler A, Englert D, Schulz R (2011) Ecotoxicological impact of the fungicide tebuconazole on an aquatic decomposer-detritivore system. Environ Toxicol Chem 30:2718–2724
Acknowledgments
The authors acknowledge S. Hartmann, T. Schell, and S. Schneider for actually performing the experiments as well as the Fix-Stiftung Landau for the financial support regarding the research infrastructure. J.P. Zubrod received funding through a scholarship of the German Federal Environmental Foundation (Deutsche Bundesstiftung Umwelt).
Author information
Authors and Affiliations
Corresponding author
Additional information
Responsible editor: Philippe Garrigues
Rights and permissions
About this article
Cite this article
Bundschuh, M., Newman, M.C., Zubrod, J.P. et al. Misuse of null hypothesis significance testing: would estimation of positive and negative predictive values improve certainty of chemical risk assessment?. Environ Sci Pollut Res 20, 7341–7347 (2013). https://doi.org/10.1007/s11356-013-1749-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11356-013-1749-z