Environmental Management

, Volume 62, Issue 2, pp 183–189 | Cite as

On Abandoning Hypothesis Testing in Environmental Standard Compliance Assessment

  • Song S. QianEmail author
  • Robert J. Miltner


We use basic characteristics of statistical significance test to argue the abandonment of hypothesis testing in environmental standard (or criterion) compliance assessment. The typical sample size used for environmental assessment is small, and the natural variation of many water quality constituent concentrations is high. These conditions lead to low statistical power of the hypothesis tests used in the assessment process. As a result, using hypothesis testing is often inefficient in detecting noncompliance. When a noncompliance is detected, it is frequently due to sampling or other types of error. We illustrate the problems using two examples, through which we argue that these problems cannot be resolved under the current practice of assessing compliance one water at a time. We recommend that the hypothesis testing framework be replaced by a statistical estimation approach, which can more effectively leverage information from assessments on similar waters using a probabilistic assessment approach.


303(d) listing Compliance assessment Nutrient criteria Statistics 



We thank Lei Zheng, David Pfeifer, and two anonymous reviewers for helpful comments and discussion.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.


  1. Chambers PA, McGoldrick DJ, Brua RB, Visa C, Culp JM, Benoy GA (2012) Development of environmental thresholds for nitrogen and phosphorus in streams. J Environ Qual 41(1):7–20CrossRefGoogle Scholar
  2. Efron B, Morris C (1973a) Stein’s estimation rule and its competitors – an empirical Bayes approach. J Am Stat Assoc 68(341):117–130Google Scholar
  3. Efron B, Morris C (1973b) Combining possibly related estimation problems. J R Stat Soc Ser B (Methodol) 35(3):379–421Google Scholar
  4. Efron B, Morris C (1975) Data analysis using Stein’s estimator and its generalizations. J Am Stat Assoc 70(350):311–319CrossRefGoogle Scholar
  5. FDEP Overview of approaches for numeric nutrient criteria development in ma371 rine waters (2011) Technical report, Florida Department of Environmental Protection, Division of Environmental Assessment and Restoration, Standards and Assessment Section, Tallahassee, FLGoogle Scholar
  6. Halsey LG, Curran-Everett G, Vowler SL, Drummond GB (2015) The fickle P value generates irreproducible results. Nat Methods 12:179–185CrossRefGoogle Scholar
  7. Keller AA, Cavallaro L (2008) Assessing the us Clean Water Act 303(d) listing process for determining impairment of a waterbody. J Environ Manag 86:699–711CrossRefGoogle Scholar
  8. Lazzarotto P, Prasuhn V, Butscher E, Crespi C, Flühler H, Stamm C (2005) Phosphorus export dynamics from two swiss grassland catchments. J Hydrol 304(1):139–150CrossRefGoogle Scholar
  9. McElreath R (2016) Statistical rethinking: A Bayesian course with examples in R and Stan. Chapman & Hall/CRC Press, CRC Press, Boca Raton, FLGoogle Scholar
  10. Miltner R (2010) A method and rationale for deriving nutrient criteria for small rivers and streams in Ohio. Environ Manag 45:842–855CrossRefGoogle Scholar
  11. Miltner RJ, Rankin ET (1998) Primary nutrients and the biotic integrity of rivers and streams. Freshw Biol 40:145–158CrossRefGoogle Scholar
  12. Ott WR (1995) Environmental statistics and data analysis. Lewis Publishers, Boca RatonGoogle Scholar
  13. Qian SS (2015) The frequency component of water quality criterion compliance assessment should be data driven. Environ Manag 56(1):24–33CrossRefGoogle Scholar
  14. Qian SS (2016) Environmental and ecological statistics with R, 2nd edn. Chapman and Hall/CRC Press, CRC Press, Boca Raton, FLGoogle Scholar
  15. Qian SS, Stow CA, Cha YK (2015) Implications of Stein’s Paradox for environmental standard compliance assessment. Environ Sci Technol 49(10):5913–5920CrossRefGoogle Scholar
  16. Smith EP, Ye K, Hughes C, Shabman L (2001) Statistical assessment of violations of water quality standards under section 303(d) of the clean water act. Environ Sci Technol 35(3):606–612CrossRefGoogle Scholar
  17. Stephan CE, Mount DI, Hansen DJ, Gentile JR, Chapman GA, Brungs WA (1985) Guidelines for deriving numerical water quality criteria for the protection of aquatic organisms and their uses. US Environmental Protection Agency, Office of Research and Development, Cincinnati, OH, Number 822R85100Google Scholar
  18. Townsend CR, Uhlmann SS, Matthaei CD (2008) Individual and combined responses of stream ecosystems to multiple stressors. J Appl Ecol 45(6):1810–1819CrossRefGoogle Scholar
  19. Wang L, Robertson DM, Garrison PJ (2007) Linkages between nutrients and assemblages of macroinvertebrates and fish in wadeable streams: Implication to nutrient criteria development. Environ Manag 39(2):194–212CrossRefGoogle Scholar
  20. Yoder CO, Rankin ET (1998) The role of biological indicators in a state water quality management process. Environ Monit Assess 51(1–2):61–88CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Environmental SciencesThe University of ToledoToledoUSA
  2. 2.Ohio Environmental Protection AgencyGroveportUSA

Personalised recommendations