, 29:648

Validation of a wetland Rapid Assessment Method: Use of EPA’s level 1-2-3 framework for method testing and refinement

  • Eric D. Stein
  • A. Elizabeth Fetscher
  • Ross P. Clark
  • Adam Wiskind
  • J. Letitia Grenier
  • Martha Sutula
  • Joshua N. Collins
  • Cristina Grosso


Wetland rapid assessment has become popular in a variety of applications. Because rapid assessments rely on observable field indicators as surrogates for direct measures of condition, they must be validated against independent data. Here we present a case study of the validation of the riverine and estuarine modules of the California Rapid Assessment Method (CRAM). We evaluated responsiveness of the method to “good” vs. “poor” wetland condition, ability to represent a range of conditions, internal redundancy, alternative combination rules for constituent metrics, and reproducibility of results. Because no independent, concurrently collected measure of condition directly reflecting the same elements comprising CRAM was available for validation, we demonstrate the use of existing monitoring and assessment data on avian diversity, benthic macroinvertebrate indices, and plant community composition. Results indicate that CRAM is an effective tool for assessing general riverine and estuarine wetland condition based on its correspondence with multiple independent assessments of condition. Reproducibility analysis revealed several problematic metrics where ambiguous language or metric construction led to high inter-team error rates. Addressing these issues improved overall average error to within 5%. This study demonstrates that, when validated, rapid assessment methods provide a meaningful and reliable tool for assessing wetland condition.

Key Words

calibration CRAM method evaluation user consistency weight of evidence wetland condition wetland monitoring 

Literature Cited

  1. Ambrose, R. F., J. C. Callaway, and S. F. Lee. 2006. An evaluation of compensatory mitigation projects permitted under Clean Water Act Section 401 by the California State Water Quality Control Board, 1991–2002. Report to the State Water Resources Control Board. University of California, Los Angeles, CA, USA.Google Scholar
  2. Andreas, B. K., J. J. Mack, and J. S. McCormac. 2004. Floristic Quality Assessment Index (FQAI) for vascular plants and mosses for the State of Ohio. Ohio Environmental Protection Agency, Division of Surface Water, Wetland Ecology Group, Columbus, OH, USA.Google Scholar
  3. Baber, M. J., E. Fleishman, K. J. Babbitt, and T. L. Tarr. 2004. The relationship between wetland hydroperiod and nestedness patterns in assemblages of larval amphibians and predatory macroinvertebrates. Oikos 107: 16–27.CrossRefGoogle Scholar
  4. Bedford, B. L. and E. M. Preston. 1988. Developing a scientific basis for assessing cumulative effects of wetland loss and degradation on landscape functions: status, perspectives, and prospects. Environmental Management 12: 751–72.CrossRefGoogle Scholar
  5. Brinson, M. M. and R. Rheinhardt. 1996. The role of reference wetlands in functional assessment and mitigation. Ecological Applications 6: 69–76.CrossRefGoogle Scholar
  6. Brooks, R. P., T. J. O’Connell, D. H. Wardrop, and L. E. Jackson. 1998. Towards a regional index of biological integrity: the example of forested riparian ecosystems. Environmental Monitoring and Assessment 51: 131–43.CrossRefGoogle Scholar
  7. Brown, M. T. and M. B. Vivas. 2005. Landscape development intensity index. Environmental Monitoring and Assessment 101: 289–309.CrossRefPubMedGoogle Scholar
  8. Bryce, S. A., R. M. Hughes, and P. R. Kaufmann. 2002. Development of a bird integrity index: using bird assemblages as indicators of riparian condition. Environmental Management 30: 294–310.CrossRefPubMedGoogle Scholar
  9. Burton, G. A., Jr., P. M. Chapman, and E. P. Smith. 2002. Weight-of-evidence approaches for assessing ecosystem impairment. Human and Ecological Risk Assessment 8: 1657–73.CrossRefGoogle Scholar
  10. Cohen, M. J., C. R. Lane, K. C. Reiss, J. A. Surdick, E. Bardi, and M. T. Brown. 2005. Vegetation based classification trees for rapid assessment of isolated wetland condition. Ecological Indicators 5: 189–206.CrossRefGoogle Scholar
  11. Collins, J. N., E. D. Stein, M. Sutula, R. Clark, A. E. Fetscher, L. Grenier, C. Grosso, and A. Wiskind. 2006. California rapid assessment method (CRAM) for wetlands and riparian areas. Version 4.5, San Francisco Estuary Institute. Oakland, CA. www.cramwetlands.org.Google Scholar
  12. Cowardin, L., V. Carter, F. Golet, and E. Laroe. 1979. Classification of wetlands and deepwater habitats of the United States. Washington, D. C.: United States Fish and Wildlife Service, Biological Services Program. FWS/OBS-79/31.Google Scholar
  13. DeZwart, D., S. D. Dyer, L. Posthuma, and C. P. Hawkins. 2006. Predictive models attribute effects on fish assemblages to toxicity and habitat alteration. Ecological Applications 16: 1295–1310.CrossRefGoogle Scholar
  14. Fennessy, M. S., A. D. Jacobs, and M. E. Kentula. 2004. Review of rapid methods for assessing wetland condition. National Health and Environmental Effects Research Laboratory, Office of Research and Development, Research Triangle Park, North Carolina, EPA/620/R-04/009.Google Scholar
  15. Fennessy, M. S., A. D. Jacobs, and M. E. Kentula. 2007. An evaluation of rapid methods for assessing the ecological condition of wetlands. Wetlands 27: 543–60.CrossRefGoogle Scholar
  16. Griffith, M. B., B. H. Hill, F. H. McCormick, P. R. Kaufmann, A. T. Herlihy, and A. R. Selle. 2005. Comparative application of indices of biotic integrity based on periphyton, macroinvertebrates, and fish to southern Rocky Mountain streams. Ecological Indicators 5: 117–36.CrossRefGoogle Scholar
  17. Harrington, J. M. 1999. California stream bioassessment procedure. California Department of Fish and Game, Water Pollution Control Laboratory. Rancho Cordova, CA.Google Scholar
  18. Hauer, F. R., B. J. Cook, M. C. Gilbert, E. J. Clairain, Jr., and R. D. Smith. 2002. A regional guidebook for applying the hydrogeomorphic approach to assessing wetland functions of intermontane prairie pothole wetlands in the Northern Rocky Mountains. U.S. Army Engineer Research and Development Center, Vicksburg, MS, Technical Report ERDC/EL TR-02-7.Google Scholar
  19. Hawkins, C. P. 2006. Quantifying biological integrity by taxonomic completeness: its utility in regional and global assessment. Ecological Applications 16: 1277–94.CrossRefPubMedGoogle Scholar
  20. Hill, A. J., V. S. Neary, and K. L. Morgan. 2006. Hydrologic modeling as a development tool for HGM functional assessment models. Wetlands 6: 161–80.CrossRefGoogle Scholar
  21. Hruby, T., T. Granger, K. Brunner, S. Cooke, K. Dublanica, R. Gersib, L. Reinelt, K. Richter, D. Sheldon, E. Teachout, A. Wald, and F. Weinmann. 1999. Methods for assessing wetland functions volume I: riverine and depressional wetlands in the lowlands of western Washington. Washington State Department Ecology Publication #99–115.Google Scholar
  22. Institute for Bird Populations (IBP). 2006. The monitoring avian productivity and survivorship (MAPS) program, Point Reyes Station, CA, http://www.birdpop.org/maps.htm.Google Scholar
  23. Janssen, P. H. M. and P. S. C. Heuberger. 1995. Calibration of process-oriented models. Ecologic Modeling 83: 55–66.CrossRefGoogle Scholar
  24. Karr, J. R. and E. W. Chu. 1999. Restoring life in running waters: better biological monitoring. Island Press, Washington, DC.Google Scholar
  25. Kentula, M. E. 2007. Monitoring wetlands at the watershed scale. Wetlands 27: 412–15.CrossRefGoogle Scholar
  26. Lee, L. C., P. L. Fiedler, S. R. Stewart, D. J. Partridge, J. A. Mason, E. M. Inlander, and M. C. Rains. 2003. Draft operational guidebook for assessment of the functions of riverine waters/wetlands in the Santa Margarita watershed, Riverside & San Diego counties, California. San Diego Regional Water Quality Control Board, Technical Publication. San Diego, CA.Google Scholar
  27. Linthurst, R. A., L. A. Mulkey, M. W. Slimak, G. D. Veith, and B. M. Levinson. 2000. Ecological research in the office of research and development at the U.S. environmental protection agency: an overview of new directions. Environmental Toxicology and Chemistry 19: 1222–29.CrossRefGoogle Scholar
  28. Mack, J. J. 2001. Vegetation Indices of Biotic Integrity (VIBI) for Wetlands: Ecoregional, Hydrogeomorphic, and Plant Community Comparison with Preliminary Aquatic Life Use Designations. Final Report to U.S. EPA Grant No. CD985875, Volume 1. Wetland Ecology Group, Division of Surface Water, Ohio Environmental Protection Agency, Columbus, OH.Google Scholar
  29. Mack, J. J. 2006. Landscape as a predictor of wetland condition: an evaluation of the landscape development index (LDI) with a large reference wetland dataset from Ohio. Environmental Monitoring and Assessment 120: 221–41.CrossRefPubMedGoogle Scholar
  30. Micacchion, M. 2004. Integrated wetland assessment program. part 7: amphibian index of biotic integrity (AmphIBI) for Ohio wetlands. Ohio Environmental Protection Agency, Wetland Ecology Group, Division of Surface Water, Columbus, OH, Ohio EPA Technical Report WET/2004-7.Google Scholar
  31. Miller, S. W., Q. D. Skinner, and K. J. Reddy. 2004. Stream assessment using biotic indices: responses to physicochemical variables. Journal of the American Water Resources Association 40: 1173–87.CrossRefGoogle Scholar
  32. Ode, P. R., A. C. Rehn, and J. T. May. 2005. A quantitative tool for assessing the integrity of southern coastal California streams. Environmental Management 35: 493–504.CrossRefPubMedGoogle Scholar
  33. Oreskes, N., K. Shraderfrechette, and K. Belitz. 1994. Verification, validation, and confirmation of numerical-models in the earth-sciences. Science 263: 641–46.CrossRefPubMedGoogle Scholar
  34. Reiss, K. C. and M. T. Brown. 2007. Evaluation of Florida palustrine wetlands: application of USEPA levels 1, 2, and 3 assessment methods. EcoHealth 4: 206–18.CrossRefGoogle Scholar
  35. Resh, V. H. and J. K. Jackson. 1993. Rapid assessment approaches to biomonitoring using benthic macroinvertebrates. p. 195–233. In D. M. Rosenberg and V. H. Resh (eds.) Freshwater Biomonitoring and Benthic Macroinvertebrates. Chapman and Hall, New York, NY, USA.Google Scholar
  36. Smith, R. D., A. Ammann, C. Bartoldus, and M. M. Brinson. 1995. An approach for assessing wetland functions using hydrogeomorphic classification, reference wetlands, and functional indices. Waterways Experiment Station, U.S. Army Corps of Engineers, Vicksburg, MS, Technical Report WRP-DE-9.Google Scholar
  37. Smith, E. P., I. Lipkovich, and K. Ye. 2002. Weight-of-evidence (WOE): quantitative estimation of probability of impairment for individual and multiple lines of evidence. Human and Ecological Risk Assessment 8: 1585–96.CrossRefGoogle Scholar
  38. Stapanian, M. A., T. A. Waite, G. Krzys, J. J. Mack, and M. Micacchion. 2004. Rapid assessment indicator of wetland integrity as an unintended predictor of avian diversity. Hydrobiologia 520: 119–26.CrossRefGoogle Scholar
  39. Stoddard, J. L., D. P. Larsen, C. P. Hawkins, R. K. Johnson, and R. H. Norris. 2006. Setting expectations for the ecological condition of streams: the concept of reference condition. Ecological Applications 16: 1267–76.CrossRefPubMedGoogle Scholar
  40. Sutula, M., J. N. Collins, J. Callaway, T. Parker, M. Vasey, and E. Wittner. 2001. Quality assurance project plan (QAPP) for environmental monitoring and assessment program, west coast pilot 2002 intertidal assessment: California intensification. Southern California Coastal Water Research Project. Costa Mesa, CA.Google Scholar
  41. Sutula, M. A., E. D. Stein, J. N. Collins, A. E. Fetscher, and R. Clark. 2006. A Practical Guide for the Development of a Wetland Assessment Method: The California Experience. Journal of the American Water Resources Association 42: 157–75.CrossRefGoogle Scholar
  42. Talmage, P. J., J. A. Perry, and R. M. Goldstein. 2002. Relation of instream habitat and physical conditions to fish communities of agricultural streams in the northern Midwest. North American Journal of Fisheries Management 22: 825–33.CrossRefGoogle Scholar
  43. U.S. Environmental Protection Agency (USEPA). 2002. Research strategy: environmental monitoring and assessment program. Office of Research and Development. Research Triangle Park. p. 78, http://www.epa.gov/emap/html/pubs/docs/resdocs/ resstrat02.html.Google Scholar
  44. Wardrop, D. H., M. E. Kentula, D. L. StevensJr, S. F. Jensen, and R. P. Brooks. 2007. Assessment of wetland condition: an example from the Upper Juniata watershed in Pennsylvania, USA. Wetlands 27: 416–30.CrossRefGoogle Scholar
  45. Whittier, T. R., R. M. Hughes, J. L. Stoddard, G. A. Lomnicky, D. V. Peck, and A. T. Herlihy. 2007. A structured approach for developing indices of biotic integrity: three examples from streams and rivers in the western USA. Transactions of the American Fisheries Society 136: 718–35.CrossRefGoogle Scholar

Copyright information

© Society of Wetland Scientists 2009

Authors and Affiliations

  • Eric D. Stein
    • 1
  • A. Elizabeth Fetscher
    • 1
  • Ross P. Clark
    • 2
  • Adam Wiskind
    • 3
  • J. Letitia Grenier
    • 4
  • Martha Sutula
    • 1
  • Joshua N. Collins
    • 4
  • Cristina Grosso
    • 4
  1. 1.Southern California Coastal Water Research ProjectCosta MesaUSA
  2. 2.California Coastal CommissionSanta CruzUSA
  3. 3.Moss Landing Marine LaboratoriesMoss LandingUSA
  4. 4.San Francisco Estuary InstituteOaklandUSA

Personalised recommendations