Performance Criteria

  • Valerie J. HarwoodEmail author
  • Donald M. Stoeckel


The establishment of rigorous, consistent performance criteria for microbial source tracking (MST) methods is essential for their usefulness and widespread acceptance as research and regulatory tools. In this chapter, we focus on performance criteria for library-independent methods, although many aspects of the discussion are applicable to both library-independent and library-dependent methods. We separate these criteria into three levels for ease of discussion: (1) the intrinsic characteristics of the “marker” (target), (2) protocols for generating laboratory data, and (3) field applications. By ensuring that a consistent set of metrics for characteristics such as accuracy and precision be applied to field studies and published works, we can begin to circumscribe the set of MST tools that will be most useful for discriminating among fecal pollution sources in environmental waters.


qPCR Performance Efficiency Accuracy Precision Error 


  1. Ahmed, W., Goonetilleke, A., Powell, D., Chauhan, K., and Gardner, T. (2009a). Comparison of molecular markers to detect fresh sewage in environmental waters. Water Res 43(19), 4908–17.PubMedCrossRefGoogle Scholar
  2. Ahmed, W., Goonetilleke, A., Powell, D., and Gardner, T. (2009b). Evaluation of multiple sewage-associated Bacteroides PCR markers for sewage pollution tracking. Water Res 43(19), 4872–7.PubMedCrossRefGoogle Scholar
  3. Albert, J. M., Munakata-Marr, J., Tenorio, L., and Siegrist, R. L. (2003). Statistical evaluation of bacterial source tracking data obtained by rep-PCR DNA fingerprinting of Escherichia coli. Environ Sci Technol 37(20), 4554–60.PubMedCrossRefGoogle Scholar
  4. Balleste, E., Bonjoch, X., Belanche, L. A., and Blanch, A. R. (2010). Molecular indicators used in the development of predictive models for microbial source tracking. Appl Environ Microbiol 76(6), 1789–95.PubMedCrossRefGoogle Scholar
  5. Bernhard, A. E., and Field, K. G. (2000). A PCR assay To discriminate human and ruminant feces on the basis of host differences in Bacteroides-Prevotella genes encoding 16S rRNA. Appl Environ Microbiol 66(10), 4571–4.PubMedCrossRefGoogle Scholar
  6. Boehm, A. B., Fuhrman, J. A., Mrse, R. D., and Grant, S. B. (2003). Tiered approach for identification of a human fecal pollution source at a recreational beach: case study at Avalon Bay, Catalina Island, California. Environ Sci Technol 37(4), 673–80.PubMedCrossRefGoogle Scholar
  7. Cankar, K., Stebih, D., Dreo, T., Zel, J., and Gruden, K. (2006). Critical points of DNA quantification by real-time PCR--effects of DNA extraction method and sample matrix on quantification of genetically modified organisms. BMC Biotechnol 6, 37.PubMedCrossRefGoogle Scholar
  8. Carson, C. A., Shear, B. L., Ellersieck, M. R., and Schnell, J. D. (2003). Comparison of ribotyping and repetitive extragenic palindromic-PCR for identification of fecal Escherichia coli from humans and animals. Appl Environ Microbiol 69(3), 1836–9.PubMedCrossRefGoogle Scholar
  9. Dombek, P. E., Johnson, L. K., Zimmerley, S. T., and Sadowsky, M. J. (2000). Use of repetitive DNA sequences and the PCR To differentiate Escherichia coli isolates from human and animal sources. Appl Environ Microbiol 66(6), 2572–7.PubMedCrossRefGoogle Scholar
  10. Field, K. G., Chern, E. C., Dick, L. K., Fuhrmann, J., Griffith, J., Holden, P., LaMontagne, M. G., Le, J., Olson, B., and Simonich, M. T. (2003). A comparative study of culture-independent, library-independent genotypic methods of fecal source tracking. Journal of Water and Health 01(4), 181–94.Google Scholar
  11. Gordon, D. M., Bauer, S., and Johnson, J. R. (2002). The genetic structure of Escherichia coli populations in primary and secondary habitats. Microbiology 148(Pt 5), 1513–22.PubMedGoogle Scholar
  12. Gregory, J. B., Litaker, R. W., and Noble, R. T. (2006). Rapid one-step quantitative reverse transcriptase PCR assay with competitive internal positive control for detection of enteroviruses in environmental samples. Appl Environ Microbiol 72(6), 3960–7.PubMedCrossRefGoogle Scholar
  13. Griffith, J. F., Weisberg, S. B., and McGee, C. D. (2003). Evaluation of microbial source tracking methods using mixed fecal sources in aqueous test samples. Journal of Water and Health 01, 141–151.Google Scholar
  14. Hagedorn, C., Robinson, S. L., Filtz, J. R., Grubbs, S. M., Angier, T. A., and Reneau, R. B., Jr. (1999). Determining sources of fecal pollution in a rural Virginia watershed with antibiotic resistance patterns in fecal streptococci. Appl Environ Microbiol 65(12), 5522–31.PubMedGoogle Scholar
  15. Harwood, V. J. (2007). Assumptions and limitations of microbial source tracking methods. In “Microbial Source Tracking” (J. Santo-Domingo, Sadowsky, M., Ed.). ASM Press, Washington, D.C.Google Scholar
  16. Harwood, V. J., Brownell, M., Wang, S., Lepo, J., Ellender, R. D., Ajidahun, A., Hellein, K. N., Kennedy, E., Ye, X., and Flood, C. (2009). Validation and field testing of library-­independent microbial source tracking methods in the Gulf of Mexico. Water Res 43(19), 4812–9.PubMedCrossRefGoogle Scholar
  17. Harwood, V. J., Whitlock, J., and Withington, V. (2000). Classification of antibiotic resistance patterns of indicator bacteria by discriminant analysis: use in predicting the source of fecal contamination in subtropical waters. Appl Environ Microbiol 66(9), 3698–704.PubMedCrossRefGoogle Scholar
  18. Harwood, V. J., Wiggins, B., Hagedorn, C., Ellender, R. D., Gooch, J., Kern, J., Samadpour, M., Chapman, A. C. H., Robinson, B. J., and Thompson, B. C. (2003). Phenotypic library-based microbial source tracking methods: Efficacy in the California collaborative study. J. Water Health 01, 153–66.Google Scholar
  19. Helsel, D. R. (1990). Less than obvious: Statistical treatment of data below detection limit. Environmental Science and Technology (24), 1767–1744.CrossRefGoogle Scholar
  20. Hundesa, A., Bofill-Mas, S., Maluquer de Motes, C., Rodriguez-Manzano, J., Bach, A., Casas, M., and Girones, R. (2010). Development of a quantitative PCR assay for the quantitation of bovine polyomavirus as a microbial source-tracking tool. J Virol Methods 163(2), 385–9.PubMedCrossRefGoogle Scholar
  21. Karlen, Y., McNair, A., Perseguers, S., Mazza, C., and Mermod, N. (2007). Statistical significance of quantitative PCR. BMC Bioinformatics 8, 131.PubMedCrossRefGoogle Scholar
  22. Kildare, B. J., Leutenegger, C. M., McSwain, B. S., Bambic, D. G., Rajal, V. B., and Wuertz, S. (2007). 16S rRNA-based assays for quantitative detection of universal, human-, cow-, and dog-specific fecal Bacteroidales: a Bayesian approach. Water Res 41(16), 3701–15.PubMedCrossRefGoogle Scholar
  23. Kirs, M., Harwood, V. J., Fidler, A. E., Gillespie, P. A., Fyfe, W. R., Blackwood, A. D., and Cornelisen, C. C. (2011). Source tracking faecal contamination in an urbanized and a rural waterway in the Nelson-Tasman region, New Zealand. New Zealand J Freshwater Res 45:53–8.Google Scholar
  24. Koike, S., Krapac, I. G., Oliver, H. D., Yannarell, A. C., Chee-Sanford, J. C., Aminov, R. I., and Mackie, R. I. (2007). Monitoring and source tracking of tetracycline resistance genes in lagoons and groundwater adjacent to swine production facilities over a 3-year period. Appl Environ Microbiol 73(15), 4813–23.PubMedCrossRefGoogle Scholar
  25. Korajkic, A., Badgley, B. D., Brownell, M. J., and Harwood, V. J. (2009). Application of microbial source tracking methods in a Gulf of Mexico field setting. J Appl Microbiol 107(5), 1518–27.PubMedCrossRefGoogle Scholar
  26. Layton, A., McKay, L., Williams, D., Garrett, V., Gentry, R., and Sayler, G. (2006). Development of Bacteroides 16S rRNA gene TaqMan-based real-time PCR assays for estimation of total, human, and bovine fecal pollution in water. Appl Environ Microbiol 72(6), 4214–24.PubMedCrossRefGoogle Scholar
  27. Lebuhn, M., Effenberger, M., Garces, G., Gronauer, A., and Wilderer, P. A. (2004). Evaluating real-time PCR for the quantification of distinct pathogens and indicator organisms in environmental samples. Water Sci Technol 50(1), 263–70.PubMedGoogle Scholar
  28. Lu, J., Santo Domingo, J. W., Hill, S., and Edge, T. A. (2009). Microbial diversity and host-specific sequences of Canada goose feces. Appl Environ Microbiol 75(18), 5919–26.PubMedCrossRefGoogle Scholar
  29. McLain, J. E., Ryu, H., Kabiri-Badr, L., Rock, C. M., and Abbaszadegan, M. (2009). Lack of specificity for PCR assays targeting human Bacteroides 16S rRNA gene: cross-amplification with fish feces. FEMS Microbiol Lett 299:38–43.Google Scholar
  30. McLellan, S. L., Daniels, A. D., and Salmore, A. K. (2003). Genetic characterization of Escherichia coli populations from host sources of fecal pollution by using DNA fingerprinting. Appl Environ Microbiol 69(5), 2587–94.PubMedCrossRefGoogle Scholar
  31. McQuaig, S. M., Scott, T. M., Harwood, V. J., Farrah, S. R., and Lukasik, J. O. (2006). Detection of human-derived fecal pollution in environmental waters by use of a PCR-based human polyomavirus assay. Appl Environ Microbiol 72(12), 7567–74.PubMedCrossRefGoogle Scholar
  32. McQuaig, S. M., Scott, T. M., Lukasik, J. O., Paul, J. H., and Harwood, V. J. (2009). Quantification of human polyomaviruses JC Virus and BK Virus by TaqMan quantitative PCR and comparison to other water quality indicators in water and fecal samples. Appl Environ Microbiol 75(11), 3379–88.PubMedCrossRefGoogle Scholar
  33. Myoda, S. P., Carson, C. A., Fuhrmann, J. J., Hahm, B.-K., Hartel, P. G., Yampara-Iquise, H., Johnson, L., Kuntz, R. L., Nakatsu, C. H., Sadowsky, M. J., and Samadpour, M. (2003). Comparison of genotypic-based microbial source tracking methods requiring a host origin database. Journal of Water and Health 01(4), 167–180.Google Scholar
  34. Noble, R. T., Allen, S. M., Blackwood, A. D., Chu, W., Jiang, S. C., Lovelace, G. L., Sobsey, M. D., Stewart, J. R., and Wait, D. A. (2003). Use of viral pathogens and indicators to differentiate between human and non-human fecal contamination in a microbial source tracking comparison study. J Water Health 1(4), 195–207.PubMedGoogle Scholar
  35. Noble, R. T., Griffith, J. F., Blackwood, A. D., Fuhrman, J. A., Gregory, J. B., Hernandez, X., Liang, X., Bera, A. A., and Schiff, K. (2006). Multitiered approach using quantitative PCR to track sources of fecal pollution affecting Santa Monica Bay, California. Appl Environ Microbiol 72(2), 1604–12.PubMedCrossRefGoogle Scholar
  36. Parveen, S., Portier, K. M., Robinson, K., Edmiston, L., and Tamplin, M. L. (1999). Discriminant analysis of ribotype profiles of Escherichia coli for differentiating human and nonhuman sources of fecal pollution. Appl Environ Microbiol 65(7), 3142–7.PubMedGoogle Scholar
  37. Ramakers, C., Ruijter, J. M., Deprez, R. H., and Moorman, A. F. (2003). Assumption-free analysis of quantitative real-time polymerase chain reaction (PCR) data. Neurosci Lett 339(1), 62–6.PubMedCrossRefGoogle Scholar
  38. Ritter, K. J., Carruthers, E., Carson, C. A., Ellender, R. D., Harwood, V. J., Kingsley, K., Nakatsu, C., Sadowsky, M., Shear, B., West, B., Whitlock, J. E., Wiggins, B. A., and Wilbur, J. D. (2003). Assessment of statistical methods used in library-based approaches to microbial source tracking. J Water Health 1(4), 209–23.PubMedGoogle Scholar
  39. Robinson, B. J., Ritter, K. J., and Ellender, R. D. (2007). A statistical appraisal of disproportional versus proportional microbial source tracking libraries. J Water Health 5(4), 503–9.PubMedCrossRefGoogle Scholar
  40. Sen, K., Schable, N. A., and Lye, D. J. (2007). Development of an internal control for evaluation and standardization of a quantitative PCR assay for detection of Helicobacter pylori in drinking water. Appl Environ Microbiol 73(22), 7380–7.PubMedCrossRefGoogle Scholar
  41. Seurinck, S., Defoirdt, T., Verstraete, W., and Siciliano, S. D. (2005). Detection and quantification of the human-specific HF183 Bacteroides 16S rRNA genetic marker with real-time PCR for assessment of human faecal pollution in freshwater. Environ Microbiol 7(2), 249–59.PubMedCrossRefGoogle Scholar
  42. Shanks, O. C., Domingo, J. W., Lu, J., Kelty, C. A., and Graham, J. E. (2007). Identification of bacterial DNA markers for the detection of human fecal pollution in water. Appl Environ Microbiol 73(8), 2416–22.PubMedCrossRefGoogle Scholar
  43. Shanks, O. C., Kelty, C. A., Sivaganesan, M., Varma, M., and Haugland, R. A. (2009). Quantitative PCR for genetic markers of human fecal pollution. Appl Environ Microbiol 75(17), 5507–13.PubMedCrossRefGoogle Scholar
  44. Shanks, O. C., White, K., Kelty, C. A., Sivaganesan, M., Blannon, J., Meckes, M., Varma, M., and Haugland, R. A. (2010). Performance of PCR-based assays targeting Bacteroidales genetic markers of human fecal pollution in sewage and fecal samples. Environ Sci Technol 44(16), 6281–8.PubMedCrossRefGoogle Scholar
  45. Siefring, S., Varma, M., Atikovic, E., Wymer, L., and Haugland, R. A. (2008). Improved real-time PCR assays for the detection of fecal indicator bacteria in surface waters with different instrument and reagent systems. J Water Health 6(2), 225–37.PubMedCrossRefGoogle Scholar
  46. Stoeckel, D. M., and Harwood, V. J. (2007). Performance, design, and analysis in microbial source tracking studies. Appl Environ Microbiol 73(8), 2405–15.PubMedCrossRefGoogle Scholar
  47. Stoeckel, D. M., Mathes, M. V., Hyer, K. E., Hagedorn, C., Kator, H., Lukasik, J., O’Brien, T. L., Fenger, T. W., Samadpour, M., Strickler, K. M., and Wiggins, B. A. (2004). Comparison of seven protocols to identify fecal contamination sources using Escherichia coli. Environ Sci Technol 38(22), 6109–17.PubMedCrossRefGoogle Scholar
  48. Stoeckel, D. M., Stelzer, E. A., and Dick, L. K. (2009). Evaluation of two spike-and-recovery controls for assessment of extraction efficiency in microbial source tracking studies. Water Res 43(19), 4820–7.PubMedCrossRefGoogle Scholar
  49. U.S. Environmental Protection Agency (2004). Quality assurance/quality control guidance for laboratories performing PCR analyses on environmental samples. U.S. Environmental Protection Agency. EPA 815-B-04–001.Google Scholar
  50. U.S. Environmental Protection Agency (2005). Microbial source tracking guide document. U.S. Environmental Protection Agency. EPA/600/R-05/064. June 2005.Google Scholar
  51. Vogel, J. R., Stoeckel, D. M., Lamendella, R., Zelt, R. B., Santo Domingo, J. W., Walker, S. R., and Oerther, D. B. (2007). Identifying fecal sources in a selected catchment reach using multiple source-tracking tools. J Environ Qual 36(3), 718–29.PubMedCrossRefGoogle Scholar
  52. Weidhaas, J. L., Macbeth, T. W., Olsen, R. L., Sadowsky, M. J., Norat, D., and Harwood, V. J. (2010). Identification of a Brevibacterium marker gene specific to poultry litter and development of a quantitative PCR assay. J Appl Microbiol 109:334  –  47.PubMedGoogle Scholar
  53. Wiggins, B. A. (1996). Discriminant analysis of antibiotic resistance patterns in fecal streptococci, a method to differentiate human and animal sources of fecal pollution in natural waters. Appl Environ Microbiol 62(11), 3997–4002.PubMedGoogle Scholar
  54. Wiggins, B. A., Cash, P. W., Creamer, W. S., Dart, S. E., Garcia, P. P., Gerecke, T. M., Han, J., Henry, B. L., Hoover, K. B., Johnson, E. L., Jones, K. C., McCarthy, J. G., McDonough, J. A., Mercer, S. A., Noto, M. J., Park, H., Phillips, M. S., Purner, S. M., Smith, B. M., Stevens, E. N., and Varner, A. K. (2003). Use of antibiotic resistance analysis for representativeness testing of multiwatershed libraries. Appl Environ Microbiol 69(6), 3399  –  405.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Department of Integrative BiologyUniversity of South FloridaTampaUSA

Personalised recommendations