Skip to main content

Can Volunteers Collect Data that are Comparable to Professional Scientists? A Study of Variables Used in Monitoring the Outcomes of Ecosystem Rehabilitation

Abstract

Having volunteers collect data can be a cost-effective strategy to complement or replace those collected by scientists. The quality of these data is essential where field-collected data are used to monitor progress against predetermined standards because they provide decision makers with confidence that choices they make will not cause more harm than good. The integrity of volunteer-collected data is often doubted. In this study, we made estimates of seven vegetation attributes and a composite measure of six of those seven, to simulate benchmark values. These attributes are routinely recorded as part of rehabilitation projects in Australia and elsewhere in the world. The degree of agreement in data collected by volunteers was compared with those recorded by professional scientists. Combined results showed that scientists collected data that was in closer agreement with benchmarks than those of volunteers, but when data collected by individuals were analyzed, some volunteers collected data that were in similar or closer agreement, than scientists. Both groups’ estimates were in closer agreement for particular attributes than others, suggesting that some attributes are more difficult to estimate than others, or that some are more subjective than others. There are a number of ways in which higher degrees of agreement could be achieved and introducing these will no doubt result in better, more effective programs, to monitor rehabilitation activities. Alternatively, less subjective measures should be sought when developing monitoring protocols. Quality assurance should be part of developing monitoring methods and explicitly budgeted for in project planning to prevent misleading declarations of rehabilitation success.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. Agresti A (1984) Analysis of ordinal categorical data. Wiley, New York

    Google Scholar 

  2. Andrew MH, Noble IR, Lange RT (1979) A non-destructive method for estimating the weight of forage on shrubs. Australian Rangeland Journal 1:225–231

    Article  Google Scholar 

  3. Anonymous (1997) New South Wales State of the Environment 1997. Environment Protection Authority, Sydney, NSW

  4. Anonymous (2003) Integrated Catchment Management Plan for the Hunter Catchment 2002. NSW Department of Land and Water Conservation

  5. Anonymous (2006) Hunter-Central Rivers Catchment Management Authority annual report 2005–2006. http://www.hcr.cma.nsw.gov.au/pubs/annualreport_0506.pdf. Accessed 17 Nov 2009

  6. Anonymous (2007) Keeping a project journal—guidelines for planning, monitoring and evaluating a land management project. Hunter-Central Rivers Catchment Management Authority. http://www.hcr.cma.nsw.gov.au/ep_resources.php3. Accessed 17 Nov 2009

  7. Ashcroft M, Gollan JR, Batley M (2012) Combining citizen science, bioclimatic envelope models and observed habitat preferences to determine the distribution of an inconspicuous, recently detected introduced bee (Halictus smaragdulus Vachal Hymenoptera: Halictidae) in Australia. Biological Invasions 14:515–527

    Article  Google Scholar 

  8. Boudreau SA, Yan ND (2004) Auditing the accuracy of a volunteer-based surveillance program for an aquatic invader Bythotrephes. Environmental Monitoring and Assessment 91:17–26

    Article  Google Scholar 

  9. Boylen CW, Howe EA, Bartkowski JS, Eichler LW (2004) Augmentation of a long-term monitoring program for Lake George, NY by citizen volunteers. Lake and Reservoir Management 20(2):121–129

    Article  Google Scholar 

  10. Brandon A, Spyreas G, Molano-Flores B, Carroll C, Ellis J (2003) Can volunteers provide reliable data for forest vegetation surveys? Natural Areas Journal 23:254–262

    Google Scholar 

  11. Brooks SS, Lake PS (2007) River restoration in Victoria, Australia: Change is in the wind, and none too soon. Restoration Ecology 15(3):584–591

    Article  Google Scholar 

  12. Carr AJL (2004) Why do we all need community science? Society and Natural Resources 17:841–849

    Article  Google Scholar 

  13. Cohn JP (2008) Citizen science: can volunteers do real research? BioScience 58(3):192–197

    Article  Google Scholar 

  14. Crall AW, Newman GJ, Jarnevich CS, Stohlgren TJ, Waller DM, Graham J (2010) Improving and integrating data on invasive species collected by citizen scientists. Biological Invasions 12(10):3419–3428

    Article  Google Scholar 

  15. Curtis A, Robertson A, Race D (1998) Lessons from recent evaluations of natural resource management programs in Australia. Australian Journal of Environmental Management 5:109–119

    Google Scholar 

  16. Danson FM, Hetherington D, Morsdorf F, Koetz B, Allgöwer B (2007) Forest canopy gap fraction from terrestrial laser scanning. IEEE Geoscience and Remote Sensing 4:157–160

    Article  Google Scholar 

  17. Delaney DG, Sperling CD, Adams CS, Leung B (2008) Marine invasive species: validation of citizen science and implications for national monitoring networks. Biological Invasions 10(1):117–128

    Article  Google Scholar 

  18. Engel SR, Voshell JR (2002) Volunteer biological monitoring: can it accurately assess the ecological condition of streams? American Entomologist 48:164–177

    Google Scholar 

  19. Environmental Protection Agency, EPA (2002) Methods for evaluating wetland condition: volunteers and wetland biomonitoring. EPA, Washington, D.C

  20. Forbes GA, Korva KT (1994) The effect of using a Horsfall–Barratt scale on precision and accuracy of visual estimation of potato late blight severity in the field. Plant Pathology 43:675–682

    Article  Google Scholar 

  21. Fore LS, Paulsen K, O’Laughlin K (2001) Assessing the performance of volunteers in monitoring streams. Freshwater Biology 46:109–123

    Google Scholar 

  22. Gardner H (1993) Multiple intelligences: the theory in practice. Basic Books, New York

    Google Scholar 

  23. Gibbons P, Freudenberger D (2006) An overview of methods used to assess vegetation condition at the scale of the site. Ecological Management and Restoration 7(S1):S10–S17

    Article  Google Scholar 

  24. Gilbert M, Gregoire JC (2003) Visual, semi-quantitative assessments allow accurate estimates of leafminer population densities: an example comparing image processing and visual evaluation of damage by the horse chestnut leafminer Cameraria ohridella (Lep., Gracillariidae). Journal of Applied Entomology 127:354–359

    Article  Google Scholar 

  25. Giller PS (2005) River restoration: seeking ecological standards, Editor’s introduction. Journal of Applied Ecology 42:201–207

    Article  Google Scholar 

  26. Gollan JR, Ashcroft MB, Cassis G, Donnelly AP, Lassau SL (2009) Testing common habitat based surrogates in a semi arid rangeland. Biodiversity and Conservation 18:1147–1159

    Article  Google Scholar 

  27. Gorrod EJ, Keith DA (2009) Observer variation in field assessments of vegetation condition: implications for biodiversity conservation. Ecological Management and Restoration 10(1):31–40

    Article  Google Scholar 

  28. Jansen A, Robertson A, Thompson L, Wilson A (2004) Development and application of a method for the rapid appraisal of riparian condition. River management technical guideline no. 4, land and water Australia, Canberra, ACT

  29. Kershner JL (1997) Monitoring and adaptive management. In: Williams JE, Dombeck MP, Wood CA (eds) Watershed restoration: principles and practices. American Fisheries Society Special Publication, Bethesda, pp 116–134

    Google Scholar 

  30. King WH, Woolmington ER (1960) The role of the river in the development of settlement in the lower Hunter Valley. Australian Geographer 8:3–16

    Article  Google Scholar 

  31. Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174

    Article  CAS  Google Scholar 

  32. Lemmon PE (1956) A spherical densiometer for estimating forest overstorey density. Forest Science 2:314–332

    Google Scholar 

  33. McElhinny C, Gibbons P, Brack C, Bauhus J (2005) Forest and woodland stand structural complexity: its definition and measurement. Forest Ecology and Management 218:1–24

    Article  Google Scholar 

  34. McLaren MA, Cadman MD (1999) Can novice volunteers provide credible data for bird surveys requiring song identification? Journal of Field Ornithology 70(4):481–490

    Google Scholar 

  35. Middleton JV (2001) The stream doctor project: community-driven stream restoration. BioScience 51:293–296

    Article  Google Scholar 

  36. Miles I, Sullivan WC, Kuo FE (1998) Ecological restoration volunteers: the benefits of participation. Urban Ecosystems 2(1):27–41

    Article  Google Scholar 

  37. Motulsky HJ (2003) Prism 4 statistics guide—statistical analyses for laboratory and clinical researchers. GraphPad Software Inc., San Diego

    Google Scholar 

  38. Nerbonne JF, Vondracek B (2003) Volunteer macroinvertebrate monitoring: assessing training needs through examining error and bias in untrained volunteers. Journal of the North American Benthological Society 22(1):152–163

    Article  Google Scholar 

  39. Newman C, Buesching CD, Macdonald W (2003) Validating mammal monitoring methods and assessing the performance of volunteers in wildlife conservation–”Sed quis custodiet ipsos custodies?”. Biological Conservation 113:189–197

    Article  Google Scholar 

  40. Newman G, Crall A, Laituri M, Graham J, Stohlgren T, Moore JC, Kodrich K, Holfelder KA (2010) Teaching citizen science skills online: implications for invasive species training programs. Applied Environmental Education and Communication 9(4):276–286

    Article  Google Scholar 

  41. Noss RF (1990) Indicators for monitoring biodiversity: a hierarchical approach. Conservation Biology 4:355–364

    Article  Google Scholar 

  42. Oliver I, Pearce S, Greenslade PM, Britton DR (2006) Contribution of paddock trees to the conservation of terrestrial invertebrate biodiversity within grazed native pastures. Austral Ecology 31:1–12

    Article  Google Scholar 

  43. Ruiz-Jaen MC, Mitchell Aide T (2005) Rehabilitation success: how is it being measured? Rehabilitation Ecology 13(3):569–577

    Article  Google Scholar 

  44. Scion Corporation (2000) Scion Image for Windows. http://www.scioncorp.com/pages/scion_image_windows.htm. Accessed 17 Nov 2009

  45. Seidel D, Fleck S, Leuschner C (2012) Analyzing forest canopies with ground-based laser scanning: A comparison with hemispherical photography. Agricultural and Forest Meteorology 154–155:1–8

    Article  Google Scholar 

  46. Stadel AV, Nelson JG (1995) The role of citizen participation in ecosystem monitoring. In: Herman T, Bondrup-Nelson S, Martin Willison JH, Munro NWP (eds). Second International conference on science and the management of protected areas. Dalhousie University, Halifax, Nova Scotia, Canada, pp 447–453

  47. Stokes P, Havas M, Bridges T (1990) Public participation and volunteer help in monitoring programs: An assessment. Environmental Monitoring and Assessment 15:225–229

    Article  Google Scholar 

  48. Story R (1963) Vegetation of the Hunter Valley. Commonwealth Scientific and Industrial Research Organization, Land Research Series Rep. No. 8

  49. Ten Kate K, Bishop J, Bayon R (2004) Biodiversity offsets: views, experience, and the business case—executive summary. IUCN, Gland, Switzerland and Cambridge, UK and Insight Investment, London, UK. http://www.forest-trends.org/publication_details.php?publicationID=661. Accessed 17 Nov 2009

  50. Thompson AA, Mapstone BD (1997) Observer effects and training in underwater visual surveys of reef fishes. Marine Ecology Progress Series 154:53–63

    Article  Google Scholar 

  51. Underwood AJ, Chapman MG (2002) Conservation of coastal organisms depends on scientific realism, not community ‘monitoring’. In: Lunney D, Dickman C, Burgin S (eds) A clash of paradigms: community and research-based conservation. Royal Zoological Society of New South Wales, Mosman, pp 20–37

    Google Scholar 

  52. USDA (2003) Conservation reserve program. US Department of Agriculture, Farm Service Agency, Washington, DC. http://www.fsa.usda.gov/FSA/webapp?area=homeandsubject=coprandtopic=cr. Accessed 17 Nov 2009

  53. Vos P, Meelis E, Keurs WJT (2000) A framework for the design of ecological monitoring programs as a tool for environmental and nature management. Environmental Monitoring and Assessment 61:317–344

    Article  CAS  Google Scholar 

  54. Wells SM (1995) Reef assessment and monitoring using volunteers and non-professionals. Publication of Rosential School of Marine and Atmospheric Science, University of Miami, Miami, p 57

    Google Scholar 

  55. Whitelaw G, Vaughan H, Craig B, Atkinson D (2003) Establishing the Canadian community monitoring network. Environmental Monitoring and Assessment 88:409–418

    Article  Google Scholar 

  56. Wilson A, Jansen A, Curtis A, Robertson A (2006) Measuring riparian condition: a comparison of assessments by landholders and scientists. Ecological Management and Restoration 7:123–129

    Article  Google Scholar 

  57. Zar J (1999) Biostatistical analysis, 4th ed. Prentice Hall, Upper Saddle River

Download references

Acknowledgments

This project was supported by the NSW Environmental Trust (Grant 2003/RD/001) to L.W. The Coal and Allied Community Trust also provided financial support as part of the BugWise project. We thank Conservation Volunteers Australia, Green Corp volunteers and Australian Museum scientists for their time in collecting data. The Upper Hunter River Rehabilitation Initiative provided in-kind support. M Bulbert and H Smith assisted in the field. We also thank three anonymous reviewers for providing comments that led to improvements to the manuscript.

Conflict of interest

All work presented complied with current Australian law and the authors declare that they have no conflict of interest.

Author information

Affiliations

Authors

Corresponding author

Correspondence to John Gollan.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOC 440 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Gollan, J., de Bruyn, L.L., Reid, N. et al. Can Volunteers Collect Data that are Comparable to Professional Scientists? A Study of Variables Used in Monitoring the Outcomes of Ecosystem Rehabilitation. Environmental Management 50, 969–978 (2012). https://doi.org/10.1007/s00267-012-9924-4

Download citation

Keywords

  • Benchmark
  • Citizen science
  • Cost-effective
  • Data collection
  • Data credibility