Skip to main content
SpringerLink
Log in

Can Volunteers Collect Data that are Comparable to Professional Scientists? A Study of Variables Used in Monitoring the Outcomes of Ecosystem Rehabilitation

  • Published:
Environmental Management Aims and scope Submit manuscript

Abstract

Having volunteers collect data can be a cost-effective strategy to complement or replace those collected by scientists. The quality of these data is essential where field-collected data are used to monitor progress against predetermined standards because they provide decision makers with confidence that choices they make will not cause more harm than good. The integrity of volunteer-collected data is often doubted. In this study, we made estimates of seven vegetation attributes and a composite measure of six of those seven, to simulate benchmark values. These attributes are routinely recorded as part of rehabilitation projects in Australia and elsewhere in the world. The degree of agreement in data collected by volunteers was compared with those recorded by professional scientists. Combined results showed that scientists collected data that was in closer agreement with benchmarks than those of volunteers, but when data collected by individuals were analyzed, some volunteers collected data that were in similar or closer agreement, than scientists. Both groups’ estimates were in closer agreement for particular attributes than others, suggesting that some attributes are more difficult to estimate than others, or that some are more subjective than others. There are a number of ways in which higher degrees of agreement could be achieved and introducing these will no doubt result in better, more effective programs, to monitor rehabilitation activities. Alternatively, less subjective measures should be sought when developing monitoring protocols. Quality assurance should be part of developing monitoring methods and explicitly budgeted for in project planning to prevent misleading declarations of rehabilitation success.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Agresti A (1984) Analysis of ordinal categorical data. Wiley, New York

    Google Scholar 

  • Andrew MH, Noble IR, Lange RT (1979) A non-destructive method for estimating the weight of forage on shrubs. Australian Rangeland Journal 1:225–231

    Article  Google Scholar 

  • Anonymous (1997) New South Wales State of the Environment 1997. Environment Protection Authority, Sydney, NSW

  • Anonymous (2003) Integrated Catchment Management Plan for the Hunter Catchment 2002. NSW Department of Land and Water Conservation

  • Anonymous (2006) Hunter-Central Rivers Catchment Management Authority annual report 2005–2006. http://www.hcr.cma.nsw.gov.au/pubs/annualreport_0506.pdf. Accessed 17 Nov 2009

  • Anonymous (2007) Keeping a project journal—guidelines for planning, monitoring and evaluating a land management project. Hunter-Central Rivers Catchment Management Authority. http://www.hcr.cma.nsw.gov.au/ep_resources.php3. Accessed 17 Nov 2009

  • Ashcroft M, Gollan JR, Batley M (2012) Combining citizen science, bioclimatic envelope models and observed habitat preferences to determine the distribution of an inconspicuous, recently detected introduced bee (Halictus smaragdulus Vachal Hymenoptera: Halictidae) in Australia. Biological Invasions 14:515–527

    Article  Google Scholar 

  • Boudreau SA, Yan ND (2004) Auditing the accuracy of a volunteer-based surveillance program for an aquatic invader Bythotrephes. Environmental Monitoring and Assessment 91:17–26

    Article  Google Scholar 

  • Boylen CW, Howe EA, Bartkowski JS, Eichler LW (2004) Augmentation of a long-term monitoring program for Lake George, NY by citizen volunteers. Lake and Reservoir Management 20(2):121–129

    Article  Google Scholar 

  • Brandon A, Spyreas G, Molano-Flores B, Carroll C, Ellis J (2003) Can volunteers provide reliable data for forest vegetation surveys? Natural Areas Journal 23:254–262

    Google Scholar 

  • Brooks SS, Lake PS (2007) River restoration in Victoria, Australia: Change is in the wind, and none too soon. Restoration Ecology 15(3):584–591

    Article  Google Scholar 

  • Carr AJL (2004) Why do we all need community science? Society and Natural Resources 17:841–849

    Article  Google Scholar 

  • Cohn JP (2008) Citizen science: can volunteers do real research? BioScience 58(3):192–197

    Article  Google Scholar 

  • Crall AW, Newman GJ, Jarnevich CS, Stohlgren TJ, Waller DM, Graham J (2010) Improving and integrating data on invasive species collected by citizen scientists. Biological Invasions 12(10):3419–3428

    Article  Google Scholar 

  • Curtis A, Robertson A, Race D (1998) Lessons from recent evaluations of natural resource management programs in Australia. Australian Journal of Environmental Management 5:109–119

    Google Scholar 

  • Danson FM, Hetherington D, Morsdorf F, Koetz B, Allgöwer B (2007) Forest canopy gap fraction from terrestrial laser scanning. IEEE Geoscience and Remote Sensing 4:157–160

    Article  Google Scholar 

  • Delaney DG, Sperling CD, Adams CS, Leung B (2008) Marine invasive species: validation of citizen science and implications for national monitoring networks. Biological Invasions 10(1):117–128

    Article  Google Scholar 

  • Engel SR, Voshell JR (2002) Volunteer biological monitoring: can it accurately assess the ecological condition of streams? American Entomologist 48:164–177

    Google Scholar 

  • Environmental Protection Agency, EPA (2002) Methods for evaluating wetland condition: volunteers and wetland biomonitoring. EPA, Washington, D.C

  • Forbes GA, Korva KT (1994) The effect of using a Horsfall–Barratt scale on precision and accuracy of visual estimation of potato late blight severity in the field. Plant Pathology 43:675–682

    Article  Google Scholar 

  • Fore LS, Paulsen K, O’Laughlin K (2001) Assessing the performance of volunteers in monitoring streams. Freshwater Biology 46:109–123

    Google Scholar 

  • Gardner H (1993) Multiple intelligences: the theory in practice. Basic Books, New York

    Google Scholar 

  • Gibbons P, Freudenberger D (2006) An overview of methods used to assess vegetation condition at the scale of the site. Ecological Management and Restoration 7(S1):S10–S17

    Article  Google Scholar 

  • Gilbert M, Gregoire JC (2003) Visual, semi-quantitative assessments allow accurate estimates of leafminer population densities: an example comparing image processing and visual evaluation of damage by the horse chestnut leafminer Cameraria ohridella (Lep., Gracillariidae). Journal of Applied Entomology 127:354–359

    Article  Google Scholar 

  • Giller PS (2005) River restoration: seeking ecological standards, Editor’s introduction. Journal of Applied Ecology 42:201–207

    Article  Google Scholar 

  • Gollan JR, Ashcroft MB, Cassis G, Donnelly AP, Lassau SL (2009) Testing common habitat based surrogates in a semi arid rangeland. Biodiversity and Conservation 18:1147–1159

    Article  Google Scholar 

  • Gorrod EJ, Keith DA (2009) Observer variation in field assessments of vegetation condition: implications for biodiversity conservation. Ecological Management and Restoration 10(1):31–40

    Article  Google Scholar 

  • Jansen A, Robertson A, Thompson L, Wilson A (2004) Development and application of a method for the rapid appraisal of riparian condition. River management technical guideline no. 4, land and water Australia, Canberra, ACT

  • Kershner JL (1997) Monitoring and adaptive management. In: Williams JE, Dombeck MP, Wood CA (eds) Watershed restoration: principles and practices. American Fisheries Society Special Publication, Bethesda, pp 116–134

    Google Scholar 

  • King WH, Woolmington ER (1960) The role of the river in the development of settlement in the lower Hunter Valley. Australian Geographer 8:3–16

    Article  Google Scholar 

  • Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174

    Article  CAS  Google Scholar 

  • Lemmon PE (1956) A spherical densiometer for estimating forest overstorey density. Forest Science 2:314–332

    Google Scholar 

  • McElhinny C, Gibbons P, Brack C, Bauhus J (2005) Forest and woodland stand structural complexity: its definition and measurement. Forest Ecology and Management 218:1–24

    Article  Google Scholar 

  • McLaren MA, Cadman MD (1999) Can novice volunteers provide credible data for bird surveys requiring song identification? Journal of Field Ornithology 70(4):481–490

    Google Scholar 

  • Middleton JV (2001) The stream doctor project: community-driven stream restoration. BioScience 51:293–296

    Article  Google Scholar 

  • Miles I, Sullivan WC, Kuo FE (1998) Ecological restoration volunteers: the benefits of participation. Urban Ecosystems 2(1):27–41

    Article  Google Scholar 

  • Motulsky HJ (2003) Prism 4 statistics guide—statistical analyses for laboratory and clinical researchers. GraphPad Software Inc., San Diego

    Google Scholar 

  • Nerbonne JF, Vondracek B (2003) Volunteer macroinvertebrate monitoring: assessing training needs through examining error and bias in untrained volunteers. Journal of the North American Benthological Society 22(1):152–163

    Article  Google Scholar 

  • Newman C, Buesching CD, Macdonald W (2003) Validating mammal monitoring methods and assessing the performance of volunteers in wildlife conservation–”Sed quis custodiet ipsos custodies?”. Biological Conservation 113:189–197

    Article  Google Scholar 

  • Newman G, Crall A, Laituri M, Graham J, Stohlgren T, Moore JC, Kodrich K, Holfelder KA (2010) Teaching citizen science skills online: implications for invasive species training programs. Applied Environmental Education and Communication 9(4):276–286

    Article  Google Scholar 

  • Noss RF (1990) Indicators for monitoring biodiversity: a hierarchical approach. Conservation Biology 4:355–364

    Article  Google Scholar 

  • Oliver I, Pearce S, Greenslade PM, Britton DR (2006) Contribution of paddock trees to the conservation of terrestrial invertebrate biodiversity within grazed native pastures. Austral Ecology 31:1–12

    Article  Google Scholar 

  • Ruiz-Jaen MC, Mitchell Aide T (2005) Rehabilitation success: how is it being measured? Rehabilitation Ecology 13(3):569–577

    Article  Google Scholar 

  • Scion Corporation (2000) Scion Image for Windows. http://www.scioncorp.com/pages/scion_image_windows.htm. Accessed 17 Nov 2009

  • Seidel D, Fleck S, Leuschner C (2012) Analyzing forest canopies with ground-based laser scanning: A comparison with hemispherical photography. Agricultural and Forest Meteorology 154–155:1–8

    Article  Google Scholar 

  • Stadel AV, Nelson JG (1995) The role of citizen participation in ecosystem monitoring. In: Herman T, Bondrup-Nelson S, Martin Willison JH, Munro NWP (eds). Second International conference on science and the management of protected areas. Dalhousie University, Halifax, Nova Scotia, Canada, pp 447–453

  • Stokes P, Havas M, Bridges T (1990) Public participation and volunteer help in monitoring programs: An assessment. Environmental Monitoring and Assessment 15:225–229

    Article  Google Scholar 

  • Story R (1963) Vegetation of the Hunter Valley. Commonwealth Scientific and Industrial Research Organization, Land Research Series Rep. No. 8

  • Ten Kate K, Bishop J, Bayon R (2004) Biodiversity offsets: views, experience, and the business case—executive summary. IUCN, Gland, Switzerland and Cambridge, UK and Insight Investment, London, UK. http://www.forest-trends.org/publication_details.php?publicationID=661. Accessed 17 Nov 2009

  • Thompson AA, Mapstone BD (1997) Observer effects and training in underwater visual surveys of reef fishes. Marine Ecology Progress Series 154:53–63

    Article  Google Scholar 

  • Underwood AJ, Chapman MG (2002) Conservation of coastal organisms depends on scientific realism, not community ‘monitoring’. In: Lunney D, Dickman C, Burgin S (eds) A clash of paradigms: community and research-based conservation. Royal Zoological Society of New South Wales, Mosman, pp 20–37

    Google Scholar 

  • USDA (2003) Conservation reserve program. US Department of Agriculture, Farm Service Agency, Washington, DC. http://www.fsa.usda.gov/FSA/webapp?area=homeandsubject=coprandtopic=cr. Accessed 17 Nov 2009

  • Vos P, Meelis E, Keurs WJT (2000) A framework for the design of ecological monitoring programs as a tool for environmental and nature management. Environmental Monitoring and Assessment 61:317–344

    Article  CAS  Google Scholar 

  • Wells SM (1995) Reef assessment and monitoring using volunteers and non-professionals. Publication of Rosential School of Marine and Atmospheric Science, University of Miami, Miami, p 57

    Google Scholar 

  • Whitelaw G, Vaughan H, Craig B, Atkinson D (2003) Establishing the Canadian community monitoring network. Environmental Monitoring and Assessment 88:409–418

    Article  Google Scholar 

  • Wilson A, Jansen A, Curtis A, Robertson A (2006) Measuring riparian condition: a comparison of assessments by landholders and scientists. Ecological Management and Restoration 7:123–129

    Article  Google Scholar 

  • Zar J (1999) Biostatistical analysis, 4th ed. Prentice Hall, Upper Saddle River

Download references

Acknowledgments

This project was supported by the NSW Environmental Trust (Grant 2003/RD/001) to L.W. The Coal and Allied Community Trust also provided financial support as part of the BugWise project. We thank Conservation Volunteers Australia, Green Corp volunteers and Australian Museum scientists for their time in collecting data. The Upper Hunter River Rehabilitation Initiative provided in-kind support. M Bulbert and H Smith assisted in the field. We also thank three anonymous reviewers for providing comments that led to improvements to the manuscript.

Conflict of interest

All work presented complied with current Australian law and the authors declare that they have no conflict of interest.

Author information

Authors and Affiliations

  1. Australian Musuem, 6 College St, Sydney, NSW, 2010, Australia

    John Gollan & Lance Wilkie

  2. School of the Environment, University of Technology, Sydney, PO Box 123, Sydney, NSW, 2007, Australia

    John Gollan

  3. Ecosystem Management, School of Environmental and Rural Science, University of New England, Armidale, NSW, Australia

    Lisa Lobry de Bruyn & Nick Reid

Authors
  1. John Gollan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lisa Lobry de Bruyn
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nick Reid
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lance Wilkie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to John Gollan.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOC 440 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gollan, J., de Bruyn, L.L., Reid, N. et al. Can Volunteers Collect Data that are Comparable to Professional Scientists? A Study of Variables Used in Monitoring the Outcomes of Ecosystem Rehabilitation. Environmental Management 50, 969–978 (2012). https://doi.org/10.1007/s00267-012-9924-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00267-012-9924-4

Keywords

Search

Navigation