Environmental Management

, Volume 50, Issue 5, pp 969–978

Can Volunteers Collect Data that are Comparable to Professional Scientists? A Study of Variables Used in Monitoring the Outcomes of Ecosystem Rehabilitation

  • John Gollan
  • Lisa Lobry de Bruyn
  • Nick Reid
  • Lance Wilkie


Having volunteers collect data can be a cost-effective strategy to complement or replace those collected by scientists. The quality of these data is essential where field-collected data are used to monitor progress against predetermined standards because they provide decision makers with confidence that choices they make will not cause more harm than good. The integrity of volunteer-collected data is often doubted. In this study, we made estimates of seven vegetation attributes and a composite measure of six of those seven, to simulate benchmark values. These attributes are routinely recorded as part of rehabilitation projects in Australia and elsewhere in the world. The degree of agreement in data collected by volunteers was compared with those recorded by professional scientists. Combined results showed that scientists collected data that was in closer agreement with benchmarks than those of volunteers, but when data collected by individuals were analyzed, some volunteers collected data that were in similar or closer agreement, than scientists. Both groups’ estimates were in closer agreement for particular attributes than others, suggesting that some attributes are more difficult to estimate than others, or that some are more subjective than others. There are a number of ways in which higher degrees of agreement could be achieved and introducing these will no doubt result in better, more effective programs, to monitor rehabilitation activities. Alternatively, less subjective measures should be sought when developing monitoring protocols. Quality assurance should be part of developing monitoring methods and explicitly budgeted for in project planning to prevent misleading declarations of rehabilitation success.


Benchmark Citizen science Cost-effective Data collection Data credibility 

Supplementary material

267_2012_9924_MOESM1_ESM.doc (440 kb)
Supplementary material 1 (DOC 440 kb)


  1. Agresti A (1984) Analysis of ordinal categorical data. Wiley, New YorkGoogle Scholar
  2. Andrew MH, Noble IR, Lange RT (1979) A non-destructive method for estimating the weight of forage on shrubs. Australian Rangeland Journal 1:225–231CrossRefGoogle Scholar
  3. Anonymous (1997) New South Wales State of the Environment 1997. Environment Protection Authority, Sydney, NSWGoogle Scholar
  4. Anonymous (2003) Integrated Catchment Management Plan for the Hunter Catchment 2002. NSW Department of Land and Water ConservationGoogle Scholar
  5. Anonymous (2006) Hunter-Central Rivers Catchment Management Authority annual report 2005–2006. http://www.hcr.cma.nsw.gov.au/pubs/annualreport_0506.pdf. Accessed 17 Nov 2009
  6. Anonymous (2007) Keeping a project journal—guidelines for planning, monitoring and evaluating a land management project. Hunter-Central Rivers Catchment Management Authority. http://www.hcr.cma.nsw.gov.au/ep_resources.php3. Accessed 17 Nov 2009
  7. Ashcroft M, Gollan JR, Batley M (2012) Combining citizen science, bioclimatic envelope models and observed habitat preferences to determine the distribution of an inconspicuous, recently detected introduced bee (Halictus smaragdulus Vachal Hymenoptera: Halictidae) in Australia. Biological Invasions 14:515–527CrossRefGoogle Scholar
  8. Boudreau SA, Yan ND (2004) Auditing the accuracy of a volunteer-based surveillance program for an aquatic invader Bythotrephes. Environmental Monitoring and Assessment 91:17–26CrossRefGoogle Scholar
  9. Boylen CW, Howe EA, Bartkowski JS, Eichler LW (2004) Augmentation of a long-term monitoring program for Lake George, NY by citizen volunteers. Lake and Reservoir Management 20(2):121–129CrossRefGoogle Scholar
  10. Brandon A, Spyreas G, Molano-Flores B, Carroll C, Ellis J (2003) Can volunteers provide reliable data for forest vegetation surveys? Natural Areas Journal 23:254–262Google Scholar
  11. Brooks SS, Lake PS (2007) River restoration in Victoria, Australia: Change is in the wind, and none too soon. Restoration Ecology 15(3):584–591CrossRefGoogle Scholar
  12. Carr AJL (2004) Why do we all need community science? Society and Natural Resources 17:841–849CrossRefGoogle Scholar
  13. Cohn JP (2008) Citizen science: can volunteers do real research? BioScience 58(3):192–197CrossRefGoogle Scholar
  14. Crall AW, Newman GJ, Jarnevich CS, Stohlgren TJ, Waller DM, Graham J (2010) Improving and integrating data on invasive species collected by citizen scientists. Biological Invasions 12(10):3419–3428CrossRefGoogle Scholar
  15. Curtis A, Robertson A, Race D (1998) Lessons from recent evaluations of natural resource management programs in Australia. Australian Journal of Environmental Management 5:109–119Google Scholar
  16. Danson FM, Hetherington D, Morsdorf F, Koetz B, Allgöwer B (2007) Forest canopy gap fraction from terrestrial laser scanning. IEEE Geoscience and Remote Sensing 4:157–160CrossRefGoogle Scholar
  17. Delaney DG, Sperling CD, Adams CS, Leung B (2008) Marine invasive species: validation of citizen science and implications for national monitoring networks. Biological Invasions 10(1):117–128CrossRefGoogle Scholar
  18. Engel SR, Voshell JR (2002) Volunteer biological monitoring: can it accurately assess the ecological condition of streams? American Entomologist 48:164–177Google Scholar
  19. Environmental Protection Agency, EPA (2002) Methods for evaluating wetland condition: volunteers and wetland biomonitoring. EPA, Washington, D.CGoogle Scholar
  20. Forbes GA, Korva KT (1994) The effect of using a Horsfall–Barratt scale on precision and accuracy of visual estimation of potato late blight severity in the field. Plant Pathology 43:675–682CrossRefGoogle Scholar
  21. Fore LS, Paulsen K, O’Laughlin K (2001) Assessing the performance of volunteers in monitoring streams. Freshwater Biology 46:109–123Google Scholar
  22. Gardner H (1993) Multiple intelligences: the theory in practice. Basic Books, New YorkGoogle Scholar
  23. Gibbons P, Freudenberger D (2006) An overview of methods used to assess vegetation condition at the scale of the site. Ecological Management and Restoration 7(S1):S10–S17CrossRefGoogle Scholar
  24. Gilbert M, Gregoire JC (2003) Visual, semi-quantitative assessments allow accurate estimates of leafminer population densities: an example comparing image processing and visual evaluation of damage by the horse chestnut leafminer Cameraria ohridella (Lep., Gracillariidae). Journal of Applied Entomology 127:354–359CrossRefGoogle Scholar
  25. Giller PS (2005) River restoration: seeking ecological standards, Editor’s introduction. Journal of Applied Ecology 42:201–207CrossRefGoogle Scholar
  26. Gollan JR, Ashcroft MB, Cassis G, Donnelly AP, Lassau SL (2009) Testing common habitat based surrogates in a semi arid rangeland. Biodiversity and Conservation 18:1147–1159CrossRefGoogle Scholar
  27. Gorrod EJ, Keith DA (2009) Observer variation in field assessments of vegetation condition: implications for biodiversity conservation. Ecological Management and Restoration 10(1):31–40CrossRefGoogle Scholar
  28. Jansen A, Robertson A, Thompson L, Wilson A (2004) Development and application of a method for the rapid appraisal of riparian condition. River management technical guideline no. 4, land and water Australia, Canberra, ACTGoogle Scholar
  29. Kershner JL (1997) Monitoring and adaptive management. In: Williams JE, Dombeck MP, Wood CA (eds) Watershed restoration: principles and practices. American Fisheries Society Special Publication, Bethesda, pp 116–134Google Scholar
  30. King WH, Woolmington ER (1960) The role of the river in the development of settlement in the lower Hunter Valley. Australian Geographer 8:3–16CrossRefGoogle Scholar
  31. Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174CrossRefGoogle Scholar
  32. Lemmon PE (1956) A spherical densiometer for estimating forest overstorey density. Forest Science 2:314–332Google Scholar
  33. McElhinny C, Gibbons P, Brack C, Bauhus J (2005) Forest and woodland stand structural complexity: its definition and measurement. Forest Ecology and Management 218:1–24CrossRefGoogle Scholar
  34. McLaren MA, Cadman MD (1999) Can novice volunteers provide credible data for bird surveys requiring song identification? Journal of Field Ornithology 70(4):481–490Google Scholar
  35. Middleton JV (2001) The stream doctor project: community-driven stream restoration. BioScience 51:293–296CrossRefGoogle Scholar
  36. Miles I, Sullivan WC, Kuo FE (1998) Ecological restoration volunteers: the benefits of participation. Urban Ecosystems 2(1):27–41CrossRefGoogle Scholar
  37. Motulsky HJ (2003) Prism 4 statistics guide—statistical analyses for laboratory and clinical researchers. GraphPad Software Inc., San DiegoGoogle Scholar
  38. Nerbonne JF, Vondracek B (2003) Volunteer macroinvertebrate monitoring: assessing training needs through examining error and bias in untrained volunteers. Journal of the North American Benthological Society 22(1):152–163CrossRefGoogle Scholar
  39. Newman C, Buesching CD, Macdonald W (2003) Validating mammal monitoring methods and assessing the performance of volunteers in wildlife conservation–”Sed quis custodiet ipsos custodies?”. Biological Conservation 113:189–197CrossRefGoogle Scholar
  40. Newman G, Crall A, Laituri M, Graham J, Stohlgren T, Moore JC, Kodrich K, Holfelder KA (2010) Teaching citizen science skills online: implications for invasive species training programs. Applied Environmental Education and Communication 9(4):276–286CrossRefGoogle Scholar
  41. Noss RF (1990) Indicators for monitoring biodiversity: a hierarchical approach. Conservation Biology 4:355–364CrossRefGoogle Scholar
  42. Oliver I, Pearce S, Greenslade PM, Britton DR (2006) Contribution of paddock trees to the conservation of terrestrial invertebrate biodiversity within grazed native pastures. Austral Ecology 31:1–12CrossRefGoogle Scholar
  43. Ruiz-Jaen MC, Mitchell Aide T (2005) Rehabilitation success: how is it being measured? Rehabilitation Ecology 13(3):569–577CrossRefGoogle Scholar
  44. Scion Corporation (2000) Scion Image for Windows. http://www.scioncorp.com/pages/scion_image_windows.htm. Accessed 17 Nov 2009
  45. Seidel D, Fleck S, Leuschner C (2012) Analyzing forest canopies with ground-based laser scanning: A comparison with hemispherical photography. Agricultural and Forest Meteorology 154–155:1–8CrossRefGoogle Scholar
  46. Stadel AV, Nelson JG (1995) The role of citizen participation in ecosystem monitoring. In: Herman T, Bondrup-Nelson S, Martin Willison JH, Munro NWP (eds). Second International conference on science and the management of protected areas. Dalhousie University, Halifax, Nova Scotia, Canada, pp 447–453Google Scholar
  47. Stokes P, Havas M, Bridges T (1990) Public participation and volunteer help in monitoring programs: An assessment. Environmental Monitoring and Assessment 15:225–229CrossRefGoogle Scholar
  48. Story R (1963) Vegetation of the Hunter Valley. Commonwealth Scientific and Industrial Research Organization, Land Research Series Rep. No. 8Google Scholar
  49. Ten Kate K, Bishop J, Bayon R (2004) Biodiversity offsets: views, experience, and the business case—executive summary. IUCN, Gland, Switzerland and Cambridge, UK and Insight Investment, London, UK. http://www.forest-trends.org/publication_details.php?publicationID=661. Accessed 17 Nov 2009
  50. Thompson AA, Mapstone BD (1997) Observer effects and training in underwater visual surveys of reef fishes. Marine Ecology Progress Series 154:53–63CrossRefGoogle Scholar
  51. Underwood AJ, Chapman MG (2002) Conservation of coastal organisms depends on scientific realism, not community ‘monitoring’. In: Lunney D, Dickman C, Burgin S (eds) A clash of paradigms: community and research-based conservation. Royal Zoological Society of New South Wales, Mosman, pp 20–37Google Scholar
  52. USDA (2003) Conservation reserve program. US Department of Agriculture, Farm Service Agency, Washington, DC. http://www.fsa.usda.gov/FSA/webapp?area=homeandsubject=coprandtopic=cr. Accessed 17 Nov 2009
  53. Vos P, Meelis E, Keurs WJT (2000) A framework for the design of ecological monitoring programs as a tool for environmental and nature management. Environmental Monitoring and Assessment 61:317–344CrossRefGoogle Scholar
  54. Wells SM (1995) Reef assessment and monitoring using volunteers and non-professionals. Publication of Rosential School of Marine and Atmospheric Science, University of Miami, Miami, p 57Google Scholar
  55. Whitelaw G, Vaughan H, Craig B, Atkinson D (2003) Establishing the Canadian community monitoring network. Environmental Monitoring and Assessment 88:409–418CrossRefGoogle Scholar
  56. Wilson A, Jansen A, Curtis A, Robertson A (2006) Measuring riparian condition: a comparison of assessments by landholders and scientists. Ecological Management and Restoration 7:123–129CrossRefGoogle Scholar
  57. Zar J (1999) Biostatistical analysis, 4th ed. Prentice Hall, Upper Saddle RiverGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • John Gollan
    • 1
    • 2
  • Lisa Lobry de Bruyn
    • 3
  • Nick Reid
    • 3
  • Lance Wilkie
    • 1
  1. 1.Australian MusuemSydneyAustralia
  2. 2.School of the EnvironmentUniversity of Technology, SydneySydneyAustralia
  3. 3.Ecosystem Management, School of Environmental and Rural ScienceUniversity of New EnglandArmidaleAustralia

Personalised recommendations