Skip to main content

Advertisement

SpringerLink
Log in

Accounting for Results: How Conservation Organizations Report Performance Information

  • Published:
Environmental Management Aims and scope Submit manuscript

Abstract

Environmental program performance information is in high demand, but little research suggests why conservation organizations differ in reporting performance information. We compared performance measurement and reporting by four private-land conservation organizations: Partners for Fish and Wildlife in the US Fish and Wildlife Service (national government), Forest Stewardship Council—US (national nonprofit organization), Land and Water Conservation Departments (local government), and land trusts (local nonprofit organization). We asked: (1) How did the pattern of performance reporting relationships vary across organizations? (2) Was political conflict among organizations’ principals associated with greater performance information? and (3) Did performance information provide evidence of program effectiveness? Based on our typology of performance information, we found that most organizations reported output measures such as land area or number of contracts, some reported outcome indicators such as adherence to performance standards, but few modeled or measured environmental effects. Local government Land and Water Conservation Departments reported the most types of performance information, while local land trusts reported the fewest. The case studies suggest that governance networks influence the pattern and type of performance reporting, that goal conflict among principles is associated with greater performance information, and that performance information provides unreliable causal evidence of program effectiveness. Challenging simple prescriptions to generate more data as evidence, this analysis suggests (1) complex institutional and political contexts for environmental program performance and (2) the need to supplement performance measures with in-depth evaluations that can provide causal inferences about program effectiveness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Boyd J, Banzhaf S (2007) What are ecosystem services? The need for standardized environmental accounting units. Ecol Econ 63:616–626

    Article  Google Scholar 

  • Brechin SR, Murray G, Mogelgaard K (2010) Conceptual and practical issues in defining protected area success: the political, social, and ecological in an organized world. J Sustain For 29:362–389

    Article  Google Scholar 

  • Carman JG (2009) Nonprofits, funders, and evaluation accountability in action. Am Rev Public Adm 39:374–390

    Article  Google Scholar 

  • Caro T, Gardner TA, Stoner C, Fitzherbert E, Davenport TRB (2009) Assessing the effectiveness of protected areas: paradoxes call for pluralism in evaluating conservation performance. Divers Distrib 15:178–182

    Article  Google Scholar 

  • Cashore B (2002) Legitimacy and the privatization of environmental governance: how non-state market-driven (NSMD) governance systems gain rule-making authority. Governance 15:503–529

    Article  Google Scholar 

  • Christensen J (2003) Auditing conservation in an age of accountability. Conserv Pract 4:12–18

    Article  Google Scholar 

  • Clark MR, Kozar JS (2011) Comparing sustainable forest management certifications standards: a meta-analysis. Ecol Soc 16:3

    Google Scholar 

  • Conservation Measures Partnership (2007) Open standards for the practice of conservation, version 2.0

  • Cook CN, Hockings M, Carter R (2009) Conservation in the dark? The information used to support management decisions. Front Ecol Environ 8:181–186

    Article  Google Scholar 

  • Dane County Land and Water Resources Department (2008) Dane County Land and Water Resource Management Plan. Dane County Land and Water Resources Department, Madison

    Google Scholar 

  • Ebrahim A (2003) Accountability in practice: mechanisms for NGOs. World Dev 31:813–829

    Article  Google Scholar 

  • Fairfax SK, Gwin L, King MA, Raymond L, Watt LA (2005) Buying nature: the limits of land acquisition as a conservation strategy, 1780-2004. The MIT Press, Cambridge

    Google Scholar 

  • Feltham GA, Xie J (1994) Performance measure congruity and diversity in multi-task principal/agent relations. Acc Rev 69:429–453

    Google Scholar 

  • Ferraro PJ, Pattanayak SK (2006) Money for nothing? A call for empirical evaluation of biodiversity conservation investments. PLoS Biol 4:482–488

    Article  CAS  Google Scholar 

  • Forest Stewardship Council—United States (2010) FSC-US forest management standard v1.0. Forest Stewardship Council, Minneapolis

  • Frederickson DG, Frederickson HG (2006) Measuring the performance of the hollow state. Georgetown University Press, Washington, DC

    Google Scholar 

  • Genskow KD, Wood DM (2011) Improving voluntary environmental management programs: facilitating learning and adaptation. Environ Manag 47:907–916

    Article  Google Scholar 

  • Gibson CC, Williams JT, Ostrom E (2005) Local enforcement and better forests. World Dev 33:273–284

    Article  Google Scholar 

  • Harris GP, Heathwaite AL (2012) Why is achieving good ecological outcomes in rivers so difficult? Freshwat Biol 57:91–107

    Article  Google Scholar 

  • Hatry HP (2006) Performance measurement: getting results, 2nd edn. The Urban Institute, Washington, DC

    Google Scholar 

  • Heinrich CJ, Marschke G (2010) Incentives and their dynamics in public sector performance management systems. J Policy Anal Manag 29:183–208

    Article  Google Scholar 

  • Henri JF, Journeault M (2008) Environmental performance indicators: an empirical study of Canadian manufacturing firms. J Environ Manag 87:165–176

    Article  Google Scholar 

  • Hildén M (2009) Time horizons in evaluating environmental policies. New Dir Eval 2009:9–18

    Article  Google Scholar 

  • Imperial MT (2005) Collaboration and performance management in network settings: lessons from three watershed governance efforts. In: Kamensky JM, Morales A (eds) Managing for results. Rowman & Littlefield Publishers Inc, Oxford, pp 379–424

    Google Scholar 

  • Jennings ET Jr, Hall JL (2011) Evidence-based practice and the use of information in state agency decision-making. J Publ Adm Res Theory 22:245–266

    Article  Google Scholar 

  • Kamal S, Grodzińska-Jurczak M, Brown G (2014) Conservation on private land: a review of global strategies with a proposed classification system. J Environ Plan Man. doi:10.1080/09640568.2013.875463

  • Keene M, Pullin AS (2011) Realizing an effectiveness revolution in environmental management. J Environ Manag 92:2130–2135

    Article  Google Scholar 

  • King MA, Fairfax SK (2004) Beyond bucks and acres: land acquisition and water. Tex Law Rev 83:1941

    Google Scholar 

  • Knight RL (1999) Private lands: the neglected geography. Conserv Biol 13:223–224

    Article  Google Scholar 

  • Koontz TM, Thomas CW (2012) Measuring the performance of public-private partnerships. Public Perform Manag Rev 35:769–786

    Article  Google Scholar 

  • Kroll A, Moynihan DP (in press) Creating public value using performance information. In: Bryson J, Crosby B, Bloomberg L (eds) Valuing public value: approaches to discerning, measuring, and assessing the public sphere, public values, and the creation of public value. Georgetown University Press, Washington, DC

  • Land Trust Alliance (2004) Land trust standards and practices. Land Trust Alliance, Washington, DC

    Google Scholar 

  • Land Trust Alliance (2011) 2010 national land trust census report. Land Trust Alliance, Washington, DC

    Google Scholar 

  • Lockwood M (2010) Good governance for terrestrial protected areas: a framework, principles and performance outcomes. J Environ Manag 91:754–766

    Article  Google Scholar 

  • McLaughlin JA, Jordan GB (1999) Logic models: a tool for telling your program’s performance story. Eval Program Plann 22:65–72

    Article  Google Scholar 

  • Moynihan DP (2008) The dynamics of performance management: constructing information and reform. Georgetown University Press, Washington, DC

    Google Scholar 

  • Moynihan DP, Hawes DP (2012) Responsiveness to reform values: the influence of the environment on performance information use. Public Admin Rev 72:95–105

    Article  Google Scholar 

  • Muñoz-Erickson TA, Cutts BB, Larson EK, Darby KJ, Neff M, Wutich A, Bolin B (2010) Spanning boundaries in an Arizona watershed partnership: information networks as tools for entrenchment or ties for collaboration. Ecol Soc 15:22

    Google Scholar 

  • Nichols JD, Williams BK (2006) Monitoring for conservation. Trends Ecol Evol 21:668–673

    Article  Google Scholar 

  • O’Rourke D (2014) The science of sustainable supply chains. Science 344:1124–1127. doi:10.1126/science.1248526

    Article  Google Scholar 

  • Overdevest C, Rickenbach MG (2006) Forest certification and institutional governance: an empirical study of forest stewardship council certificate holders in the United States. Forest Policy Econ 9:93–102

    Article  Google Scholar 

  • Partners for Fish and Wildlife and Coastal Program (2004) Habitat Information Tracking System. United States Fish and Wildlife Service. http://www.era.noaa.gov/pdfs/habits.pdf. Accessed Nov 2012

  • Partners for Fish and Wildlife Program (2010) Regional showcase accomplishments: Fiscal Year 2010. United States Fish and Wildlife Service. http://www.fws.gov/partners/docs/PFW_Accomplishments_2010.pdf. Accessed Nov 2012

  • Provan KG, Milward HB (2001) Do networks really work? A framework for evaluating public-sector organizational networks. Public Admin Rev 61:414–423

    Article  Google Scholar 

  • Pullin AS, Knight TM (2001) Effectiveness in conservation practice: pointers from medicine and public health. Conserv Biol 15:50–54

    Article  Google Scholar 

  • Pullin AS, Knight TM, Watkinson AR (2009) Linking reductionist science and holistic policy using systematic reviews: unpacking environmental policy questions to construct an evidence-based framework. J Appl Ecol 46:970–975

    Article  Google Scholar 

  • Radin BA (2006) Challenging the performance movement: accountability, complexity, and democratic values. Georgetown University Press, Washington, DC

    Google Scholar 

  • Raymond CM, Brown G (2011) Assessing conservation opportunity on private land: socio-economic, behavioral, and spatial dimensions. J Environ Manag 92:2513–2523

    Article  Google Scholar 

  • Rissman AR (2011) Evaluating conservation effectiveness and adaptation in dynamic landscapes. Law Contemp Probl 74:145–173

    Google Scholar 

  • Rissman AR, Carpenter SR (2015) Progress on nonpoint pollution: barriers and opportunities. Daedalus (in press)

  • Robertson M, BenDor TK, Lave R, Riggsbee A, Ruhl J, Doyle M (2014) Stacking ecosystem services. Front Ecol Environ 12:186–193

    Article  Google Scholar 

  • Salafsky N, Margoluis R, Redford KH, Robinson JG (2002) Improving the practice of conservation: a conceptual framework and research agenda for conservation science. Conserv Biol 16:1469–1479

    Article  Google Scholar 

  • Sanger MB (2013) Does measuring performance lead to better performance? J Policy Anal Manag 32:185–203

    Article  Google Scholar 

  • Sarewitz D (2004) How science makes environmental controversies worse. Environ Sci Policy 7:385–403

    Article  Google Scholar 

  • Srebotnjak T (2007) The role of environmental statisticians in environmental policy: the case of performance measurement. Environ Sci Policy 10:405–418. doi:10.1016/j.envsci.2007.02.002

  • Stem C, Margoluis R, Salafsky N, Brown M (2005) Monitoring and evaluation in conservation: a review of trends and approaches. Conserv Biol 19:295–309

    Article  Google Scholar 

  • Sutherland WJ, Pullin AS, Dolman PM, Knight TM (2004) The need for evidence-based conservation. Trends Ecol Evol 19:305–308

    Article  Google Scholar 

  • Wardropper CB, Chang C, Rissman AR (in press) Fragmented water quality governance: constraints to spatial targeting for nutrient reduction in a Midwestern USA watershed. Landsc Urban Plan

  • Wisconsin Land and Water Conservation Association (2012) What is the WLWCA? Wisconsin Land and Water Conservation Association. http://www.wlwca.org/whatiswlwca.html. Accessed 10 Nov 2012

  • Yin RK (2009) Case study research: design and methods, 4th edn. Sage Publications Inc, Thousand Oaks

    Google Scholar 

Download references

Acknowledgments

We thank all the conservation staff who contributed and S. Gillon, D. Moynihan, M. Rickenbach, and our research group for helpful feedback. Funding was provided by National Science Foundation Water Sustainability and Climate DEB-1038759, McIntire-Stennis Act 0229961 and University of Wisconsin-Madison.

Author information

Authors and Affiliations

  1. Department of Forest and Wildlife Ecology, University of Wisconsin-Madison, 1630 Linden Drive, Madison, WI, 53706, USA

    Adena R. Rissman

  2. Nelson Institute for Environmental Studies, University of Wisconsin-Madison, 550 North Park Street, Madison, WI, 53706, USA

    Robert Smail

Authors
  1. Adena R. Rissman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robert Smail
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adena R. Rissman.

Appendices

Appendix 1

Interview questionnaire

  1. 1.

    Introductory questions

    1. a.

      What organization do you work for?

    2. b.

      What are your job title and responsibilities?

    3. c.

      How long have you worked in this agency?

    4. d.

      How long have you been in the resource management field?

    5. e.

      What is the primary resource conservation goal in your work?

    6. f.

      How many projects do you monitor?

    7. g.

      How many projects do you initiate?

    8. h.

      What policy instruments do you use (legal authorities, funding sources, incentives, acquisitions)?

  2. 2.

    Monitoring

    1. a.

      Do you monitor participants or personnel in your programs/easements?

      1. i.

        If yes, who do you monitor?

      2. ii.

        How frequently do you monitor?

      3. iii.

        How active or intensive is the monitoring?

      4. iv.

        Does your monitoring involve a visit to the land in question?

      5. v.

        Does your monitoring involve modeling or measuring environmental variables?

    2. b.

      Is monitoring required? If so by whom?

    3. c.

      Is formal monitoring done?

    4. d.

      What format is used for recording monitoring results?

      1. 1.

        Do you have a monitoring template?

      2. 2.

        Could we get a copy of a recently completed monitoring report or blank form?

    5. e.

      Are there any informal ways you monitor your programs?

    6. f.

      What capacities or authorities do you have to change, alter or terminate a program, funding source or staff position if the program intentions are not being followed?

    7. g.

      Do you believe that your monitoring efforts are capable of capturing the specifics of the locale and the resources you manage?

  3. 3.

    Reporting

    1. a.

      Do you provide formal reports to others to account for your activities or expenditures?

      1. i.

        If yes, to whom?

      2. ii.

        What do you report?

      3. iii.

        What is the format of the reporting?

      4. iv.

        How often are these reports required?

    2. b.

      Do the entities to which you report have the capacity to change, alter or terminate a program, funding source or staff position if the program intentions are not being satisfactorily met?

    3. c.

      Do you feel your reporting requirements are well tailored to the specifics of your locale and the resources you manage?

    4. d.

      Are there others that you feel you need to report to or are accountable to even though it may not be formally required?

  4. 4.

    Environmental measurement

    1. a.

      Do you engage in any on the ground monitoring or measurement of environmental variables?

      1. i.

        If so, what types or methods?

      2. ii.

        If not, does anyone else?

    2. b.

      Do you manage or execute any programs that have a goal with results which are difficult to quantify or demonstrate?

      1. i.

        What are they?

      2. ii.

        What makes them difficult to quantify?

      3. iii.

        What steps do you take to quantify results in these cases?

      4. iv.

        Do you feel like these programs are at an disadvantage when compared to other programs with more easily quantified results?

    3. c.

      Do you think there is more pressure for monitoring and reporting in private-land conservation programs, versus public lands?

  5. 5.

    Workload

    1. a.

      How much time do you spend on monitoring?

      1. i.

        Is this sufficient? Excessive?

    2. b.

      How much time do you spend reporting?

      1. i.

        Is this sufficient? Excessive?

  6. 6.

    Change in accountability pressures over time

    1. a.

      Since you began working in resource management have you experienced increased pressure for monitoring and reporting?

    2. b.

      Has pressure to formalize monitoring and reporting increased?

    3. c.

      If yes to either, where has pressure for this increase come from?

  7. 7.

    Rationale for monitoring and effects of monitoring

    1. a.

      Does monitoring improve or protect resources?

      1. i.

        If yes, how?

    2. b.

      Does it document changes in the resource?

    3. c.

      Do you perceive your monitoring and reporting efforts as more procedural or substantive in nature?

    4. d.

      To what extent are they done to improve performance?

    5. e.

      To what extent do you see your overall accountability demands as designed to make better programs?

    6. f.

      To what extent are they designed to punish/reward performance?

    7. g.

      Do you see the demand for accountability as coming mostly from inside your organization/project or from those outside of it?

    8. h.

      Is there high conflict among those you report to?

    9. i.

      Are there ways in which you think accounting for results could be changed to improve results or efficiency?

  8. 8.

    Collaboration

    1. a.

      Do you collaborate with other agencies or NGO’s?

    2. b.

      Do demands for accountability facilitate or hinder your ability to collaborate with others?

    3. c.

      Do you report or monitor the efforts of those collaborators with whom you share equal standing?

    4. d.

      Do you feel like collaborative efforts tend to under or over account for their accomplishments?

  9. 9.

    Closing Questions

    1. a.

      Are we measuring what matters in private-land conservation?

    2. b.

      If not, what can we do better?

Appendix 2

Acronyms

ASI:

Accreditation Services International

BARNY:

Wisconsin Barnyard Runoff Model

CMP:

Conservation Measures Partnership

DATCP:

Wisconsin Department of Agriculture, Trade, and Consumer Protection

DNR:

Wisconsin Department of Natural Resources

DOA:

Wisconsin Department of Administration

EPA:

Environmental Protection Agency

FSA:

Farm Service Agency

FSC:

Forest Stewardship Council

GPRA:

Government Performance and Results Act

HabITS:

Habitat Information Tracking System

IRS:

Internal Revenue Service

ISO:

International Organization for Standardization

LCC:

Land Conservation Committee

LTA:

Land Trust Alliance

LWC Board:

Land and Water Conservation Board

LWCD:

Land and Water Conservation Department

NGO:

Nongovernmental Organization

NRCS:

Natural Resources Conservation Service

OMB:

Office of Management and Budget

PFW:

Partners for Fish and Wildlife

SWAT:

Soil and Water Assessment Tool

WALCE:

Wisconsin Association of Land Conservation Employees

WLWCA:

Wisconsin Land and Water Conservation Association

USAID:

U.S. Agency for International Development

USFWS:

U.S. Fish and Wildlife Service

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rissman, A.R., Smail, R. Accounting for Results: How Conservation Organizations Report Performance Information. Environmental Management 55, 916–929 (2015). https://doi.org/10.1007/s00267-014-0435-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00267-014-0435-3

Keywords

Search

Navigation