Introduction

Globally, most biodiversity conservation programmes are not currently evaluated in terms of their costs and benefits, or their rate of return on the original investment (Haddock et al. 2007), though there have been some notable exceptions (see: Moran et al. 1996; Metrick and Weitzman 1998; Cullen et al. 1999; Cullen et al. 2001; Finn et al. 2009; Laycock et al. 2009, 2011; Perkins et al. 2011; Baker et al. 2012; Pannell et al. 2013; Shwiff et al. 2013; Cullen and White 2013). Assessing the cost-effectiveness of conservation programmes can be challenging for a number of reasons. First, there is often a lack of quantitative ecological monitoring data on which to estimate the effectiveness of interventions. Second, the availability of data on the costs of the intervention can be limited. Third, the effectiveness of any set of biodiversity programmes can be dependent on many factors, including (but not limited to) the suitability of the interventions themselves, the nature of the targets which have been set, how the spending has been targeted or implemented and the ecology of the species or habitats under consideration (OECD 2010, 2012; Cullen 2013).

Due to a general lack of empirical data, informal knowledge from experts or stakeholders is being used increasingly in the assessment of conservation programmes (Cullen 2013). Changes to policies based on information provided by stakeholder participation can not only improve the likelihood of implementation and effectiveness (Prager and Freese 2009) but also result in decisions that are better adapted to local socio-cultural and environmental conditions (Reed 2008).

In the UK, there is considerable ecological evidence, but limited economic evidence, about the returns on spending on UK biodiversity programmes. Notable exceptions include Laycock et al. (2009, 2011, 2013) and Christie et al. (2011) which are specific to species and habitat action plans and Wynn (2002) and Macmillan et al. (1998) who considered the cost-effectiveness of broader agri-environmental schemes. However, these studies were confined to using quantitative assessments only. While these can provide a useful metric for their evaluation, they offer little insight into the underlying ecological, socio-economic or political factors that could have influenced success. Assessments that combine quantitative techniques with the addition of qualitative data will not only provide a more complete understanding of these factors but also lead to possible improvements for the benefit of the conservation schemes.

Here, we evaluate the cost-effectiveness of a selection of species and habitat conservation schemes in Scotland, UK. We use cost-effectiveness analysis (CEA), which is applied routinely in health economics (Gold et al. 1996), but has so far only been applied to conservation programmes on a handful of occasions (e.g. Laycock et al. 2009, 2011, 2013; Montgomery et al. 1994; Macmillan et al. 1998; Fairburn et al. 2004). In addition, unlike previous studies, we use expert qualitative data to assess and explain variations in effectiveness and cost-effectiveness and as the basis for recommendations for improvements, both for conservation of individual species and habitats and also across biodiversity conservation programmes. Although the conservation schemes available within Scotland form the focus of the study, the approaches used and interpretations drawn are relevant to the assessment of any regional, national or international biodiversity conservation programmes.

Methods

Policy background

The species and habitat conservation schemes we examined were undertaken through a number of elements of the Scotland Rural Development Programme (SRDP), which helps to deliver the European Union’s Common Agricultural Policy in Scotland, as well as other funding programmes such as the Scottish Natural Heritage’s (SNH) Natural Care programme. Within the European Union (EU), schemes which encourage farmers to manage their land for the benefit of wildlife and the environment (agri-environment schemes) are predominantly funded under the Common Agricultural Policy (CAP). Payment rates and scheme design are regulated by the EU and expenditure has to be planned over several years through Rural Development Programmes. The SRDP allocated around one billion Euros to agri-environment schemes over the period 2007–2013, with funding distributed via both non-competitive (e.g. ‘Land Managers Options’) and competitive (e.g. ‘Rural Priorities’) mechanisms (see: http://www.scotland.gov.uk/Topics/farmingrural/SRDP/Background). SNH’s Natural Care Strategy was launched in 2001 to encourage positive management of Sites of Special Scientific Interest (SSSIs) and EU’s Natura 2000 sites (see: http://www.snh.gov.uk/land-and-sea/managing-the-land/farming-crofting/grants-and-funding/natural-care-programme/). Each scheme offered a range of management options with standard payments attached. In 2009, Natural Care Schemes began to be phased out as the conservation options concerned became more integrated into the SRDP.

Identifying target species and habitats

In order to help deliver the Scottish Biodiversity Strategy, SNH recognised that there was a need to prioritise the way to manage species, focusing on those where significant gains to overall biodiversity were expected. As a result, the Species Action Framework (SAF) produced in 2007 set out a strategic approach to species management in Scotland. It also identified a ‘Species Action List’ of 32 species which were the focus of new, targeted management interventions for five years between 2007 and 2012 (http://www.snh.gov.uk/protecting-scotlands-nature/species-action-framework/).

The species selected for use in this study were drawn from the SAF and include a mix of native bird, mammal, amphibian, insect, fungi and plant species of conservation interest (black grouse, capercaillie, hen harrier, sea eagle, red squirrel, great crested newt, marsh fritillary butterfly, slender scotch burnet moth, hazel gloves fungus, and water vole), and invasive species (grey squirrel, rhododendron ponticum, American mink) which action aims to reduce. Only one of our selected study species (corncrake) was not included in the SAF, but was included in our study due to the targeted conservation action taking place including targeted options within the SRDP. Our list of five habitats (hedgerows, arable fields, wetland, native woodland, uplands heath and moorland) was drawn from a selection of habitats deemed to be important for future land management requirements for the UK (Cao et al. 2009), on the basis of their biodiversity importance and on information on their distribution being reasonably well known.

Identifying scheme expenditure data for selected species and habitats

We identified the funding that was directly related to our study species and habitats or linked to the species and habitats through published scheme literature. We chose 2005 as the starting date and identified the main programmes containing biodiversity conservation schemes running in Scotland from 2005 onwards. These often involved long lists of funding options relating to either the SRDP or Natural Care Schemes. Because our focus was the cost-effectiveness of schemes linked to the Scottish Government funding, no other sources of funding, for example from non-governmental organisations (NGOs), have been included. Information on previous actual and committed future spending on the identified schemes was extracted from data supplied by the Scottish Government’s Rural Payments and Inspectorate Directorate (RPID).

Survey development and implementation

We developed a survey based on semi-structured interviews with key species and habitat advisors. Key contacts were identified for each species and habitat by the project team and comprised species leads and advisors from public agencies (SNH, Forestry Commission), conservation NGOs (RSPB, Game and Wildlife Conservation Trust, Butterfly Conservation Scotland), land owners and other stakeholder groups (Scottish Land and Estates, SAC Consulting). These contacts were identified as they were likely to have knowledge of the scheme and conservation status of the individual species and habitats.

We used the survey to ask the interviewees a range of questions regarding the cost, effectiveness, conservation status and wider impacts of the species and habitat-specific interventions (Table 1). However, the information regarding the effectiveness (extent to which the conservation objectives have been met) and cost (species and habitat-specific expenditure) is the focus of this particular paper. Participants were asked to comment only on the cost and effectiveness of schemes funded by the Scottish government as listed above and not of those from other funding sources.

Table 1 Information required and how this was obtained at expert interview
Full size table

The conservation objectives on which the effectiveness scores were based do vary in scope between species and are listed in Supplementary Information A. These species objectives can be broadly categorised as: (1) maintaining current populations and ranges; or (2) extending populations and ranges. There is also variation in the extent to which the objectives are quantified. The objectives for habitats (Supplementary Information A) also typically relate to maintaining or expanding the extent, or improving condition.

All species objectives are taken from the Species Action Framework (SAF, Scottish Natural Heritage, 2007) apart from the Corncrake objectives which were taken from the Rural Priorities package website (http://www.scotland.gov.uk/Topics/farmingrural/SRDP/RuralPriorities/Packages/Corncrakes). All habitat objectives are taken from Scotland’s Targets (drawn from Biodiversity Action Reporting System which holds target information: http://ukbars.defra.gov.uk/archive/default.asp). Where more than six objectives were given for a habitat, we chose to focus on those most important for biodiversity. The full list of species and habitat objectives shown at interview and questions asked can be found in Supplementary Information A and B.

A total of 28 interviews were conducted between October and December 2012. Each interview typically lasted between 1 and 2 h depending on the number of species/habitats that the participant was being interviewed about. The interviewees were sent some information regarding the interview questions and topic areas prior to the interview. The interviews were conducted during face-to-face meetings, but when this was not possible, they were conducted via telephone or video conference. The interviews were recorded with the permission of the participants to support the extensive notes that were taken at the time of interview.

Data analysis

Total effectiveness and cost-effectiveness

We used the following equation to calculate the Total Effectiveness of SRDP spending on each species or habitat (after Laycock et al. 2009):

$$E_{i} = \sum\nolimits_{n = 1}^{N} {\left[ {M_{n} \left( {I_{n} /100} \right)} \right]}$$
(1)

where E i is the total effectiveness i ; each species or habitat has a total of N objectives; M n is the percentage by which objective n has been met; and I n is the percentage importance of objective n to the overall effectiveness of spending on that species or habitat. We then calculated the efficiency of spending on each species or habitat using Eq. (2), where C i /E i is the Present Value (PV) Cost-Effectiveness Ratio, i.e. the discounted cost per percent effectiveness, of species i or habitat i ; the spending on species i or habitat i has been implemented for a total of T years; C it is the spending on species i or habitat i in year t; and d is the discount rate.

$${{C_{i} } \mathord{\left/ {\vphantom {{C_{i} } {E_{i} }}} \right. \kern-0pt} {E_{i} }} = \frac{{\sum\nolimits_{t = 0}^{T} {\left[ {C_{it} \left( {1 + d} \right)^{t} } \right]} }}{{E_{i} }}$$
(2)

Discounting is a commonly used process that collapses cost/benefit streams over time to Present Value equivalents (HM Treasury 2003). Here, the process allows different SRDP spending profiles to be compared on a consistent basis. In cases where participants had estimated the percentage of the total amount that was spent over blocks of several years rather than single years, we assumed that the cost was distributed evenly across the individual years within these blocks. In addition, because the different species and habitat schemes were not all implemented at the same time, the only time point common to all schemes is the end of the approved spending (2015). Thus, this was taken as the reference date for discounting, which means that we actually compounded rather than discounted, taking 2015 as Year 0 and the first year that any programmes were implemented (2005, if some Natural Care Schemes applied) as Year 10. Here, we use a discount rate of 3.5 %, as this is the rate the HM Treasury (2003) currently advises for social projects.

Qualitative analysis

The effectiveness of the scheme could be dependent on many factors including suitability of the scheme, suitability of the objectives, how the spending has been targeted or implemented and the ecology of the species. Therefore, in order to provide a background context to the quantitative data for each species or habitat and to provide information for further recommendations for improvements, a qualitative analysis was performed using field notes taken during the interviews.

We coded the field notes from each interview into two pre-defined categories: (1) the barriers to the uptake and efficiency of schemes for each species or habitat; and (2) where improvements could be made and how schemes could be more cost-effective. Where needed, the recordings of the interviews were double-checked against the field notes to ensure all the information was captured consistently between interviews. This information was then summarised and is presented alongside the quantitative results.

Results

For all but one species, only one participant (per species or habitat) was able to give information on the financial costs relating to the amount spent within the SRDP for that species or habitat. The majority of the cost and cost-effectiveness estimates were therefore based on the information given by one respondent per species or habitat. Where more than one estimate was given, the average was used. However, for some species and habitats, effectiveness data were given by more than one respondent. Where this was the case, the effectiveness score, and therefore cost-effectiveness ratio, has been calculated based on each complete dataset (incomplete answers have not been included) and a range given (for effectiveness and cost-effectiveness) to reflect any differences between participants. Although we did ask participants to estimate the change in conservation status of species and habitats (see Table 1), the majority of participants were unable to answer this question.

The effectiveness of species and habitat conservation programmes in relation to specific objectives

For the species, the effectiveness scores range from 0 % (lower estimate, black grouse and capercaillie) to 100 % (upper estimate, sea eagle) (Table 2). In terms of the habitats, effectiveness scores range from 28 % (lower estimate, hedgerows) to 95 % (upper estimate, arable fields) (Table 3).

Table 2 The effectiveness scores, along with the total costs for SRDP spend on the species (calculated using a social discount rate of 0 and 3.5 %), and the present value (PV) cost-effectiveness ratio (calculated using a social discount rate of 3.5 %) are presented
Full size table
Table 3 The effectiveness scores, along with the total costs for SRDP spend on the habitat (calculated using a social discount rate of 0 and 3.5 %), and the present value (PV) cost-effectiveness ratio (calculated using a social discount rate of 3.5 %) are presented
Full size table

There were five species (great crested newt, marsh fritillary butterfly, slender scotch burnet moth, rhododendron ponticum, water vole) and two habitats (upland heath and moorland and native woodland) for which we were unable to estimate effectiveness (and therefore cost-effectiveness) due to participants being unable to supply this information (Tables 2, 3). The reasons given by participants included a lack of monitoring data on which to base the effectiveness estimates and in some cases the participant stating that associated SRDP options were not actually being applied to those species and that funding for any conservation actions were coming from other sources.

Costs and cost-effectiveness of species and habitat conservation programmes

For the species, the present value costs (d = 3.5 %) range from £79,000 (hazel gloves fungus) to £10,603,600 (corncrake). The cost-effectiveness estimates range from £3,500 (lower estimate, sea eagle) to £4,564,800 (upper estimate, black grouse) (Table 2). The ratio is a measure of the ‘cost per unit of effectiveness’, therefore, the higher the value, the higher the cost of each unit of effectiveness gained.

In terms of the habitats, present value costs (d = 3.5 %) range from £12,516,000 (arable fields) to £50,403,000 (hedgerows). Cost-effectiveness estimates range from £131,700 (lower estimate, arable fields) to £1,800,100 (upper estimate, hedgerows) (Table 3).

The qualitative interview data summarised in the final column of Tables 2 and 3 offers potential reasons behind these differences in cost-effectiveness ratios between species and habitats. For example, black grouse management has relatively high costs per unit of effectiveness (upper estimate) but interviewees for this species stated that funding through the black grouse SRDP package has often been spent in areas where populations are too low for the work to be beneficial. In addition, external factors such as weather and alternative neighbouring land-uses (such as afforestation) can have a large impact on the breeding success of the species and directly impact on whether or not the stated objectives are achievable.

In contrast, the sea eagle has relatively low costs per unit management. This is likely to reflect both the relatively low cost of the scheme and the high effectiveness. Interviewees for this species also stated that since the reintroduction of the species, Natural Care schemes have enabled conflict reduction via positive management of livestock for both sea eagle populations (East and West) which has been beneficial for at least partly achieving the set objectives for the management of that species.

In terms of the habitats, hedgerow management has relatively high costs per unit of effectiveness (upper estimate). This may be because (as stated by interviewees) much of the funding allocated for this habitat has been spent on the creation of new hedges and not the management of existing ones, despite the fact that many of the habitat objectives relate to the management of existing hedgerows.

In contrast, arable field management under this scheme has relatively low costs per unit of effectiveness. During the interview, the participants said this was because schemes have had impacts for this habitat but there has not been sufficient uptake of these options (Table 3), i.e. not enough of those interventions were established in the right places at the right scale.

Species and habitat specific stakeholder recommendations for improving cost-effectiveness

In addition to commenting on the current cost and effectiveness of the species and habitat programmes, participants were also encouraged to discuss how the programmes could be altered to improve cost-effectiveness. These comments are species and habitat specific (listed in Tables 2 and 3) but there are some common themes that occur. For example, ‘more accurate geographical targeting’ of resources was mentioned for the majority of species and habitats as a way of improving effectiveness of the schemes. In addition, ‘more advice and support for landowners’ was frequently mentioned as an important mechanism for improving cost-effectiveness for species and habitats (Tables 2 and 3).

For most species and habitats, adequate monitoring of the impacts of the schemes was limited. This was reflected in the recommendations to improve effectiveness for some species. For the red squirrel for example, it was stated that effectiveness could be improved through ‘long term monitoring of population levels’ to establish the minimum amount of control (of grey squirrels) needed to achieve the conservation objectives (Table 2). Adapting schemes to encourage collaborative working (instead of being targeted only at the individual) was another recommendation made for several species and habitats. Enabling management at the landscape level was mentioned as particularly important for habitats such as native woodland (Table 3).

Discussion

In this paper, we have used cost-effectiveness analysis in an evaluation of biodiversity conservation schemes funded under the SRDP. In addition, we have used qualitative information from experts to place these evaluations in a broader context of the other factors affecting these schemes.

Due to differences in the way that objectives have been set, units of effectiveness cannot be standardised and hence, direct comparisons across different species and habitats based on the quantitative results alone need to be interpreted with caution. However, the qualitative data offer additional insight into underlying ecological, socio-economic or political factors that could have influenced success and are important in identifying many species or habitat-specific factors that could be impacting on the cost-effectiveness of the different biodiversity conservation schemes.

Our findings highlight, in particular, a lack of geographical targeting in terms of where the funding might achieve the most conservation benefit, which is likely to be contributing to high costs per unit of effectiveness for certain species and habitats. Another recurrent theme was the need for improved advice on appropriate management and meaningful monitoring programmes linked closely to the objectives which have been set – sentiments that echo findings in other recent studies (e.g. Hart et al. 2011; Perkins et al. 2011; Armsworth et al. 2012).

Although conservation schemes within Scotland were used as the focus of the study, the approaches used, interpretations drawn and improvements identified as being required could be applied to the assessment of any regional, national or international biodiversity conservation programmes. We have focused on a selection of species and habitats, but this technique could be applied to other examples if funding expenditure and effectiveness against funding objectives are identifiable. However, we found that obtaining such information is not always straightforward and presents a number of challenges when conducting the cost-effectiveness assessments.

First, the complex inter-connectedness of ecosystems means that identifying a discrete set of relevant funding streams for each species/habitat can be challenging. For the SRDP, information is often freely available on which options are aimed specifically for the species or habitat in question, but their effectiveness may be conditional upon a number of other, less directly relevant but nonetheless supporting interventions. Often such interventions are supporting the management of a number of species or habitats. Hence, for our study, we needed to combine the best available information but also expert advice to identify all of the relevant interventions for each species and habitat. This may not always be possible for other types of funding for other biodiversity conservation programmes and therefore making this type of information transparent and widely available would enhance the feasibility of other future assessments of cost-effectiveness.

Second, even if a set of relevant interventions can be identified, attaching a cost to them is not always straightforward. This partly reflects unexpected difficulties in accessing funding data, but also that funding does not necessarily equate to expenditure (the latter is often less, and lags behind, the former) and that any given intervention may support more than one species or habitat and thus funding needs to be apportioned between them. In this study, this apportionment was further reliant on the perceptions of our survey participants. In most cases, only one participant (per species or habitat) was able to give information relating to the amount spent within the SRDP on conservation for that species or habitat. Although we feel this was the best approach to determining where categories of funding have been targeted, greater availability of where the expenditure has gone for each intervention would enable greater accuracy regarding cost data of these conservation programmes.

We measured effectiveness using expert opinion in relation to stated policy objectives. However, the stated policy objectives vary across individual species and habitats in terms of their ambition, clarity and initial conditions. This makes it difficult to compare effectiveness directly across species and habitats. Perceived effectiveness could be due to easily attainable objectives and/or to well designed and implemented schemes. Conversely, objectives might be unrealistic and/or schemes could be poorly designed and implemented. A number of previous studies have highlighted the importance of setting clear objectives and subsequently monitoring progress against those objectives when seeking to assess the cost-effectiveness of biodiversity conservation programmes (e.g. OECD 2010, 2012).

In addition, although interviewees were selected for their expert knowledge, many of them acknowledged information gaps and limits to the accuracy of their quantitative estimates. This emphasises further the need for routine and repeated monitoring to be set in place, and for this monitoring to be matched with objectives, so that if there is a change (negative or positive) in status, the data are sufficient for it to be detectable. Objectives should be set such that it is possible that data being collected will be able to determine whether it has been met or not. The problems arising from a lack of monitoring and an inability to ascribe outcomes to schemes is not unique to the SRDP and have been noted in similar contexts elsewhere (see for example Hanley et al. 1999; Welsh Assembly Government 2008; OECD 2012).

Although comprehensive monitoring of baseline and changing conditions can be expensive and attributing observed changes to policy can be difficult, it is possible. For example, in the case of farmland birds, Perkins et al. (2011) report how bespoke monitoring was used to assess the effectiveness of a specific scheme in Scotland whilst Baker et al. (2012) use more routinely collected monitoring data to identify spatial variation in scheme impacts across the UK. For more complex schemes involving multiple potential benefits, Mauchline et al. (2012) and Pannell et al. (2013) separately advocate the involvement of scheme participants in not only scheme design but also in undertaking monitoring activities. Failure to establish meaningful baselines or reporting procedures inevitably hampers any subsequent assessment of policy effectiveness and, whilst more could be made of existing data, some additional effort is required.

Conclusions

This work has demonstrated the insights that can be gained into biodiversity conservation programmes through a quantitative analysis of their outcomes relative to investment. Moreover, it has highlighted how additional qualitative information can be used to inform this quantitative analysis, identify existing constraints and propose potential solutions or alternative approaches. An over-reliance on quantitative assessments in isolation for biodiversity conservation assessment can have shortcomings, especially where there are uncertainties in the quality of the data used for calculations. For cost-effectiveness analysis, problems in deriving empirical estimates of effectiveness (and in some cases, cost) mean that cost denominators and effectiveness numerators can be subject to a high degree of uncertainty. Hence, any cost-effectiveness estimate is subject to a number of caveats. There is therefore not only a need to focus on improving the cost-effectiveness of biodiversity programmes—to which the qualitative findings in this study supports many previous studies in its findings and recommendations—but also a need to improve the robustness of cost-effectiveness measures, in terms of better availability and accuracy of baseline data.