Introduction

1.4 billion people depend directly on forest products for some portion of their livelihoods and household consumables (World Bank 2008)—and thousands of tree species are instrumental to global diets, health, shelter, fuel and incomes of the world’s poor (Arnold et al. 2011). In some settings, products from forests account for a higher proportion of livelihood and income benefits than does agriculture (Leakey et al. 2005).

Central Asia is considered a centre of origin and diversity for many globally-important temperate fruit tree species. According to Vavilov (1931), Central Asia is the richest region in specific and intraspecific diversity and belongs to one of the five most important centres of origin of cultivated plants. Despite the erosion of natural resources, there are still 8100 plant species in the region, of which 890 species are endemic. About 400 of them are endangered, included in IUCN’s Red Data Book of endangered species.Footnote 1

Among Central Asian countries, Kazakhstan, Kyrgyzstan, Tajikistan, Turkmenistan and Uzbekistan are particularly rich in highly-variable fruit and nut species that have global commercial and nutritional importance, such as apple (Malus domestica), apricot (Armeniaca vulgaris), peach (Persica vulgaris), pear (Pyrus communis), plum (Prunus domestica), grape (Vitis vinifera), almond (Amygdalus communis), pistachio (Pistacia vera), pomegranate (Punica granatum), and fig (Ficus corica) (Johns et al. 2013). Moreover, many wild relatives of these fruit species are still found in forests throughout the region and mainly used as rootstocks. Thus, the region contains important and highly diverse gene pools that are valuable both locally and globally. This diversity plays a lead role in facing environmental and agricultural challenges, representing a key aspect in the sustainable management of the agroforestry systems (Frison et al. 2011; Mijatović et al. 2013), providing both public and private ecosystem services (Baumgärtner and Quaas 2010; Zytynska et al. 2011). Agro and tree diversity use and conservation sustain food production over time and, therefore, contribute to alleviating food security concerns for present and future generations (Gore 1992; Thrupp 2000; Esquinas-Alcázar 2005). The sustainable use of the genetic resources is widely considered to be the key source of the technological innovations in agriculture (including forestry): numerous studies have indicated productivity gains in the agriculture and forestry sectors, resulting from crop and tree genetic improvement (Johnson et al. 2003; Tilman et al. 2005) while genetic erosion increases household vulnerability to pedoclimatic stresses and to fluctuations of price and production, especially in developing countries (Thrupp 2000; Arnold et al. 2011). The broadening of diversity provides important assistance to farmers addressing changing climatic conditions (Howden et al. 2007; Cavatassi et al. 2011), as well as improving their diets (Arnold et al. 2011), income diversification (Bellon 2004) and overall livelihoods (Belcher et al. 2005; Nabahungu and Visser 2011; Gotor et al. 2013).

However, over the years, the native genetic diversity of fruit tree species has been eroded in many Central Asian countries, mainly due to increased overgrazing, deforestation, logging and industrialization (Sunderland 2011). Indeed, the agroforestry sector has become more market-oriented, generating private incentives that hamper the conservation and sustainable use of genetic diversity in favor of economic activities that threaten this diversity, resulting in biodiversity loss risk (Drucker et al. 2001; Leakey et al. 2005; Bellon et al. 2015a). While several economic values (whether direct or indirect and option values) are associated to the agro and tree biodiversity access, exchange and use, they are only partially captured in the market place (Pearce and Moran 1994). Although tree genetic diversity generates societal benefits, its nature as a public good results in a tendency for it to be under-maintained relative to regional or global needs (Baumgärtner and Quaas 2010; Ferrarro and Hanauer 2011). The rise of these private incentives prompted the development of possible external interventions needed to obtain a sustainable-level use of these resources (Bezabih 2008).

Despite the short-term outcomes of external interventions in areas such as genetic diversity conservation, wherein yields and agronomic management are well-documented, there is a lack of structured and analytic assessment of the success of these projects in broader and longer gains beyond immediate training—development efficacy—(Lutz and Munasingheb 1994), while the majority of existing impact studies fail to recognize that a key aspect of the development efficacy is the sustainability of the impact (White 2010). The latter is pivotal for assessing the long-term efficacy of activities and their real impact on rural livelihoods (Bellon et al. 2015a). Tracing impact over time presents problems that require rethinking or adapting existing methodologies or creating new impact assessment approaches across temporal scales (White 2010).

This paper attempts to fill this gap analyzing the impact at the household level of a development program related to conservation and use of fruit species (cultivated and wild relatives) in Central Asia, and specifically in Uzbekistan. The research-for-development programme on in situ/on-farm conservation and use of fruit species in Uzbekistan analysed in this paper was initiated in 2005. The programme, financially supported by UNEP-GEF, was part of a larger framework of multi-country research projects in Kazakhstan, Kyrgyzstan, Tajikistan, Turkmenistan, and Uzbekistan aimed at promoting the conservation of genetic diversity and sustainable use of horticultural crops and fruit species.

As documented by Bellon et al. (2015a, b) an effective development program should provide interventions with the purpose of creating private incentives for farmers (through the transfer of technologies, practices, skills etc.) to continue maintaining on-farm diversity beyond the end of the project activities. A well-informed and trained farmer should, in fact, can improve his/her income and livelihood opportunities while contributing to the conservation and improvement of the resources he/she has access to (Nabahungu and Visser 2011).

This paper thus assesses the effects of the interventions in the areas of the chain of processes for enhancing use/consumption of fruit species (cultivated and wild) improving their marketing value and enhancing household food security and livelihoods. More in detail, a difference-in-differences propensity score matching is used to assess the impact of household participation in development programs on conservation and use of fruit tree species. Thus, the robustness of the implemented statistical method may contribute to the discussion of the utility and soundness of quasi-experimental approaches in this area of investigation, where randomized control trials are not feasible, since self-selection, households motivation, and capacity to participate are key elements of the process that need to be explicitly considered (Barrett and Carter 2010).

The remainder of the document is organized as follows: “Program description and conceptual framework” section illustrates program activities and the conceptual approach followed in this study. “Methodology” section describes the data sources and the empirical strategy used in the investigation, while program impact estimates are discussed in “Results” section. “Conclusion” section concludes by summarizing findings and discussing implications for the conservation of fruit tree species in the future.

Program description and conceptual framework

The development program on fruit tree diversity analyzed in this paper was implemented over a period of five years between 2005 and 2010. The program was a part of a larger framework of a multi-country research UNEP-GEF project, emerging from the need to test a comprehensive framework for sustainable conservation and use of fruit (cultivated and wild) and horticulture crops with a livelihoods perspective. In order to be effective and leave a sustainable impact on people’s livelihoods, the promotion of agricultural biodiversity needs to be based on holistic approaches, be highly participatory, apply gender-sensitive interventions and pursue inter-disciplinary close collaborations (Padulosi et al. 2014).

The program aimed at ensuring that farmers, institutes, and local communities were provided with the knowledge, methodologies, and policies to conserve and use in situ fruit species in Central Asia.Footnote 2 Key project objectives were to: (1) provide options to policy-makers for strengthening legal and policy frameworks; (2) assess, document, and manage local varieties of wild fruit species sustainably; (3) promote broad stakeholder participation, representative decision-making, and strong partnerships among them; and (4) strengthen the capacity to implement all aspects of fruit species genetic diversity conservation at local, national and regional levels. Specifically, the program aimed to strengthen the capacity of farmers in agronomic techniques, improving their access to varieties through nurseries and pushing for benefit-sharing policies. At the same time the program facilitated the development of policy in support of the sustainable management of fruit tree genetic diversity (cultivated and wild), the participation of farmers and local communities in conservation, and the improved capacity of stakeholders to implement legal, scientific, and social aspects of fruit species genetic diversity conservation (Fig. 1). Thus, the implemented program followed a holistic approach, implementing an array of several types of activities tailored to different stakeholders and levels of the production system. The program addressed different aspects of the production consumption and marketing of targeted fruit species, providing diverse pathways for create private incentives for households to continue to preserve and use fruit diversity, and related practices under varying conditions. Moreover, the use of a basket of activities offers diverse alternatives to households, some of which may be more relevant than others, conditional on households’ specific needs.

Fig. 1
figure 1

Project areas of interventions and list of activities

Full size image

The target sites and the selected species were identified though a series of multi-stakeholder consultations undertaken in the 2005. The target sites were chosen on the basis of their being representative of the beneficiary groups, as well as the socio-economic, cultural and geographic contexts surrounding target species. The project involved more than 200 villages overall within 101 districts in 29 provinces, covering a population of around 160,000 households and an area of around 42,000 ha across Kazakhstan, Kyrgyzstan, Tajikistan, Turkmenistan, and Uzbekistan (Table 1).

Table 1 Designated project areas in Central Asia
Full size table

Activities carried out have produced several tangible outcomes: over 50 fruit tree nurseries have been established as a result of the program, producing more than 1.5 million traditional seedling varieties annually of apple, grape, pomegranate and other fruit and nut trees. The project trained 300 farmers each year in soil, water and crop management practices with the aim of improving the production systems, while the establishment of farmer associations aimed at improving local incomes and livelihoods.

The effective impact of the implemented activities, including their interactions, needs to be evaluated all together, rather than by isolating the individual task. The valuation of the impact of each intervention has little importance considering the overall strategy of the development program. The specific objective of this impact assessment analysis is to assess the extent to which the overall program contributed to (i) increased tree diversity, (ii) and to the creation of livelihood benefits, which in turn encourage farmers and communities to conserve diversity, creating a feedback loop that ensures both diversity and its continuing benefit to present and future generations. The assessment is based on the application of the theory of change developed by Bellon et al. (2015a, b).

Figure 2 represents the sustainable pathway of an effective in situ, on-farm conservation project. According this framework, the following four hypotheses should be tested in order to assess the success of in situ, on-farm conservation project (Bellon et al. 2015a):

Fig. 2
figure 2

The effectiveness of an on-farm conservation project

Full size image
  1. a)

    Participation in project interventions leads farmers to apply options provided by the interventions;

  2. b)

    The application of these options leads to farmers maintaining higher levels of wild fruit diversity than would have been possible without the interventions;

  3. c)

    Farmers with higher levels of diversity obtain additional benefits from this diversity;

  4. d)

    The higher levels of fruit diversity linked with the application of these options are associated with higher levels of genetic diversity than would have occurred otherwise.

While the first three hypotheses constitute the research questions around which the whole study is built, the latter deals with issues pertaining to crop population genetics and biogeography, and is not directly analyzed in this study.

Methodology

This paper aims to analyze the impact of a set of interventions related to on-farm/in situ conservation and use of wild fruit species on farmer livelihoods in Central Asia, specifically in Uzbekistan. More specifically, in order to appropriately answer the above-mentioned research hypotheses, a quantitative analysis is performed, aiming to provide the following two outputs:

  1. 1)

    Evidence of whether the application of project activities leads to an increase in in situ diversity;

  2. 2)

    Evidence of the extent to which the application of project activities, through the conservation of diversity in situ, contributes to the improvement of household livelihoods.

These outcomes must take place at household level in order to highlight the causal relations between the participation in project activities and their measurable impact on the individuals. Providing statistically-significant evidence of project impact is necessary to deal with program evaluation challenges and to successfully assess these outcomes (Barrett and Carter 2010; Gotor et al. 2013). However, this task faces the common quandaries of ex-post evaluation studies with non-experimental design (Lewis et al. 2011): The challenges regard the correct identification of the real causal impact of a project within the context of observational data, due to the potential presence of endogeneity, sample selection bias and other confounding effects (Bellon et al. 2015b). However, many technical options exist to address these problems including, amongst others, propensity score matching and difference-in-difference methodologies (Guo and Fraser 2010) that were jointly used in this analysis.

Statistical evidence of a treatment effect will be given by comparing outcomes (or inter-temporal changes in outcomes) of households participating in the program activities with a counterfactual given by a control group drawn from households that did not participate. More in detail, the estimation of causal effects by difference-in-difference methods (DID) (Card and Krueger 1994) is performed by comparing inter-temporal changes in the outcomes from “baseline” to “endline” between households participating in project activities and those not participating. This procedure entails the collection of data before the start of the project (baseline, collected in 2005) and after its completion (endline, collected in 2013).

The first step entails the definition of fruit diversity and livelihood outcomes that project interventions were (ex-post) expected to achieve, and developing measurable indicators of those outcomes. These measures have to be case-specific and functionally consistent with the project aim. On-farm diversity will be measured at household level using the species richness (R), Simpson index of diversity (D) and EquitabilityFootnote 3 (E) (Peet 1974). In particular, they are defined as:

  1. i)

    Richness (R h ), for the h-household, count of different species of fruit;

  2. ii)

    Simpson index of diversity (D h ), for the h-household \(D_{h} = \frac{1}{{\mathop \sum \nolimits_{i = 1}^{{R_{h} }} p_{hi}^{2} }}\) where p hi is the land under cultivation for the i- fruit species for the h-household.

  3. iii)

    Equitability (\(E_{h}\)): for the h-household \(E_{h} = D_{h} \times \frac{1}{{R_{h} }}\).

As concerns outcomes related to the household wellbeing, livelihood indicators have been identified from the theoretical framework developed by the program coordinator at the beginning of the implementation of the activities, tailoring standard survey instruments usually used by the practitioners and researchers to analyze livelihood outcomes (Hong et al. 2006; Bellon et al. 2016; Schnitzer 2016) to the socio-economic context. Four livelihood measures were defined:

  1. a)

    Self- consumption (SCh), for the h-household it measures the share of fruit production for self-consumption;

  2. b)

    Marketing (MKh), for the h-household it indicates the share of fruit production sold through the market;

  3. c)

    Livestock index (LIh): for the h-household it proxies the ownership of different livestock species;

  4. d)

    Appliance index (AIh): it proxies the presence/availability of different appliances (i.e. tv-color, dvd, personal computer) into the h-household.

Once the outcomes are measured, the impact of the program can be assessed as the variation in outcomes that can attributed to the interventions. The DID estimator is given by the \(\hat{\beta }\) parameter interpreted as follows:

$$\hat{\beta } = \, \left( {{\text{Y}}_{ 1}^{\text{T1}} - {\text{ Y}}_{ 1}^{{{\text{T}}0}} } \right) \, - \, \left( {{\text{Y}}_{0}^{\text{T1}} - {\text{ Y}}_{0}^{{{\text{T}}0}} } \right)$$
(1)

with Y T11 is the outcome potentially associable to the project measured at the “endline”, for the participants households, Y T10 the outcome for the households non participants, Y T01 and Y T00 are the outcomes measured at the baseline for respectively the participants and not participants. Parameters \(\hat{\beta }\) can be estimated from the following weighted least squares regression estimation (WLS):

$$\Delta {\text{Y}}_{\text{h}} = \, \alpha \, + \, \beta {\text{W}}_{\text{h}} + {\text{ e}}_{\text{h}}$$
(2)

where for the h-th individual, ΔYh is the difference of the outcome measured at the endline minus the initial value of the outcome measured at the “baseline”, while Wh = 1 for project participants households and Wh = 0 for non participants.

Following (Heckman et al. 1998) in this study, we employ the conditional difference-in- differences estimator by defining outcomes conditional on the pre-treatment characteristics (Xh) of the households for controlling for other confounding effects, and using weights derived from the Kernel Propensity Score Matching (Guo and Fraser 2010) for controlling the source of inconsistency given by potential selection bias.

$$\Delta {\text{Y}}_{\text{h}} = \, \alpha \, + \gamma {\text{X}}_{\text{h}} + \beta {\text{W}}_{\text{h}} + {\text{ e}}_{\text{h}}$$
(3)

As previously stated, the project involved overall more 200 villages within the 101 districts in 29 provinces, covering a population of around 160,000 households and an area of around 42,000 ha across Kazakhstan, Kyrgyzstan, Tajikistan, Turkmenistan, Uzbekistan.

As regards the impact assessment study, Uzbekistan was selected not only for the country importance in terms of households and area covered by the project but also according the size of the data sample and the quality of the collected information. Table 2 presents the characteristics of the project sites selected in Uzbekistan.

Table 2 Description of UNEP-GEF project sites in Uzbekistan
Full size table

In Uzbekistan two household surveys were carried out, one of which was a baseline survey. Trained project staff interviewed a total of 206 households divided by those that were randomly selected for the upcoming participation to the program (participation to the program was open to all households and participation was voluntary) and those that were not.Footnote 4 The second survey ended after the project completion, where the same households interviewed in the baseline were interviewed again. However, some slight differences in terms of the number of households interviewed in the two rounds exist due to the fact that not all households could be interviewed again, since they had either moved from the community or were simply unavailable. Only 141 households were in both rounds (baseline and endline). Figure 3 shows the spatial distribution of the sampled households.

Fig. 3
figure 3

Spatial distribution of sampled households

Full size image

Tables 3 and 4 summarizes socio-demographic characteristics of the two groups of households, participants and non-participants. Generic information on the household head (including age, sex, education), the household (including the number of individuals, residence and home garden ownership) and farm characteristics (including land area, altitude, and tractor availability) were included.

Table 3 Description of the variables used in the quantitative analysis
Full size table
Table 4 Sample description
Full size table

Results

A participation model was used to calculate for each h-households the propensity scores p(Z h ) or the conditional probability of participated (or propensity score):

$${\text{p}}\left( {{\mathbf{Z}}_{{\mathbf{h}}} } \right) \, = {\text{ Prob}}\left( {{\text{W}}_{\text{h}} = 1} \right) \, = F({\mathbf{Z}}_{{\mathbf{h}}} ,{\varvec{\uptheta}}) + {\text{u}}_{\text{h}}$$
(4)

where θ is the parameters vector of the participation model, and Wh = 1 if the h-th household participates in the program or Wh = 0 if it does not participate. The most frequently used functional forms for F are the normal or logistic probability distribution functions (Guo and Fraser 2010). The propensity scores is here estimated p(Z h ) using a logit model with the dependent variable coded as 1 for participant households and 0 for non-participants.

As concerns the Z h vector, all the explanatory variables which could be used to motivate the household decision to participate in the program were included. However, only variables considered statistically significant (at least 10%) were kept in the model to ensure the best model fit (Table 5).

Table 5 Project participation model
Full size table

Land areas in hectares, the availability of a tractor, are all factors positively influencing participation, while land ownership negatively influences project participation. As concerns age, the model found the existence of a non-linear (quadratic) relation Fig. ( 4).

Fig. 4
figure 4

Estimated relationship between age, farm size and household probability to participate in the project

Full size image

According to the sample data collected, several wild fruit species are cultivated and maintained by farmers. Apricot is the most frequent fruit, and it is cultivated by more than 40% of the interviewed households. Other widely-cultivated fruits include grape, pear, pomegranate, fig and alycha (with a frequency bigger than 10%) Fig. ( 5).

Fig. 5
figure 5

Relative frequency of fruit presence among the interviewed households

Full size image

As concerns the use of the target wild fruits, fresh fruit marketing and self-consumption are the most frequent uses for all the cultivated species, with the exclusion of the mulberry where the processing of the fruits exceeds other types of valorization Fig. 6.

Fig. 6
figure 6

Revealed use of the fruit (%)

Full size image

Table 6 reports the estimate of parameters \(\hat{\beta }\) measuring the impact at household level of the project for the different outcomes implemented (see appendix 1 for technical details): results strongly support the hypothesis that the participation to the activities implemented by the project had a positive and significant effect on two of the three indices measuring the species diversity of wild fruit. More specifically, the results show a positive effect of the application of interventions on Equitability (Evenness) and Simpson Index of Diversity, increasing respectively +61 and +39%. The area cultivated in targeted fruits, as well as the consumption and marketing thereof also increased, by 5.3 and 10% respectively. As concerns the second category of outcomes—those related to livelihood measures—the participation in the project and the interventions themselves had a positive impact on the appliance and livestock indices (around 3%), but these were not statistically significant.

Table 6 Estimates of differences in differences with kernel matching
Full size table

To conclude, while the impact assessment analysis demonstrated a positive and significant impact of project interventions on diversity of fruit tree species, it seems unable to demonstrate statistically significant impact in the household livelihoods sphere. One possibility is that not enough time elapsed between the end of the project and the evaluation, wherein more time may be necessary to accurately capture the impact on livelihoods.

Conclusion

The scope of this article was to analyze and evaluate the impact of a program dealing with in situ/on-farm conservation and use of fruit species (including wild relatives) in Uzbekistan. The project assessment was developed using the theoretical framework developed by Bellon et al. (2015a), that provides a series of linked hypotheses needed to assess the success of a in situ/on-farm conservation program. One of the most common objective faced by institutions and scholars that implement development programs aiming to promote the sustainable use of agricultural diversity in rural settings is the identification of the intervention pathways needed to reach a significant and tangible impact. The assessment presented in this study considers the sustainability of the impact (White 2010) as a key aspect of the development efficacy. From an empirical point of view, the assessment was carried out using a quasi-experimental (observational) design for capturing the impact of project participation on target fruit tree diversity, self-consumption, marketing and household livelihoods.

Based on the quantitative results of this study, the following conclusions can be summarized: species crop diversity maintained by farmers before and after the project increased as a result of the interventions. An increase in the Simpson index of diversity (+39%) and Equitability index (+61%) were demonstrated, a clear sign of the value of the intervention promoted by the projects. These results are not unexpected, as the program under investigation was specifically designed to promote the conservation and use of wild fruit diversity. However, an understanding of the extent of this impact, and its statistical validation is important. Moreover, this outcome can be considered in order to provide useful insights about the effectiveness of different interventions to policymakers. As discussed by Bellon et al. (2015b) an effective and sustainable program for supporting farmers to maintain in situ/on-farm diversity on-farm has to generate positive livelihood outcomes as well. Estimates show that participation in the UNEP-GEF program increased both household propensity of marketing (+8%) and self-consumption of target wild fruit (+11%). While higher and more reliable levels of consumption and/or marketing of these fruits are relevant prerequisites to generating benefits from the wild fruit diversity for farmers, no significant impact was observed in broader livelihood outcomes. Indeed, in terms of outcomes related to livelihood measures, participation in the program showed only a small positive impact on the appliance of asset-based indicators of wealth and livestock (+3%). But this observable impact was not statistically significant. Two possible explanations of this lack of observable evidence of program impact is the short time elapsed between the end of the project and the impact assessment, likely too brief to capture the impact on livelihoods and the sample size too small to evaluate the outcome. However, these are only two possible explanations of the results. What is beyond doubt is that the effectiveness of a program focusing on the sustainable use of agro and tree diversity depends on all stakeholders both directly involved in the interventions (households) and indirectly involved (institutions and research), and by their interactions (Caracciolo et al. 2011).

Previous attempts in the region in the promotion of genetic diversity of fruit species failed for a series of reasons, including lack of information on the benefits derived from genetic resources, limited social and economic resources and the requirement of a collaborative integrated approach among main stakeholders. Further research should also take these aspects into account. Finally, recommendations can be made on the applicability and generalizability (external validity) of these findings. Generalizability cannot be addressed trying to apply the same interventions and associated activities over many villages and households, but rather needs a critical re-organization and contextualization of the interventions in which diverse activities are assembled and targeted to fit different contexts, letting households select which ones fit best under their own situations. This also means that self-selection, households motivation, and capacity to participate are key elements of the process that need to be explicitly considered.