1 Introduction

Smallholder farmers, particularly in developing countries, are among the groups experiencing the most direct impacts of climate change (Morton, 2007; WMO, 2019). These direct impacts might disrupt or reverse development achievements such as poverty reduction and food security (Beg et al., 2002; J. Hansen et al., 2022; WMO, 2019). Climate services are essential for achieving almost all Sustainable Development Goals (Griggs et al., 2021; Machingura et al., 2018). In agriculture, agro-climate services (ACS) can provide support to SDG1 (i.e., ending poverty) and SDG2 (i.e., ending hunger) by better informing farmers’ decision-making and therefore reducing their vulnerability and safeguarding their farm productivity and income (J. Hansen et al., 2022; Machingura et al., 2018; WMO, 2019). For example, ACS can provide information about local climatic conditions and support strategic agricultural planning. ACS can also provide medium to short-term climate and weather information to support daily to seasonal agricultural decisions (Born et al., 2021; FAO, 2019a; J. Hansen et al., 2019; Loboguerrero et al., 2017). Depending on the context, the returns on investment generated by investing in ACS may vary. Nevertheless, the benefits often outweigh the costs (Ferdinand et al., 2021; Luu et al., 2022; WMO, 2015). Typical benefits of ACS include increased harvests, efficient use of agricultural inputs, reduced harvest and input losses, farmer empowerment, improved food security, gross domestic product growth, cleaner water and reduced greenhouse gas emissions (Ambani & Percy, 2014; Ferdinand et al., 2021; Luu et al., 2022; WMO, 2015). In the context of climate services, “last-mile” delivery of information is considered one of the strategic priorities for improving climate change adaptation in the agriculture sector at scale (FAO, 2019; Ferdinand et al., 2021; J. Hansen et al., 2019; J. W. Hansen et al., 2019; WMO, 2019)

The “last-mile” provision of ACS, however, faces critical challenges regarding the delivery of actionable advice to smallholder farmers, especially in developing countries (J. Hansen et al., 2019; Simelton & McCampbell, 2021; WMO, 2019) Agro-climate information may be disseminated, but it is not necessarily received or used by farmers. For example, a recent project in Myanmar aimed to provide agricultural advice and emergency alerts to 150,000 subscribers of the Site Pyo app. While the project actually met this target, the share of active users was below 20% (Simelton & McCampbell, 2021). Moreover, information that reaches farmers might not be used effectively. While, in general, the provision of climate services has increased, there is often a mismatch between service providers and users about what constitutes useful information. This difference can be depicted as a usability gap (Lemos et al., 2012). Related to this gap, Perrels et al. (2013) and Pilli-Sihvola et al. (2014) highlight the risk of decaying value of climate information services, which may lead to a gradual decline of potential benefits during information delivery and uptake (i.e., from forecast generation and accuracy → user orientation → users’ access to information → users’ comprehension → users’ ability to respond → effectiveness of users’ response). For example, if the value of the climate information was mainly dependent on the accuracy of a forecast, very high potential benefits might be realized. However, as the climate information value might be dependent on the appropriateness of the information to different user groups, moderately high potential benefits may remain. Furthermore, when the realization of the final benefits is also dependent on uptake dynamics, including access to information, comprehension of information, application of advice and effectiveness of the application by users, the realized benefits may fall far short of the initial potential (Perrels et al., 2013; Pilli-Sihvola et al., 2014).

The use of diffusion of innovation theory often fails to recognize the dynamic and complex nature of innovation processes in dynamic real-world contexts. Oversimplified and linear views in innovation diffusion theory hinder understanding of adoption processes (Geels & Schot, 2010). ACS innovations, especially digital ACS, can suffer from the use of a narrow, supply-driven approach (Daniels et al., 2020). Simelton and McCampbell (2021) reveal the weaknesses in designing digital climate services, which may fail to effectively involve farmers and thus overlook the needs and social settings of farmers adopting new technologies. In development contexts, climate service design is largely modeled after traditional development project design, generally based on a logical framework, a framework often used to define the key hierarchical changes given the project interventions (Bong, 2014; Springer-Heinze et al., 2003). Intervention inputs such as data, model development and capacity building, are expected to create outputs (e.g., number of agro-climate bulletins). These outputs are then assumed to generate outcomes, e.g., a particular number of innovation adopters. However, strong assumptions about the linear causal mechanisms behind the logical framework can be problematic in complex contexts (Springer-Heinze et al., 2003). When the project impact pathway is not made explicit, important processes that might trigger or block expected and unexpected changes may be neglected (Bong, 2014; Springer-Heinze et al., 2003). In consequence, influencing factors in the intervention delivery processes may not be reflected and addressed (Bong, 2014; Springer-Heinze et al., 2003; Vogel, 2012). As delivery and adoption pathways evolve and grow in complexity (Haigh et al., 2018), the importance of understanding the dynamics of these pathways increases.

Previous studies on “last-mile” delivery and adoption have outlined important factors that can influence the success of climate services. Examples of these factors include meeting farmers’ need for information, supporting access and use of information, co-production of climate services and capacity building for climate service stakeholders (Alexander & Dessai, 2019; Born et al., 2021; Hansen et al., 2019; Nkiaka et al., 2019; Rossa et al., 2020; Simelton & McCampbell, 2021). However, last-mile delivery and adoption studies have generally had a limited focus on adoption pathways. A better understanding of the causal pathways by which farmers access and adopt ACS may help in designing and adjusting ACS interventions. They may also reveal the social dynamics of climate service adoption, delivering answers to critical questions: Do farmers find their needs satisfied by ACS? Do they reject any specific ACS advice? Do farmers recommend ACS to their peers? Is there a critical mass within a social group that supports the outscaling of interventions?

Impact pathways offer a flexible approach to understanding the processes behind agricultural innovations by explicitly including the logical and ordered sequence of events leading to outcomes. Thus they provide the opportunity to elucidate the potential outcomes of changes to processes (Springer-Heinze et al., 2003). Links within these impact pathways represent hypotheses that can be further validated (Springer-Heinze et al., 2003; Vogel, 2012). Findings from the validation can serve as important guides for improving development interventions (Vogel, 2012).

In this study, we apply the impact pathway approach to improve our understanding of farmers’ decisions for or against receiving and adopting climate services. We assess a case study on the implementation of co-produced ACS for two different farmer group settings in Muong Phang and Pa Khoang communes in Dien Bien District, Vietnam. We demonstrate a novel approach that integrates a desk review with participatory exploratory discussions to map out impact pathways of farmers’ decision-making and to reveal insights about the delivery and adoption of ACS. Based on a farmer survey, we validate and determine the strength of relationships between components of the impact pathways and draw recommendations to improve ACS interventions in a development context.

2 Materials and methods

2.1 The agro-climate service projects in Dien Bien district

The study took place in Dien Bien District where eight ethnic groups live together (Dien Bien People’s Committee, 2009). The district has a high poverty rate of 17.1% (Dien Bien People’s Committee, 2019) and the majority of residents are rice farmers, known for producing high-quality rice (Agrifood Consulting International, 2006). In recent years, the region has been experiencing major changes in climate and weather patterns. According to data from the Vietnamese Institute of Meteorology, Hydrology and Climate Change, the average annual temperature in Dien Bien increased by 0.74 °C between 1961–1990 and 1991–2018. Farmers have reported increased occurrence of cold spells, flash floods, landslides, drought, hail, floods, erratic rainfall, frost, extended heavy rain and early start of the rainy season (Luu et al., 2022).

In our study, we rely on three multi-stakeholder projects led by CARE International in Vietnam (CVN), which provided ACS in Dien Bien District from 2015 until 2021. These ACS interventions aimed to support farmers with information about adapting rice farming systems to changing climatic conditions to improve system outputs and reduce farmers’ vulnerability to climate change.

In an attempt to address the “last-mile” delivery, CVN’s projects focused on the co-generation of weather forecasts and agricultural advice through seasonal participatory scenario planning (PSP) workshops (CARE, 2018). The stakeholders that CVN engaged in PSP workshops included weather forecasters from the Provincial Hydro-Meteorological Station, agricultural planners and agricultural extensionists from the Provincial and District Departments of Agricultural and Rural Development, NGOs/Women’s Unions and farmer champions (CARE, 2018; Simelton et al., 2019). Farmer champions included village leaders and the head of the Village Saving and Loan Association (VSLA). VSLAs are self-selected groups of 20–30 women in each project village (CARE in Vietnam, 2018), who meet almost every week. Based on the PSP outputs, CARE released a printed seasonal bulletin containing indigenous and scientific seasonal forecasts, analysis of the climate impacts on rice farming and advice on the seasonal calendar and farming practices. Since 2018, weekly bulletins were produced by weather forecasters, agricultural extensionists and NGOs without the participation of farmer champions.

The project shared ACS interventions through two different methods, through non-structured/conventional or through structured processes. In the non-structured/conventional ACS intervention, printed seasonal bulletins were distributed at conventional village meetings to both VSLA members and non-members. In the structured-processes ACS intervention, meetings with VSLA group members included additional and structured communications and discussion to explain and exchange on the ACS bulletin contents in detail. Weekly bulletins were also sent to VSLA members via text messages (Table 1).

Table 1 Type, generation and communication mechanisms of agro-climate services (ACS) to Village Saving and Loan Association (VSLA) and conventional farmer groups (non-VSLA) in Muong Phang and Pa Khoang communes, Dien Bien District, Vietnam
Full size table

CVN also integrated gender activities into its projects. These activities included training on gender equality for participants engaging in the PSP. In the VSLA groups, activities such as gender norm realization, reflection and norm change dialogues were also integrated into the groups’ activities.

2.2 Adoption pathway development

We conceptualized an impact pathway for CVN's project based on the project’s design, the project’s logical framework, innovation diffusion literature and personal communication with key informants from the project. We held one focus group discussion (FGD) with 12 VSLA farmers and another with 12 non-VSLA farmers to capture their views on possible impact pathways of ACS. During these focus group discussions, we asked farmers to individually reflect on their experiences in accessing and applying ACS. After that, we encouraged farmers to share their views in group discussions. We then synthesized the information provided by the farmers and integrated it into the previously drafted version of the pathway to derive a consolidated impact pathway model.

The resulting model comprised interacting factors and their relationships in the form of nodes and arrows (directed links). Each arrow connecting two interacting nodes represents one sub-hypothesis. One sub-hypothesis consists of a hypothesized causal and a resulting event (e.g., Read → Understand), both binomially distributed (causal event [yes/no] → resulting event [yes/no]). The aggregation of sub-hypotheses constitutes the larger system hypothesis (e.g., Access information → Read → Understand → Adopt). We tested each sub-hypothesis to construct the overall understanding of the system hypothesis, using a farmer survey and our proposed testing procedure.

2.3 Farmer survey

After developing the impact pathway, we developed a household questionnaire (see supplementary information) with questions about farmers’ rice production, about the impact of weather and climate on farming and about farmers’ access and practice of ACS to conduct a farmer survey. Our overall approach to determining the number of farmers to survey was based on the available resources and the feasibility of conducting the survey (i.e., in our case, it was conducted during the COVID-19 pandemic). We consider that even with a limited sample size, the application and interpretation of confidence intervals (as mentioned in Sect. 2.4) can still offer valuable insights. In the specific context of the VSLA and non-VSLA farmers, the number of farmers to survey was determined to ensure a balanced representation of both groups. We randomly selected one household from each of the 41 VSLA and 41 non-VSLA groups. We based the household selection on the lists of households in the villages and the VSLA member lists by selecting the 10th name from each list. In case of any issue arising with these selected households (e.g., failure to meet the interviewee), the next household on the list was selected. Since members of VSLA groups were all women, we selected women as the respondents for both VSLA and non-VLSA households. The survey, therefore, reflects women’s perceptions of the adoption of ACS in their households. We collected additional information during surveys on contextual variables such as age, gender, household size and income (Table 2).

Table 2 Socio-economic characteristics of 41 Village Saving and Loan Association (VSLA) and 41 non-VSLA households in Vietnam’s Muong Phang and Pa Khoang communes in Dien Bien District
Full size table

We employed six local enumerators to conduct surveys. We trained them in data collection including sampling, interviewee identification, questionnaire content, gender sensitivity and techniques to avoid potential interference from other respondents or peers. We used the KoboCollect app (Harvard Humanitarian Initiative, 2020) to gather data on phones and tablets directly or from notes written in the field and checked the collected data for consistency and completeness at the end of each day. If necessary (i.e., if collected data are inconsistent or incomplete), we followed up with the respondents.

2.4 Testing relationships in the adoption pathway

We used the survey data to validate the relationships in the adoption pathway identified before and during the workshop. We quantified the overall “success” rate in terms of the adoption aspects for every node of the adoption pathway. We further tested each relationship between sets of two nodes by comparing the success rate in terms of adoption aspects in the presence and absence of the hypothesized causal event. For example, we wanted to test if understanding would lead to a positive perception of ACS (Understand → Perceive ACS positively). We then examined the difference in positive perception rates (successful event [yes]) among farmers who understood (hypothesized causal event [yes]) and did not understand ACS (hypothesized causal event [no]).

To implement the test, we first calculated the two probabilities p1 and p2 of the successful event [yes] attributable to [yes] and [no] observations of the hypothesized causal event. We calculated p1 as the probability of observing successful event [yes] together with hypothesized causal event [yes] observations and p2 to be the probability of observing successful event [yes] occurrence together with hypothesized causal event [no] observations. The response rates p1 and p2 were estimated from the sample proportions x1/n1 and x2/n2, in which

n1:

Number of [yes] observations of the hypothesized causal event

n2:

Number of [no] observations of the hypothesized causal event

x1:

Number of successful event [yes] attributable to [yes] observations of the hypothesized causal event

x2:

Number of successful event [yes] attributable to [no] observations of the hypothesized causal event

Since the occurrence of all events was binomially distributed, we quantified the 95% confidence interval (CI) for the difference between the two success rates p1 and p2 (δ = p1−p2) using the ciBinomial function of the gsDesign package (Anderson, 2021) for the R programming language (R Core Team, 2020). All the data, functions, tests and scripts are provided in a public repository (https://github.com/ThiThuGiangLuu/ACS-adoption-decision-pathway). The CI served as the hypothesis test and also displayed the probability of the population’s parameter with a specific level of confidence (Sim & Reid, 1999). This practice moves beyond the traditional strategy of testing for statistically significant differences, which is often based on a null value (Sim & Reid, 1999). The interpretation of the CI value should be considered in practical contexts. For example, when comparing two ratios, the statistical test might indicate a statistically significant difference between the two ratios. Nevertheless, the CI value may reveal that the probable difference is too small to be meaningful. In that case, the difference might not be important in a practical context. In contrast, the CI might indicate a non-statistically significant difference. However, a large range of the CI might make us reluctant to conclude that it is not important (Sim & Reid, 1999). Based on that premise, we interpreted the relationship by comparing the width of the 95% CI with a pre-defined range of practical indifference, i.e., the reference range used to infer, by comparing its quantitative value with the 95% CI, if the difference between two ratios is practically important or not. If the CI lay completely within the range of practical indifference, the difference was considered practically irrelevant or trivial. In this study, we chose a lower and upper limit of the practical indifference at −20% and 20% to interpret the CI value. We also considered other possibilities of the CI value when compared with the range of practical indifference that might lead to other possible interpretations. Depending on the range of the CI, we considered the hypothesized causal and resulting events to have no, potential, weak, moderate or strong relationships. We adapted the matrix of the strength of evidence by McBride et al. (2013) and proposed our interpretation of the CI for the difference between the two proportions, as illustrated in Fig. 1.

Fig. 1
figure 1

Source Adapted from McBride et al. (2013)

Quantification of the strength of relationships between binomially distributed (hypothesized) causal and resulting events. The quantification is based on the interpretation of a 95% confidence interval (CI) for the difference of the probabilities p1 and p2 of the “successful event” [yes] attributable to [yes] and [no] observations of the hypothesized causal event. The width of the CI is compared with a pre-defined range of practical indifference. Depending on the range of the CI, we considered the hypothesized causal and resulting events to have no, potential, weak, moderate or strong relationships. Multiple arrows in the same row indicate different possibilities of differences between p1 and p2 that would lead to similar interpretations.

Full size image

The modeled events are generally binomially distributed. However, there were cases where the number of [yes]/[no] observations of the hypothesized causal event in the sample size returned 0 (n1 = 0 or n2 = 0). In that case, we could not calculate the ratios x1/n1 or x2/n2 and were unable to test the relationship.

3 Results

3.1 Agro-climate service delivery and adoption impact pathway

Our hypothesized impact pathway of the ACS delivery and adoption decision processes (Fig. 2a) illustrates the most important interacting factors that influence the uptake of ACS. The impact pathway shows how occurrence of climate risks motivates farmers’ risk perception (Risk occurrence → Perceived risk). This increased perception of climate and weather risks raises the likelihood that they will see a need for access to ACS (Perceived risk → Need). Having this need leads farmers to access ACS (Need → Access ACS). As a result of accessing ACS, farmers are presumed to read/listen to ACS (Access ACS → Read/Listen). After reading/listening, farmers are expected to understand the forecasts and advice (Read/Listen → Understand). Farmers will then have a positive perception of ACS (Understand → Perceive ACS positively). Positive perception is expected to trigger the intention to adopt ACS (Perceive ACS positively → Intend to adopt ACS) and then the adoption of ACS (Intend to adopt ACS → Adopt ACS).

Fig. 2
figure 2

Hypothesized agro-climate service (ACS) delivery and adoption impact pathway (a) and dynamics of ACS adoption in Village Saving and Loan Association (VSLA) and conventional farmer groups (non-VSLA) (b) in Muong Phang and Pa Khoang communes, Dien Bien District, Vietnam

Full size image

During and after the initial adoption, farmers may decide whether they would recommend ACS to peers (Adopt ACS → Recommend to peers) and if they confirm their continued need for ACS in the future (Adopt ACS → Confirm continued need). They may also decide to reject specific ACS advice, i.e., they may either refrain from implementing certain recommendations regarding seeds, fertilizer, pesticides and water management, or they may initially follow the recommendations but then decide to stop following specific advice from ACS (Adopt ACS → Reject ACS specific advice). Since we are interested in understanding the dynamics of recommending, confirming need and rejecting advice (on specific seeds, fertilizer, plant protection and water management) among adopters, we test the relationships between VSLA vs. non-VSLA adopters and the possibility of recommending ACS, of confirming the need for ACS and of rejecting some specific ACS advice (Fig. 2b).

During the whole process, farmers may discuss with peers, change agents and other stakeholders to share and exchange information at any time. Due to our limited resources (i.e., we relied on the enumerators to collect data during the COVID-19 pandemic), we did not collect data to test all the relations of “Discuss” with all other observed events; only the relations between Read/listen → Discuss → Understand (Fig. 3) were selected for testing.

3.2 Impact pathway validation

The household survey served to validate the hypothesized ACS delivery and adoption impact pathway (Fig. 3). The results show that the adoption impact pathways differ between VSLA and non-VSLA groups (Fig. 3a). For VSLA groups, tests indicate potentially relevant relations for six connections (Perceived risk → Need; Need → Access ACS; Read/Listen → Discuss; Discuss → Understand; Understand → Perceive ACS positively; Intend to adopt → Adopt).

In non-VSLA groups (Fig. 3b), the tests show potentially relevant relations for five connections (Risk occurrence → Perceived risk; Perceived risk → Need; Need → Access ACS; Access ACS → Read/Listen; Read/Listen → Discuss). In two cases, tests indicate moderate relationships (Understand → Perceive ACS positively; Intend to adopt → Adopt). In two other cases, tests reveal strong relationships (Discuss → Understand; Perceive ACS positively → Intend to adopt).

Fig. 3
figure 3

Testing results of an impact pathway describing the farmer’s decision-making processes in adopting agro-climate services in Muong Phang and Pa Khoang communes, Dien Bien District, Vietnam

Full size image

Detailed results of all the tests, including the sample, observation and success rates, are available in a public repository (https://github.com/ThiThuGiangLuu/ACS-adoption-decision-pathway).

3.3 VSLA versus non-VSLA adopters

In addition to the adoption impact pathway results, our survey also revealed some differences in attitudes and behavior between VSLA and non-VSLA adopters, including the confirmed need for continued ACS, peer-to-peer scaling and intention to reject specific seed, fertilizer, plant protection and water management advice (Fig. 4). Our tests indicate no potentially relevant relations for two connections (VSLA vs. non-VSLA adoption → Confirm need; VSLA vs. non-VSLA adoption → Reject plant protection advice), one moderately relevant relation for the connection between VSLA vs. non-VSLA adoption → Scale out (recommend to peers), and three potentially relevant relations (VSLA vs. non-VSLA adoption → Reject seed advice; VSLA vs. non-VSLA adoption → Reject fertilizer advice; VSLA vs. non-VSLA adoption → Reject water management advice).

Fig. 4
figure 4

Testing results of the dynamics of agro-climate service adoption among Village Saving and Loan Association (VSLA) groups and non-VSLA groups in Muong Phang and Pa Khoang communes, Dien Bien District, Vietnam

Full size image

Our survey revealed some further insights regarding the scaling out among households as well as the reasons for rejecting seed, fertilizer, plant protection and water management advice.

Regarding scaling out, among 79 VSLA and non-VSLA households that responded to the scaling question, 61 households stated that they recommended ACS to other farmers. In total, VSLA and non-VSLA households shared ACS with at least 263 others. Altogether, 36 out of 39 VSLA households recommended ACS. They shared the advice with at least 176 other farmers, of which 169 reportedly followed the recommendation. Among non-VSLA households, 25 out of 40 recommended ACS advice to other farmers. They reported sharing with at least 87 other farmers. As a result, 92 farmers (which includes adopters who received indirect recommendations) were reported to have applied the recommendation. The number of recommendations ranged from 0 to more than 5 other farmers per household. In both groups, most recommendations went to neighbors (i.e., people living near or next door to the interviewee), followed by neighboring farm households (i.e., people having farms near or next to the farm of the interviewee), relatives and close friends. Farmers did not report recommendations to any other groups. Regarding rejection of ACS advice, some farmers reported their intention to reject specific seed, fertilizer, plant protection and water management advice, citing various reasons (Table 3).

Table 3 Reported reasons for the rejection of some agro-climate advice among Village Saving and Loan Association (VSLA) and non-VSLA households in Vietnam’s Muong Phang and Pa Khoang communes in Dien Bien District
Full size table

4 Discussion

The results reflect the complexity of the social processes in response to the impacts of climate change and its implications for decision-making in development interventions (Morton, 2007; Rickards & Howden, 2012; Wise et al., 2014). Our impact pathway and test results offer several insights into the decision-making process involved in adopting ACS in different farmer group settings. The results show a relatively similar “starting” point (i.e., level of perceived risk occurrence before the project) followed by similar levels of “adoption” (i.e., level of ACS adoption after 5 years) in VSLA as well as non-VSLA groups (Fig. 3). However, analysis of the delivery and adoption pathway indicated different dynamics in the two group settings. VSLA farmers were more likely to perceive climate-related risks, have a perceived need for and access, read/listen to, discuss and understand ACS, perceive ACS positively, intend to adopt and ultimately adopt ACS. In the non-VSLA groups, farmers had a lower need and were less likely to understand ACS. Nevertheless, the likelihood to adopt was surprisingly high and similar (98%) between the VSLA and non-VSLA groups. These results support indications that the linear assumption often involved in innovation diffusion models (Geels & Schot, 2010; Hoffmann, 2007) and the conventional use of a logical framework (Springer-Heinze et al., 2003) may be oversimplifications. Our results highlight the importance of understanding the processes behind adoption, including interpersonal relationships and the influence of pioneers within societies implementing innovations (Springer-Heinze et al., 2003; Vogel, 2012). Our method for testing these processes offers a novel approach to identifying and quantifying the relationships between events in an impact pathway. Our findings on causal relationships yield insights about the triggers of and barriers to adoption along the impact pathway. These insights can help innovation projects in complex contexts (e.g., development projects) to reflect, learn and adjust their interventions. They also offer an understanding of possibilities and implications for the critical mass needed for outscaling.

The results of our survey reveal a difference in farmers’ perceived needs before and after experiencing ACS. Before CVN’s project implementation, farmers’ perceived need for ACS in VSLA and non-VSLA groups was 75% (30/41) and 54% (22/41), respectively. After experiencing ACS, all households in the two groups stated a need for ACS (39/39). Our findings support previous studies highlighting the importance of realizing the needs of farmers as a critical factor in supporting the “last-mile” delivery of ACS (J. W. Hansen et al., 2019; Nkiaka et al., 2019; Simelton & McCampbell, 2021). However, our results also suggest a potential difference between the stated and the revealed need. Farmers might have a real need for agro-climate information, but may not communicate it or even be aware of it. This might be true in contexts where farmers have limited access to information about climate change. The low need for ACS among farmers may also imply that climate services might not have been clearly defined and communicated to farmers (Lourenço et al., 2016). Awareness-raising on the concept of ACS might be needed to fill the gap between stated need and revealed need. In this case, CVN did create awareness and demand, reaching almost all non-VSLA farms, yet the initial need remained at 54%. Scaling efforts therefore should aim at understanding farmers’ revealed needs.

The high level of comprehension of ACS among VSLA farmers (83%) suggests that structured communications in VSLA groups are effective in supporting farmers’ comprehension of ACS. This finding supports Hansen et al., (2019), who showed that structured communications supporting farmers’ understanding enabled them to relate complex information to their specific contexts and decisions. This was not the case among non-VSLA farmers (i.e., the level of comprehension remained at 49% after five years of project implementation), who did not have frequent contact with project activities and stakeholders. The unstructured communications they received were less effective in increasing their understanding of ACS. This may be explained by the ACS bulletins often featuring technical language and expressions of uncertainty, which may not be easily understood by the remaining 51% of non-VSLA farmers without additional explanation. Past studies have pointed out the difficulties in communicating technical information about weather, climate, uncertainties and other agriculture-related terminologies and principles to farmers (Duong et al., 2020; Lourenço et al., 2016; Simelton & McCampbell, 2021; WMO, 2019). Illiterate people found it particularly challenging to understand the bulletins if they were not supported by others. Ethnic minority farmers may face additional barriers, since they may have to interpret the bulletins in their local languages (CARE in Vietnam, 2013; Nguyen et al., 2021). The results highlight the importance of peer-to-peer learning (Tran et al., 2017). Such personal exchange might be able to partially address the barriers to understanding ACS.

Lack of comprehension, however, does not necessarily prevent the subsequent up-taking process. For example, while the understanding rate remained at 49% in the non-VSLA groups, 85% of those farmers still held a positive perception of ACS. This positive attitude appears to have an important influence on the intention to adopt ACS. The strong relationship between ‘Perceive ACS positively → Intend to adopt’ (Fig. 2) among non-VSLA farmers, might be attributable to the influence of stakeholder involvement in ACS production, peer exchange and opinion leaders. The ACS production and delivery mechanism may effectively make the information ‘credible, salient and legitimate’. ACS is co-produced with credible and legitimate agencies, which have a clear legal mandate to provide such information and local NGOs that have operated in the community for a long time and have gained the farmers’ trust. Contacts of those involved in developing the bulletin are provided, along with their phone numbers. The perception of ACS could be further strengthened through peer exchange and the influence of opinion leaders, such as the heads of the villages and VSLAs. Involvement of village and VSLA leaders might enhance the salience of provided information since these people are close to villagers and they might potentially reflect farmers’ needs and concerns in ACS provision. Ensuring the credibility, salience and legitimacy of ACS advice is crucial to enhancing the use of climate information in decision-making (Cash et al., 2003; Vincent et al., 2018). All this may raise the odds of farmers using the advice to take appropriate action. Development efforts should consider farmers’ attitudes and perceptions in designing and monitoring ACS interventions.

The equally high adoption rates between VSLA and non-VSLA groups despite different levels of understanding have multiple implications. First, the value of ACS benefits might not fully follow a gradual decay process as suggested in the studies by Perrels et al. (2013) and Pilli-Sihvola et al. (2014), i.e., we cannot confirm that the ultimate benefits of ACS adoption are always low if the comprehension level is low. Second, scaling out is likely to be possible given that farmers can learn or simply imitate climate-informed actions through social networks (Tran et al., 2017). In this study, each VSLA household recommended ACS services to at least 4.5 peers, resulting in at least 4.3 new applications of ACS advice. A typical non-VSLA household recommended ACS to at least 2.2 peers, resulting in at least 2.3 new applications of ACS information (including adopters who received indirect recommendations). Third, the high adoption rate, especially when farmers do not understand information, may lead to over-adoption or misunderstood and mistaken adoption. For example, farms in different regions may experience different drought risks. Thus, simply imitating the drought response of another farmer might not be a good strategy. Fourth, adoption rates are promising in both VSLA and non-VSLA groups, suggesting an important role of interpersonal relations and a potential role of a critical mass in outscaling ACS. The high adoption rates also suggest the continuation of such interventions within CVN’s projects. Yet it is still impossible to know at this stage if either of the farmer group settings is ideal for scaling out and scaling up beyond CVN’s project context. Monitoring the quality and consequences of adoption is necessary to understand the overall impacts of innovation and its diffusion (Rogers, 2003). Recommendations on scaling would be more comprehensive if they were preceded by a socio-economic valuation of such ACS interventions.

We found the chance of rejection for fertilizer and plant protection measures to be low, while it was fairly high for some specific advice on seeds (43%) and water management (34%). The reasons for the rejection of seed and water management advice are mostly related to the mismatch between advice and the households’ traditional practices, interests, and resources. Smallholder farmers are diverse, and services are not usually formulated to cater to all the different needs of farmers (Simelton & McCampbell, 2021; VNIFIP et al., 2018). Resources should be invested in understanding the typology of farmers (Cruz et al., 2021; Shukla et al., 2019) and in adjusting the interventions accordingly.

Previous adoption studies on agro-climate services have largely been rooted in variance theory, in which the adoption variable is explained by a linear combination of independent variables. The limitation of the variance approach is the neglect of history and time ordering of events (Geels & Schot, 2010), even though these may have important implications for adoption outcomes. The impact pathway testing approach that we used offers us the chance to understand the dynamics of the hypothesized causal processes, as well as potential blockage or trigger points. The impact pathway insights also offer a concrete way to support reflections on the process from inputs to outputs and outcomes in development interventions. These insights are important in identifying and prioritizing further development research and interventions. Thus, impact pathway testing is crucial to supporting ACS design, monitoring, reflection and learning, and ultimately for creating sustainable impacts.

The use and interpretation of the CI value provide various advantages in both statistical and practical contexts (Brosi & Biber, 2009; McBride et al., 2013; Sim & Reid, 1999). Even with the limited sample size of 41 for each type of farmer group, the CI value still offers a lot of insights into the relationships between the events. This differs from the traditional variance approach, which requires a large sample size (Geels & Schot, 2010). Thus, using the CI interpretation approach is helpful, especially when resources are limited. The traditional interpretation of statistical tests for significance can make it difficult to apply results in decision-making. The interpretation of the CI in a practical context, on the other hand, gives concrete meaning to support decision-making (Brosi & Biber, 2009; Sim & Reid, 1999).

Although we have made efforts to understand the impact pathway of ACS adoption, some limitations and caveats should be considered. For example, we did not provide a concrete definition of ACS to the respondents. The interviewees stated whether they mostly understood or did not understand ACS bulletins. There may have been varying interpretations of ACS and the level of understanding may not have been fully captured. Second, while all the variables in the impact pathway are binomially distributed, there may have been some more qualitative or quantitative information (e.g., the level of detail in understanding) that could have provided further insights. In the results of the impact pathway testing, inferences on causal relations should also be interpreted with caution. Rather than being accepted as proven, they should be treated as improved hypotheses for continued learning, reflection and improvement (Vogel, 2012).

5 Conclusions

Understanding the dynamics of “last-mile” delivery and adoption of ACS is critical for decision-making in sustainability-oriented development interventions. Impact pathway development and testing is a novel approach to generating an explicit overview of the impact pathway relationships in ACS adoption processes. The testing procedure developed in our study offers a robust and rapid tool to validate hypotheses for development interventions. These hypotheses might otherwise be overly simplistic and remain largely untested and unvalidated. The testing procedure also allows for quantifying the strength of relationships, which can be useful in formulating recommendations to support resource prioritization in decision-making. The impact pathway development and testing method may support filling in ACS adoption gaps. Our case study shows that structured communications in farmer groups, demand awareness creation, enhancing peer-to-peer exchange and influencing farmers’ attitudes appear to be crucial to delivering and spreading ACS effectively. Efforts to scale out ACS should consider these important aspects. Future research may focus on studying the impacts of ACS adoption, possibly using this impact pathway testing procedure combined with other ACS impact evaluation methodologies, to support further scaling of ACS. The use of the impact pathway development and testing approach is not limited to adoption or ACS contexts. We expect it to find successful applications in a host of other cause-and-effect processes as well as in outscaling of sustainable development interventions and innovations.