Methodological Pre-analysis on Goodness of Fit (Prediction Improvement Value analysis)
Figure 2 shows the adoption rates, Error Rates and PIVs of the BBNs for each of our eleven practices of interest. As discussed in the methods section, the Error Rate highly depends on the adoption rate of the practice, where it is highest for practices with adoption rate close to 50%. Correspondingly, the PIV is also highest for those practices as the improvement potential is larger. Regarding the direction of the PIV, we observe that most target nodes have positive PIV, meaning that prediction with the BBN is better than predicting independently of the influence variables, based only on adoption rate. Nevertheless, there are three practices with negative PIV, which can happen, for example, due to too little data, especially when adoption rate is so low that the predictor variables cannot distinguish adopters from non-adopters. We excluded ‘Cover crops’, ‘Pressure bomb’ and ‘Variable rate GPS’ from the following analyses as they have PIV < 0%; but we assume our model to have explanatory power for the remaining practices. Values indicating special reliability (PIV > 10%) were found for ‘Leaf testing’, ‘Soil testing’, ‘Fertigation’, ‘Foliar application’ and ‘Moisture probe’, so these were selected for the policy experiments in Section “Effect of engagement-related policy experiments across different farm types”.
Importance of Influence Factors for BMP Adoption
First, we examined the general influence of the predictor variables on practice adoption (research question R1). We investigated the mean sensitivities of the target node to the predictor variables, which are depicted in Fig. 3, together with the direction of influence, i.e. whether a variable increases or decreases adoption probability. A table with the sensitivity data for each individual practice can be found in the Online Resource E.
The three categories of variables defined in the methods section will guide the closer analysis.
Farm characteristics are consistently the most important of the three categories.’Irrigation System’ and’Farm Size’ demonstrate similarly high sensitivity values and rank second and third, respectively, as influence factors; ‘Crop Type‘ ranks fourth in influence.
This confirms results found in Rudnick et al. (2021) that highlight the importance of structural farm variables in determining the feasibility of BMP adoption. It also corroborates past farmer adoption literature that explains the relationship between structural farm variables and adoption: Farm size correlates with access to capital and technical expertise and provides an economies of scale advantage in adoption (Caswell et al. 2001; Daberkow and McBride 2003). Pressurized irrigation systems show more innovation and access to technical capital, additionally some of the considered BMPs were co-evolved with pressurized irrigation systems (Hanson et al. 2009; Taylor and Zilberman 2017). Perennial cropping systems indicate long-term thinking as well as greater access to financial resources (Blank 2001; Ghadim et al. 2005; Marra et al. 2003).
The farmer characteristics show varying importance: ‘Income’ is the most reliable indicator for adoption of all considered variables. ‘Education’ and ‘Years in Farming’ are less consistent predictors of adoption, and rank behind the structural farm characteristics variables.
Income not only indicates access to financial capital to overcome the upfront cost of adoption, but also strongly correlates with farm size (Prokopy et al. 2019). The finding about ‘Education’ and ‘Years in Farming’ is consistent with the broad literature reviews of farmer adoption literature in the USA, where these variables don’t always have consistent impact on predicting adoption (Baumgart-Getz et al. 2012; Prokopy et al. 2008; Prokopy et al. 2019).
The engagement variables rank in the lower half of all predictor variables, regarding their sensitivities. ‘Number of Information Sources’ still shows a similar sensitivity value as ‘Crop Type’, followed by ‘Self-Certification’. ‘Consultant’ is one of the least important influence factors of all variables considered.
Access to (and quality of) information has been found a particularly important variable across many studies (Baumgart-Getz et al. 2012; Prokopy et al. 2019). The rather low influence of ‘Self-Certification’ and ‘Consultant’ can partially be explained together with the finding that these variables show great heterogeneity across management practices. We hypothesize that not all the considered practices receive attention in the ‘Self-Certification’ courses or in the consulting of the N management advisors, respectively. Thus, these variables show high sensitivity values only for some of the practices.
In a second step, we compared the sensitivities of adoption to the predictor variables for different management practices. The sensitivities for each practice are presented in the Online Resource E. Most of the findings above are consistent over all the practices; for example, ‘Farm Size’ and ‘Income’ are almost always among the top three most decisive variables, whereas ‘Years in Farming’ and ‘Education’ never play a notable role. Nevertheless, for some practices significant deviations from the general influence pattern could be observed, especially when it comes to the role of the engagement variables. For example, for ‘Foliar application’, the sensitivities are quite low in general, but ‘Number of Information Sources’ has a surprisingly high sensitivity value. For the ‘Soil testing’ practice, ‘Crop Type’ and ‘Irrigation System’ are not as important, but ‘Self-Certification’ has a comparatively high influence on adoption. This heterogeneity is due to technical idiosyncrasies of the individual practices and the (practical) knowledge spread in the different forms of engagement possibilities. For instance, the finding for ‘Soil testing’ may be explained by the fact that testing the soil for residual nitrate is independent of the crop type and irrigation system while it requires expertise to interpret the data accordingly. This specialized knowledge may be gained mainly from the progressive Self-Certification courses.
We can conclude that in our case the basic influence dynamics towards BMP adoption can be described as follows: ‘Income’ and the structural farm characteristics are the most important variables and mainly suffice for prediction. Engagement plays a complementary role. However, we focus the policy experiments on the engagement variables as they are easiest to influence and do still have a notable influence on adoption.
Effect of Engagement-related Policy Experiments Across Different Farm Types
Next, we present the results of the policy experiments described in Section “Analysis tools and policy experiments”. First, we investigated the general effect of increasing farmers’ engagement on adoption of different practices using the previously mentioned six different policy interventions; second, we compared the effect across the different types of farms. The five considered practices are the ones with particularly convincing modeling performance of the BBN (see Section “Methodological pre-analysis on goodness of fit (Prediction Improvement Value analysis)”).
Figure 4 shows the absolute change of the adoption rate for each practice for the full intervention (where all farmers are set to the ‘engaged’ state of the resp. variable).
The influence of the interventions is heterogeneous across different types of practices. Increasing the number of information sources strongly increases the adoption of ‘Foliar Application’ and ‘Soil testing’ but has almost no effect on the adoption of ‘Moisture probe’. The ‘Self-Certification’ full intervention seems to be fruitful for ‘Soil testing’ and ‘Fertigation’; on the other hand, it doesn’t increase the adoption of ‘Foliar Application’. Hiring an N management consultant leads to much higher adoption of ‘Leaf testing’ and ‘Soil testing’ but has almost no effect on the adoption of ‘Fertigation’. The picture for the normalized interventions (where only 10% of farmers change to the ‘engaged’ state) looks similar, but with smaller absolute impacts (see Online Resource F). This mirrors our earlier finding that the single engagement variables can vary strongly in their influence on adoption across practices. The reasons lie in the idiosyncrasies of the different forms of engagement. For instance, the last finding about the ‘Consultant’ intervention may be explained by the fact that consultants are often the ones to take samples for ‘Leaf testing’ and ‘Soil testing’ and make their product recommendations based on the tests. ‘Fertigation’ instead doesn’t require a consultant, and rather is implemented directly by the farm or irrigation manager; thus, adding a consultant may do little to influence adoption of this practice.
This suggests that policy interventions should be tailored to the specific challenges and features of different practices.
Finally, we look at the effect of policy interventions across the different types of farms, averaged over all considered practices and engagement channels, but for both the full and the normalized interventions (Fig. 5; the full individual data for each practice and each intervention can be found in the Online Resource F). For this we use the typology of farms developed in the methods section classifying farms along two dimensions: large farms versus small farms (L- vs. S- farms) and farms with perennial crops and a pressurized irrigation system versus farms with annual crops and non-pressurized irrigation (-PP vs. -AN farms), resulting in the four different farm types LPP, LAN, SPP, and SAN.
Comparing -PP with -AN farms, the effect on adoption rates of all interventions is always greater in the -PP farms than the -AN farms. According to the authors’ experience, farms that have a pressurized irrigation system and grow perennial crops are more technologically and economically sophisticated than -AN farms, potentially indicating higher all-around levels of innovation. Our overall finding on the effect of interventions suggests that innovative and well-resourced farmers may be more responsive to increased engagement activities than less-resourced farmers in terms of BMP adoption rates. This may be due to the fact that there is more outreach tailored to -PP farms, with practices adapted to best suit -PP farms and field days often featuring trials on -PP farms, so these better respond to an increase of farmers’ engagement. Another explanation may be that by virtue of their overall attitude towards and capacity for innovation, innovative farmers are more capable of learning and incorporating new knowledge. In this way, increased engagement for innovative farms leads to heightened learning on the BMPs under consideration, thereby increasing adoption.
If one compares large and small farms under the policy interventions, the picture is less clear. The effect of the normalized interventions on BMP adoption is not consistently higher for either. However, farmers with large farms already show a greater engagement (79% of the farmers with large farms are self-certified vs. 47% of the farmers with small farms, ‘Number of Information Sources’: 54% vs. 36%, ‘Consultant’: 38% vs. 42%). Thus, putting all farmers in the ‘engaged’ state of the respective engagement variable is less of a change from the observed data for large farms than for small farms. Therefore, regarding the full interventions, the farmers with small farms respond better to the measures.
Altogether our results suggest that it is essential to fit the policy intervention to the targeted nitrogen management strategy and to the farm type that is targeted for adoption. Strengthening the engagement possibilities in their current form has more effect on farmers with -PP farms than on farmers with -AN farms, indicating that an adaptation of these possibilities may be needed to better suit the -AN farms. Such a general statement cannot be made about different sizes of farms, but farmers with small farms show more unexploited potential in increasing their engagement and thus adoption.
Discussion of the Method
BBNs were a helpful tool for investigating the influence factors on BMP adoption. However, they require a cautious approach for various reasons. We discuss the advantages and limitations of using BBNs in the given research context.
BBNs always take the whole causal structure of the system into account. In contrast to more conventional statistical methods such as regression models, interdependencies between predictor variables are modeled explicitly. Thus, BBNs align with farm systems thinking, i.e. thinking of the multitude of agronomic, economic, ecological, and socio-behavioral factors influencing decisions on a farming operation.
BBNs enabled us to quantify the effect of external policy interventions on practice adoption in a natural, reasonable way. They are particularly useful for prediction of explicit probability distributions of the target variable in various scenarios (Celio and Grêt-Regamey 2016; Drees and Liehr 2015). This focus on explicit prediction of adoption probability has several other advantages: For example, it allows for applying a fully calibrated BBN to predict adoption in new contexts where fewer data are available. Moreover, BBNs could be used to model the farmers’ decision-making part in related agent-based models (ABMs) (Pope and Gimblett 2015; Sun and Müller 2013). Such an incorporation in an ABM would allow an investigation of the dynamics over time and, by coupling with biophysical models, to explicitly consider the social-ecological feedbacks in the system, such as the evolvement of N content in soil or groundwater over time, depending on specific policy interventions.
At the same time as entering the policy interventions in the model, we could also specify characteristics of the farm under consideration. Hence, the method allowed us to compare the effect of policy measures across different farm types and therefore to carry out predictive analyses that are limited under traditional linear statistical analysis. Moreover, we could assess the importance of influence factors across different practices by just changing the target node. Therefore, we found BBNs a flexible tool for different kinds of heterogeneity analyses.
While conducting our study, we discovered another analysis method BBNs offer which is the comparison of the predictive power of different network structures (Done and Wooldridge 2004). Though in the end we didn’t include them in our study, we initially investigated multiple network structures and consider this a promising approach for further research. By analyzing which network structures better explain the observed data, new knowledge about the role of and interactions between predictor variables may be derived.
Finally, as often, when survey data is used, we had to cope with a lot of incomplete data cases. For the artificial intermediate nodes there was no data at all. It turned out an important advantage of BBNs that there are reliable algorithms for learning the CPTs which are able to handle cases with missing data (Uusitalo 2007). Nevertheless, it should be kept in mind that extrapolating data can have a large impact on the simulated outcomes of the model and replaces a complete data set only at the prize of higher uncertainty.
The most difficult step in the construction of our BBN was fixing the network structure. The structure influences the results in a non-negligible way; however, often multiple structures are found equally reasonable. Compared with a conventional statistical analysis, this adds a dimension of uncertainty to the results. In our case, we relied heavily upon the use of theory and expert knowledge for fixing the structure. This agrees with the experience made in other studies dealing with BBNs in an environmental context (Kleemann et al. 2017a; Uusitalo 2007). Though we tested an algorithm to learn the structure of the BBN from the data, the resulting structures were not reasonable in any way (e.g. ‘Farm Size’ pointing on ‘Education’) and did not show better goodness of fit values either. Other sources recommend to use structure learning only in combination with expert knowledge and when a high amount of data is available (Alameddine et al. 2011; Kleemann et al. 2017b; Sun and Müller 2013). Another technical issue we encountered while constructing the network structure is the need for avoiding too many links pointing onto the same node due to the risk of overfitting (see 3.1). Artificial intermediary nodes partially solved this problem in our study (Drees and Liehr 2015).
Care has to be taken not only while building the network structure but also when comparing different networks in the analysis. We already pointed out that the Error Rate is a good indicator for the predictive power, but not that appropriate for evaluating the goodness of fit, if networks with different target nodes are compared. Instead we introduced a new measure, the PIV, which we consider more meaningful when it comes to the question which networks represent the influence dynamics well. The idea of comparing the Error Rate of the model with the prediction accuracy of a comparative method was already established in Drees and Liehr 2015; yet, without explicitly explaining the methodological issue behind it. The PIV gives a natural way to overcome the bias of the Error Rate and is ready to be used in further research due to the detailed description provided in this paper. However, the PIV is still biased by the adoption rate of the target node (see Section “Goodness of fit and model validation”).
Additionally, the sensitivities of the target node to the variables in different network structures should not be compared without further ado. They strongly depend on the number of layers as well as the total number of variables in the network (Ticehurst 2010).
Further disadvantages of using BBNs in the given context include the need to discretize continuous variables such as farm size and income which reduces information granularity (Uusitalo 2007). It is also quite difficult to model feedback loops and add temporal and spatial dimension (Landuyt et al. 2013). This becomes important if one wants to closer investigate the long-term biophysical impacts of political interventions on the environment and could be better dealt with in an ABM (Pope and Gimblett 2015; Sun and Müller 2013).