Skip to main content
Log in

Behavioral and Experimental Agri-Environmental Research: Methodological Challenges, Literature Gaps, and Recommendations

  • Published:
Environmental and Resource Economics Aims and scope Submit manuscript

Abstract

Insights from behavioral and experimental economics research can inform the design of evidence-based, cost-effective agri-environmental programs that mitigate environmental damages and promote the supply of environmental benefits from agricultural landscapes. To enhance future research on agri-environmental program design and to increase the speed at which credible scientific knowledge is accumulated, we highlight methodological challenges, identify important gaps in the existing literature, and make key recommendations for both researchers and those evaluating research. We first report on four key methodological challenges—underpowered designs, multiple hypothesis testing, interpretation issues, and choosing appropriate econometric methods—and suggest strategies to overcome these challenges. Specifically, we emphasize the need for more detailed planning during the experimental design stage, including power analyses and publishing a pre-analysis plan. Greater use of replication studies and meta-analyses will also help address these challenges and strengthen the quality of the evidence base. In the second part of this paper, we discuss how insights from behavioral and experimental economics can be applied to improve the design of agri-environmental programs. We summarize key insights using the MINDSPACE framework, which categorizes nine behavioral effects that influence decision-making (messenger, incentives, norms, defaults, salience, priming, affect, commitment, and ego), and we highlight recent research that tests these effects in agri-environmental contexts. We also propose a framework for prioritizing policy-relevant research in this domain.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Notes

  1. Ellis (2010) provides a helpful review of concepts including statistical power, effect sizes, and meta-analysis.

  2. Lakens et al. (2018) emphasize that researchers should also justify their alpha level (i.e. the statistical significance level) along with other decisions when designing a study.

  3. See Brown et al. (2019) for a discussion about statistical methods that can be used to interpret null findings, which can be meaningful and policy-relevant when derived from a well-designed study.

  4. This module accommodates within and between-subject experimental designs that may involve complications such as continuous treatment variables, order effects, and repetition. Further, power calculations can be based on tests derived from a range of econometric estimators, including tobit, probit, and common panel data estimators.

  5. See Feiveson (2002) for additional guidance on estimating the power of statistical tests using Stata.

  6. There is not a universally employed method of standardization. Common alternatives include Glass’s Δ and Hedge’s g.

  7. To be clear, if the standard deviation of the outcome variable is 2.5, a 0.10 standard deviation effect size refers to an unstandardized treatment effect of 0.25.

  8. We have compiled a table of standardized effect sizes reported in experimental economics papers that present behavioral insights which can inform agri-environmental research and program design. The table will be periodically updated to reflect new research and can be found at https://osf.io/cf259/.

  9. Power analyses are already an important component of proposals to fund clinical research, and an inadequate description of power calculations is considered to be a major issue during the review process (Inouye and Fiellin 2005).

  10. Another approach is to control the familywise error rate (FWER)—the probability of falsely rejecting even one hypothesis (i.e., the probability of at least one Type I error). List et al. (2019) present an approach to control the FWER that the authors assert leads to gains in power over Bonferroni-type procedures.

  11. If one can identify the various channels through which an intervention works, it may be possible to design an experiment that turns various channels on and off. See, for example, Ferraro and Hanauer (2014).

  12. For an overall discussion on the usefulness and limitations of randomized controlled trials, see Deaton and Cartwright (2018). For discussion about the need to further develop theoretical frameworks that can be used to generate testable hypotheses, see Muthukrishna and Henrich (2019).

  13. Currently, the AEA registry is set up primarily for the registration of RCTs, but we recommend that registries be updated to accommodate pre-analysis plans for both laboratory and field experiments.

  14. Behavioral nudges were popularized by Thaler and Sunstein (2008) in their book, Nudge, which presents a behavioral economics toolkit for designing more-effective private and government programs and policies. They define a nudge as “any aspect of the choice architecture that alters people's behavior in a predictable way without forbidding any options or significantly changing their economic incentives. To count as a mere nudge, the intervention must be easy and cheap to avoid. Nudges are not mandates.” (Thaler and Sunstein 2008, p. 6).

  15. See Mason and Phillips (1997), Messer et al. (2007), Shogren and Taylor (2008), Kotani et al. (2010), Messer and Murphy (2010), Shogren et al. (2010), Gsottbauer and van den Bergh (2011), Osbaldiston and Schott (2012), Friesen and Gangadharan (2013), Schultz (2014), List and Price (2016), Delaney and Jacobson (2016), Hobbs and Mooney (2016), Ferraro et al. (2017), Reddy et al. (2017), Zarghamee et al. (2017), Brent et al. (2017) and Kesternich et al. (2017).

  16. Although consumer preferences and behavior can drive change in agriculture, we include only studies analyzing producer behavior. This is consistent with the aim of agri-environmental programs, which is to achieve permanent changes in how producers manage impure public goods.

  17. We only review papers that are published in peer-reviewed journals, but we acknowledge that there is a growing body of experimental literature in the agri-environmental domain, and many of these papers are in the review process or in working paper form. We do not believe, however, that including this body of in-progress work would change our overall conclusions.

  18. Duflo et al. (2011) analyze farmer investments in fertilizer and is, therefore, not categorized as an Ag-E paper.

  19. A large portion of this literature tests the outcomes of various tax and subsidy mechanisms to reduce nonpoint source (NPS) pollution (Alpízar et al. 2004; Poe et al. 2004; Spraggon 2004, 2013; Cochard et al. 2005; Vossler et al. 2006; Suter et al. 2008; Spraggon and Oxoby 2010; Cason 2010; Cason and Gangadharan 2013; Suter and Vossler 2014; Miao et al. 2016; Palm-Forster et al. 2017), improve extraction of ground water for irrigation (Gardner, Moore, and Walker 1997; Suter et al. 2012, 2018; Li et al. 2014; Liu et al. 2014), and incentivize land conservation and ecosystem service provision (Parkhurst et al. 2002; Cason and Gangadharan 2004; Parkhurst and Shogren 2007, 2008; Arnold, Duke, and Messer 2013; Banerjee et al. 2014, 2015, 2017; Fooks et al. 2015, 2016; Messer et al. 2017; Duke et al. 2017; Banerjee 2018; Reeling et al. 2018).

  20. See Ostrom (2000) for more discussion on the evolution of rules and social norms. There is also broad literature on the role that communication and voting have on improving the performance of groups in public good and common pool resource settings (Messer et al. 2017, 2008); however, these studies have not focused on agri-environmental decision-making.

  21. As a frequently cited example of the power of defaults, organ donor rates are much higher in countries where the default option is that everyone is an organ donor (Johnson and Goldstein 2003).

  22. See Harrison and List (2004), Messer, Duke, and Lynch (2014), and Higgins et al. (2017) for definitions of types of experiments.

  23. We recommend publishing pre-analysis plans on a public experiment registry, like those maintained by the American Economic Association and the Open Science Framework.

References

Download references

Acknowledgements

The authors acknowledge financial support from the USDA Economic Research Service (ERS), the Center for Behavioral and Experimental Agri-environmental Research (CBEAR), and conference funding from the USDA National Institute of Food and Agriculture through the Agriculture and Food Research Initiative Foundational Program (NIFA-AFRI Grant No. 12234087). The authors appreciate feedback and helpful comments from Simanti Banerjee, Tim Cason, Lata Gangadharan, Jordan Suter, Tim Wojan, attendees at the 2018 Brown Bag Lunch Series on Behavioral Science in Agri-environmental Program Design hosted by USDA ERS, and participants of the 2018 Appalachian Experimental and Environmental Economics Workshop and the 2017 Conference for Behavioral and Experimental Agri-environmental Research: Methodological Advancements and Applications to Policy (CBEAR-MAAP). Dr. Janusch contributed to this article in his personal capacity. The views expressed are his own and do not necessarily represent the views of the California Energy Commission.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Leah H. Palm-Forster.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Palm-Forster, L.H., Ferraro, P.J., Janusch, N. et al. Behavioral and Experimental Agri-Environmental Research: Methodological Challenges, Literature Gaps, and Recommendations. Environ Resource Econ 73, 719–742 (2019). https://doi.org/10.1007/s10640-019-00342-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10640-019-00342-x

Keywords

JEL Classification

Navigation