Behavioral and Experimental Agri-Environmental Research: Methodological Challenges, Literature Gaps, and Recommendations

  • Leah H. Palm-ForsterEmail author
  • Paul J. Ferraro
  • Nicholas Janusch
  • Christian A. Vossler
  • Kent D. Messer


Insights from behavioral and experimental economics research can inform the design of evidence-based, cost-effective agri-environmental programs that mitigate environmental damages and promote the supply of environmental benefits from agricultural landscapes. To enhance future research on agri-environmental program design and to increase the speed at which credible scientific knowledge is accumulated, we highlight methodological challenges, identify important gaps in the existing literature, and make key recommendations for both researchers and those evaluating research. We first report on four key methodological challenges—underpowered designs, multiple hypothesis testing, interpretation issues, and choosing appropriate econometric methods—and suggest strategies to overcome these challenges. Specifically, we emphasize the need for more detailed planning during the experimental design stage, including power analyses and publishing a pre-analysis plan. Greater use of replication studies and meta-analyses will also help address these challenges and strengthen the quality of the evidence base. In the second part of this paper, we discuss how insights from behavioral and experimental economics can be applied to improve the design of agri-environmental programs. We summarize key insights using the MINDSPACE framework, which categorizes nine behavioral effects that influence decision-making (messenger, incentives, norms, defaults, salience, priming, affect, commitment, and ego), and we highlight recent research that tests these effects in agri-environmental contexts. We also propose a framework for prioritizing policy-relevant research in this domain.


Behavioral insights Conservation Effect size Environmental economics Experimental design Power analysis Subject pools 

JEL Classification

C9 D9 Q58 Q52 



The authors acknowledge financial support from the USDA Economic Research Service (ERS), the Center for Behavioral and Experimental Agri-environmental Research (CBEAR), and conference funding from the USDA National Institute of Food and Agriculture through the Agriculture and Food Research Initiative Foundational Program (NIFA-AFRI Grant No. 12234087). The authors appreciate feedback and helpful comments from Simanti Banerjee, Tim Cason, Lata Gangadharan, Jordan Suter, Tim Wojan, attendees at the 2018 Brown Bag Lunch Series on Behavioral Science in Agri-environmental Program Design hosted by USDA ERS, and participants of the 2018 Appalachian Experimental and Environmental Economics Workshop and the 2017 Conference for Behavioral and Experimental Agri-environmental Research: Methodological Advancements and Applications to Policy (CBEAR-MAAP). Dr. Janusch contributed to this article in his personal capacity. The views expressed are his own and do not necessarily represent the views of the California Energy Commission.


  1. Allcott H (2011) Social norms and energy conservation. J Public Econ 95:1082–1095. CrossRefGoogle Scholar
  2. Alpízar F, Requate T, Schram A (2004) Collective versus random fining: an experimental study on controlling ambient pollution. Environ Resour Econ 29:231–252. CrossRefGoogle Scholar
  3. Armitage P, McPherson CK, Rowe BC (1969) Repeated significance tests on accumulating data. J R Stat Soc Ser Gen 132:235–244. CrossRefGoogle Scholar
  4. Arnold MA, Duke JM, Messer KD (2013) Adverse selection in reverse auctions for ecosystem services. Land Econ 89:387–412CrossRefGoogle Scholar
  5. Banerjee S (2018) Improving spatial coordination rates under the agglomeration bonus scheme: a laboratory experiment with a pecuniary and a non-pecuniary mechanism (NUDGE). Am J Agric Econ 100:172–197. CrossRefGoogle Scholar
  6. Banerjee S, de Vries FP, Hanley N, van Soest DP (2014) The impact of information provision on agglomeration bonus performance: an experimental study on local networks. Am J Agric Econ 96:1009–1029. CrossRefGoogle Scholar
  7. Banerjee S, Kwasnica AM, Shortle JS (2015) Information and auction performance: a laboratory study of conservation auctions for spatially contiguous land management. Environ Resour Econ 61:409–431. CrossRefGoogle Scholar
  8. Banerjee S, Cason TN, de Vries FP, Hanley N (2017) Transaction costs, communication and spatial coordination in payment for ecosystem services schemes. J Environ Econ Manag 83:68–89. CrossRefGoogle Scholar
  9. Bellemare C, Bissonnette L, Kröger S (2016) Simulating power of economic experiments: the powerBBK package. J Econ Sci Assoc 2:157–168. CrossRefGoogle Scholar
  10. Benjamini Y, Hochberg Y (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Ser B Methodol 57:289–300Google Scholar
  11. Benjamini Y, Yekutieli D (2001) The control of the false discovery rate in multiple testing under dependency. Ann Stat 29:1165–1188. CrossRefGoogle Scholar
  12. Bernedo M, Ferraro PJ (2017) Behavioral economics and climate change adaptation: insights from experimental economics on the role of risk and time preferences. In: Botelho A (ed) World scientific reference on natural resources and environmental policy in the era of global change: volume 4: experimental economics. World Scientific, pp 151–177Google Scholar
  13. Brent DA, Friesen L, Gangadharan L, Leibbrandt A (2017) Behavioral insights from field experiments in environmental economics. Int Rev Environ Resour Econ 10:95–143. CrossRefGoogle Scholar
  14. Brown JP, Lambert DM, Wojan TR (2019) The effect of the conservation reserve program on rural economies: deriving a statistical verdict from a null finding. Am J Agric Econ 101:528–540. CrossRefGoogle Scholar
  15. Burke MA, Young HP (2011) Social norms. In: Benhabib J, Bisin A, Jackson MO (eds) Handbook of social economics. North-Holland, Amsterdam, pp 311–338CrossRefGoogle Scholar
  16. Butler JM, Fooks JR, Messer KD, Palm-Forster LH (2019) Addressing social dilemmas with mascots, information, and graphics. Econ Inq.
  17. Button KS, Ioannidis JPA, Mokrysz C et al (2013) Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci 14:365–376. CrossRefGoogle Scholar
  18. Cameron AC, Miller DL (2015) A practitioner’s guide to cluster-robust inference. J Hum Resour 50:317–372CrossRefGoogle Scholar
  19. Cason TN (2010) What can laboratory experiments teach us about emissions permit market design? Agric Resour Econ Rev 39:151–161. CrossRefGoogle Scholar
  20. Cason TN, Gangadharan L (2004) Auction design for voluntary conservation programs. Am J Agric Econ 86:1211–1217. CrossRefGoogle Scholar
  21. Cason TN, Gangadharan L (2013) Empowering neighbors versus imposing regulations: an experimental analysis of pollution reduction schemes. J Environ Econ Manag 65:469–484. CrossRefGoogle Scholar
  22. Cason TN, Wu SY (2018) Subject pools and deception in agricultural and resource economics experiments. Environ Resour Econ. CrossRefGoogle Scholar
  23. Cason TN, Gangadharan L, Duke C (2003) A laboratory study of auctions for reducing non-point source pollution. J Environ Econ Manag 46:446–471. CrossRefGoogle Scholar
  24. Cherry TL, Kroll S, Shogren JF (eds) (2008) Environmental economics, experimental methods. Routledge, New YorkGoogle Scholar
  25. Cochard F, Willinger M, Xepapadeas A (2005) Efficiency of nonpoint source pollution instruments: an experimental study. Environ Resour Econ 30:393–422. CrossRefGoogle Scholar
  26. Cohen J (1988) Statistical power analysis for the behavioral sciences, 2nd edn. Lawrence Erlbaum Associates, HillsdaleGoogle Scholar
  27. Czap NV, Czap HJ, Khachaturyan M et al (2013) Smiley or Frowney: the effect of emotions and empathy framing in a downstream water pollution game. Int J Econ Finance 5:9. CrossRefGoogle Scholar
  28. Deaton A, Cartwright N (2018) Understanding and misunderstanding randomized controlled trials. Soc Sci Med 210:2–21. CrossRefGoogle Scholar
  29. Delaney J, Jacobson S (2016) Payments or persuasion: common pool resource management with price and non-price measures. Environ Resour Econ 65:747–772. CrossRefGoogle Scholar
  30. Dolan P, Hallsworth M, Halpern D et al (2012) Influencing behaviour: the mindspace way. J Econ Psychol 33:264–277. CrossRefGoogle Scholar
  31. Doucouliagos C, Stanley TD (2013) Are all economic facts greatly exaggerated? Theory competition and selectivity. J Econ Surv 27:316–339CrossRefGoogle Scholar
  32. Duflo E, Kremer M, Robinson J (2011) Nudging farmers to use fertilizer: theory and experimental evidence from Kenya. Am Econ Rev 101:2350–2390. CrossRefGoogle Scholar
  33. Duke JM, Messer KD, Lynch L, Li T (2017) The effect of information on discriminatory-price and uniform-price reverse auction efficiency: an experimental economics study of the purchase of ecosystem services. Strateg Behav Environ 7:41–71. CrossRefGoogle Scholar
  34. Duquette E, Higgins N, Horowitz J (2012) Farmer discount rates: experimental evidence. Am J Agric Econ 94:451–456. CrossRefGoogle Scholar
  35. Ehmke MD, Shogren JF (2009) Experimental methods for environment and development economics. Environ Dev Econ 14:419–456. CrossRefGoogle Scholar
  36. Ellis PD (2010) The essential guide to effect sizes: statistical power, meta-analysis, and the interpretation of research results. Cambridge University Press, New YorkCrossRefGoogle Scholar
  37. FAO (2016) FAOSTAT land domain. Accessed 13 Mar 2019
  38. Feiveson AH (2002) Power by simulation. Stata J 2:107–124. CrossRefGoogle Scholar
  39. Ferraro PJ, Hanauer MM (2014) Quantifying causal mechanisms to determine how protected areas affect poverty through changes in ecosystem services and infrastructure. Proc Natl Acad Sci. CrossRefGoogle Scholar
  40. Ferraro PJ, Miranda JJ (2013) Heterogeneous treatment effects and mechanisms in information-based environmental policies: evidence from a large-scale field experiment. Resour Energy Econ 35:356–379. CrossRefGoogle Scholar
  41. Ferraro PJ, Price MK (2013) Using nonpecuniary strategies to influence behavior: evidence from a large-scale field experiment. Rev Econ Stat 95:64–73. CrossRefGoogle Scholar
  42. Ferraro PJ, Miranda JJ, Price MK (2011) The persistence of treatment effects with norm-based policy instruments: evidence from a randomized environmental policy experiment. Am Econ Rev 101:318–322CrossRefGoogle Scholar
  43. Ferraro PJ, Messer KD, Wu S (2017) Applying behavioral insights to improve water security. Choices 32:1–6Google Scholar
  44. Fink G, McConnell M, Vollmer S (2014) Testing for heterogeneous treatment effects in experimental data: false discovery risks and correction procedures. J Dev Eff 6:44–57. CrossRefGoogle Scholar
  45. Fooks JR, Messer KD, Duke JM (2015) Dynamic entry, reverse auctions, and the purchase of environmental services. Land Econ 91:57–75CrossRefGoogle Scholar
  46. Fooks JR, Higgins N, Messer KD et al (2016) Conserving spatially explicit benefits in ecosystem service markets: experimental tests of network bonuses and spatial targeting. Am J Agric Econ 98:468–488. CrossRefGoogle Scholar
  47. Fowlie M, Wolfram C, Spurlock CA et al (2017) Default effects and follow-on behavior: evidence from an electricity pricing program. National Bureau of Economic Research, CambridgeCrossRefGoogle Scholar
  48. Friesen L, Gangadharan L (2013) Environmental markets: what do we learn from the lab? J Econ Surv 27:515–535. CrossRefGoogle Scholar
  49. Gardner R, Moore MR, Walker JM (1997) Governing a groundwater commons: a strategic and laboratory analysis of western water law. Econ Inq 35:218–234. CrossRefGoogle Scholar
  50. Gelman A, Carlin J (2014) Beyond power calculations: assessing type S (sign) and type M (magnitude) errors. Perspect Psychol Sci 9:641–651. CrossRefGoogle Scholar
  51. Griesinger MR, Palm-Forster LH, Messer KD et al (2017) Stewardship signaling and the power of using social pressures to reduce nonpoint source pollution. Agricultural and Applied Economics Association, ChicagoGoogle Scholar
  52. Gsottbauer E, van den Bergh JCJM (2011) Environmental policy theory given bounded rationality and other-regarding preferences. Environ Resour Econ 49:263–304. CrossRefGoogle Scholar
  53. Harrison GW, List JA (2004) Field experiments. J Econ Lit 42:1009–1055CrossRefGoogle Scholar
  54. Higgins N, Hellerstein D, Wallander S, Lynch L (2017) Economic experiments for policy analysis and program design: a guide for agricultural decisionmakers. US Department of Agriculture, Economic Research Service, WashingtonGoogle Scholar
  55. Hobbs JE, Mooney S (2016) Applications of behavioral and experimental economics to decision making in the agricultural, food, and resource sectors: an introduction. Can J Agric Econ Can Agroecon 64:593–597. CrossRefGoogle Scholar
  56. Inouye SK, Fiellin DA (2005) An evidence-based guide to writing grant proposals for clinical research. Ann Intern Med 142:274–282. CrossRefGoogle Scholar
  57. Ioannidis JPA, Stanley TD, Doucouliagos H (2017) The power of bias in economics research. Econ J 127:F236–F265. CrossRefGoogle Scholar
  58. Jacquemet N, Luchini S, Shogren JF, Zylbersztejn A (2018) Coordination with communication under oath. Exp Econ 21:627–649. CrossRefGoogle Scholar
  59. Johnson EJ, Goldstein D (2003) Do defaults save lives? Science 302:1338–1339. CrossRefGoogle Scholar
  60. Kahneman D (2003) Maps of bounded rationality: psychology for behavioral economics. Am Econ Rev 93:1449–1475CrossRefGoogle Scholar
  61. Kesternich M, Reif C, Rübbelke D (2017) Recent trends in behavioral environmental economics. Environ Resour Econ 67:403–411. CrossRefGoogle Scholar
  62. Kotani K, Messer KD, Schulze WD (2010) Matching grants and charitable giving: why people sometimes provide a helping hand to fund environmental goods. Agric Resour Econ Rev 39:324–343CrossRefGoogle Scholar
  63. Lakens D, Adolfi FG, Albers CJ et al (2018) Justify your alpha. Nat Hum Behav 2:168–171. CrossRefGoogle Scholar
  64. Leiser D, Azar OH (2008) Behavioral economics and decision making: applying insights from psychology to understand how people make economic decisions. J Econ Psychol 29:613–618. CrossRefGoogle Scholar
  65. Li J, Michael HA, Duke JM et al (2014) Behavioral response to contamination risk information in a spatially explicit groundwater environment: experimental evidence. Water Resour Res 50:6390–6405. CrossRefGoogle Scholar
  66. List JA, Price MK (2016) The use of field experiments in environmental and resource economics. Rev Environ Econ Policy. CrossRefGoogle Scholar
  67. List JA, Sadoff S, Wagner M (2011) So you want to run an experiment, now what? Some simple rules of thumb for optimal experimental design. Exp Econ 14:439–457. CrossRefGoogle Scholar
  68. List JA, Shaikh AM, Xu Y (2016) Multiple hypothesis testing in experimental economics. National Bureau of Economic Research, CambridgeCrossRefGoogle Scholar
  69. List JA, Shaikh A, Xu Y (2019) Multiple hypothesis testing in experimental economics. Exp Econ. CrossRefGoogle Scholar
  70. Liu Z, Suter JF, Messer KD et al (2014) Strategic entry and externalities in groundwater resources: evidence from the lab. Resour Energy Econ 38:181–197. CrossRefGoogle Scholar
  71. Lokhorst AM, Werner C, Staats H et al (2013) Commitment and behavior change: a meta-analysis and critical review of commitment-making strategies in environmental research. Environ Behav 45:3–34. CrossRefGoogle Scholar
  72. Mason CF, Phillips OR (1997) Mitigating the tragedy of the commons through cooperation: an experimental evaluation. J Environ Econ Manag 34:148–172. CrossRefGoogle Scholar
  73. Messer KD, Borchers AM (2015) Choice for goods under threat of destruction. Econ Lett 135:137–140. CrossRefGoogle Scholar
  74. Messer KD, Murphy JJ (2010) Special issue on experimental methods in environmental, natural resource, and agricultural economics. Econ Rev 39:iii–vi. CrossRefGoogle Scholar
  75. Messer KD, Kaiser HM, Poe GL (2007) Voluntary funding for generic advertising using a provision point mechanism: an experimental analysis of option assurance. Appl Econ Perspect Policy 29:612–631. CrossRefGoogle Scholar
  76. Messer KD, Duke JM, Lynch L (2014) Applying experiments to land economics: public information and auction efficiency in ecosystem service markets. In: Duke JM, Wu J (eds) The Oxford handbook of land economics. Oxford University Press, Oxford, pp 481–546Google Scholar
  77. Messer KD, Duke JM, Lynch L, Li T (2017) When does public information undermine the efficiency of reverse auctions for the purchase of ecosystem services? Ecol Econ 134:212–226. CrossRefGoogle Scholar
  78. Miao H, Fooks JR, Guilfoos T et al (2016) The impact of information on behavior under an ambient-based policy for regulating nonpoint source pollution. Water Resour Res 52:3294–3308. CrossRefGoogle Scholar
  79. Miguel E, Casey K, Glennerster R (2012) Reshaping institutions: evidence on aid impacts using a preanalysis plan. Q J Econ 127:1755–1812. CrossRefGoogle Scholar
  80. Muthukrishna M, Henrich J (2019) A problem in theory. Nat Hum Behav 10:10. CrossRefGoogle Scholar
  81. OECD (2017) Tackling environmental problems with the help of behavioural insights. Organisation for Economic Cooperation and Development, Paris. CrossRefGoogle Scholar
  82. Olken BA (2015) Promises and perils of pre-analysis plans. J Econ Perspect 29:61–80. CrossRefGoogle Scholar
  83. Osbaldiston R, Schott JP (2012) Environmental sustainability and behavioral science: meta-analysis of proenvironmental behavior experiments. Environ Behav 44:257–299. CrossRefGoogle Scholar
  84. Ostrom E (2000) Collective action and the evolution of social norms. J Econ Perspect 14:137–158. CrossRefGoogle Scholar
  85. Palm-Forster LH, Swinton SM, Shupp RS (2017) Farmer preferences for conservation incentives that promote voluntary phosphorus abatement in agricultural watersheds. J Soil Water Conserv 72:493–505. CrossRefGoogle Scholar
  86. Palm-Forster LH, Suter JF, Messer KD (2019) Experimental evidence on policy approaches that link agricultural subsidies to water quality outcomes. Am J Agric Econ 101:109–133. CrossRefGoogle Scholar
  87. Parkhurst GM, Shogren JF (2007) Spatial incentives to coordinate contiguous habitat. Ecol Econ 64:344–355. CrossRefGoogle Scholar
  88. Parkhurst GM, Shogren JF (2008) Smart subsidies for conservation. Am J Agric Econ 90:1192–1200. CrossRefGoogle Scholar
  89. Parkhurst GM, Shogren JF, Bastian C et al (2002) Agglomeration bonus: an incentive mechanism to reunite fragmented habitat for biodiversity conservation. Ecol Econ 41:305–328. CrossRefGoogle Scholar
  90. Poe GL, Schulze WD, Segerson K et al (2004) Exploring the performance of ambient-based policy instruments when nonpoint source polluters can cooperate. Am J Agric Econ 86:1203–1210CrossRefGoogle Scholar
  91. Reeling C, Palm-Forster LH, Melstrom RT (2018) Policy instruments and incentives for coordinated habitat conservation. Environ Resource Econ. CrossRefGoogle Scholar
  92. Reddy SMW, Montambault J, Masuda YJ et al (2017) Advancing conservation by understanding and influencing human behavior. Conserv Lett 10:248–256. CrossRefGoogle Scholar
  93. Schultz PW (2014) Strategies for promoting proenvironmental behavior: lots of tools but few instructions. Eur Psychol 19:107–117. CrossRefGoogle Scholar
  94. Shogren JF, Taylor LO (2008) On behavioral-environmental economics. Rev Environ Econ Policy 2:26–44. CrossRefGoogle Scholar
  95. Shogren JF, Parkhurst GM, Banerjee P (2010) Two cheers and a qualm for behavioral environmental economics. Environ Resour Econ 46:235–247. CrossRefGoogle Scholar
  96. Simmons JP, Nelson LD, Simonsohn U (2011) False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci 22:1359–1366. CrossRefGoogle Scholar
  97. Smaldino PE, McElreath R (2016) The natural selection of bad science. R Soc Open Sci 3:160384. CrossRefGoogle Scholar
  98. Spraggon J (2004) Testing ambient pollution instruments with heterogeneous agents. J Environ Econ Manag 48:837–856. CrossRefGoogle Scholar
  99. Spraggon JM (2013) The impact of information and cost heterogeneity on firm behaviour under an ambient tax/subsidy instrument. J Environ Manag 122:137–143. CrossRefGoogle Scholar
  100. Spraggon J, Oxoby RJ (2010) Ambient-based policy instruments: the role of recommendations and presentation. Agric Resour Econ Rev 39:262–274CrossRefGoogle Scholar
  101. Stuart D, Benveniste E, Harris LM (2014) Evaluating the use of an environmental assurance program to address pollution from United States cropland. Land Use Policy 39:34–43. CrossRefGoogle Scholar
  102. Sturm B, Weimann J (2006) Experiments in environmental economics and some close relatives. J Econ Surv 20:419–457. CrossRefGoogle Scholar
  103. Suter JF, Vossler CA (2014) Towards an understanding of the performance of ambient tax mechanisms in the field: evidence from upstate new york dairy farmers. Am J Agric Econ 96:92–107. CrossRefGoogle Scholar
  104. Suter JF, Vossler CA, Poe GL, Segerson K (2008) Experiments on damage-based ambient taxes for nonpoint source polluters. Am J Agric Econ 90:86–102CrossRefGoogle Scholar
  105. Suter JF, Duke JM, Messer KD, Michael HA (2012) Behavior in a spatially explicit groundwater resource: evidence from the lab. Am J Agric Econ 94:1094–1112. CrossRefGoogle Scholar
  106. Suter JF, Collie S, Messer KD et al (2018) Common pool resource management at the extensive and intensive margins: experimental evidence. Environ Resour Econ. CrossRefGoogle Scholar
  107. Thaler RH, Sunstein CR (2008) Nudge: improving decisions about health, wealth, and happiness. Yale University Press, New HavenGoogle Scholar
  108. Thalheimer W, Cook S (2002) How to calculate effect sizes from published research: a simplified methodology. Work-Learning Research, SomervilleGoogle Scholar
  109. U.S. Environmental Protection Agency (2018) National summary of state information water quality assessment and TMDL information. Accessed 27 Aug 2018
  110. Vossler CA, Poe GL, Schulze WD, Segerson K (2006) Communication and incentive mechanisms based on group performance: an experimental study of nonpoint pollution control. Econ Inq 44:599–613. CrossRefGoogle Scholar
  111. Wallander S, Ferraro P, Higgins N (2017) Addressing participant inattention in federal programs: a field experiment with the conservation reserve program. Am J Agric Econ. CrossRefGoogle Scholar
  112. Xie J, Cai TT, Maris J, Li H (2011) Optimal false discovery rate control for dependent data. Stat Interface 4:417–430CrossRefGoogle Scholar
  113. Yekutieli D (2008) Hierarchical false discovery rate-controlling methodology. J Am Stat Assoc 103:309–316. CrossRefGoogle Scholar
  114. Zarghamee HS, Messer KD, Fooks JR et al (2017) Nudging charitable giving: three field experiments. J Behav Exp Econ 66:137–149. CrossRefGoogle Scholar
  115. Zhang L, Ortmann A (2013) Exploring the meaning of significance in experimental economics. Social Science Research Network, RochesterCrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.Department of Applied Economics and StatisticsUniversity of DelawareNewarkUSA
  2. 2.Bloomberg School of Public Health, Carey Business School and Whiting School of EngineeringJohns Hopkins UniversityBaltimoreUSA
  3. 3.Energy Assessments DivisionCalifornia Energy CommissionSacramentoUSA
  4. 4.Department of Economics and Howard H. Baker Jr. Center for Public PolicyUniversity of TennesseeKnoxvilleUSA

Personalised recommendations