Abstract
How can a decision-maker assess the potential of environmental policies when a group of experts provides divergent estimates on their effectiveness? To address this question, we propose and analyze a variant of the well-studied \(\alpha \)-maxmin model in decision theory. In our framework, and consistent to the paper’s empirical focus on renewable-energy R&D investment, experts’ subjective probability distributions are allowed to be action-dependent. In addition, the decision maker constrains the sets of priors to be considered via a parsimonious measure of their distance to a benchmark “average” distribution that grants equal weight to all experts. While our model is formally rooted in the decision-theoretic framework of Olszewski (Rev Econ Stud 74:567–595, 2007), it may also be viewed as a structured form of sensitivity analysis. We apply our framework to original data from a recent expert elicitation survey on solar energy. The analysis suggests that more aggressive investment in solar energy R&D is likely to yield significant dividends even, or rather especially, after taking expert ambiguity into account.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
See Gilboa and Marinacci (2012) for a comprehensive recent survey.
Indeed, there seems to be experimental evidence suggesting that lab subjects actually reason in a way that is consistent to models of ambiguity aversion when confronted with inconclusive expert judgments (Baillon et al. 2012).
An important additional concern that some authors have raised is that probabilistic estimates elicited from experts may not always represent probabilities in the actuarial sense [see, e.g., Millner et al. (2013); Kriegler et al. (2009)]. This concern seems to apply to the empirical application of this paper as well, at least in part (see “Appendix”). The decision-theoretic literature has begun to address such imprecise probabilities [see, e.g., Gajdos et al. (2008) and Gajdos and Vergnaud (2013)] in the context of decision making under ambiguity. Extending our framework to account for expert imprecision would be a very interesting avenue of future research.
Note, however, that our formal model generalizes to many alternative contexts of decision making under uncertainty.
Note, however, that endogenous probabilities are common in Decision Analysis and dynamic programming.
We thank an anonymous referee for suggesting this alternative interpretation of our work.
In contrast to the more general setting of our paper, these uncertain parameters were exogenous to the choice of policy.
Note that, despite the term “ambiguity”, the parameter \(b\) does not characterize a state or amount of knowledge. Knowledge, as represented by the set of experts’ pdfs, remains fixed.
The latter statement holds in light of the fact that values of \(b>\sqrt{\frac{N-1}{N}}\) cannot enlarge the feasible set. This is because the maximizers of \(\left[ \sum _{n=1}^{N} \left( p_n-\frac{1}{N}\right) ^2\right] ^{\frac{1}{2}}\) over the set of probability vectors concentrate all probability mass on one expert, leading to an aggregation ambiguity of \(\left[ \left( 1-\frac{1}{N}\right) ^2+(N-1)\cdot \left( \frac{1}{N}\right) ^2\right] ^{\frac{1}{2}}=\sqrt{\frac{N-1}{N}}.\)
It is important to note that the parameterization (4) may, in some cases, obscure important information. For instance, suppose one wishes to make a decision on the basis of a water container’s temperature. There are two experts, one of which provides a pdf suggesting that the water is freezing and the other that it is boiling hot. Suppose, further, that we know that one of the two experts must be correct. Then it is clear that if we consider the set of pdfs (4) for some small value of \(b\), we will be taking into account a whole set of pdfs implying that the water is tepid and none that it is either freezing or boiling. Clearly, in such cases, our framework will likely lead to suboptimal decisions. To avoid this from happening, we should either (a) set \(b\approx \sqrt{\frac{1}{2}}\) that allows for the unmixed consideration of both expert pdfs, or (b) consider a different parametric framework altogether. We thank an anonymous referee for bringing this issue to our attention.
On a related note, the theoretical machinery of the Savage framework (involving state spaces, consequence spaces, and acts) seems overwrought for our context (to this effect, see also the discussion in the introduction of Ahn (2008)).
Note that the presence of \(r\) as an argument of utility function \(u\) compels us to consider this broader outcome space (instead of just \({\mathcal {C}}\)) and somewhat awkward reframing of pdfs \(\pi _n\), even though there is no direct randomness associated with \(r\) (once the decision maker chooses it, it is fixed).
Skipping some of the formal details, the Hausdorff distance of two lotteries \(A\) and \(B\) is given by
$$\begin{aligned} d_H(A,B)=\max \left\{ \max _{a \in A}\min _{b\in B} d(a,b),\quad \max _{b \in B}\min _{a\in A}d(a,b) \right\} . \end{aligned}$$The survey is part of a 3-year ERC-funded project on innovation in carbon-free technologies (ICARUS - Innovation for Climate chAnge mitigation: a study of energy R&D, its Uncertain effectiveness and Spillovers www.icarus-project.org).
By “coherent” we mean non-contradictory responses which do not violate the laws of probability. In particular, 2 out of the 16 elicited experts provided probabilistic estimates implying non-monotonic cumulative distribution functions, which is clearly in conflict with probability theory and thus unusable for our purposes. As mentioned in the introduction, such incoherent expert estimates have been discussed in the literature (Millner et al. 2013; Kriegler et al. 2009), and addressing them in the context of expert elicitations is deserving of further study.
Please refer to the “Appendix” for more information on how expert pdfs were constructed from the ICARUS survey data.
The analysis is carried out using the World Induced Technical Change Hybrid (WITCH) model (Bosetti et al. 2006), an energy-economy-climate model that has been used extensively for economic analysis of climate change policies. See www.witchmodel.org for a list of applications and papers.
All simulations are performed in Mathematica. The optimizations are facilitated by the fact that right-hand-side of Eqs. (5) and (6) can be rewritten as \(V_{min}(r|b)=\min _{\mathbf {p} \in \mathrm P (b)}\sum _{n=1}^{N}p_n u_n(r)\) and \(V_{max}(r|b)=\max _{\mathbf {p} \in \mathrm P (b)}\sum _{n=1}^{N}p_n u_n(r)\), where \(u_n(r) \equiv \int _{\mathcal {C}} u(c,r)\mathrm {d}\pi _n(c|r)\). These equalities are valid by the linearity of the expectation operator under mixtures of distributions.
Indeed, constructing plausible approximations of experts’ \(u_n(r)\) functions over an interesting range of \(r\) will likely require further engagement with the experts.
We thank an anonymous referee for this suggestion.
References
Ahn D (2008) Ambiguity without a state space. Rev Econ Stud 75:3–28
Arrow K, Hurwicz L (1972) An optimality criterion for decision-making under ignorance. In: Carter CF, Ford LJ (eds) Uncertainty and expectations in economics: essays in Honour of G.L.S. Shackle. Basil Blackwell, Oxford, pp 1–11
Asano T (2010) Precautionary principle and the optimal timing of environmental policy under ambiguity. Environ Resour Econ 47:173–196
Athanassoglou S, Xepapadeas A (2012) Pollution control with uncertain stock dynamics: when, and how, to be precautious. J Environ Econ Manag 63:304–320
Baillon A, Cabantous L, Wakker P (2012) Aggregating imprecise or conflicting beliefs: an experimental investigation using modern ambiguity theories. J Risk Uncertain 44(2):115–147
Baker E, Chon H, Keisler J (2009a) Advanced solar R&D: combining economic analysis with expert elicitations to inform climate policy. Energy Econ 31:S37–S49
Baker E, Chon H, Keisler J (2009b) Carbon capture and storage: combining economic analysis with expert elicitations to inform climate policy. Clim Change 96:379–408
Baker E, Keisler J (2011) Cellulosic biofuels: expert views on prospects for advancement. Energy 36:595–605
Bosetti V, Carraro C, Galeotti M, Massetti E, Tavoni M (2006) WITCH: a world induced technical change hybrid model. Energy J 2:13–38
Bosetti V, Catenacci M, Fiorese G, Verdolini E (2012) The future prospect of PV and CSP solar technologies: an expert elicitation survey. Energy Policy 49:308–319
Clemen R, Winkler R (1999) Combining probability distributions from experts in risk analysis. Risk Anal 19:187–203
Clemen R, Winkler R (2007) Aggregating probability distributions. In: Edwards W, Miles R, von Winterfeldt D (eds) Advances in decision analysis. Cambridge University Press, Cambridge, pp 154–176
Cooke R (1991) Experts in uncertainty: opinion and subjective probability in science. Oxford University Press, USA
Curtright AE, Morgan G, Keith DW (2008) Expert assessments of future photovoltaic technologies. Environ Sci Technol 42:9031–9038
Eichberger J, Grant S, Kelsey D, Koshevoye G (2011) The \(\alpha \)-MEU model: a comment. J Econ Theory 146:1684–1698
Fiorese G, Catenacci M, Verdolini E, Bosetti V (2013) Advanced biofuels: future perspectives from an expert elicitation survey. Energy Policy 56:293–311
Gajdos T, Hayashi T, Tallon JM, Vergnaud JC (2008) Attitude toward imprecise information. J Econ Theory 140:27–65
Gajdos T, Vergnaud JC (2013) Decisions with conflicting and imprecise information. Soc. Choice Welfare 41:427–452
Ghirardato P (2001) Coping with ignorance: unforeseen contingencies and non-additive uncertainty. Econ Theor 17:247–276
Ghirardato P, Maccheroni F, Marinacci M (2004) Differentiating ambiguity and ambiguity attitude. J Econ Theory 118:133–173
Gilboa I, Marinacci M (2012) Ambiguity and the Bayesian paradigm. In: Acemoglu D, Arellano M, Dekel E (eds) Advances in Economics and Econometrics. Tenth World Congress of the Econometric Society, Economic Theory, vol I, Econometric Society Monographs, pp 179–242
Gilboa I, Schmeidler D (1989) Maxmin expected utility with non-unique prior. J Math Econ 18:141–153
Jaffray J-Y (1989) Linear utility theory for belief functions. Oper Res Lett 8:107–112
Klibanoff P, Marinacci M, Mukerji S (2005) A smooth model of decision making under ambiguity. Econometrica 73:1849–1892
Knutti R, Furrer R, Tebaldi C, Cermak J, Meehl GA (2010) Challenges in combining projections from multiple climate models. J Clim 23:2739–2758
Kriegler E, Hall JW, Held H, Dawson R, Schellnhuber HJ (2009) Imprecise probability assessment of tipping points in the climate system. Proc Natl Acad Sci 106(13):5041–5046
Ludwig D, Hilborn R, Walters C (1993) Uncertainty, resource exploitation, and conservation: lessons from history. Science 260:17–18
Meinshausen M, Meinshausen N, Hare W, Raper SCB, Frieler K, Reto K, Frame DJ, Allen MR (2009) Greenhouse-gas emission targets for limiting global warming to 2 C. Nature 458(7242):1158–1162
Millner A, Calel R, Stainforth DA, MacKerron G (2013) Do probabilistic expert elicitations capture scientists uncertainty about climate change? Clim Change 116(2):427–436
Millner A, Dietz S, Heal G (2013) Scientific ambiguity and climate policy. Environ Resour Econ 55:21–46
Morgan MG, Henrion M (1990) Uncertainty: a guide to dealing with uncertainty in quantitative risk and policy analysis. Cambridge University Press, New York
Morgan MG, Pitelka I, Shevliakova E (2001) Elicitation of expert judgments of climate change impacts on forest ecosystems. Clim Change 49:279–307
Morgan MG, Adams PJ, Keith DW (2006) Elicitation of expert judgments of aerosol forcing. Clim Change 75:195–214
Nordhaus W (1994) Expert opinion on climatic change. Am Sci 82:45–51
Nordhaus W (2008) A question of balance: weighing the options on global warming policies. Yale University Press, New Haven
O’Hagan A, Buck CE, Daneshkhan A, Eiser JR, Garthwaite PH, Jenkinson DJ, Oakey JE, Rakow T (2006) Uncertain judgments: eliciting experts probabilities. Wiley, Chichester
Olszewski W (2007) Preferences over sets of lotteries. Rev Econ Stud 74:567–595
Popp D (2004) ENTICE: endogenous technological change in the dice model of global warming. J Environ Econ Manag 48:742–768
Rostek M (2010) Quantile maximization in decision theory. Rev Econ Stud 77:339–371
Savage LJ (1954) The foundations of statistics. Wiley, New York
Treich N (2010) The value of a statistical life under ambiguity aversion. J Environ Econ Manag 59:15–26
Tversky A, Kahneman D (1974) Judgment under uncertainty: heuristics and biases. Science 185:1124–1131
Tversky A, Kahneman D (1983) Extensional vs. intuitive reasoning: the conjunction fallacy in probability judgment. Psychol Rev 90:293–315
U.S. Epa (2011) Expert elicitation task force white paper. http://www.epa.gov/stpc/pdfs/ee-white-paper-final
Vardas G, Xepapadeas A (2010) Model uncertainty, ambiguity and the precautionary principle: implications for biodiversity management. Environ Resour Econ 45:379–404
Vaughan D, Spouge J (2002) Risk estimation of collapse of the west Antarctic ice sheet. Clim Change 52:65–91
Woodward RT, Bishop RC (1997) How to decide when experts disagree: uncertainty-based choice rules in environmental policy. Land Econ 73:492–507
Yaari ME (1969) Some remarks on measures of risk aversion and on their uses. J Econ Theory 1(3):315–329
Zickfeld K, Levermann A, Morgan MG, Kuhlbrodt T, Rahmstorf S, Keith DW (2007) Expert judgements on the response of the Atlantic meridional overturning circulation to climate change. Clim Change 82:235–265
Zickfeld K, Morgan MG, Frame D, Keith D (2010) Expert judgments about transient climate response to alternative future trajectories of radiative forcing. Proc Natl Acad Sci 107:12451–12456
Author information
Authors and Affiliations
Corresponding author
Additional information
We are grateful to Michela Catenacci and Giulia Fiorese for help with data collection and to Gauthier de Maere. We also thank seminar participants at FEEM and ETH Zurich, for useful comments. The research leading to these results has received funding from the European Research Council under the European Commission’s Seventh Framework Programme (FP7/2007-2013)/ERC Grant agreement #240895, project ICARUS. The views expressed herein are purely those of the authors and may not in any circumstances be regarded as stating an official position of the European Commission.
Appendix: Constructing Expert Pdfs for the Three R&D Scenarios from ICARUS Survey Data
Appendix: Constructing Expert Pdfs for the Three R&D Scenarios from ICARUS Survey Data
In the ICARUS survey, experts were asked to provide values for the 10th, 50th, and 90th percentile of their distributions for the 2030 cost of solar technology conditional on all three Scenarios. In addition, they were asked to provide values for the probability of this cost being less than or equal to the following three values: 11.3, 5.5, and 3 c$/kWh. These “threshold” cost levels correspond to projections of the costs of electricity from fossil fuels or nuclear in 2030. The first (11.27 c$/kWh) corresponds to the 2030 projected cost of electricity from traditional coal power plants in the presence of a specific policy to control CO2 emissions (thus effectively doubling electricity costs from fossil sources). The second threshold cost (5.5 c$/kWh) is the projected cost of electricity from traditional fossil fuels in 2030, without considering any carbon tax. Finally, the third (3 c$/kWh) reflects a situation in which solar power becomes competitive with the levelized cost of electricity from nuclear power.
Asking experts the follow up question on the likelihood of reaching threshold cost targets allowed the survey authors to guard against the cognitive pitfalls associated with direct elicitation of subjective probabilities, to increase the amount of elicited information, and to deepen the discussion with the expert, hence improving their perception of his/her beliefs. In cases where the two sets of answers (percentile values and threshold probabilities) were inconsistent, we contacted the expert in order to obtain coherent estimates. Moreover, we asked all experts to give values for the upper and lower limits of their distribution’s support in order to pinpoint the intervals over which their implied probability distributions range.
Such corrected estimates were obtained from 14 out of the original 16 experts, and therefore the analysis of Sect. 4 focuses solely on them. Among the respondents, not all provided values on the left and right endpoints of their distributions’ support. As a result, we deduced between 6 and 8 points of 14 experts’ cumulative distribution functions (cdf) of the 2030 cost of solar electricity, given the aforementioned three R&D investment Scenarios. From these points a probability distribution function (pdf) was constructed using linear interpolation in the following way. First of all, and in accordance with the experts’ answers, we considered cost levels \(c\) lying in [2c$/kWh, 30c$/kWh] and discretized this interval on a scale of 0.5 (30c$/kWh represents an estimate of the technology’s current cost). Now, suppose an expert reported the values of his/her cdf \(F_n\) at two successive points \(c_1\) and \(c_2\) where \(c_2>c_1\) and gave no further information on cost levels between \(c_1\) and \(c_2\). Assuming right-continuity of \(F_n\) we took the probability mass \(F_n(c_2)-F_n(c_1)\) to be distributed uniformly among the cost levels \(\{c_1+.5, c_1+1,\ldots ,c_2\}\). For experts who did not provide values for the lower limit of their distribution’s support we assumed that whatever probability mass remained to be allocated (always less than .1) was distributed uniformly between the smallest argument of the cdf and two cost levels below it. For example, if an expert only indicated that \(c_l\) was his \(y\)’th percentile and gave no further points of the cdf below this, we assumed that a probability mass of \(y\) was distributed evenly across \(\{c_l-1,c_l-.5, c_l\}\). In the case of an unknown upper limit, if an expert only indicated that \(c_u\) was his \(y\)th percentile and gave no further arguments for the cdf above it, we assumed that a probability mass of \(1-y\) was distributed evenly across \(\{c_u+.5, c_u+1\}\).
Following this procedure we arrived at probability distribution functions for all 14 experts conditional on all three Scenarios. The implied cumulative distribution functions are depicted in Fig. 3.
Rights and permissions
About this article
Cite this article
Athanassoglou, S., Bosetti, V. Setting Environmental Policy When Experts Disagree. Environ Resource Econ 61, 497–516 (2015). https://doi.org/10.1007/s10640-014-9804-x
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10640-014-9804-x