Introduction

Energy systems have been marked by disruptive events that changed the contours of the system and the markets that developed around them: light bulbs overtaking gas lighting in the 1890s, George Westinghouse’s victory over Thomas Edison in the war of the currents, the New Deal’s rapid development of federally funded hydropower to provide electricity in rural areas, the oil embargo of 1973, restructuring and liberalization of gas and electricity markets in the late twentieth century, the unconventional gas boom of the early twenty-first century, and the recent remarkable declines in generation costs from solar and wind are just a few examples of such disruptive events.

As an old Danish proverbFootnote 1 warns, prediction is very difficult, especially about the future. Disruptive events, which change the course of how the future unfolds, are often unexpected and are seldom forecasted. Though formal and rigorous modeling and forecasting of the energy sector did not emerge until the 1970s, we can already examine numerous examples of how challenging it is to forecast the development of socio-technical systems that are complex, sensitive to exogenous factors, and subject to peculiar, historically contingent regulatory and market structures. In this paper we explore the difficulties of forecasting in the presence of disruptive events by studying the US shale gas “revolution” that rapidly transformed the US energy map and policy landscape in a few short years, and has raised new uncertainties and complexities in the international geopolitics of energy production as traditional oil-producing states respond to new sources of supply in both gas and oil. As is seen in energy forecasts before, during, and after the fact, the boom in shale gas production was largely unpredicted by forecasters, commentators, companies, or other experts in the energy sector, regardless of whether they might have conceptualized shale gas as an opportunity, a threat, or something more complicated. Indeed, while many were tangentially aware of the potential of shale gas development, it remained an abstract potentiality that resisted quantification—and thus also tended to resist serious consideration. Also unexpected were the subsequent sectoral realignments in resource extraction, chemicals, electric power production, and other industries around the newfound abundance of gas.

The intent of this analysis is to highlight possible lessons learned from this particular event in time, not to castigate the efforts of the energy modeling community, many of whom are well-versed in the topics discussed here. For decades, both the US Energy Information Administration (EIA) and International Energy Agency (IEA) have provided credible analyses of energy trends, historically from the perspective of energy-consuming states. On the supply side, the Organization of the Petroleum Exporting Countries (OPEC) has also developed robust analytical and forecasting outputs that are critical to the economies that rely on energy exports. Obviously, energy geopolitics directs massive flows of capital and activity, and energy-producing and energy-consuming states alike expend enormous resources to develop forecasts and data that aim to be accurate and germane to the needs of their respective economies. But the necessary abstractions to those ends and the inability of anyone—regardless of their expertise—to accurately predict the future are not always recognized by the general public. Thus our aim here is focused on informing decision-makers—industry analysts, policymakers and policy thought leaders, advocates, and the broader scientific community—who do not generally deal with the internal mechanics of models, but nevertheless rely on them. Our hope is that by discerning the difference between what we thought would happen—given the realization of certain key assumptions—and what actually did happen, we can begin to improve the manner in which forecasters communicate uncertainties to decision-makers and the manner in which decision-makers understand and use long-term energy forecasts.

We explore two challenges to forecasting complex systems which can lead to large forecast errors. Though not an exhaustive list, these two challenges lead to a significant fraction of large forecast errors and are of central importance to energy system modeling. The first challenge is that in complex systems, there are more variables than can be considered. Often described as epistemic uncertainty, these un-modeled variables—the unknown unknowns—can lead to reality diverging dramatically from forecasts. The second challenge in forecasting complex systems is from the inherently nonlinear nature of many such systems. This results in a compounding of stochastic uncertainties—the known unknowns—which in turn can result in real-world outcomes that deviate significantly from forecasts.Footnote 2

Large forecast errors due to epistemic uncertainty are often referred to as “black swan” events.Footnote 3 As one cannot reject the existence of a black swan from only seeing white swans; one cannot neglect the possibility of a large disruptive event in the future just because of an absence of a disruptive event at present. Here we further analogize large forecast errors due to compounded stochastic uncertainties as “dying swan” events, the notion being that small deviations in the health of a swan can compound to result in a large deviation of whether the swan lives (see “Black swans and dying swans”).

Both epistemic and stochastic uncertainty—black swans and dying swans—are present in the case of the US shale gas revolution. Understanding what happened, and how forecasts deviated from eventual reality, through the lens of epistemic uncertainties and compounded stochastic uncertainties allows us to better see where the uncertainties inherent in forecasting lie. This in turn can help inform investors and policymakers as to when and how forecasts may fail to provide accurate predictions. We first provide an overview of the US shale gas revolution and describe the large forecast errors during this period (“A (hydraulically) fractured crystal ball”). We next discuss how epistemic and stochastic uncertainties resulted in these forecast errors (“Black swans and dying swans” and “Statistical and epistemic uncertainty in the shale gas revolution”). We then describe ways to account for each type of uncertainty (“Forecast errors and modeler apologetics in context” and “Incorporating uncertainty in energy forecasts”). We conclude by discussing the import of our analysis for forecasts and decision-making in the natural gas sector and beyond (“Conclusions”).

A (hydraulically) fractured crystal ball

In this section we examine the ways in which prominent energy forecasts for natural gas deviated from the future that was ultimately realized in the past decade. We focus on predictions from the EIA, the primary US agency responsible for the collection and analysis of energy data, due to its prominence, well-respected reputation within the energy modeling community, and global coverage. EIA’s results are also generally well aligned with the modeling efforts of private companies and academic analytics. Other notable examples of missed forecasts—and their reasons—are explored in Mai et al. [2].

In its Annual Energy Outlook (AEO) for 2008 [4], the EIA predicted that shale gas would supply approximately 2.2 Tcf annually in 2030. Actual production rapidly outpaced the EIA’s growth predictions. Figure 1 shows the dramatic revisions made by the EIA in subsequent years to come to grips with the reality of the shale gas revolution. By 2013 [5], EIA’s estimate for shale gas production in 2030 had increased by 535% to 14.2 Tcf, and by 2018 [6] the estimate increased an additional 90% to 26.9 Tcf.

Fig. 1
figure 1

(Adapted from AEO data)

Changes in AEO projections for US shale gas production 2008–2018

Full size image

As early as 2004 [7], there was some acknowledgement by EIA of the large potential role of unconventionals—though this was largely related to production from coal bed methane and tight gas. Shale, even then, was seen as a fairly minor contributor. The 2004 forecasts for 2015 of 1024 Bcf were roughly an order of magnitude off from the more than 15,000 Bcf of shale gas actually produced in 2015. Other forecast organizations were similarly caught off guard. Four years later, even as US shale gas production was starting to accelerate, IEA’s World Energy Outlook 2008 (which did not separate shale gas from conventional gas development) forecasted a negative 0.1% average annual growth rate in US natural gas production overall from 2006 to 2030 in its reference scenario [8]. In fact, the realized average annual growth rate from 2006 to 2019 was 5.7%, and US gas production overall has grown 69% since 2006 [9].

A closer look at the individual shale plays gives a more detailed view of the nonlinear growth since 2008 (Fig. 2). Shale gas has not only emerged as the largest source of gas in the US, but has arrested a decline in overall gas production that had seemed all but certain. The implications of the shale gas revolution are apparent in North America: local land use conflicts, debate regarding the environmental impact and “true” life-cycle greenhouse gas emissions associated with shale gas [11,12,13], and a dramatic increase in electricity generated with natural gas, which climbed from 13% of total generation in 2001 to more than 35% in 2018 [14].

Fig. 2
figure 2

Dry shale gas production estimates by play through May 2019 [10]

Full size image

Finally, the potential exposure of the US economy to declining levels of natural gas production and growing reliance on imports in 2005 forecasts cannot be overemphasized (Fig. 3). In that year, forecasts expected net imports to climb to 9 Tcf annually in 2025, while 10 years later, the US was expected to be a net exporter of 4 Tcf by 2025, a difference of 13 Tcf. In addition to the geopolitical differences of this shifting trade outcome, the economic impacts would have been significant: at a price of $5 per 1000 cubic feet, this 13 Tcf would have had a value of $650 billion annually.

Fig. 3
figure 3

Estimates of natural gas imports and exports in 2005 and 2015 forecasts [15]

Full size image

How could major energy forecasts have missed such a momentous development? The answer is a timely reminder of an epistemological truth about forecasting and its role in policy development and investment decisions. This truth, oft-repeated by forecasters themselves and oft-forgotten by their audience, is a modern Cassandra myth, but in reverse.Footnote 4 Energy forecasters cannot in fact tell the future, but everyone believes that they can. Forecasting—the extrapolation of potential future conditions from assumptions about present conditions and their causes—can become confused with foresight, a presumed actual knowledge of the future that may only be confirmed after the fact. But when the realized world diverges from the prediction so dramatically, the distinction becomes apparent and, in hindsight, obvious.

What is perhaps most interesting about the EIA’s inability to accurately forecast the pace and volume of the shale gas revolution is that in previous years, it clearly understood the tremendous potential energy locked in unconventional gas formations, and the major uncertainties inherent in both the cost of extracting the resource and its precise size. In 1993 the EIA noted: “It is widely recognized, for example, that gas from coal beds and from very-low-permeability reservoirs may exist in very large quantities, but neither the potential volumes nor the associated extraction costs are understood with any precision” [16].

Forecasts typically struggle with disruptive technologies. In the case of the shale gas boom, the widespread use of a disruptive technology—hydraulic fracturing combined with horizontal drilling and an assortment of other enabling advances—has provided access to an entirely new class of energy resource and fundamentally changed the supply economics and risk profile of gas development [17, 18].Footnote 5 In the case of the United States, those technological advances were met aboveground with a property rights regime and restructured energy sector that encouraged development. Policymakers and the energy sector itself were unprepared for the disruption, and the country’s energy and environmental policies are just now beginning to keep pace with environmental concerns related to, among others, groundwater impacts, associated methane emissions, and possibilities of induced seismicity.

Black swans and dying swans

A “black swan” event is any event with the attributes of “rarity, extreme impact, and retrospective (though not prospective) predictability” [20]. In other words, a black swan is both unexpected and influential. The phrase is a reference to the philosophical problem of induction. As the author of The Black Swan, Nassim Nicholas Taleb, writes, “No amount of observations of white swans can allow the inference that all swans are white, but the observation of a single black swan is sufficient to refute that conclusion” [21].Footnote 6

Edison inventing the lightbulb was a black swan event for the gas lighting industry. Similarly, the Arab oil embargo was both shocking and had profound market consequences. Mathematically, black swans can be understood with the following simple equation:

$$y = X + \in + z$$
(1)

A forecaster predicts \(y\) using knowledge of \(X\) and facing uncertainty from the known unknowns \(\in\) and the unknown unknowns \(z\). Modeling errors due to \(\in\) are stochastic errors and can be incorporated into the model. Errors from \(z\) are epistemic errors and are due to all of the variables which were not incorporated into the model but which affect \(y\). Black swan events can be thought of as \(z\) rearing its ugly head. If the effects of \(z\) are large but occur infrequently, then the model forecasts could closely conform to reality until a large \(z\) draw leads to a rapid and drastic divergence.

While black swans can result in poor forecasts, another cause of poor forecasts is the highly nonlinear nature of many complex systems. In nonlinear systems, even small stochastic errors can compound and lead to large errors in prediction. We call this effect a “dying swan” event, referring to the following metaphor. You are trying to forecast whether a swan will be alive or dead tomorrow. The death of the swan is certainly a disruptive event, especially for the swan. Furthermore, all historical time periods have shown a live prediction to be correct, so death is a sharp deviation from the norm. However, the death of a bird is by no means unexpected, especially if the bird is sick or old. Instead, the compounding of the ordinary uncertainties of life results in a forecast, either alive or dead, which has the chance of being significantly in error.

While a dying swan event has many of the same attributes as a black swan event, the underlying causes and resulting implications are different. Take George Westinghouse’s victory over Thomas Edison in the war of the currents. It clearly had a disruptive effect on the future of electricity, and contradicted the forecasts of everyone who backed Edison, but was it unexpected? The economies of scale required for mass adoption of electric power meant that the grid would be powered predominantly by either AC or DC, so the fact that one technology won out was by no means a black swan event. The fact that one technology triumphed is no surprise, but the uncertainties surrounding which would win out made reliably accurate forecasts near impossible.

Mathematically, a dying swan event can be understood through Eq. (2). Here, the forecaster is trying to predict whether \(y\) will be large or small using knowledge of \(X\) and \(h\).Footnote 7

$$y = \left( {\frac{X + \in }{h}} \right)^{100}$$
(2)

When \(X\) is much smaller than \(h\), then \(y\) will likely be small, and when \(X\) is much larger than \(h\), then \(y\) will likely be large. However, when \(X\) is close in size to \(h\), the statistical uncertainty compounds in such a way that either forecast is likely to be inaccurate. In other words, the forecast errors for nonlinear systems are not constant for all input values. Instead, forecasts can be precise for some values and highly inaccurate for other values.

Statistical and epistemic uncertainty in the shale gas revolution

Hydraulic fracturing, horizontal drilling, and related technologies and regulations did not just make it cheaper to drill for gas. They allowed access to vast formations that had been technically or economically infeasible before, and in areas where no one had previously drilled for gas in substantial amounts. EIA knew about the possibility of shale gas for many decades, and we saw mentions of it in the Annual Energy Outlook reports of the early 1990s. But it remained, in the language of Daniel Kahneman [23], a “known unknown,” with accompanying “unknown unknowns.” We had no definite notion of when it might become available, if ever, and how much might be recoverable. On the latter, we still do not know.

The scale of production was unlikely and was not prospectively predicted, and the economic, environmental and political impacts of hydraulic fracturing were far-reaching. Thus, the shale gas revolution can be viewed through the lens of a black swan event. Especially in the years before the boom started, few if any forecasters or market participants could envision the scale of market disruption. While the potential for shale gas was understood, hydraulic fracturing was seen as uneconomic, and forecasters in the early 2000s certainly could not imagine credibly that US shale gas production would account for 63% of total US dry gas production by 2018 [6].

The other lens through which to view the shale gas revolution is as a dying swan event. Figure 4 displays average gas production for wells drilled in the Barnett shale formation for the years 1998, 2006, and 2014. What is apparent is the remarkable increase in well productivity over time. This is due to a learning-by-doing effect, where drillers learn from each well drilled how to make future wells cheaper and more productive [18].

Effects such as learning-by-doing result in a positive feedback loop, where each well drilled makes future wells more economic. This in turn leads to exponential growth in the number of economically viable wells. As Fig. 4 shows, the linear increases forecasted by the EIA drastically underestimated the resulting exponential growth in production. Such positive feedback loops result in diverging potential futures, with uncertainty surrounding whether and when a critical mass will be reached, but little doubt that significant growth will eventually occur.

Fig. 4
figure 4

(Data from [18])

Barnett shale average production per well over time

Full size image

Forecast errors and modeler apologetics in context

We must acknowledge that forecast errors are inevitable due to extraneous factors that the model did not or could not consider. This process is recognizable to philosophers of science: it is a small-scale application of Kuhnian theories related to the resilience of scientific paradigms [24]. Here, the paradigm is the self-contained world of the model as a theory about inter-sectoral connections in the economy. There is, at present, no way around this reality. Forecasting inevitably means being “wrong” in an absolute sense: the world is irreducibly complex, and any attempt to model it requires simplification and abstraction that leads to divergence between forecasts and reality [25,26,27,28].

Most of the time, forecasting errors are matters of degree—for example, energy demand increased by a greater amount than forecasters predicted, but the forecast still predicted an increase within some bounds for error. Policymakers regularly use forecasts in the formulation of energy policy: the EIA, for example, exists for the express purpose of constructing such forecasts in order to inform US energy policy and both public- and private-sector investment. Forecasts prepared for policymakers and the energy sector often deal simultaneously with two domains of prediction. The first domain consists of the model itself, a simplified and abstracted approximation of reality with its own internal logic, coherence, and probabilities that follow the modeler’s beliefs about the importance of various drivers. The second domain consists of the modeler’s meta-consideration of epistemic uncertainties, which include extraneous factors that resist abstraction and quantification and thus cannot exist within the model itself.Footnote 8

While the degree of error in forecasts of future shale gas production was large, the shale gas revolution was not the first time that models have failed to predict a major event. Historically, major energy-economic models have tended to underestimate the impacts of technological change, market structure, and behavioral responses among producers and consumers, at times resulting in narrower forecast ranges that under-represent uncertainty and situational flexibility [29, 30]. In other words, it is an inherent quality of commonly used models to downplay both the potential for dramatic, game-changing developments in technology and market dynamics, and the potential efficacy of policy actions taken to shape future energy production and consumption.

Forecast errors have not been equal across all regions of the energy sector. Indeed, in a 25-year review of Schurr [31] and Landsberg [32], the two seminal energy forecasts that established the energy policy forecasting field, Joskow [33] found that

The RFF [Resources for the Future] and Ford studies have stood the test of time quite well; and “quite well” is hardly a poor grade when projecting trends and articulating policy recommendations and their effects predicated on the complex interplay of demographic, economic, technological, and environmental factors over long time periods.

Natural gas, however, has an especially ignominious history of fooling forecasts. Joskow, again discussing the RFF and Ford studies from 1979, notes that both of those venerable studies

…completely missed the increasingly important role of natural gas and the central role of efficient combined-cycle gas turbine (CCGT) generating technology using natural gas in electricity generation and its role in evolving competitive wholesale power markets. These developments in turn were stimulated by the decontrol of natural gas prices and the subsequent restructuring of the natural gas industry [33]. [italics ours]

Joskow notes that all of these developments were “in a sense, a direct but unforseen consequence of [Schurr’s and Landsberg’s] recommendations to decontrol oil and natural gas prices, to bring electricity prices to market levels, to stimulate cogeneration, and to rely more on competitive forces.” In other words, both Schurr and Landsberg, despite their expertise and best efforts, struggled to imagine the ultimate effects of complex structural changes and market dynamics that in fact emerged from their own policy recommendations.

Likewise, EIA forecasts at the turn of the millennium also missed the mark on natural gas. The EIA’s Annual Energy Outlook Evaluation from 2003 states:

Natural gas generally has been the fuel with the least accurate forecasts in consumption, production, and prices. As regulatory reforms that increased the role of competitive markets were implemented beginning in the mid-1980s, the behavior of natural gas in competitive markets was especially difficult to predict. In earlier forecasts, EIA’s technology improvement expectations proved conservative, as technological advances made natural gas less costly to produce, while in the more recent forecasts, EIA overestimated technology improvements [34].

We might call these assessments examples of “modeling apologetics,” wherein forecasters explain the reasons behind forecast errors in a post hoc analysis. By explaining the failures of the model as stemming from extraneous uncertainties, the model itself remains useful for the vast majority of future forecasts. The same logic has often been applied to the shale gas boom, where, now that we look back and see the shale gas boom as a one-off unpredictable event, we can incorporate newfound gas reserves and technologies and get on to crunching the numbers for the upcoming years. This happened with EIA forecasts for gas over the years, which incorporated the new market dynamics created by the deregulation of wholesale prices and improved forecast accuracy year by year.

In many cases, modeling apologetics are insightful and accurate, and meaningfully contribute to improved future forecasts. But there are reasons to question the universality of this narrative. Explaining away modeling errors as due to one-off unlikely events misses the prevalence of errors caused by such events, and may lure us (especially those of us who are non-modelers but rely upon model outputs) toward a heuristic of naturalistic equilibrium: a belief that “now things are normal,” or that they will soon be. The history of the energy system teaches us that the future is often directed by unlikely events, and that there is value in questioning whether naturalistic analogies of equilibrium are appropriate in many cases. Energy systems may experience multiple years, or even decades, of disequilibrium due to complex and shifting market rules, uncertainties of technological or economic feasibility at nonlinear scales of deployment, and extraordinary diversity in market structure, composition, and actors. The enormity of such extraneous uncertainties places any forecaster in very deep water.

Incorporating uncertainty in energy forecasts

Generally speaking, most analytical approaches in the energy sector do not consider disruptions well. In the case of the shale gas revolution, modelers were aware of the uncertainties associated with shale gas development and even noted it in their forecasts (e.g., [6]), but had no coherent method for inserting such knowledge into the definitive outputs of the work itself. Of course, no one is more aware of this than modelers themselves, who have gone to great lengths to address the problem. There is a wide body of literature on dealing with uncertainty in general [35], and “jumps” in commodity markets (mainly electricity) specifically (e.g., [36,37,38,39,40,41]). The more common approach is to use some family of autoregressive conditional heteroscedasticity models (see, e.g., [42,43,44,45,46,47]).

These models rely either on econometric techniques that use the past to approximate the future, making them inherently incapable of capturing future structural disruptions, or on equilibrium or optimization techniques in some combination, using mostly linear or mixed integer approaches. While there are analytical techniques that can be employed to look at radical departures from incremental change, often the ranges of possible outcomes they provide are simply too vast to be useful for a policy or investment perspective. Still, it is clear that more needs to be done with sensitivity or scenario analysis that can help inform decisions by helping to better understand boundary conditions or other factors from which unlikely futures might spring.

Just as important as a quantitative approach to incorporating uncertainty is a qualitative understanding of how large forecast errors can occur. This responsibility lies primarily not with modelers but with forecast users: policymakers, investors, and analysts who use forecasts to make decisions. Policy decisions are full of value judgments. They require decision-makers to weigh options across multiple communities and species of knowledge, and to anticipate things which cannot be quantified. These problems are complex and require expert interpretation of scientific and technical information as it relates to future developments. Forecasts are useful and necessary tools for illuminating the nature of trade-offs and hazards in such decisions. And yet there is a tension. Just as forecasting expands our knowledge of the possible consequences of our actions, it also potentially constricts the universe of considered options when it is misunderstood or—worse—misused in the service of conclusions that have been prematurely formulated and for which forecasts may be sought for post hoc rationalization. In gaining more insight about some paths, we may make other paths seem less likely, and so we are prone to prepare less diligently for those possibilities.

We are confronted with a dilemma—good decision-making requires both tractable models which can provide understandable forecasts, and the use of outputs that do not unintentionally discount the uncertainties inherent in forecasts. The way out seems to lie in a better understanding—particularly on the part of forecast users who are not themselves modeling experts—of where models can go awry and why. Especially for long-term forecasts, an understanding of epistemic uncertainty becomes increasingly important. Both forecasting and planning for the energy landscape for long time horizons require more than just an extrapolation of current trends. They require an understanding of the policies, trends, and technologies which have not yet been realized but which have a chance of changing the energy landscape in the coming decades. Additionally, in many nonlinear systems, and especially in energy systems, forecast errors begin to compound once a threshold has been passed. By acknowledging this fact, we can begin to identify which predictions are likely accurate and which have a high chance of significantly missing the mark.

To that end, we could list a series of familiar recommendationsFootnote 9:

Forecasts should emphasize that they do not predict the future, but rather provide scenarios through which sensitivities might be compared. But of course, many forecasts already do this, and policymakers, who must make decisions in the face of uncertainty, require predictions even if they lead to uses of forecasts that forecasters themselves lament. Indeed, in a hypothetical case of forecast misuse, policymakers may demand that modelers produce outputs that are deliberately oversimplified in order to support a position they have already adopted.Footnote 10 While forecasts may not predict the future, neither are they insulated from affecting it, and the potential for an influential forecast to become a self-fulfilling prophecy can present a moral hazard to policymakers beholden to powerful constituencies. No amount of scenario analysis or modeling technique can even begin to address such a problem, which is fundamentally political rather than mathematical.

Even leaving moral hazard aside and presuming that our politicians are acting with Rawlsian perfection, Morgan and Keith [49] have argued that the use of scenarios to explore feasible alternatives without respect to subjective probabilities is itself problematic and possibly untenable, as “feasibility” and “plausibility” may be substituted as synonyms for “likelihood”:

Absent a supernatural ability to foresee the future, what could be meant by a statement that one scenario is feasible and another infeasible but that the first is (subjectively) more probable than the second? [49]

In other words, Morgan and Keith caution that semantically shrouding predictions in scenarios risks embedding even more unconscious assumptions in the minds of policymakers and other model users. They cite evidence from psychology studies that experts and laypersons alike are vulnerable to the availability heuristic, wherein subjective probability assessments are biased toward how easily the subject can remember past examples or imagine futures. This heuristic, they assert, “can lead people to overestimate the probability of a scenario or story line when the detail with which it is specified is increased” [49].

Another common aphorism: forecasts should aim to identify emerging trends that could alter energy landscapes in the medium and long term. Many forecasts do this as well, with no apparent impact on collective uptake or improved accuracy, as Sanchez [32] explores with respect to EIA forecasts of the late 1990s. Or perhaps: forecasts should provide the widest possible range of options and outcomes. But such an approach risks overwhelming policymakers and inducing calls for further research—itself never opposed by researchers, but of questionable value when decisions need to be made now.Footnote 11

It is ironic that we can search the heavens and the depths for sufficient integrating capacities and miss the very capacity that is itself doing the searching. Surely the knowledge contained in even the most complex forecasting model is dwarfed by the learned intuition of modelers themselves. Of course, this intuitive knowledge is disfavored because it is nonempirical and subjective, slippery and resistant to quantification, just like the epistemic uncertainties with which we grapple. Surely, say our careful empirical sensibilities, we should not casually invite the nonobjective opinions of forecasters into the policymaking sphere. And yet, it is precisely the strong statement of opinions that leads an educated audience to be most critical and inquisitive, not least because our first impulse upon hearing a strong proclamation should be to see how it might be wrong.

It may be, then, that the best means of incorporating uncertainty into forecasts is the fostering of a robust and lively meta-discursive space composed of both modeling experts and decision-makers and focused on the second domain of modeling’s inherent epistemic uncertainties rather than further tinkering within the first domain of model structures themselves. Of course, the opinions of forecasters are as subject to availability heuristics as those of policymakers, but by displaying and revealing such biases in the light of day, we give ourselves and policymakers a greater chance at recognizing them.

Conclusions

While cheap shale gas production is now part of the energy reality, important policy decisions regarding shale gas development and use remain. What are the best technical pathways and sectoral uses of natural gas, and how can the associated build-out of related infrastructure be done most efficiently and with minimal risk? How should electricity grid operators handle the rapid increase in generation from natural gas plants to ensure grid stability? How will the evolving knowledge of fugitive emissions and life-cycle greenhouse gas emissions associated with shale gas production and use impact decision-making? How these policy decisions play out will affect the future production and price of natural gas. Thus, the future of natural gas still has a great deal of uncertainty, and any prediction should come with significant error bars and be met with a healthy degree of skepticism.

The questions raised here, and the types of forecast errors described, should be expanded to other sectors. The rapid pace of advancement and interconnectedness of the world means that epistemic uncertainty is larger than ever [50]. For many newer technologies, the degree of uncertainty regarding future generation has increased in recent years. Cost declines in technologies such as wind, solar, and energy storage place them on competitive terms with conventional generation technologies. These technologies have shown even more stark learning-by-doing effects than shale gas production. Markets for electric cars and demand response, to name a few, similarly pose the possibility for dramatic shifts. Even moderate changes in cost and policies can lead to large changes in the future adoption of these technologies.

The US shale experience has had international implications: foreign investment has flowed to US plays in an effort to unlock the techniques of hydraulic fracturing. It has also rearranged the geopolitical energy chessboard: the production of oil from shale resources has created new uncertainties in the international geopolitics of petroleum production, particularly for OPEC and non-OPEC producer states that seek to respond to new US-based production with production cuts of their own. The primacy of oil as both energy and economic driver means that these decisions produce further feedbacks that ripple throughout the global economy.

Decision-makers and investors would benefit from learning more about why they were caught unawares by the shale revolution, and how they can be better prepared the next time such a surprise occurs. The answer to that second question is not immediately apparent. Tautologically, if we were prepared for them, surprises would cease to be surprises. But perhaps a start is for decision-makers to adapt to an increasingly uncertain and dynamic world by creating a more imaginative discourse, one that welcomes nuance and doubt as spaces for opportunity and transformative change, and sees forecasts as the beginning of a policy or investment discussion rather than the end, and forecasters not as Delphic oracles of outcome, but as the people who know best why attempts at prediction must fall short.