1 Introduction

In 2006, California passed the Global Warming Solutions Act (AB32) which set the limit on greenhouse gas (GHG) emissions at 1990 levels by 2020. California Governors Schwarzenegger and Brown both passed Executive Orders providing further goals of limiting state-wide GHG emissions to at most 20 % of 1990 levels by 2050. Like many jurisdictions throughout the world with long-term GHG targets, California is now focusing on developing post-2020 climate strategies (CARB 2014). To assist in this process, several research groups have built energy planning models for California that estimate the future trajectories of technologies, fuels, infrastructure, and/or economic impacts (Roland-Holst 2008; Williams et al. 2012; Greenblatt 2015; Jacobson et al. 2014; Nelson et al. 2014; Wei et al. 2014; Yang et al. 2015).

However, policymakers sometimes have difficulty distilling the pertinent insights from energy models because of the diversity of model structures, differences in scenario specifications, opaque underlying assumptions, and medley of model results. In this paper, we perform a comparison of nine California energy planning models with projections to 2050 that include 50 scenarios (including business-as-usual (BAU) and “deep reduction” scenariosFootnote 1). Our focus is on the model results and we highlight important model inputs in the Online Resource (OR) 1. Among many benefits, inter-model comparisons can help policymakers by providing a range of conceivable technology deployment rates and GHG trajectories. These comparisons also can be useful to model developers in identifying model deficiencies and future modeling needs.

The outline of this paper is as follows: Section 2 provides the methodology of the comparison and an introduction to the nine models. In Section 3, we examine GHG trajectories, the electricity sector, the transportation sector, biofuel use, air quality, and non-energy emissions. Finally, in Section 4 we provide a discussion on future modeling needs from both the policymaker and model developer perspectives.

2 Methodology

2.1 Model comparisons

Past inter-model comparisons fall into two broad categories. The first category, “model discovery,” uses common scenario assumptions (or projections) at a specific point in time and compares the behaviors of the models (IPCC 2014a). Perhaps the most well-known and enduring model discovery exercise is the Energy Modeling Forum (e.g., Fawcett et al. 2014). Another type of comparison – as done here – is to use existing model scenarios and projections to synthesize findings across models. These model reviews help identify common insights and deficiencies across modeling platforms but require less coordination than model discovery exercises (e.g., Beaver and Huntington 1992). To our knowledge, ours is the first formal model review of California-specific energy models. This review was not possible until now because many of the models have been developed in the last 2 or 3 years.

To conduct this comparison, we distributed questionnaires to and solicited feedback from 65 policymakers and energy stakeholders on the key topics, practices, and modeling needs in the fall of 2013. We then held a two-day forum in December, 2013 (Morrison et al. 2014). Representation at the forum included high-level individuals from the California Governor’s Office, California Air Resources Board, California Energy Commission, electric utilities, public utility commission, and academia. We limited attendance to a relatively small set of stakeholders so that the discussion could be targeted and manageable. The bulk of the work for this paper and the collaboration between modeling teams occurred after the forum. We focused this paper on five topics of greatest interest to the policymakers: electricity, passenger transportation, biofuels, criteria pollutants, and non-energy emissions. Other sectors or topics, such as economic impacts, the industrial sector, or sustainability, were raised as possible dimensions of comparison, but were saved for future exercises.

2.2 Background on models

These nine models are scenario-based tools built to understand the merits, constraints, and timing of different mixes of policies, technologies, and energies in the future. All but the Wind, Water, Solar (WWS) model (Jacobson et al. 2014) focus on achieving the state’s 2020 and 2050 climate goal. WWS examines the pathway to a 100 % renewable energy system by 2050. No new scenarios were developed specifically for this model comparison.

The exact composition of future scenarios depends largely on the assumptions, storylines, and analytical underpinnings of the scenarios. Some scenarios in these models emphasize immature technologies such as hydrogen fuel cell vehicles, while others explore shifts in energy service demand (e.g., reduction in vehicle miles traveled), or changes to key input parameters (e.g., price elasticity of energy demand). Because most models have been developed over a number of years and have multiple versions, we limit this comparison to the most recent model version (Table 1) unless otherwise noted.

Table 1 Model versions used for this comparison
Full size table

Table 2 summarizes key characteristics of these nine models. Note that LEAP and SWITCH are soft-linked and are typically run together. For this paper, we also reviewed the 2008 and 2014 AB32 Scoping Plan from CARB (2008, 2014) and the CCST-Bioenergy report (Youngs 2013).

Table 2 Comparison of nine models across multiple dimensions
Full size table

The models in this paper can be categorized into three broad model structures: optimization, equilibrium, and inventory models (also known as “accounting frameworks”). A model’s structure is indicative of both the types of research questions that can be addressed with the model, as well as the caveats to bear in mind when interpreting results. As others have noted (Beaver 1993), heterogeneous model structures make model comparisons more difficult, but also lead to a greater number of insights.

3 Comparison

3.1 GHG trajectories

Across models, reference scenarios have a wide range of emissions by 2050: over 800 million metric tonnes CO2-equivalent per year (MMT CO2e/year) in the CCST and PATHWAYS models to under 500 MMT CO2e/year in CA-TIMES.Footnote 2 Reference scenarios help capture the underlying assumptions of the models: for example, the models with the highest GHG trajectories (PATHWAYS and CCST) had 10–20 % higher income and population assumptions by 2050 than more recently developed models,Footnote 3 . Footnote 4

In scenarios that achieve deep reductions in GHGs by 2050, the GHG trajectories also vary widely. Annual emissions decline to 8–46 % below 1990 levels by 2030 and 59–84 % by 2050, or to 230–396 MMTCO2e per year and 68–175 MMTCO2e per year, respectively (Fig. 1a). Most deep reduction scenarios have lagged emission reductions, meaning they wait until later years to make the steepest declines in reductions. Rates of change in annual GHG emissions vary between −0.3 and −5.3 % per year from today to 2030 (average across scenarios of −1.7 %), and −0.8 to −19.9 % per year from 2020 to 2050 (average of −5.2 %).

Fig. 1
figure 1

Annual GHG emissions (1a) in MMT CO2e /year and cumulative emissions from 2010 onward (1b) in total CO2e for the select deep reduction scenarios (highest and lowest from each model)

Full size image

For ease of viewing in Fig. 1a/b, we only show the highest and lowest deep reduction scenario from each model that projects GHG emissions. Also shown are the linear and constant-percent reductions between the 2020 GHG target of 431 MMT CO2e/yearFootnote 5 to the 2050 target of 86 MMT CO2e/year (black lines). A linear interpolation was used between time steps. LEAP-SWITCH explores scenarios that achieve greater than 80 % reduction by 2050, but these scenarios are not presented in this paper,Footnote 6 . Footnote 7

Figure 1a and b help demonstrate the importance of early emissions reductions using a side-by-side comparison of annual and cumulative emissions.Footnote 8 For example, CALGAPS (S3) only achieves a 59 % emission reduction in annual emissions by 2050 (Fig. 1a) but has the lowest cumulative emissions between 2010 and 2050 (Fig. 1b). Conversely, the PATHWAYS (Hi Renew) scenario achieves an 80 % reduction by 2050 but has the highest cumulative emissions in 2050 due to its lagged reduction schedule. Others have shown that the cumulative emissions of a scenario are a robust indictor for whether that scenario stays below a particular level of global warming (Meinshausen et al. 2009). Of course, Fig. 1a/b use 2050 as the end point – if these trajectories were extended to 2070 the PATHWAYS scenario may have cumulatively lower emissions than the CALGAPS scenario.

3.2 Power sector

Between 2001 and 2013, electricity generation in California (including both in-state and net imports before transmission losses) increased from 267 TWh to 296 TWh and the corresponding renewable fraction of generated energy increased from 14 to 20 %,Footnote 9 . Footnote 10 In the same years, the capacity of the grid powering California expanded from 60.8 GW to 88.5 GW (CEC 2014).

The future expansion of the electricity grid poses both spatial and temporal challenges to energy planners (Hart and Jacobson 2011; Williams et al. 2012; Nelson et al. 2012; Wei et al. 2013). The models examined here differ widely in their geographic scope and resolution. For example, the SWITCH model includes a multi-state region (the Western Electricity Coordinating Council) which allows for optimal solutions across state boundaries. Other models assume a certain fraction of out-of-state generation is always available or, like CA-TIMES, assume all power generation after a certain year is generated in-state. SWITCH also is the only model that determines the geographic location and capacity of future power plants and transmission lines. The time dimension also differs widely between models. The models that include time-of-day dispatch models to better understand renewable intermittency problems include CA-TIMES, PATHWAYS, SWITCH, and WWS.Footnote 11

In CCST, SWITCH,Footnote 12 and WWS, demand for electricity is driven exogenously. PATHWAYS estimates demand using a “bottom-up” approach in which the electricity requirements of each individual end-use is first estimated then summed. CALGAPS estimates demand in a similar way as PATHWAYS, but electricity requirements are determined at the sector level rather than by end use. In CA-TIMES, electricity demand is determined endogenously based on the need to meet the 2050 GHG goal.

Across BAU scenarios, the total power generation from in-state plus imported electricity generation increases by 20–31 % above the 2013 level by 2030 and 45–75 % by 2050. In all deep reduction scenarios, the electricity grid shifts towards renewable generation – particularly after 2030 – and most end-uses are electrified by 2050. Because some sectors cannot be electrified or are difficult to decarbonize (e.g., aviation, marine, heavy duty road freight, agricultural fertilizer, etc.), GHG emissions from the electricity grid will likely need to be reduced beyond 80 % (Williams et al. 2012; Nelson et al. 2014; Yang et al. 2015).

As shown in Fig. 2, across deep reduction scenarios the total power generation increases by −2 to +40 % by 2030 and 8 to 226 % by 2050, relative to 2013. For WWS, these increases are much larger: 334 and 465 %. The renewable fraction of total generation is 30–58 % by 2030 and 30–89 % by 2050, with the majority of new generation coming from wind and solar. For WWS, the renewable fractions are 85 and 100 %, respectively. These renewable ranges include small but exclude large hydroelectric generation and therefore are roughly consistent with California’s definition of “renewable electricity” in its Renewable Portfolio Standard (RPS). In general, the lower values in these ranges reflect scenarios with greater nuclear and/or CCS deployment. Across scenarios, the implied build-out rate of in-state plus imported renewable electricity (mostly solar and wind) ranges between 0.2 and 4.2 GW per year from 2013 until 2030, with an average of 0.83 GW per year. The renewable build-out rate increases to between 1.5 and 10.4 GW per year from 2030 until 2050, with an average of 3.9 GW per year. Faster rates of grid expansion are assumed in the WWS model: an average of 17 GW of nameplate renewable capacity added per year from 2013 to 2050 to reach 652 GW of total renewable capacity by 2050. For perspective, from 2001 to 2013 the renewable capacity used by the state (in-state and imported electricity) expanded by 0.67 GW per year while non-renewable capacity expanded by 1.6 GW per year (CEC 2014).

Fig. 2
figure 2

2030 and 2050 electricity generation (TWh/year) in deep reduction scenarios. Figure includes in-state production and imported generation. Individual scenarios shown. Percentages below points are percent renewable (excluding large hydro) across mitigation scenarios. Note: CCST did not report 2030 generation

Full size image

3.3 Passenger transportation sector

A standard practice for modeling the transportation sector in energy models is to make exogenous assumptions about future energy service demand (e.g., statewide vehicle-miles travelled (VMT)) and then allow the model to estimate future fuel mix, vehicle/technology mix, and emissions. The models in this study all follow this practice. The lower the future demand assumptions, the less the need for low-GHG emitting fuels.

For example, in deep reduction scenarios statewide VMT for light-duty vehiclesFootnote 13 is assumed to change from 293 billion miles per year in 2010 (CARB 2012) to 226–600 billion miles in 2050. Therefore, the amount of near-zero CO2e emission energy used across these models differs widely. Figure 3 shows the passenger light-duty vehicle (LDV) energy projections (stacked columns) and the total transportation sector energy (red triangles) for the model reporting detailed LDV-specific results. Across deep reduction scenarios, total LDV energy use ranges from 8.6 to 25.2 billion gallons of gasoline equivalent (BGGE) in 2030 (1.1–3.3 exajoules (EJ)) and 8.1–19.6 BGGE in 2050 (1.1–2.6 EJ).

Fig. 3
figure 3

Passenger LDV energy projections (stacked columns) and the total transportation sector energy (red triangles) for the model reporting detailed LDV-specific results. Note that each fuel provides a different energy intensity of travel (e.g., electric vehicles go 2–3 times as far as a gasoline vehicle per MJ of energy)

Full size image

With the exception of the PATHWAYS-mitigation scenario, the passenger LDV energy drops from 2010 to 2030, and again from 2030 to 2050 in deep reduction scenarios. This decline results mainly from (1) the underlying assumptions about lower energy service demand in future years and (2) the improved efficiency of LDV technology. Across deep reduction scenarios, petroleum consumption declines 15–72 % by 2030 and 39–100 % by 2050 as the light-duty-vehicle fleet moves primarily to battery electric, plug-in hybrid electric, and hydrogen fuel cell vehicles, although the composition and magnitude of change varies between scenarios. For example, in CA-TIMES the combination of battery electric and hydrogen fuel cell vehicles makes up between 50 and 96 % of the LDV fleet in 2050. In the ARB VISION model’s mitigation scenario, these same technologies comprise over 80 % of the LDV fleet in 2050. Regardless of the exact fleet composition, hydrogen and electricity with near-zero life-cycle GHGs (e.g., from wind, solar, biomass, NG with CCS) are needed to power virtually all of the LDV fleet by 2050.

3.4 Contribution from bioenergy

Bioenergy assumptions are important drivers in energy planning models (Wei et al. 2013; Rose et al. 2014). The more “low-carbon” bioenergy assumed to exist, the fewer mitigation strategies that are needed in other sectors and technologies. Across models reviewed here (except WWS), between 4 and 13 billion gallons of gasoline equivalent (BGGE) (0.4–1.7 exajoules) are used in 2050 – up from about 1.0 BGGE today.Footnote 14 Models utilize biomass supply curves from Parker et al. (2010) or De la Torre Ugarte and Ray (2000).

Most models make simple assumptions regarding the carbon content of bioenergy. For example, SWITCH assumes bioenergy has 30 % lower carbon intensity than petroleum-based fuels today and improves to 80 % lower by 2050. PATHWAYS only includes biomass feedstocks produced in the U.S. that have a “net-zero” carbon intensityFootnote 15 on a lifecycle basis including corn stover, wheat straw, forest residues, forest thinning, and switchgrass (Williams et al. 2012). CA-TIMES assumes a carbon intensity of 75–80 gCO2e/MJ for corn ethanol, 25–30 gCO2e/MJ for cellulosic ethanol, and 13–30 gCO2e/MJ for waste-based or Fischer-Tropsch diesel. CALGAPS estimates net life-cycle GHG emissions for biofuels that include offsets based on the assumed in-state portion of biofuels produced. ARB VISION assumes that the average carbon intensity of all biofuel declines from 67 to 41 gCO2e/MJ. It should be emphasized that all models here assume point estimates rather than distributions in carbon intensity. A number of studies suggest that these carbon intensities are highly uncertain (e.g., Plevin et al. 2010).

Setting aside this concerns and assuming that bioenergy with low lifecycle GHG emissions will in fact exist in the future, CA-TIMES suggests these fuels are best utilized in the transportation sector (rather than in other sectors). This is mainly because fewer mitigation options are available in the transportation sector compared to other sectors, in particular in aviation, marine, and heavy duty road transport. Across scenarios, bioenergy accounts for a maximum of about 40 % of transportation energy in 2050. Not all long-term energy modeling suggests large quantities of biofuels are needed in the transportation sector. The WWS model, presents a vision of 2050 without bioenergy, relying instead solely on batteries and hydrogen. Biofuel and bioelectricity production with CCS are modeled in sensitivity analyses in the CA-TIMES and SWITCH models but their results are not presented here.

3.5 Non-CO2 GHG and criteria emissions

Reducing non-CO2 GHGs and criteria emissions is a major policy focus in California; however most energy planning models spend relatively little effort on characterizing their future levels. The relative contribution of non-energy and High Global Warming Potential (HGWP) GHGs to overall emissions levels is likely to increase in the coming decades. Greenblatt (2015) and Wei et al. (2013) find that, absent further policy, these emissions alone could exceed the 2050 emission goal.

For criteria emissions, California policymakers are contemplating how to transform the energy system to simultaneously meet GHG targets and the near-term (2023) and midterm (2032) National Ambient Air Quality Standards (NAAQS) for ozone. In particular, meeting the 2032 legally-binding NAAQS deadline will be challenging given historical vehicle turnover rates and higher costs of clean technology. CARB (2014) reports that additional strategies, early action, and more rapid development and adoption of zero-emission technologies is needed. How to do this while also meeting GHG targets remains a lively policy debate. Table OR.2 of the OR-1 compares (1) the criteria pollutants estimated by each model and (2) the spatial resolution of these estimations. The table shows that only three of the nine models estimate NOx, ROG, and PM2.5 and none do so at a highly resolute level.Footnote 16

4 Discussion and conclusions

Twenty states in the U.S. and the District of Columbia have adopted 2020 and/or longer-term GHG targets (see table OR.3 in the Online Resources for a complete list). While the results from this model comparison are California-specific, the insights may be useful to other jurisdictions that have long-term climate targets or those considering such targets. Our conclusions fall into two broad categories: (1) lessons for policymakers and (2) lessons for model developers and those interested in facilitating model comparisons.

One of the more lucid takeaways for policymakers is the need to consider both annual and cumulative emissions when setting GHG targets and building climate strategies. Scenarios with aggressive and early emission reductions achieve far lower cumulative emissions by 2050. In some BAU scenarios, California has more than twice the cumulative emissions in 2050 as in the mitigation case. From a climate perspective, the obvious implication is that near-term reductions are preferable to delayed reductions (IPCC 2014b).

The model results also highlight how achieving deep GHG emission reductions in the state will require further policy commitments beyond the current policy measures (many of which stop in 2020). Rates of technology transitions in these models – such as deployment of better vehicles or renewable electricity – exceed the historical rates of change in California, often by two or three times. The optimization models (CA-TIMES, SWITCH) suggest that the least expensive path to make these transitions includes aggressive decarbonization of our electricity supply, electrification of most end-uses, increases in energy efficiency, and deployment of low-carbon transportation fuels and technologies.

Policymakers should also recognize their important role in model development. For example, modelers at the December forum requested more up-to-date information about upcoming policies and more access to the latest state-collected data to improve model calibration/validation and to strengthen analysis of existing and future policies. The December forum also served as a reminder to policymakers about limitations of energy modeling. One cogent example given at the forum was that air quality models adequate for state policy planning and regulation need to be highly spatially-disaggregated whereas most energy models are at the regional or state-level. For example, California’s statewide raw emissions inventory had a spatial resolution of 4 × 4 km on a Lambert Conformal projection of the earth’s surface. The South Coast Air Quality Management District (SCAQMD) raw emissions inventory had a spatial resolution of 5x5 km on a Universal Transverse Mercator (UTM) coordinate system. Exposure assessment studies that seek to measure human health impacts are evaluated at even finer resolutions (e.g., 1 × 1 km or at street or block level) (Zapata et al. 2013). Although some integrated assessment models have linked energy projections with air quality to understand co-benefits of climate policies (e.g., McCollum et al. 2013), the resolution of these models is at most 1x1° (approximately 111 × 111 km at the equator). Despite requests from policymakers, attempting to integrate air quality and energy models at a finer resolution may not be practical in terms of model development and computing time.

There are also a number of important lessons for model developers and for those who conduct model comparisons. Yang et al. (2015) report that results of scenarios in CA-TIMES are highly sensitive to the assumed baseline, technology costs, and availability of breakthroughs (such as advanced bio-liquids, nuclear, and CCS). By treating these drivers of the models with distributions rather than point estimates, one can better understand their impact on the future and design policy to be more robust to potential outcomes.

Additionally, there is a need to better understand the shape of the GHG trajectories between 2020 and 2050 and how the shape contributes to the cumulative cost of scenarios. “Bending” the annual GHG curve downward in early years may provide technological, economic, and risk management options if certain measures prove more difficult than anticipated. However, this has not been adequately researched.

Model developers could also consider standardizing input assumptions in models, such as population, income, elasticities, costs, emission factors, etc. This could be achieved through close coordination between modeling teams, making the models open-source, and by increasing the interoperability between research models and government models used by the state. Greater coherence/transparency of technology adoption/diffusion assumptions, and more extensive data sharing, would likely improve the insights gained from model comparisons. Indeed, having a set of standardized scenarios as done by the Intergovernmental Panel on Climate Change in their Share Socioeconomic Pathways (SSPs) could simplify the often-opaque black boxes of the modeling world (O’Neill et al. 2014).

A final set of lessons for model developers was expressed by policymakers at the December, 2013 forum. Policymakers involved asked for more modeling of: (1) individual policies (i.e., rather than generic climate policies) in order to better understand the spatial, temporal, and socio-economic effects of regulations, (2) interactive effects between two or more policies, (3) non-emission impacts like water, land-use, and economic equity, and (4) the optimal sequencing (i.e., timing) and prioritization of policies and technology deployment. Lastly, policymakers requested that model output be reported in the same metrics as those used in the policy arena. Policymakers specifically requested greater reporting of performance metrics (such as gCO2e/mile for vehicles, average gCO2e/MJ for fuels, gCO2e/kWh for electricity, percent renewables by year) and economic metrics (such as $/metric ton CO2e, percent change of household expenditure on energy, and lifecycle costs of travel in $/vehicle miles traveled).