Introduction and background

Climate change is expected to reveal vulnerabilities in the US energy system, and the US power sector in particular, in several ways (Dell et al. 2014; U.S. DOE 2010; Wilbanks et al. 2012). Rising air temperatures are projected to increase the electricity demand for air conditioning over more days of the year, over more areas of the country and at greater intensity during peak times. This greater demand would coincide with reductions in effective thermal plant capacity. Higher temperatures reduce the availability of cooling water (Dodder 2014) and reduce the efficiency of dry cooling (ICF 1995). Additionally, elevated temperatures restrict the capacity of transmission lines. Higher sea levels and storm surge place many coastal plants at risk of inundation. Changing precipitation patterns may alter the capacity and operation of existing hydropower facilities. Finally, extreme weather events such as drought, heat waves, and intense storms may exacerbate the all of these vulnerabilities.

This study focuses on the effect of changes in ambient air temperature and climate policy on electricity demand and supply. An important and novel aspect of this study is feeding a consistent set of temperature data and climate policy assumptions through three electricity demand and supply models—the U.S. version of the Global Climate Assessment Model (GCAM-USA) developed by the Joint Global Change Research Institute at Pacific Northwest National Labs, the Regional Electricity Deployment System model (ReEDS) from the National Renewable Energy Laboratory and the Integrated Planning Model (IPM®) of ICF Resources, Inc. The use of multiple models allows for the comparison of methods and aims to produce robust results. The three electricity modeling groups translated changes in surface air temperatures into changes in electricity demand and supply for scenarios which differ by temperature pathway and climate policy. Although the other impact channels are no less important than rising air temperatures, the modeling capacity to look at multiple impacts remains an area of continuing development.

Changes to building energy demand from rising temperatures, have been examined in several studies at the national level (e.g., Rosenthal et al. 1995; Hadley et al. 2006; Zhou et al. 2013, 2014). Changes in the primary energy demand in these studies are mixedFootnote 1 likely due to several factors including different assumed temperature pathways, spatial resolutions, energy models, and because of the partially offsetting effects between lower winter heating demand and higher summer cooling demand from rising temperatures. However, electricity demand in these studies is uniformly higher to meet the increased need for air conditioning. Preliminary work by Sue Wing (2013) using an econometric model found a 7.6 % increase in annual electricity demand in 2050. At the regional level, Dirks et al. (2014) project using a highly detailed buildings model across the Eastern Interconnect of the U.S. project an increase of over 7 % in annual electricity and a 60 % increase in cooling demand. Hamlet et al. (2010) explore changes in energy demand in the Pacific Northwest and find the share of residential energy demand for cooling (i.e., electricity) nearly quadruples from about 1 % today to 3.8 % by mid-century. Franco and Sanstad (2008) in a study of climate change on California’s electricity demand show mid-century increases from 1.6 to 8.1 % depending upon the emissions scenario and climate model.

Section 2 of the paper describes the study design, models and methods. Section 3 presents the demand-side results. The supply-side results follow in Section 4 and Section 5 concludes.

Study design, models and methods

Study design

This analysis is part of the Climate Change Impacts and Risk Analysis (CIRA) project (Waldhoff et al. 2014, this issue) which aims to quantify the physical and economic impacts of climate change in the United States. One of the central features of the CIRA project is the use of consistent socio-economic and climatic projections. To that end, this analysis uses temperature pathways developed by the MIT IGSM-CAM model as described in Monier et al. (2014, this issue). The socioeconomic and emissions projections underlying these pathways may be found in Paltsev et al. (2013, this issue).

This study compares the electricity demand and supply results from three electricity models (GCAM, IPM, and ReEDS) across six scenarios that differ by temperature pathway and climate policy (see Online Resource Table 1). The scenarios are summarized below.

  • Control scenario - No temperature change, no policy. This scenario holds ambient air temperatures constant over time. The Control scenario reflects a typical baseline simulation of each model in which electricity demand is unaffected by temperature change.

  • REF CS3 - Reference temperature change with climate sensitivity of 3°. This scenario incorporates the effects of rising temperatures under reference (i.e., no policy) emission levels using the same global reference GHG emission pathways at equilibrium climate sensitivities (CS) of 3° Celsius.

  • REF CS6 - Reference temperature change with climate sensitivity of 6°. The higher climate sensitivity represents a low probability, higher temperature scenario.

  • POL4.5 CS3 - Emission reduction policy and temperature pathway consistent with a radiative forcing target of 4.5 W/m2. Cumulative power sector emissions from 2015 to 2050 are reduced by 8.9 %.

  • POL3.7 CS3 - Emission reduction policy and temperature pathway consistent with a radiative forcing target of 3.7 W/m2. Cumulative power sector emissions from 2015 to 2050 are reduced by 21.3 %.

  • TEMP 3.7 CS3 - Temperature pathway consistent with a radiative forcing target of 3.7 W/m2, but without the emission reduction policy. This scenario isolates the effect of a small temperature change under a low emission scenario from the combined policy and temperature effects in POL3.7 CS3.

The two climate policy scenarios (POL4.5 CS3 and POL3.7 CS3) represent the emissions reductions required in the U.S. electric power sector consistent with global emissions pathways required to achieve equilibrium levels of radiative forcing of 4.5 and 3.7 W/m2 in 2100. The power sector emissions pathways for the two policy scenarios, POL4.5 CS3 and POL3.7 CS3, are based on the change in power sector emissions from the global GCAM simulations of these scenarios (see Calvin et al. 2014, this issue). We apply the percent reduction in cumulative power sector emissions from 2015 to 2050 of the GCAM model to the three models in this analysis. The percentage change in emissions is appropriate to use instead of an absolute value because GCAM-USA, IPM, and ReEDS have different emissions pathways in the control scenario. Cumulative emissions versus the Control scenarios fall by 8.9 and 21.3 % in the POL4.5 CS3 and POL3.7 CS3 scenarios, respectively. To meet this cumulative target, the 2050 annual emissions fall by roughly 23 and 55 % below reference emissions in 2050 for the respective POL4.5 CS3 and the POL3.7 CS3 scenarios.

Model descriptions

This section provides a description of the models used in the analysis including an overview of the model, the translation of temperature change into electricity demand, and implementation of the policy scenarios. See Table 1 for a summary of the models’ attributes. To aid in comparing the results across models with different native geographic resolution and make the analysis more tractable, results are aggregated—using population weighting where appropriate (e.g., heating and cooling degree days)—to six national regions as shown in Table 2.

Table 1 Attributes of temperature-sensitive demand and supply–demand models
Table 2 Reporting regions


The GCAM-USA model is a detailed, service-based building energy model for the 50 U.S. states (Kyle et al. 2010; Zhou et al. 2014). Nested within the global GCAM model (Kim et al. 2006), it allows for greater spatial representation of U.S. buildings sector while maintaining the full interaction with other U.S. sectors and other global regions. GCAM is a recursive dynamic model that projects greenhouse gas emissions and energy trends to the end of the century, and it includes partial equilibrium economic models of the global energy system and global land use.

The heating and cooling demands come from the buildings sector in each state, which is based on two representative building types: residential and commercial. Each building type demands six service categories: heating, cooling, lighting, hot water, appliances (residential) or office equipment (commercial), and others. These services are provided by end use technologies, the number of which depends on the service. These technologies use four types of fuels including electricity, natural gas, fuel oil and biomass. Other electricity demands (e.g., industrial and transportation) are modeled at the state level.

The electricity demand from buildings is a function of floor space, building shell efficiencies, end use technologies, state economic and population growth, population-weighted heating and cooling degree-days (HDD/CDDs), and other technical and calibration parameters (see Zhou et al. 2014 and the Online Resource). The model is calibrated to a base year of 2005. The change in energy demand for the non-controls is based on the changes in HDD/CDD over time from the CIRA scenarios’ temperature data.


The Regional Energy Deployment System model (ReEDS) is a deterministic, myopic, optimization model of the deployment of electric power generation technologies and transmission infrastructure for the contiguous United States. It is designed to analyze critical energy issues in the electric sector, especially power sector emissions constraints and clean energy standards. ReEDS provides a detailed treatment of electricity-generating and electrical storage technologies and specifically addresses a variety of issues related to renewable energy technologies, including accessibility and cost of transmission, regional quality of renewable resources, seasonal and diurnal generation profiles, variability of wind and solar power, and the influence of variability on the reliability of the electrical grid. ReEDS addresses these issues through a highly discretized regional structure (e.g., 134 balancing areas), explicit statistical treatment of the variability in wind and solar output over time, and consideration of ancillary service requirements and costs.

To translate temperature change to change in power demand, a temperature-sensitive econometric demand model was developed for each of ReEDS’ regions. The model estimates the change in electricity demand load as a function of HDD/CDD over a reference level of demand for each of the power control areas. The model parameters are based on detailed empirical utility load data for over 300 transmission zones over a 2 year period (see Sullivan et al. 2015 and Denholm et al. 2012). Parameter estimates are obtained for four seasons, which captures both heating and cooling seasons, and four daily time slices. Unlike the structural equations used in GCAM and IPM that specify residential and commercial heating and cooling, the ReEDS demand model aggregates all temperature-sensitive demand changes including industrial. The model assumes a fixed ratio of temperature-sensitive demand to total demand. A limitation of this approach is that the model does not capture changes in consumer preferences, shifts in population, and technological change (e.g., end-use efficiency improvements).

ReEDS, an electricity-only model, requires additional information relating a global, economy-wide GHG reduction pathway to U.S. electricity-sector CO2 emissions limits. As an estimate for the electricity sector’s share, ReEDS uses as input the electric-sector CO2 emissions from the relevant GCAM scenarios, rescaled to match ReEDS’ 2010 emissions levels. In contrast to GCAM and IPM, ReEDS assumes that carbon credits are given away to emitting sources rather than auctioned off. This policy assumption has the effect of reducing the cost of a GHG policy to utilities, thereby damping the price change seen by consumers and the demand response.


The Integrated Planning Model (IPM®) is a well-established electric sector dispatch and capacity planning model used by both the public and private sectors to inform business and regulatory policy decisions. The implementation of IPM used for this study (EPA Base Case v4.10) represents the power system of the contiguous United States and Canada in 32 model regions (see Online Resource Figure 1). The model is a fully forward-looking linear programming model that determines the least-cost method of meeting energy and peak demand requirements over the period of 2012 to 2050.Footnote 2 It provides integration of wholesale power, system reliability, environmental constraints, fuel choice, transmission, capacity expansion, and all key operational elements of generators on the power grid.

For the present work, IPM treats electricity demand as an input from a separate electricity demand model. The demand model uses structural equations that represent electricity demand as a function of activity (e.g., population or employment), structure (e.g., square footage, climate), and intensity (i.e., electricity used per unit of activity) as described in Jaglom et al. (2014). The population, employment, and square footage factors change over time, driven by assumptions about regional population growth contained in the Annual Energy Outlook (U.S. DOE 2010). Intensity can be thought of as a semi-empirical measure of the technology, efficiency, and consumer preference that drive demand in a particular region. The intensity factors are estimated based on the base-year and projected electricity consumption from the Annual Energy Outlook and the observed HDD and CDD from 2000 to 2009. The intensity factors are constant across the scenario, but change over time to capture shifts in exogenous variables such as consumer preferences and end-use efficiencies.

For the non-control scenarios in which temperature changes over time, the demand model uses HDD/CDD data to calculate the changes in temperature sensitive demand (i.e., residential and commercial heating and cooling). A 30-year centered average of temperatures is used in the HDD/CDD calculations. These changes in demand are applied to the exogenous control scenario demand that feeds into IPM. The single policy scenario run by IPM, POL3.7 CS3, is implemented as a cap-and-trade system in which banking is allowed; borrowing is not.

Comparison of heating and cooling degree-day estimates

All of the models used the same method for calculating HDD/CDD from the temperature data using a base temperature set-point of 65 °F, a common convention (see Isaac and van Vuuren (2009) and equations in the Online Resource). This convention was chosen because of its widespread use in the literature and simplicity. In doing so, we acknowledge the work of Hekkenberg et al. (2009) that suggests this method may lead to a conservative energy demand estimate.

The national, population-weighted HDD/CDD values for each model are shown in Online Resource Figure 2. The models exhibit closely aligned trends of falling HDD and rising CDD over time and the values fall within a range of 1.6 to 6 % of the median. From 2005 to 2050, HDD declines by 780 to 990 degree-days (−19 to −24 % from 2005) while CDD rises by 540 to 670 degree-days (32 to 43 % from 2005).

Snapshots of HDD and CDD broken-out by region, model, and scenario for 2005 and 2050 may be found in Online Resource Figures 3 and 4. Across the regions, the absolute decrease in HDD is greatest in the northern regions (Northeast, Midwest, and Northwest). As expected the decline is more pronounced with the higher temperature scenarios (REF CS6,3) than the policy scenario (POL3.7). The absolute increase in CDD over time is greatest in the southern regions (Southeast, South Central, and Southwest). The increase in CDD is greatly mitigated in the policy scenario.

Though the absolute differences in CDD are largest in the southern regions, the percentage change over time is greatest in the northern regions as shown in Fig. 1. The average increase in Northeast and Northwest is 70 % (averaged across the models for REF CS3) followed by 35 % in the Midwest and Southwest. The percent changes within a region are consistent across most regions with the exception of the Northeast and Northwest in which IPM is higher than the other models by over 20 percentage points. The variation in the changes between the models is attributed to differences in population assumptions, spatial resolution, and smoothing method applied to noisy temperature or HDD/CDD time series data. The percent change in heating degree day shows more uniformity across regions and models at around −20 % (REF CS3) with the Southwest slightly greater at −29 %. Under the POL3.7 scenario, the average change in CDD drops to 19 % and the change in HDD moves to −15 %.

Fig. 1
figure 1

Percent change in regional HDD/CDD from 2005 to 2050 by scenario, region and model

Temperature and policy effects on electricity demand

Effects on U.S. electricity demand to 2050

Accounting for changes in temperature over time raises the projected demand for electricity. Figure 2a shows US electricity demand over time across models and scenarios with the percentage change from the Control scenario (no temperature change) in Fig. 2b. Turning first to the REF CS3 scenario, the models show stronger divergence from the Control with rising temperatures over time. IPM exhibits the strongest response with a 6.5 % increase over the Control in 2050. The response from GCAM and ReEDS are more muted at 2.1 and 1.7 %, respectively. Though the IPM response is higher than GCAM and ReEDS, it is lower than the 7.6 % increase in annual demand found in a preliminary analysis of the U.S. power sector (Sue Wing 2013) using a scenario with lower reference emissions (i.e., SRES A2).Footnote 3 The percentage electricity demand in the higher temperature scenario, REF CS6, for both GCAM and ReEDS rises by 2.3 %. Reasons for the difference in responsiveness across the models will be discussed after the regional results.

Fig. 2
figure 2

a, b Electricity demand by model and scenario to 2050 (a) and percent change in electricity demand versus the Control scenario (b)

The change in demand is lower under the policy scenarios. Demand in the POL3.7 scenario in IPM rises by only 2.2 % versus the control because the lower temperatures depress electricity demand for cooling. Recall that in this particular implementation of IPM, demand does not respond to changes in power price (zero price elasticity of demand). In ReEDS the POL4.5 scenario shows a small increase (0.5 %) above the control in 2050. Under the more stringent POL3.7 scenario, demand in 2050 falls by 0.8 %. In GCAM demand falls by 1.2 % under POL4.5 yet falls by only 0.4 % under POL3.7. Under the more stringent reduction scenario in GCAM, the demand for low-carbon electricity increases because this sector is the least expensive to decarbonize.

Regional effect on electricity demand

Having examined how electricity demand changes over time, we turn to regional changes. Figure 3 shows the regional percentage change in electricity demand in 2050 for the REF CS3 and CS6 scenarios versus the control scenario. There is no readily discernible regional pattern across the models. GCAM has marginally stronger response in the Southeast (2.4 %) and South Central (2.1 %) regions and a very weak response in the Midwest (0.8 %). ReEDS shows a more uniform increase across the regions, from 1.4 to 2.3 %, with the exception of the Northwest, which shows a decline in electricity demand because the reduction in electricity for heating offsets the increase in cooling demand. In IPM the percentage change in demand is highest in the Northeast, Midwest, and Southwest at 7.6 to 9 % and only 4.2 to 5.8 % in the other regions. Demand changes for the policy scenarios may be found in Online Resource Figure 5.

Fig. 3
figure 3

Percent change in electricity demand by region and model, reference scenarios versus control in 2050 Blue and red represents the change for the REF CS3 and REF CS6 scenarios, respectively

Comparison of demand models

The two- to five-fold differences in the change in electricity demand across models and regions is explained by the demand sensitivity to changes in cooling and heating degree-days. To isolate the influence of CDD/HDD from changes in other factors over time (e.g., changes in population, floor space, building efficiencies within the GCAM and IPM demand models), we compute the ratio of the change in electricity demand from the control scenario to REF CS3 in 2050 to the change in CDD/HDD from 2005 to 2050. Figure 4 shows the demand sensitivity to HDD/CDD by model and region with separate breakouts for commercial and residential buildings for GCAM and IPM. The combined residential and commercial CDD demand sensitivities for IPM are up to four to five times greater than GCAM and ReEDS, which is consistent with the electricity demand response embodied in the structual demand modelused.

Fig. 4
figure 4

Electricity demand sensitivity by region, model and demand sector in 2050. The demand sensitivity isolates the effect on electricity demand from the temperature change between the control and REF CS3 scenarios. The heating sensitivity values represent the decrease in electricity demand for a change in HDD; the cooling sensitivity values represent the increase in demand for a change in CDD

The figure also shows that the main source of the difference between IPM and GCAM comes from IPM’s greater sensitivity in the commercial cooling sector; the sensitivities in the residential sector are very similar. The sensitivity differences are reflective of the differences in the cooling demand in the underlying data used to calibrate the models. GCAM uses EIA survey data (Zhou et al. 2014) while IPM calibrates to EIA’s Annual Energy Outlook (Jaglom et al. 2014). Reconciling uncertainties in the underlying historic data may help bridge these differences.

Temperature and policy effects on electricity supply

Higher temperatures affect the supply-side of the power sector in three ways. The largest effect is through higher electricity demand for space cooling as discussed in Section 3. Meeting this demand requires greater electricity supply which entails higher fuel consumption and investments in generation capacity, particularly to meet summer peak demand. The higher ambient air temperature also lowers rated capacity of thermal units slightly because of a decrease in dry cooling efficiency. For perspective, using the CIRA reference temperature change to 2050, dependable capacity is estimated to fall by 0.6 % for steam units and by 2 % for gas turbines (Jaglom et al. 2014). All three models incorporate this effect in the supply-side analysis. Higher air temperatures also reduce the capacity of transmission lines as the lines hit thermal constraints (ICF 1995). This effect is modeled in ReEDS.

Effects on generation mix and system cost

The effects of both air temperature and climate policy on the generation mix are shown in Fig. 5. As expected, the positive changes in demand versus the control in the reference scenarios, 1.6 to 6.5 % by 2050, are met by corresponding increases in generation. The policy scenarios exhibit smaller changes in demand, −1.2 to 0.8 % by 2050. The supply mix in the REF CS3 and CS6 scenarios does not differ substantially from the control for a given model. GCAM expands all generation types in roughly equal proportions. Lower projected capital costs for nuclear in GCAM (roughly half of the cost in other models) lead to nuclear expansion in GCAM in all scenarios whereas the other two models show contraction of nuclear power in the reference scenarios. ReEDS preferentially expands coal and IPM’s generation mix shifts slightly to gas and nuclear. The policy scenarios, across all three models, exhibit reductions in coal generation and expanded generation from nuclear and renewables (see Online Resource 6 for the marginal changes in the supply mix).

Fig. 5
figure 5

Electricity generation by technology and scenario in 2015 (control) and 2050 (control, REF CS3, POL3.7) with percent change from control. Percentages show change in generation vs. control scenario

Meeting higher demand or altering the mix to reduce emissions imposes additional costs on the power system. Figure 6 compares the change in system costs (cumulative from 2015 to 2050, discounted at 3 %) to the control case for each model. From the control scenario to the REF CS3 scenario system costs—comprised primarily of capital, operations and maintenance, and fuel (see Online Resource Figure 6)—rise by 1.7 to 8.3 % across the models. The POL3.7 scenario shows a similar range of cost increases of 2.3 to 10.1 %. The similar magnitudes of system cost changes highlight the importance of reflecting the effects of temperature change in electricity sector models when evaluating climate policies. Ignoring the increase in system costs attributable to rising temperatures in a reference scenario artificially inflates the relative system costs of scenarios with climate policies. For example, system costs for the POL3.7 scenario in IPM rise by nearly 10 % relative to the control, yet by only 1.2 % relative to the REF CS3 scenario. Note that electric power system costs are an imperfect proxy for the total economic costs of a policy. Yet such costs capture the necessary changes in investment behavior to meet demand and/or emission targets in a sector that is frequently shown to be responsible for a substantial portion of emission reductions.

Fig. 6
figure 6

Percent change in cumulative discounted system costs (2015–2050) vs. Control

Effects on emissions and prices

Incorporating the effect of rising temperatures in the REF CS3 scenario raises CO2 emissions slightly above the control scenario in 2050 (i.e., GCAM 1.5 %, ReEDS 4.9 %, IPM 5.4 %; see Online Resource Figure 7). Because the temperature effect on demand becomes more prominent in the later years, the rise in cumulative emissions from 2015 to 2050 is much less (i.e., GCAM 0.8 %, ReEDS 1.2 %, IPM 2.7 %).

In the policy scenarios, the models met the cumulative emission targets to within a couple of percentage points and exhibit similar emission reduction paths. IPM has slightly greater reductions in the first 15 years in the POL3.7 scenario than GCAM or ReEDS because it is forward-looking with constraints on the adoption rate of nuclear and CCS technologies. Nevertheless, by 2050 the emission reductions across all three models fall into a tight band from 54 to 56 %.

The CO2 emissions prices needed to achieve the emission reductions in the policy cases are shown in Online Resource Figure 8. In the POL3.7 scenario, emissions prices start at between $7 and $13 per ton CO2 in 2015 and reach $60 to $78 per ton CO2 by 2050. GCAM’s price path rises by a constant 5 % per year as explained in the methods section. Although IPM optimizes across time, the model shows an accelerated increase in price between 2030 and 2040 due to adoption constraints placed on nuclear and CCS capacity.

The effect of rising temperatures and climate policy on power prices is shown in Online Resource Figure 9. The temperature effect in the reference scenarios raises electricity prices in 2050 by less than 5 %. This small price change indicates that the marginal cost of producing additional power is relatively low. In the POL3.7 scenario, prices rise by 24 % in GCAM (wholesale), 17 % in IPM (retail), and 11 % in ReEDS (retail). Such price changes are within the range of projections price projections typically seen for a roughly 50 % reduction in emissions.


This study examines the effect of projected changes in air temperature due to both climate change and mitigation policy on electricity demand and supply in the contiguous United States to mid-century. A valuable and novel methodological approach of this study is the application of a common set of temperature projections and policies to three well-established models of the U.S. power sector— GCAM-USA, ReEDS, and IPM. The multi-model, multi-scenario approach aims to enhance the robustness of the results.

In a reference scenario (REF CS3) with global mean temperatures rising by 1.7 °C from 2005 to 2050, U.S. electricity demand in 2050 is 1.6 to 6.5 % higher than a control scenario with constant temperatures. In conjunction with rising electricity demand, power sector CO2 emissions also increase by a similar amount (1.5 to 5.4 %). This range of demand changes from rising temperatures is largely consistent with other research. However, the regional patterns of demand changes were not consistent across the three models. Because the models used the same temperature data, differences in the demand response of the models and variation in regional demand patterns may be attributed to the translation of temperature change into electricity demand change. This analysis identifies the underlying electricity demand data used in calibration as the primary source of variation across models.

A comparison of the control and reference scenarios with stylized emission reduction policy scenarios shows the importance of adequately reflecting projected temperature changes in policy analyses of the power sector. In the absence of global action to mitigate emissions, rising air temperatures increase the system costs (cumulative 2015–2050, discounted by 3 %) of electricity generation by 1.7 to 8.3 %. In assessing the costs and benefits of climate policies, ignoring the effects of rising temperatures in the reference scenario artificially inflates the costs of mitigation actions.

As indicated at the outset, this study focuses on a single aspect of climate change, namely changes in average ambient air temperature. Additional research is needed to fully characterize the impacts of climate change on the electric power sector. The temporal aggregation of the electricity supply models used in this study, which range from annual to 16 time slices, is too coarse to assess the impact of extreme temperature events that occur on only the very hottest days of the year. Refinements of the current models or the use of hourly dispatch models could address this issue. On the supply-side, on-going research seeks to extend the present analysis by incorporating the effects of changes in temperature and precipitation on hydropower supply and cooling water availability. Assessing the impact of climate change on the power sector using multiple climate models would improve our understanding of the robustness of the results.