1 Introduction

Climate information users (practitioners) seek out downscaled climate model projections (data) because downscaled data are available at spatial scales that are most suitable for local planning. In the past, the “practitioner’s dilemma” was a lack of downscaled products to choose from, but now, it refers to the challenge of knowing how to choose from a large supply of data in decision making (Barsugli et al. 2013). Cash et al. (2002) describe three characteristics of scientific information that are necessary for it to be usable in decision making, and they are salience (relevance), credibility (plausible and adequate), and legitimacy (institutionally unbiased). Of these three traits, we focus on the scientific credibility of downscaled climate model projections.

Practitioners want to have confidence in the climate projections they use, and credibility in climate model-based information is garnered through effective communication of its scientific uncertainty (Stainforth et al. 2007). Greater confidence may ultimately lead to the decision not to use a particular data set or to proceed with greater caution, but even that kind of confidence is important for explaining and justifying choices.

Uncertainty in climate model projections is commonly partitioned into three main categories including model uncertainty (different outcomes for the same climate processes represented by different models), scenario uncertainty (uncertainty related to future radiative forcing), and internal variability (random, internal fluctuations of the climate) (Deser et al. 2012; Giorgi et al. 2009; Hawkins and Sutton 2011; Latif 2011; Yip et al. 2011). These characterizations are necessary for providing a quantitatively based description of the uncertainty space for the model outputs (i.e., temperature or precipitation), but they do not provide explanatory information about how models arrived at those outputs and whether or not that affects the decision being made. The ways that scientists communicate uncertainty are often very different from what practitioners need to make decisions (Pidgeon and Fischhoff 2011).

Here, we pose that descriptions of climate projection uncertainty should include an assessment of the ability of the underlying climate model to represent weather-scale processes because many climate change problems require credible representations of local weather for projections to be usable (Rood 2007; Rood and Team 2014). For example, realistic representations of precipitation processes (i.e., convective precipitation) are important to practitioners who use climate projections to drive hydrologic models (Fowler et al. 2007; Hwang et al. 2011). With regard to the types of uncertainty (model uncertainty, scenario uncertainty, and internal variability), our focus is not easily categorized. Our focus is not on, for example, known deficiencies of model components as much as it is on the ability to organize the physics represented in the model components into realistic meteorological features. If a feature is present—for example, the precipitation due to mid-latitude cyclones—then information about the future can be related to changes in the behavior of that feature. If a feature is not present—for example, convection organized on scales of tens of kilometers—then it is difficult to anchor the uncertainty description between present and future climate on realistic mechanisms. This type of uncertainty is a process-based analysis of model quality, which contributes to understanding the model’s ability to represent internal variability.

This paper demonstrates how to assess the quality of statistically downscaled precipitation projections, and we use the Great Lakes region as our study domain (38–50°N and 74–94°W). Our motivation to study projections for the Great Lakes region is based on known deficiencies in the models for their representation of the lakes (Winkler et al. 2012), which has important consequences in many applications of the data. Our contribution to the description of uncertainty focuses on the ability of the models to correctly simulate dynamical processes, which enhances climate projection credibility (Barsugli et al. 2013) and confidence (Stainforth et al. 2007).

The paper is outlined as follows: In the Background section, we describe the important meteorological processes in the Great Lakes region. We also describe the sources of data used in the evaluation. Next, we present our methodology for assessing the quality of the projections and our results for how well precipitation processes are represented in the Great Lakes region. We then discuss the implications of our results for the downscaled projections and present three regionally based cases where the assumptions of bias correction are not valid in the Great Lakes region. Lastly, we summarize major process-based and downscaling uncertainties in the models, and we make recommendations for alternative downscaled products that offer improvements over statistically downscaled data.

2 Background

2.1 Observed meteorological processes

Non-lake effect (i.e., large-scale and convective) precipitation in the Great Lakes region is seasonally controlled by different mechanisms. We define the warm (May–September) and cool (October–April) seasons based on different precipitation processes that are generally dominant during those times. Our definition of the warm season is fairly consistent with others, plus or minus a month on either side (Fritsch et al. 1986; Schubert et al. 1998). Between seasons, precipitation processes transition from predominantly one type to the other. Mid-latitude or extratropical cyclones (1000 km scale) are a major mechanism for precipitation in the Great Lakes (Eichler and Higgins 2006). The most active period of extratropical cyclones is January through March with storms originating from “Alberta clippers” or from cyclogenesis in the lee of the Rocky Mountains (Eichler and Higgins 2006). The transition from the cool to warm season is accompanied by a northward shift of extratropical storms (Eichler and Higgins 2006). In the warm season, there is an increase in the amount of moisture entering the continent from the Gulf of Mexico, which is strongly linked to precipitation (Schubert et al. 1998). During the warm season, convective precipitation is the dominant form (Changnon 2001; Dai 2001), which is fueled by northwest-flow weather outbreaks (Johns 1984) and mesoscale convective systems (Ashley et al. 2003; Changnon 2001; Fritsch et al. 1986; Wang et al. 2011). The convective precipitation is not randomly distributed; rather, it is organized and, during any one summer, strongly influenced by the particular pattern of the quasi-stationary highs and lows (Jirak and Cotton 2007). Though the origin of these non-lake effect precipitation events is independent of the lakes, the lakes modify the responsible synoptic systems.

The Great Lakes define details of the distribution and types of precipitation in the Great Lakes region. During the cool season (warm season), the lakes modify regional atmospheric circulation and cause a decrease (increase) in surface pressure (Notaro et al. 2013a). Lake-induced precipitation is greatest during the autumn transition and cool season (Scott and Huff 1996) due to the relatively warm surface waters that supply heat and moisture to the atmosphere. Lake-effect snowfall is most intense during December and January and declines with the formation of lake-ice cover (Notaro et al. 2013b). The lakes’ relatively cool surface waters during the warm season suppress convective clouds and reduce rainfall directly over them (Changnon and Jones 1972; Scott and Huff 1996). Observations of over-lake precipitation have historically been derived from land-based gauge measurements (an exception is the Blust and DeCooke (1960) study), so modeling studies have been used to simulate over-lake precipitation (Holman et al. 2012). The simulations and observations agree that the ratio of over-lake precipitation to over-land precipitation is relatively low (high) during the warm season (cool season), but a regional model suggests more extreme ratios (Holman et al. 2012). Warm-season temperature gradients between air masses over the lake and land cause lake breezes that are known to generate shoreline low-level convergence, which increases thunderstorm activity over the land adjacent to the lakes (Changnon and Jones 1972).

2.2 Sources of information for assessing modeled precipitation processes

We use a set of products that, at the time this study was conducted, were from the latest release of the widely accepted World Climate Research Program’s (WCRP) Coupled Model Intercomparison Project phase 3 (CMIP3) models (Meehl et al. 2007). The downscaled version we assess, the Bias Corrected and Spatially Downscaled (BCSD) climate projections (Maurer et al. 2007), referenced in our study as the BCSD or downscaled data, was provided to the user community as potentially useful in a wide range of applications. The BCSD methodology consists of (1) a bias correction procedure that uses quantile mapping to correct model biases based on an observed 1950–1999 climatology and (2) a spatial downscaling interpolation of the monthly bias-corrected models based on a high-resolution (1/8°) observed data set (Maurer et al. 2007; Maurer et al. 2002).

The BCSD climate projections are based on 16 of the CMIP3 global climate models (GCMs) (Table 1). The GCMs are composed of multiple coupled sub-models that simulate the atmosphere, ocean, sea ice, and land surface processes. The spatial resolution of the GCMs range from over one to at most five degrees latitude/longitude, where one degree latitude is equivalent to roughly 100 km. Models of this resolution have no ability to represent precipitation associated with lake-scale circulations (Gula and Peltier 2012), such as lake breezes that penetrate inland a few ten to up to 100 km (Sills et al. 2011). Since lake-effect processes are not resolved in the models, the best the GCMs may simulate are regional-scale effects of the lakes.

Table 1 Descriptions of how each CMIP3 model included in our analysis represents the Great Lakes
Full size table

Another major limiting factor of the GCMs is the way in which they represent the Great Lakes. Table 1 summarizes each model’s spatial resolution and treatment of lakes to give an idea of the scale at which the lakes, if they exist in the model, are resolved. Fig. 1 shows maps of the land area fraction defined by each model. Blue shading indicates land is present, which is how the Great Lakes are commonly represented among the models. When the Great Lakes are covered by land as opposed to filled with water, their heat capacity and climatological “memory” is greatly reduced. Implications of the lakes as land include an underrepresentation of the moisture flux available from the surface to the atmosphere, a diminished role in modifying local air temperatures and precipitation, and reduced lag times between the seasonal cycle of the land surface and overlying atmospheric temperature. An improvement over wet soil is the recognition of a water “land” type that is used to adjust the calculation for surface fluxes of momentum, heat, and moisture (Masson et al. 2003). The most complete treatment of the Great Lakes among the CMIP3 models is the inclusion of a simple lake model in the ECHAM5 GCM (Roeckner et al. 2003). However, assumptions like constant-depth mixed layers and lake-ice fractions of only 0 or 100 % greatly limit the lake model from accurately representing the observed dynamics of the lakes and therefore their modification of precipitation processes.

Fig. 1
figure 1

Land area fraction maps for 13 of the 16 CMIP3 models based on available land-sea masks from the Intergovernmental Panel on Climate Change (IPCC) Data Distribution Centre. Red (blue) shading indicates water (land) is represented. A box is drawn around the Great Lakes region on the top center map to show our study area. Most of the models represent the Great Lakes as land (blue shading) types

Full size image

Precipitation processes in the models are studied using the CMIP3 climate of the 20th Century experiment (20C3M) output of total and convective precipitation. For our purposes, large-scale precipitation is derived from total precipitation minus convective precipitation. For our discussion, large-scale precipitation is directly related to mid-latitude cyclones (cool season), and convective precipitation is associated with the smaller scale, warm-season precipitation. The two types of precipitation are not mutually exclusive. For example, precipitation associated with mid-latitude cyclones and their associated fronts in spring might be of the convective type.

Model performance and evaluation descriptions are ultimately anchored on observations. We rely on the 1/8° Maurer et al. (2002) gridded precipitation observations that are used in the bias correction and downscaling of the BCSD projections. There are known errors and limitations (i.e., no observations over the lakes) in the Maurer et al. observations, but they must be used in our assessment to maintain consistency between our description of model bias and the biases that were corrected for in the BCSD. It is well known that observational data sets can differ from one another (Adler et al. 2001), but the data from Maurer et al. (2002) rely on the same NOAA Cooperative Observer station data as other commonly used observational products (i.e., NOAA Climate Prediction Center (CPC) US Unified Precipitation (Higgins et al. 1996)), so we are confident that our assessment of model biases is robust against major discrepancies.

3 Methodology

Our assessment of the BCSD projections starts with the GCMs. We rely on the GCMs for all physically based information, because statistical downscaling does not improve the representation of processes at higher spatial resolutions (further discussion of limitations from statistical downscaling are presented in the discussion section). We take the same approach as Brekke et al. (2009) and use monthly exceedance probability (EP) curves for the period 1950–1999 to assess total precipitation bias for the Great Lakes region (region outlined in Fig. 1). The collection of models are compared to the Maurer et al. (2002) observations of total precipitation. We can only evaluate the models’ total precipitation bias because there are no corresponding observational measurements of convective or large-scale precipitation to compare to the models. Reanalysis products (models forced with observations) provide continuous fields for historical convective and large-scale precipitation, but in our experience, there are strong differences among products for the Great Lakes region, which make it difficult to establish a baseline of “truth” for comparison.

We agree with Schaller et al. (2011) that bias alone is not a sufficient measure for evaluating models, and process information gives further insight into the quality of the model projections. Convective and large-scale precipitation simulations are available in the 20C3M model runs. We know that the GCMs do not simulate small-scale precipitation features such as lake-effect precipitation, so we look at the larger picture of precipitation across the region. For each model, we plot a climatology of its partitioning (percentage) of precipitation coming from convective and large-scale processes during each month. From this, we can gain a better understanding of which types of precipitation processes are dominant in the models at different times and if/when a transition between types occurs. We consider the set of all models—the ensemble—in our description of processes. Any similarity between models does not imply an accurate representation; however, discrepancy between models emphasizes a greater degree of uncertainty in the physical process information.

4 Results

Figure 2 shows the EP curves of monthly precipitation from the GCMs and from the Maurer et al. (2002) observations. Model bias is measured by comparing individual models to the Maurer et al. observations. The GCMs simulate precipitation amounts closest to the Maurer et al. observations during October through April (cool season). This is evident by the models’ relatively tight and even distribution of EP curves around the Maurer et al. observations which is consistent with previous findings (Deangelis et al. 2013). Model bias during the cool season increases for higher precipitation amounts, which means the GCMs have more difficulty capturing years with high precipitation. During the warm season, the models are biased more equally for dry and wet years.

Fig. 2
figure 2

Monthly distributions of total precipitation (mm/day) from 1950 to 1999 in the GCMs (colored lines) and Maurer et al. (2002) observations (thick black line) for the Great Lakes region. The ensemble means bias (average of difference between modeled and observed), and variance is printed in the upper right of each monthly plot. Bias and variance are reported for the median values of simulated precipitation (probability of exceedance = 0.5)

Full size image

The greatest individual model biases and inter-model variability occur during the warm season compared to the cool season. The ensemble mean bias and variability are reported in Fig. 2 for the median values in the precipitation distributions. The month of July is an example when the ensemble has a relatively small bias but individual models show strong positive and negative bias.

During seasonal transition months, the ensemble has consistently strong bias. The spring transition month of May is characterized by a strong positive ensemble mean bias, which indicates more precipitation is simulated than observed. The autumn transition month of September is characterized by a strong negative ensemble mean bias, which indicates less precipitation is simulated than observed.

The spring and autumn transitions coincide with the times of greatest disagreement among the models’ partitioning of convective and large-scale-based precipitation (Fig. 3). Model differences in the amount of convective precipitation are as great as 60 % from April through October, with the greatest disagreement occurring in May and September. The disagreement is due to an inconsistent (1) timing of the warm-season transition among the models and (2) magnitude of precipitation coming from convective sources. There is at least one model in every warm-season month that produces more large-scale precipitation than convective precipitation, even though the literature supports 50–80 % in nature come from thunderstorm (convective) activity (Changnon 2001; Dai 2001).

Fig. 3
figure 3

Monthly partitioning (%) of mean large-scale and convective precipitation in the GCMs from 1950 to 1999 in the Great Lakes region

Full size image

Our findings are consistent with an earlier study that showed substantial evidence that models, even at considerably higher resolution, do not represent precipitation in the eastern USA with mechanisms consistent with observations (Lee et al. 2007). In particular, the autumn dry bias in the models has been attributed to weaker southerly flow from the Gulf of Mexico into the USA, thereby decreasing the available moisture supply (Kunkel and Liang 2005). Deficiencies in the representation of convective precipitation are also large for the Great Lakes region since the models do not accurately represent the lakes or their atmospheric feedbacks.

5 Discussion

Our results show monthly total precipitation bias for sixteen CMIP3 models and their monthly partitioning of convective and large-scale precipitation in the Great Lakes region. Model total precipitation bias is weak during the cool season (April–October), strong during the warm season (May–September), and at a maximum during the seasonal transition month of May (September) when almost all models are too wet (dry) (Fig. 2).

We investigated the representation of convective and large-scale precipitation in the models to add information about the quality of the projections (Fig. 3). Precipitation during the cool season comes primarily from large-scale processes in all of the models, which is consistent with theory, thereby producing a stronger argument for high quality cool-season projections at the regional scale. At smaller spatial scales, where lake-effect precipitation becomes important, model quality is lacking. Warm-season precipitation comes primarily from convective processes in theory, but the models are highly variable in how much convective precipitation is present. Convective precipitation occurs on relatively small spatial scales, which partly explains why the coarse-resolution GCMs inconsistently represent warm-season precipitation. The seasonal transitions (May and September) have the greatest variability in how models partition convective and large-scale precipitation, which is also when model bias is greatest for total precipitation.

In addition to the model deficiencies already discussed, the bias-correction and statistical downscaling method of the BCSD also affects the quality of the projection information. The spatial detail that is gained from downscaling is a product of the historical observed climate, which can add value to the projections if there is no reason to believe that those spatial patterns will change over time. However, in the Great Lakes region, we already have examples where the patterns of the past are no longer valid for the future:

5.1 Example 1: Seasonal Transition

A comparison of the model representation of precipitation processes during the cool and warm season shows that the quality of the simulation is better for the cool season than for the warm season. We note the transition seasons are highly uncertain, since they are described by a transition from one better-represented form of precipitation (large-scale) to another poorer-represented form of precipitation (convective). We know, already, that the spring transition is occurring earlier and that the trend is expected to continue (Winkler et al. 2012). BCSD corrections in April, for example, are based on a month where model precipitation is primarily large-scale. With the earlier seasonal transition, April is likely to have more convective precipitation; hence, the April correction is no longer appropriate. There is uncertainty related to non-stationarity as well as uncertainty due to changes in the appropriate modification of the downscaling technique. The cool-season downscaling correction might be argued as an incremental correction to a resolved process. The warm-season correction accommodates greater deficiencies of simulation quality.

5.2 Example 2: Wintertime Lake-Effect Snow

An obvious challenge in the Great Lakes region follows from the poor representation of the Great Lakes in the CMIP3 models, namely, the regional and local modifications of weather caused by the lakes. Take, for example, lake-effect snow on the downwind shore of Lake Superior. Of all the variables affected by assumptions of stationarity, atmospheric general circulation and humidity attributes are likely of primary importance, but a signal of change in any variable not accounted for in the downscaling makes the downscaling less appropriate (Hewitson and Crane 2006). The amount of lake-effect precipitation is strongly influenced by the amount of ice cover on the lake (Brown and Duguay 2010; Wright et al. 2012), and lake-ice is not simulated by the GCMs. In the past, the lake has been frozen, but reductions in ice cover of more than 70 % have already been documented (Wang et al. 2012). Therefore, the source of water for the underlying weather systems is changed and this has been accompanied by large increases in lake-effect snow on the upper peninsula of Michigan (Andresen et al. 2012; Burnett et al. 2003; Ellis and Johnson 2004; Kunkel et al. 2009). Therefore, the bias correction based on past climate is not the same as present nor that which is required for the future.

5.3 Example 3: Summertime Lake-Effect Precipitation

In the summertime, the relatively cold surface waters of the Great Lakes induce higher sea level pressures across the basin (Notaro et al. 2013a). Higher sea level pressure, particularly over the lakes, creates more stable atmospheric conditions leading to decreased over-lake convective activity (Changnon and Jones 1972; Notaro et al. 2013a; Scott and Huff 1996). For Lake Superior, summer water temperatures have warmed more quickly than regional air temperatures (Austin and Colman 2007), thereby weakening the summertime air-lake temperature gradient. Even though Lake Superior’s summertime stabilizing affect has been weakened (and one might have expected the result of increased precipitation), summer precipitation downwind of Lake Superior has declined (Higgins and Kousky 2013). More specifically, records at Ironwood, MI and Marquette, MI—two near-shore weather stations—have declined −12.2 and −6.8 %, respectively, compared to the previous 30 years (GLISA 2013; GLISA 2015). Additionally, decreased precipitation is primarily responsible for increased drought in this region over the last 30 years (Ficklin et al. 2015). One possible explanation for the decrease in precipitation is a decrease in summertime mid-latitude cyclone frequency (Leibensperger et al. 2008). Regardless of the mechanisms responsible for changing lake surface temperatures, atmospheric stability, and mid-latitude storm tracks, the bias-correction and statistical downscaling procedure will not account for these physical changes to project them into the future.

One of the major issues that limits our confidence in the BCSD climate projections is that their downscaling assumes there is an unchanging relationship (stationarity) between the prevailing weather patterns and associated meteorological properties (Wilby 1997). We have documented here situations where the model deficiencies are so large as to leave important weather-scale processes as unrepresented. We have described non-stationarity in a physical process, lake ice. We have also described non-stationarity where the larger warm-season model deficiencies replace smaller cool-season deficiencies—i.e., the seasonal transition and the representation of precipitation processes. This informs the uncertainty discussion with knowledge that the prevailing weather patterns and their representation in the models is changing; hence, an underlying assumption is violated.

6 Conclusion

We provide an assessment of the Maurer et al. (2007) BCSD CMIP3 precipitation projections for the Great Lakes region. A more recent analysis of CMIP5 models suggests no significant improvement in the mean representation of North American precipitation (Sheffield et al. 2013), so the use of CMIP5 does not substantially change our conclusions. We base our assessment on the ability of the underlying climate model to represent regionally important weather-scale processes. This approach is intended to communicate local process-based information practitioners find usable in their planning.

In our analysis, we showed that the models have usable process scale (i.e., primarily large-scale precipitation) information during the cool season. This finding is supported by relatively small ensemble mean precipitation bias and variability (Fig. 2) coupled with the consistent representation of processes in the models (Fig. 3) from roughly November through March. Therefore, when the GCM projections were downscaled, usable process-scale information was incorporated at the finer spatial scale. Still, however, the downscaling does not take into account systematic changes to the lakes, which modify convective precipitation such as lake-effect snow, as discussed in Example 2 above.

Processes information is not as reliable during the warm-season and seasonal transition. Although the ensemble mean bias is relatively small during June, July, and August, there is great model variability for total (Fig. 2) and convective precipitation (Fig. 3). Model bias is greatest during May and September, coinciding with times when some models have already transitioned to the next precipitation regime while others lag behind (Fig. 3). Comparable observations of convective precipitation do not exist to evaluate the models against, so we conclude greater model variability is an indicator for greater uncertainty in the model information at those times. When the downscaling was applied to projections for the warm-season and seasonal transition, it was founded on uncertain processes, so its information is highly uncertain. This should be a flag of high uncertainty for practitioners.

In the future, we can say that there are at least three cases (Examples 1–3 above) where processes are expected to continue changing, so there is even greater uncertainty in the BCSD projections. Mainly, the underlying weather-scale processes in past and future climates will be different, so the amount of correction needed for future projections is different than for historical projections. From a statistical point of view, the weather is non-stationary. This is accompanied by a different form of non-stationarity in the climate model, namely, changes in the error characteristics of the underlying weather. In order to accommodate such changes, a statistical scheme would have to be weather sensitive or, minimally, scale sensitive.

In order to reduce uncertainty related to weather and climate processes, practitioners should consider dynamically downscaled climate projections from regional climate models (RCMs). Dynamically downscaled climate projections offer potential improvements over statistically downscaled projections if processes are better simulated (Harding et al. 2013). In the western USA, RCM simulations have not shown large improvement over the BCSD method (Wood et al. 2004), but in the Great Lakes region, the dynamic interaction of the lakes with local and regional climate requires higher-resolution modeling of the system (Gula and Peltier 2012; Notaro et al. 2013b; Wright et al. 2012). Recent modeling studies where lake models have been coupled to RCMs are advancing our understanding of complex precipitation processes in the Great Lakes region (Notaro et al. 2013a; Notaro et al. 2013b; Vavrus et al. 2013). Better representation of processes in the models allows projected climate changes to be physically explained, and practitioners can use that information to systematically think about climate impacts (Rood and Team 2014).