1 Introduction

Future warming resulting from continued increase in greenhouse gas concentrations is foreseen to be stronger in Europe than the global average (IPCC 2018). This is particularly true for northern Europe in winter and in southern Europe in summer. Climate models project changes in precipitation ranging from increases in northern Europe to decreases in the Mediterranean area. The spreads between individual model projections are generally large and for some areas, like central Europe, different projections tend to differ even in the sign of future precipitation changes (Kjellström et al. 2018). Such large spread between climate projections reveals that there is a strong degree of uncertainty in the local and regional response to global warming. This uncertainty relates not only to changes in global and regional forcing conditions, but also to the response of the climate system to these changes and how this is reflected by different climate models. Differences between models, notably related to their approximations of the physical system, lead to multi-model ensembles tending to show larger spread than single-model ensembles (Sørland et al. 2018). The uncertainty also involves a component of internal variability that regionally can be very large (Aalbers et al. 2018). Consequently, ensembles with a large number of members differing only in initial conditions give larger spread compared to smaller ensembles. Different sources of uncertainty dominate in different regions and at different future time periods (Hawkins and Sutton 2009). In some regions differences between model projections are so large that they completely mask any long term trends (Kjellström et al. 2013).

As global climate models (GCMs) generally have too coarse horizontal resolution to adequately represent regional features of orography and land-sea distribution and to resolve relevant regional processes, further downscaling is often pursued. Consequently, regional climate models (RCMs) mostly operated over limited areas at higher horizontal resolution than their driving GCMs are used (Rummukainen 2016; Giorgi 2019). Both GCMs and RCMs require very large computational resources. In recent years, large collections of downscaled simulations have been produced, in spite of this demand on resources (e.g. von Trentini et al. 2019; Bukovsky et al. 2019). Today, RCM generated climate projections have become an important source of information about details in future consequences of the anthropogenic effects on climate, and such ensembles are now fundamental components of climate services on national (e.g. Kjellström et al. 2016; Skelton et al. 2017) and regional levels (e.g. Jacob et al. 2014).

In the context of utilizing RCM information for climate services it is important to extract as valid and trustworthy information as possible for users. This involves detailed analysis of model output to map model performance in the historic period and spread between projections of future climate conditions. Methods for estimating sources of uncertainty often involve analysis of variance (ANOVA) among the ensemble members (e.g. Yip et al. 2011; Déqué et al. 2007, 2012; Evin et al. 2019). These attempts to map and understand spread between climate projections have been limited, as GCM–RCM matrices most often only constitute sparsely filled matrices. Consequently, various ways of filling non-populated GCM–RCM combinations have been put forward in these studies (e.g. Evin et al. 2019). An exceptional case with a completely filled GCM–RCM matrix is the study by Suzuki-Parker et al. (2018) analysing a 12-member matrix with four RCMs downscaling three different GCMs over East Asia.

Here we describe an analysis of which sources of variation are important for seasonal mean temperature, precipitation, and mean wind speed for a completely filled GCM–RCM matrix of downscaling simulations over Europe. We will neglect the uncertainty related to forcing conditions and concentrate on only one future scenario, the representative concentration pathway (RCP) 8.5 scenario (Moss et al. 2010). We will investigate the various contributions to the uncertainty related to choice of models, through an analysis of a set of transient simulations, where significance estimates also involve the internal inter-annual variability of seasonal average fields. This will be done through a targeted ANOVA for an almost filled combination matrix of 5 GCMs and 4 RCMs, a total of 19 available model combinations, and for a present-day as well as a future 30-year period resulting in a 2 × 5 × 4 period-GCM–RCM matrix.

2 Methods and data

2.1 Model data

In the Euro-CORDEX project (Jacob et al. 2014) a very large community effort has resulted in more than 60 simulations with durations of 120–150 years for various RCP emission scenarios in 12.5 km resolution. As part of the European Copernicus Climate Change Service (C3S), further simulations are added to the ensemble with a plan of reaching more than 120 Euro-CORDEX simulations in total. The strategy behind the pairing of a driving CMIP5 (the fifth phase of the Coupled Model Intercomparison Project, Taylor et al. 2012) GCM simulation with one of the nine RCM models employed by the C3S has been aimed at filling sub-matrices in the 3-dimensional scenario × GCM × RCM space.

We have analysed near-surface air temperature (T), precipitation (pr) and average 10-m wind speed (w10 m) as seasonal means from Euro-CORDEX RCP8.5 simulations in 12.5 km resolution, for the historical period 1981–2010 and for the future period 2071–2100. As simulations with HadGEM2 as the driving GCM do not contain the year 2100, we have instead used 2070–2099 for those. All data analysed can be freely downloaded from the ESGF data network (e.g. esgf-data.dkrz.de) or from the Copernicus Climate Data Store (cds.climate.copernicus.eu/cdsapp#!/dataset/projections-cordex-single-levels).

The five CMIP5 GCMs and four RCMs used are listed in Table 1. Each of the 20 possible combinations exists, with the exception of the MPI-ESM-LR_HIRHAM5 combination. This hole in the 5 × 4 matrix has been filled as described below. Note that one of the REMO simulations (MPI-ESM-LR_REMO2009) was performed with the older REMO2009 version while the others were made with the more recent REMO2015; potential differences in results between the two model versions have been ignored in this study, as the REMO modelling group judged that such differences between the climate of the two versions are probably small.

Table 1 Global and regional models analysed in this study
Full size table

2.2 Analysis of variance

We use a technique involving analysis of variance (ANOVA, e.g. von Storch and Zwiers 2001). The analysis is done separately for each grid point and each season. The ANOVA analyses treat a set of 1200 numbers (2 periods × 5 GCMs × 4 RCMs × 30 years), where we expand the value \(Y_{ijkl}\) from period i, GCM j, RCM k and year l as a sum of an average (M) and anomalies w.r.t. to i, j, k and l respectively, following

$$Y_{ijkl} = M + S_{i} + G_{j} + R_{k} + SG_{ij} + SR_{ik} + GR_{jk} + SGR_{ijk} + Z_{ijkl}$$
(1)

where all sums over explicit indices are constructed to be zero (Table 2). Correspondingly, we can split the total sum of squares of deviation from the grand ensemble mean into

$$\mathop \sum \limits_{ijkl}^{{}} (Y_{ijkl} - M)^{2} = \mathop \sum \limits_{ijkl}^{{}} \left( {S_{i}^{2} + G_{j}^{2} + R_{k}^{2} + SG_{ij}^{2} + SR_{ik}^{2} + GR_{jk}^{2} + SGR_{ijk}^{2} + Z_{ijkl}^{2} } \right)$$
(2)

where the Z term is the inter-annual variability for individual simulations summed for all experiments.

Table 2 Formulae for the various ANOVA terms, under the requirement that all terms sum to zero over any single index. Dots indicate mean over indices
Full size table

The single missing element (j′, k′) = (4, 1) in the matrix for both time periods has been filled as in Déqué et al. (2012) by setting the combination-specific terms to zero: \(GR_{j^{\prime}k^{\prime}} = 0;\, SGR_{ij^{\prime}k^{\prime}} = 0\) for both periods i = 1, 2, and letting the interannual variability correspond to the average variability of all existing experiments. The resulting redundant system of 3 equations with 2 unknowns for the missing GR and SGR terms is easily solvable, leading to surrogate values of the single-simulation period averages \(Y_{{ij^{\prime } k^{\prime } }}\) for both values of i. Formulae for the various terms are listed in Table 2.

The quantity M is the average over the entire ensemble, including both the historical and the future period. S is the mean deviation of each scenario from this mean, for historical and future periods separately; in other words, the average climate change is S2 − S1 = 2*S2. The G term describes the effect of each individual GCM, averaged over periods and RCMs. Correspondingly, R is the average effect of each individual RCM. The two-index combination terms are the GCM and RCM influence on climate change (SG and SR, respectively); GR is the deviation of an individual GCM–RCM combination from the expectation of mean climate from linearity, averaged over periods; similarly, SGR is the corresponding term for climate change.

3 Results and discussion

3.1 Comparison of model results

In Fig. 1 we depict the winter and summer mean temperatures in the historical climate and corresponding mean warming for the 19 simulations entering this analysis. Also, we show the standard deviation between these model results. The winter climate shows a pronounced north–south gradient with the highest temperatures over ocean areas in the south and the lowest temperatures over continental areas in the north. Also the climate change signal shows a similar north–south gradient, but now with the strongest temperature increase generally in the north and the smallest increases over the North Atlantic. While there is a general agreement in the geographical pattern across Europe and the North Atlantic, there are also considerable inter-model differences, as has been seen in several previous studies of RCM projections for the region (e.g. Déqué et al. 2007; Kjellström et al. 2013, 2018). The figure reveals the largest differences between models, expressed by the standard deviation, over oceanic areas in the far north, likely as a result of strong sensitivity to sea-ice conditions. There are also relatively large differences between models over large areas in northern Europe and in high-altitude areas in the south such as the Alps, the Pyrenees and areas in Turkey and parts of the Balkans. This is potentially related to differences in the simulation of snow cover and/or its interaction with the atmosphere in different models. There is also an area of large inter-model difference over the north Atlantic close to the western domain boundary. For other areas the standard deviation is relatively low both in the historical climate and in the climate change signal.

Fig. 1
figure 1

Mean temperatures (a, b, e, f; °C) for the present day, and the change between periods (c, d, g, h); left column (a, c, e, g): ensemble average, right column (b, d, f, h): ensemble standard deviation of 30-year means. a–d Winter. e–h Summer

Full size image

A pronounced north–south temperature gradient is seen for the historical climate also in summer. For this season, however, the lowest temperatures are generally found over the ocean instead of the continent as in winter. The climate change signal, on the other hand, differs considerably from that in winter. Similarly, there is a maximum in the far north but now there is also a pronounced wide-spread maximum in the Mediterranean area. The smallest temperature increases are confined to an area over the North Atlantic. Again, these results are very similar to what has been shown in previous studies for the region (e.g. Déqué et al. 2007; Kjellström et al. 2013, 2018). The pattern of differences between the models resembles that in winter albeit with smaller differences. Local maxima are found over the North Atlantic, some ocean areas in the north and in some mountainous regions such as the Scandinavian mountains and the Alps. Notably, also the large lakes in Russia stands out as areas with large differences between models likely reflecting different treatment of lakes in the RCMs (e.g. Samuelsson et al. 2010; Pietikäinen et al. 2018). We note that the standard deviations are generally larger for the historical climate than for the climate change signal.

In Fig. 2 winter mean temperature for the historical period is shown for each GCM–RCM combination as a deviation from the grand ensemble historical mean of all 20 GCM–RCM combinations. Also, corresponding figures for the change between periods are shown (Fig. 3). These figures provide an overview and a background to interpretation of the ANOVA results presented below. Complementing figures for summer and winter seasonal mean temperature, precipitation and wind climate averaged over both periods and for the change between the periods are shown for each GCM/RCM combination in Figs S1–S10 in the Supplementary Material.

Fig. 2
figure 2

Deviation of historical DJF temperature mean over both periods from historical grand ensemble mean (K). The missing MPI-ESM-LR-HIRHAM5 simulation has been filled in as described in the text

Full size image
Fig. 3
figure 3

Deviation of DJF temperature change from grand ensemble mean change (K). The missing MPI-ESM-LR-HIRHAM5 simulation has been filled in as described in the text

Full size image

The two figures show clearly that there are very large differences between single simulations. For the historical climate, seasonal mean temperature differences are up to ± 4 K or more in some locations. Similarly, for the climate change signal differences of up to ± 2 K are evident. Similar large differences between single GCM–RCM combinations have frequently been reported on (Christensen and Christensen 2007; Kjellström et al. 2011, 2013, 2018). One detail is obvious, particularly when looking at winter temperature and its change (Figs. 1, 2, 3): the simulations downscaling the CNRM global model are markedly different from each other. Parts of these differences may be related to the fact that while the HIRHAM5 and the RACMO22E simulations have used the correct GCM pressure level data on the boundary, consistent with sea surface chronology, the earlier REMO2015 and RCA4 simulations have used model level boundaries, which have turned out to come from an ensemble member different from the one producing sea surface data. This discrepancy will of course make the GCM–RCM cross terms GR and SGR larger. Also, the HadGEM2-ES_REMO2015 simulation seems to have an anomalous behaviour indicating a very cold north-eastern region for the historical period.

Visual inspection of Figs. 2 and 3 indicates other characteristics: the general deviation from the mean is mostly a consequence of the choice of GCM. This holds even more for the climate change. The role of the RCM is not very clear from a visual inspection, though the north-eastern part of the area shows relatively large differences between the rows for some of the columns on the climate change maps shown in Fig. 3. To clarify these preliminary findings it seems appropriate to apply a quantitative approach for analysing the role of individual models for both mean climate and for climate change. In the following we will present such results based on ANOVA.

3.2 ANOVA analysis

As an example of the ANOVA analyses, Fig. 4 summarizes results for winter temperature corresponding to the fields shown in Figs. 1, 2 and 3. Corresponding map collections for the other cases can be found in the supplementary material, Figs. S13, S16, S17, S20, and S21.

Fig. 4
figure 4

ANOVA linear terms for winter (DJF) temperature and temperature change. Top row: M; second row S1 and S2; third row Gj for the 5 GCMs; fourth row Rk for the 4 RCMs. Fifth row: SG2j sixth row: SR2k. See definitions in Table 2

Full size image

The upper row in Fig. 4 contains the ensemble and period average field. Row two are the (redundant) S terms. Since there are only two periods, and since each term must sum to zero, the two panels are identical with opposite sign. For instance, the second panel shows the difference between end-of-century fields and the total average; this means that ensemble mean climate change between the periods will be twice the field shown. Row three shows Gj, the characteristics of each GCM as averaged over the two periods and all RCMs, again with a zero sum rule. It can be seen that CNRM-CM5 has a very cold climate, in particular in northern Europe with several degrees of deviation from the mean over all GCMs, whereas NorESM1-M has a comparatively warm climate over large parts of the Atlantic, and over most of the European continent apart from Scandinavia where instead the HadGEM2-ES model is the warmest of the five GCMs. Row four correspondingly show Rk, the characteristics of each RCM for the mean climate; HIRHAM5 is generally warmer than average, and RACMO22E is colder. The SW-NE temperature gradient is stronger for REMO2015 and weaker for RCA4 and RACMO22E. Strong regional deviations from the ensemble mean can be seen notably for RACMO22E over land areas in the south-eastern part of the region and by REMO in the north-eastern part, both being more than 2 degrees cooler than average in winter.

The fifth row shows SG2j, the characteristics of each GCM relative to the average climate change (divided by 2, as discussed above); in most areas, HadGEM2-ES shows a considerably larger warming than the other GCMs while NorESM1-M shows a considerably weaker temperature increase in the east. Such differences between GCMs are commonly seen, and in these particular cases we note that: the HadGEM2-ES model has a relatively high climate sensitivity and a strong temperature increase over the mid-latitude continents (Andrews and Ringer 2014), compared to NorESM1-M with lower climate sensitivity and a relatively weak temperature response over the northern mid-latitudes (Iversen et al. 2013). There seems to be relatively little influence from RCMs on climate change for the winter temperature average (SR2k, row six) apart from some apparent differences over parts of Russia and some ocean areas in the far north; these differences may be related to different treatment of sea ice; the signal is not seen in the other three seasons. In the following we will analyse formal statistical significance of each term to substantiate these observations.

In order to summarize the importance of the various ANOVA factors, we first show the partitioning of total variability (Eq. 2) into the various percentages for the example field of winter temperature (Fig. 5). It can be seen that this field is dominated by climate change, that interannual variability is moderate compared to the statistical effects of model choice, that the choice of GCM is very important over the Atlantic ocean, and that the contribution from the RCM choice is smaller than the GCM term and most important over the eastern part of the domain and in some mountain regions. Similar relatively strong influence from RCMs in mountainous regions was found by Suzuki-Parker et al. (2018) in their analysis of precipitation over Southeast Asia including Japan. Corresponding plots of other seasons and fields can be found in the Supplementary Material Figs S11, S14, and S18. Comparing the linear terms SG and SR we obtain an indication about where the RCM effects are comparable to or larger than the GCM effect with respect to climate change. The results indicate that these areas involve regions where there is strong interaction with topography, sea ice and/or snow, or soil moisture.

Fig. 5
figure 5

Total variability partitioned over the various interaction terms for winter temperature

Full size image

Even though a contribution plays a smaller role than others, it can still be statistically significant. We now calculate the formal statistical significance of each term at each grid point, field and season, by comparing the quantities of Eq. (2) divided by the relevant degrees of freedom (1 for S, 4 for G and SG, 3 for R and SR, 12 for GR and SGR) and with the inter-annual variability per degree of freedom (29 degrees of freedom for the 30 values) with the F distribution of the appropriate kind, with a 95% significance threshold (von Storch and Zwiers 2001). In order to generalize to all seasons and fields, the percentage of points in the domain where a term is statistically significant is summarized in Table 3. For the entire area, S, G, and R are all significant. Maps of statistical significance for winter temperature are shown for all terms in Figs. S12, S15, and S19, but we highlight the terms with only partial domain-wide statistical significance in Fig. 6 (GR), Fig. 7 (SG), Fig. 8 (SR), and Fig. 9 (SGR).

Table 3 Percentage of grid points where each term is significant at a 95% level for temperature, precipitation, and wind speed, winter and summer
Full size table
Fig. 6
figure 6

Significance of GCM–RCM cross term GR for winter (left) and summer (right) for temperature (top), precipitation (middle) and mean 10 m wind speed (bottom). Values below 0.05 (green and blue colours) indicate significance at the 95% level, i.e., that the mean climate is not just a sum of an average GCM effect and an average RCM effect

Full size image
Fig. 7
figure 7

Significance of scenario-GCM term SG for winter (left) and summer (right) for precipitation (top) and mean 10 m wind speed (bottom). Values below 0.05 (green and blue colours) indicate significance at the 95% level, i.e., that the GCM choice has a significant influence on climate change. Note that the corresponding temperature plot (not shown) would be all blue with values below 0.0001

Full size image
Fig. 8
figure 8

Significance of scenario-RCM term SR for winter (left) and summer (right) for temperature (top), precipitation (middle) and mean 10 m wind speed (bottom). Values below 0.05 (green and blue colours) indicate significance at the 95% level, i.e., that the RCM choice has a significant influence on climate change

Full size image
Fig. 9
figure 9

Maps of significance for the SGR terms for winter (left) and summer (right) for temperature (top), precipitation (middle) and mean 10 m wind speed (bottom). Values below 0.05 (green and blue colours) indicate significance at the 95% level, i.e., that climate change is significantly different from the sum of average GCM and RCM contributions

Full size image

In general, significance is larger for temperature than for precipitation and wind speed, consistent with the larger interannual variability compared to model differences. The linear terms S, G, and R are almost always significant, though the fraction of points with significant climate change (term S) is smaller for summer precipitation; the reason is that the region close to the zero-line in Europe for summer precipitation does not show significant climate change (see Fig. S15, panel 8).

The specific combination of GCM and RCM (GR term, Fig. 6) is significant for large areas (> 60%) for all variables and seasons indicating that different RCMs put their specific imprint on the climate imposed from the GCMs. This result is in line with findings by Sørland et al. (2018) showing that two of the RCMs assessed also here show systematically reduced biases in seasonal mean temperature and precipitation over Europe. In Fig. 6 we summarize results for the significance of this term for all three variables over winter and summer. In general, the term shows the least significance in the western part of the domain, where the GCM has the largest influence with minor contributions from the RCM. For winter temperature the specific combination is significant also over the north-western part of the ocean as well as over most of the western and central land area. This result could very well be a consequence of the difference between CNRM-CM5-driven simulations. Winter precipitation climate is mostly independent of this term, indicating that a simple addition of GCM and RCM terms would give an error, which is small compared to interannual variability. For summer precipitation, on the other hand, most of the land areas and the Mediterranean Sea show significant contributions from the GR term.

We also note that there are relatively large areas close to the boundaries where the contribution from the GR term is significant. This may indicate differences in the treatment of the lateral boundary conditions in the different RCMs. The wind signal has less noise, but is generally similar to the precipitation signals, both in pattern and magnitude.

The GCM has a significant influence on climate change (SG term, Fig. 7) everywhere for temperature and over most of the area for precipitation and wind speed. The SR and GR terms, i.e., the influence of regional model on climate change and the deviation of mean climate from linearity, are neither entirely significant nor entirely non-significant across the domain.

In Fig. 8 we show the SR term, i.e., the significance of the RCM contribution to climate change for both seasons and all three fields investigated. The influence of the RCM on climate change is generally lower than that of the GCMs (cf. Table 3). It is significant for 40 (summer) to 50 (winter) percent of the area for temperature, somewhat smaller for wind speed and smallest for precipitation. Large parts of the land mass in the east or north-east show significance for temperature, precipitation, and wind during winter. Mountainous areas show significance for both seasons with varying amplitude, particularly large for summer. This may be a reflection of the fact that different RCMs have very different effective roughness lengths as also indicated by the R terms in Figs. 4, S12, S17 and S18. However, we note that mountainous areas also show high GR values (Fig. 6) indicating also a sensitivity to the combination of GCM and RCM. This may indicate the importance of snow related processes since it has been pointed out that a number of the GCM-driven EURO-CORDEX simulations have large biases in snow cover over the Alps (Terzago et al. 2017).

A larger importance of the RCM choice in summer than in winter on the ensemble mean was also found by Déqué et al. (2007) in an analysis covering three GCMs and 10 RCMs, although they were unable to study the interactions between the terms, since the matrix in their study was too sparse. Similarly, Mearns et al. (2013) found a relatively larger influence of the RCMs on the summer climate in their study covering North America. Also Sørland et al (2018) found a considerable imprint of the RCMs on the climate change signal in the underlying GCMs in their study for Europe.

The specific period-GCM–RCM combination (SGR; Fig. 9) is only significant in a very small fraction of the domain. This means that we can generally estimate the climate change of these seasonal mean fields very well from knowledge about the GCM and RCM separately, without having to do the specific experiment. However, for some areas in the north, there are significant contributions from the combination of the GCMs and the RCMs. One possible explanation for this is the combination of different GCM sea ice in northern winter combined with different RCM treatment of sea ice interpolation and of air temperature and roughness over sea ice. These areas were also significant for the mean-climate GR terms, but were parts of larger areas of GR significance.

The situation for wind is remarkably similar to the situation for precipitation with some exceptions. Inspecting the F value and the individual maps (not shown) reveals, however, that models have an even stronger influence on the quantity, compared to interannual variability. The GCM has its largest influence over sea, possibly due to differences in large-scale weather patterns, whereas the RCM plays a large role over land, probably due to differences in roughness parameterization. We also note that all RCMs assessed here take information about sea-surface temperature and sea-ice directly from the driving GCMs which introduces a strong imprint of the GCMs on the RCM simulated climate. Coupled atmosphere–ocean RCMs would be expected to show a stronger influence of the RCMs also over the ocean.

4 Summary and conclusions

The presence of a filled GCM–RCM matrix with only one missing combination has enabled the present ANOVA analysis of seasonal mean fields. With much better robustness than earlier studies we can examine the importance of the global and the regional component of the model system and the roles of individual GCMs and RCMs for mean climate as well as climate change. A comparison with interannual variability from the participating models enables a formal determination of statistical significance of the various linear terms and cross terms in the ANOVA analysis. We note that two of the simulations considered have known errors in the setup of boundary conditions; this may have had an influence on the results, probably increasing the GCM–RCM cross term variability in the ANOVA analysis.

For all variables and seasons addressed here, the linear contributions from both GCMs and RCMs have significant influence on the resulting matrix of results for almost all points and seasons, particularly for temperature. Further, by looking at mean climate conditions over both historical and future periods, we observe that there are clear geographical patterns of maximum GCM influence and maximum RCM influence, respectively. However, for most of the fields and seasons considered, the major part of the area has a significant influence of the GCM–RCM cross term, i.e., the specific influence of one simulation, for the mean fields considered in the present study.

The choice of GCM generally has a larger influence on climate change than the choice of RCM, showing significance for twice the area. While there is very little systematics in the geographical distribution of GCM influence on climate change, the RCM influence is generally concentrated close to the eastern and northern boundaries and in mountainous areas. Areas where the influence of the RCMs on the climate change signal is comparable to that of the GCMs are concentrated where there is strong interaction with topography, sea ice, snow, or soil moisture.

One main conclusion, which obviously only applies to the seasonal average fields examined presently, is that climate change for a particular GCM–RCM combination with very few exceptions can be determined by adding effects from the GCM and from the RCM separately; in other words, very little information is added to this particular aspect of climate change from adding a simulation, provided that the GCM and RCM effects have already been established. This conclusion points to the possibility of emulating missing simulations in partly filled matrices where the linear effects have been established and this way obtain distributions where the GCMs and RCMs have separate statistical weights. Such a procedure could yield completely filled GCM–RCM matrices. Conversely, it is likely that too sparsely filled GCM–RCM matrices that do not allow for appropriately establishing the individual GCM or RCM effects cannot easily be used for establishing the full GCM–RCM response by emulation of missing simulations. Consequently, conclusions about uncertainty inferred from analysing such sparse matrices are likely to be biased. These considerations may have important implications for existing climate services depending on sparsely filled GCM–RCM matrices. The completely filled GCM–RCM matrix presented and analyzed here are an ideal test bed for addressing such considerations. For example, gradual thinning of the matrix can be applied to test to which degree matrix sparseness influences messages about robustness and uncertainty related to climate change.

Finally we note that, while the current study as a first step focuses on seasonal mean characteristics for three variables and the climate change signal for each one of them, the ANOVA method could readily be applied also for other features. This includes: other variables and other time frequencies, including also extreme events; other forcing scenarios; and larger ensembles, sampling also internal natural variability. In addition, also model performance could be considered in such an analysis by including a term for biases in the ANOVA analysis.