Abstract
Selecting a reliable global climate model as the driving forcing in simulations with dynamic downscaling is critical for obtaining a reliable regional ocean climate. With respect to their accuracy in providing physical quantities and long-term trends, we quantify the performances of 17 models from the Coupled Model Inter-comparison Project Phase 6 (CMIP6) over the North Pacific (NP) and Northwest Pacific (NWP) oceans for 1979–2014. Based on normalized evaluation measures, each model’s performance for a physical quantity is mainly quantified by the performance score (PS), which ranges from 0 to 100. Overall, the CMIP6 models reasonably reproduce the physical quantities of the driving variables and the warming ocean heat content and temperature trends. However, their performances significantly depend on the variables and region analyzed. The EC-Earth-Veg and CNRM-CM6-1 models show the best performances for the NP and NWP oceans, respectively, with the highest PS values of 85.89 and 76.97, respectively. The EC-Earth3 model series are less sensitive to the driving variables in the NP ocean, as reflected in their PS. The model performance is significantly dependent on the driving variables in the NWP ocean. Nevertheless, providing a better physical quantity does not correlate with a better performance for trend. However, MRI-ESM2-0 model shows a high performance for the physical quantity in the NWP ocean with warming trends similar to references, and it could thus be used as an appropriate driving forcing in dynamic downscaling of this ocean. This study provides objective information for studies involving dynamic downscaling of the NP and NWP oceans.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
Dynamic downscaling, such as a regional ocean-climate model (RCM), is a powerful tool that provides regional climate information on higher spatial and temporal resolutions than those of global reanalysis and climate models, and it is particularly useful for determining processes related to complex coastal terrain (e.g., Seo et al. 2014; Teufel et al. 2017; Zhang et al. 2020; Oh and Sushama 2021). Considerable work has been conducted to improve RCM-based climate information by developing dynamical and physical schemes that are more sophisticated (e.g., Giorgi et al. 2012; Lindstedt et al. 2015; Komkoua Mbienda et al. 2021). However, it is paramount to prescribe higher-quality initial and lateral boundary conditions in the RCM (e.g., Suh and Oh 2015; Rocheta et al. 2020) as global reanalysis and climate model simulations are used as the initial and lateral boundary forcings (e.g., Eyring et al. 2016; O’Neill et al. 2016). Therefore, to provide an RCM-based reliable future climate scenario associated with global warming, an extremely reliable global climate model simulation is required as the driving forcing.
Significant efforts have been made to improve global climate model simulations. In the 1990s, the World Climate Research Programme (WCRP) promoted a set of experiments known as the Coupled Model Intercomparison Project (CMIP), with the aim of better understanding past climate changes and for making projections and uncertainty estimates about the future (e.g., Meehl et al. 2000; Annan and Hargreaves 2011; Taylor et al. 2012). The sixth phase of CMIP (CMIP6; Eyring et al. 2016), which has recently progressed, features updates to the parameterization schemes and the addition of new physical processes. It also has a somewhat higher resolution than CMIP5, which advanced our understanding of regionally heterogeneous climate warming (e.g., Taylor et al. 2012; Sun et al. 2020). CMIP6 also contains the Scenario Model Intercomparison Project (ScenarioMIP; O’Neill et al. 2016; O’Neill et al. 2020), which produces projections for new sets of emissions and land use scenarios based on Shared Socioeconomic Pathways (SSPs; Riahi et al. 2017). In this respect, the CMIP6 models provide the opportunity to investigate the climate system and perform dynamic downscaling under new scenarios.
Numerous studies have examined the performance of CMIP6 models on global and regional scales (e.g., Eyring et al. 2016; Kim et al. 2020; Lee et al. 2021; Planton et al. 2021; Tang et al. 2021; Xie et al. 2022). CMIP6 models have been reported to realistically reproduce mean and extreme climates compared to observations, and their performances have improved compared to those of previous phase CMIP models (e.g., Kim et al. 2020; Xie et al. 2022; Fan et al. 2022). These model evaluation studies have been mainly conducted in the atmospheric fields connected with the international regional-atmosphere climate model project known as the COordinated Regional climate Downscaling EXperiment (CORDEX; Oh et al. 2014; Torres-Alavez et al. 2021), and model performances have often been evaluated based on habitable land areas rather than the ocean region (e.g., Kim et al. 2020; Xie et al. 2022; Fan et al. 2022).
However, the ocean covers approximately 71% of the Earth’s surface, and it plays a key role in controlling climate change and is a very efficient carbon sink that absorbs 23% of CO2 emissions (e.g., Dobush et al. 2022). It is well known that North Pacific (NP) climate variabilities, such as the Pacific Decadal Oscillation (PDO), are closely linked to the climate over East Asia and North America through large-scale circulation changes (e.g., Nishikawa et al. 2021). The current global climate change trend will result in substantial oceanographic warming over the NP ocean by the end of the twenty-first century (e.g., IPCC 2014; Alexander et al. 2018), and NP coastal communities are facing challenges in establishing plans to mitigate the impact of climate change on their socioeconomic activities. In addition, the Northwest Pacific (NWP) is characterized by a complex local circulation and large variability, and it is mainly influenced by major ocean currents (such as the Kuroshio Current, Tsushima Current, East Korean Warm Current, and Yellow Sea Warm Current) (e.g., Seo et al. 2014). Unfortunately, the ability of the CMIP6 models to reproduce these complex climate variabilities in the NWP ocean is limited owing to their coarse horizontal resolution (Giorgi et al. 2012; Teufel et al. 2017; Zhang et al. 2020), and they thus require dynamic downscaling in this region. Therefore, it is necessary to first quantitatively evaluate how the available CMIP6 models perform with respect to dynamic downscaling in the NP and NWP regions.
The present study aims to thoroughly quantify the performance of 17 CMIP6 models over the NP and NWP oceans for 1979–2014. A particular emphasis is placed on evaluating the models for annual mean climatology using the quantified Performance Score (PS). In addition, the long-term trends of ocean temperature for the historical period are quantified and compared, because it is important to conduct accurate simulations of the substantial oceanographic warming that is predicted to occur in the future climate. This study lays the foundation for conducting dynamic downscaling over the NP and NWP oceans in modeling historical and future climate scenarios. As such, this study provides baseline information for selecting the optimal global climate models for dynamic downscaling in these regions to apply in RCM simulations.
The remainder of this paper is organized as follows. Section 2 briefly describes the data and the methods used in this study. Section 3 presents the quantified model performance based on physical quantities and long-term ocean temperature trends. A summary and discussion are then presented in Sect. 4.
2 Data and Methods
2.1 Data
The 17 CMIP6 models currently provide all the boundary variables required for running the RCM from the Earth System Grid Federation (ESGF) website at https://esgf-node.llnl.gov/search/cmip6/ (see Table 1), and these required boundary variables are summarized in Table 2. This study uses monthly mean sea surface temperature, near-surface air temperature, precipitation, and near-surface eastward and northward components of wind data from 17 CMIP6 models for the period 1979–2014 (Table 1). The variables above correspond to the surface boundary forcing used to drive the RCM. Note that the CMIP6 simulations used in this study are from historical experiments of atmosphere–ocean coupled models.
The ocean heat content in the upper 2000 m represents the energy absorbed by the ocean through the surface and lateral boundaries, and it is evaluated using the equation for the ocean heat content as follows,
where \({C}_{p0}\) is the seawater heat capacity, as defined by IOC et al. (2015); \({\rho }_{0}\) is the reference density calculated by the first-year annual mean of temperature and salinity; and \(T\) is the Conservative Temperature. An increasing ocean heat content indicates ocean warming, which varies regionally (Fox-Kemper et al. 2021; Garcia-Soto et al. 2021). The distribution of the ocean heat content and its trend need to be considered in the evaluation, especially when conducting dynamic downscaling, because the long-term regional heat content trend is influenced mainly by advective heat flux (Tian et al. 2016).
Although seawater salinity, X and Y velocities, and the sea surface height above the geoid are also required to drive an RCM, this study focuses on evaluating the physical quantity of ocean temperatures and its long-term trends in the context of global warming. In addition, although some CMIP6 models have large ensemble members of up to approximately 31 members, this study analyzes the performances of single model members to conduct a fair comparison, and the first member, r1i1p1f1, is typically used.
The 5th generation of the European Centre for Medium-Range Weather Forecast global reanalysis (ERA5; Hersbach et al. 2020) data for the satellite-era historical period (1979–2014) is used as the reference data to evaluate the surface boundary variables of the CMIP6 models. This reanalysis has a higher spatial (30 km) and temporal resolution (hourly) than previous reanalysis datasets (i.e., ERA-Interim; Dee et al. 2011; NCEP/DOE reanalysis; Kalnay et al. 2002). It implies that ERA5 data is potentially more suitable for evaluating regional climate variability. Numerous studies have used ERA5 data as a reference for the reproducibility of global climate models with near-surface variables (e.g., Kim et al. 2020; Li et al. 2021; Oh and Sushama 2021). To evaluate the estimated ocean heat content and vertical profile of seawater potential temperature, the Institute of Atmospheric Physics (IAP) ocean temperature data for the period 1979–2014 is collected (e.g., Cheng et al. 2017). These IAP data provides a 1° × 1° horizontal resolution with a monthly temporal resolution and 41 vertical levels from 1 to 2000 m.
As shown in Table 1, the horizontal resolutions of the CMIP6 models differ from one another. To summarize the multi-model ensemble statistics and conduct a one-to-one model comparison, we interpolated the surface variables of all models into a common 1° × 1° grid using bilinear remapping. Similarly, the surface variables of ERA5 data at a resolution of approximately 30 km are interpolated into a 1° × 1° grid and then used in the evaluation. Furthermore, the three-dimensional ocean temperature is vertically interpolated into standard depth levels from the World Ocean Atlas (WOA; Boyer et al. 2018) and then horizontally interpolated into a common 1° × 1° grid.
2.2 Evaluation Matrix
The model evaluations conducted in this study mainly focuses on their performances in the NP (Lat.: -20° to 65°, Lon.: 98° to 284°) and NWP (Lat.: 15° to 58°, Lon.: 113° to 165°) oceans (see Fig. 1) in terms of evaluating the physical quantities and long-term trends of annual mean climatology. The performance of each model is evaluated using the root-mean-square difference (RMSD) and Taylor skill score (TSS; Taylor 2001), and their respective equations are as follows,
In Eq. (2), n is the number of total grids in the ocean areas of the analysis domain, and Mi and Ri denote the model and reference at i th grid, respectively. In Eq. (3), SDR is the ratio of the spatial standard deviations of the model to that of the reference data, and the Correlation can be calculated as follows,
where \(\overline{M }\) and \(\overline{R }\) represent the mean values of the model and the reference for the ocean areas in the analysis domain, respectively. The TSS score quantifies the similarity between the model and reference data with respect to the distribution and amplitude of the spatial pattern. The relative RMSD and TSS values of each model are calculated based on their median values, and the combined metric of the relative RMSD and TSS is used to visually separate the CMIP6 models into superior and inferior model groups.
Model performance depends on the variables, the analysis regions, and the evaluation measures used (e.g., Taylor 2001; Eyring et al. 2016; Lee et al. 2021). The evaluation measure is also important to make a fair and standard comparison (Taylor 2001). In this study, we quantify the performance of models using the PS based on the normalized RMSD and TSS for the analyzed regions. The PS equation is as follows,
where Nvar is the number of evaluation variables (i.e., the sea surface temperature, near-surface air temperature, precipitation, near-surface eastward and northward components, and ocean heat content); NRMSD and NTSS indicate RMSD and TSS normalized by the Min–Max Normalization method, providing a linear transformation of the original range of data and maintains relationships among the original data (e.g., Patro and Sahu 2015). All of the scaled evaluation measures range from 0 to 1. The perfect values of NRMSD and NTSS are 0 and 1, respectively. In the numerator in parenthesis of Eq. (5), by performing 1 minus NRMSD, the perfect value is converted to 1 as for NTSS. Then, the denominator is multiplied by 2 to set a maximum value of the summation of the two scaled evaluation measures to 1. Subsequently, we calculate an equal-weighted average for all variables in each analysis region. To aid interpretation, these values are multiplied by 100, giving a range of the PS values for each model from 0 to 100. The PS value has the advantage that it can evaluate synthetic model performance as it is based on the RMSD and TSS of all variables considered in the evaluation. Therefore, PS is used to quantify the model performance over the analysis region.
3 Results
3.1 Evaluation of Physical Quantity for Surface and Lateral Boundary Forcing Variables
Figure 1 shows the spatial distribution of the annual mean ocean heat content, sea surface temperature, near-surface air temperature, precipitation, and near-surface eastward and northward components of wind from references and the CMIP6 multi-model ensemble over the NP ocean for the period 1979–2014. The NWP ocean is presented in each subplot as a boxed area. Compared with the references, the CMIP6 multi-model ensemble reasonably reproduces the spatial distributions of the surface and lateral boundary forcing variables. That is, the spatial ocean heat and related temperature characteristics induced by the difference in latitude, the large precipitations in the intertropical convergence zone (ITCZ) and East Asian monsoon region, and the location and strength of trade winds and westerlies are realistically reproduced, with spatial correlations of 0.85–0.99. Regionally, the simulated ocean heat contents have warm biases of + 0.4 to + 2.0 × 1010 J/m2 in the NP ocean compared to the reference, particularly for the significant warm biases in the NWP ocean. Both sea surface and near-surface-air temperatures tend to be slightly overestimated by approximately + 0.5 to + 1.5 °C in the ITCZ, but the opposite biases with a similar magnitude are found in the NWP ocean. For precipitation, relatively large and small biases ranging from -3 to + 3 mm/day and -0.5 to + 0.5 mm/day are observed in the ITCZ and the NWP ocean, respectively. Similarly, the simulated trade winds are relatively weak, whereas the strength of the westerlies is well simulated compared with the reference. These results are consistent with those of previous studies that showed CMIP multi-model ensembles have positive or negative biases on a regional scale with respect to the surface and lateral boundary variables for the historical climate but reasonably reproduce their spatial distribution (e.g., Kim et al. 2020; Zhang et al. 2020; Fan et al. 2022).
Figure 2 shows the box and whisker plots of the RMSD for the surface and lateral boundary forcing variables simulated by the CMIP6 models in the NP and NWP oceans. The RMSD ranges from 0.5 to 2.5, depending on the variable and region analyzed. There is a relatively larger model spread in the ocean heat content of the NP compared to that of the other variables. In addition, relatively lower performances are found for precipitation due to the large biases in the ITCZ precipitation zone (Figs. 1g and h). However, the simulated temperature performance in the NWP ocean is lower, and the model spread is larger than the other variables. The model resolution plays a critical role in simulating local temperature, particularly in the NWP ocean, which has complex coastal terrain, and this can result in large between-model diversity (e.g., Nishikawa et al. 2021; Oh and Sushama 2021). The results obtained here imply that the simulated temperature performance needs to be adequately considered when selecting CMIP6 models for dynamic downscaling in this region.
The relative RMSDs of the CMIP6 model for the surface and lateral boundary variables are compared in Fig. 3 using the median values of each sub-box plot shown in Fig. 2. In general, the model performance depends on the variable and region analyzed. For instance, CMCC model series show a relatively better performance with respect to temperatures than the other variables, whereas ACCESS model series show a relatively good performance in the near-surface eastward components of wind, and UK-ESM1-0-LL model shows a relatively better performance in the NP ocean but a lower performance in the NWP ocean. Overall, EC-Earth3 and CNRM model series perform relatively better in the NP and NWP oceans than the other model series.
A scatter plot based on the combined matrix of the relative RMSD and TSS is shown in Fig. 4, and the model performance is classified linearly for all variables and analysis regions. The results indicate that the combined use of RMSD and TSS enables appropriate classification of the models’ performances. This result shows that for temperature-related variables, the model performance for the magnitude of a physical quantity is of relatively greater importance for determining the driving model, as all CMIP models reproduce their spatial pattern well. For other variables, the model performance for spatial patterns is also important in determining the driving model.
Table 3 summarizes the performance of the 17 CMIP6 models based on the PS value calculated from the normalized RMSD and TSS (see Sect. 2.2). In the NP ocean, the EC-Earth-Veg model provides the best performance, with the highest PS value of 85.89 among 17 CMIP6 models. The second- and third-best models are the EC-Earth3 and EC-Earth3-Veg-LR models, with PS values of 84.97 and 84.22, respectively. These results show that EC-Earth3 model series are the good choice for use in dynamic downscaling in the NP ocean. However, in the NWP ocean, CNRM-CM6-1 and CNRM-ESM2-1 models are the best and second-best models, with PS values of 76.97 and 76.69, respectively. The MRI-ESM2-0 and EC-Earth-Veg models are also appropriate for this ocean, with PS values of 75.77 and 75.67, respectively. This result implies that different global climate models can be recommended as a driving forcing for dynamic downscaling depending on the area analyzed.
The sensitivity of the models’ performances to the variables used to compute the PS value is shown in Fig. 5. In general, the PS values of EC-Earth3 model series are higher, and the spread of the values is smaller than that of the other models for both the NP and NWP oceans (Figs. 5a and b). It indicates that the model performance of this model series is relatively less sensitive to the variables used, providing a superior performance. In contrast, the PS values of CMCC model series are widely spread according to the variables used. For example, when considering only temperature variables in computing the PS value, CMCC model series perform highly compared to the other CMIP6 models, but their performance is dramatically reduced when other boundary-forcing variables are considered when computing the PS value. The quantified model performance as a function of the variables considered is summarized in Tables 4 and 5 for the NP and NWP oceans, respectively. Overall, EC-Earth3 model series perform highly in most sensitivity tests within the NP ocean. When considering only wind, the UKESM1-0-LL and ACCESS-ESM1-5 models could be good choices for dynamic downscaling in this ocean.
In the NWP ocean, the models’ performances show a significant dependency on the variable considered. The results show that the MRI-ESM2-0 and CNRM-CM6-1 models are good choices for dynamic downscaling in this ocean considering atmospheric and oceanic variables, respectively, but the CMCC-ESM2 model is the best choice when considering only temperatures for dynamic downscaling in this ocean, and the ACCESS-ESM1-5 model may be the best choice when only wind is considered. These results therefore indicate that the selection of the CMIP6 model for use in dynamic downscaling in the NWP ocean will vary depending on the goal of the study conducted.
3.2 Evaluation of Long-Term Ocean Temperature Trend
In the context of continuous future oceanographic warming, it is critical that the long-term ocean temperature and the related heat content trends provided by CMIP6 models in relation to the historical period are accurate, because the RCM is more likely to follow that of driving forcing. In this subsection, we compare the ocean heat content, sea surface, and near-surface-air temperature trends between the 17 CMIP6 models and the reference.
Figure 6 shows the spatial distribution of the long-term annual mean near-surface-air temperature, sea surface temperature, and ocean heat content trends from the references and the CMIP6 multi-model ensemble over the NP ocean for the period 1979–2014. The western Pacific warming and eastern Pacific cooling for all three variables are observed in the references (Figs. 6a, c, and e). This result is consistent with those of previous studies using different observation datasets (Maher et al. 2018; Li et al. 2019). Spatially opposite temperature trends are closely associated with changes in circulation. For example, Maher et al. (2018) found a strengthening of the equatorial undercurrent in response to strengthened winds, which brought cooler water to the surface of the eastern Pacific and an increase in the shallow Pacific overturning cells, thereby resulting in the input of additional heat into the subsurface western Pacific. In addition, wind acceleration increases the subsequent transport of heat toward the western Pacific. This strengthening of the wind circulation can be primarily attributed to the cold tongue mode rather than the impact of the El Niño-Southern Oscillation (Li et al. 2019).
The CMIP6 multi-model ensemble fails to capture eastern Pacific cooling with all three variables. It instead shows warming trends throughout all NP oceans, with the exception of slight cooling in the ocean heat content around the Philippine Sea. These warming trends are larger in high-latitude and near-surface air temperatures than in low-latitude and ocean temperatures. The failure of the CMIP6 multi-model ensemble to capture the eastern Pacific cooling could result from its failure to reproduce the strengthening of the observed trade winds, which are known to have brought cooler water to the surface of the eastern Pacific during the historical period (not shown). This limitation of the ability of CMIP models to reproduce the spatial pattern of long-term trends, especially those of precipitation and wind variables, has been reported in previous studies (e.g., Lee et al. 2019; Vicente-Serrano et al. 2022). However, Lee et al. (2019) reported that CMIP models can reproduce regionally averaged temperature trends, depending on the region analyzed.
The time series of the observed and simulated regionally averaged annual mean near-surface-air temperature, sea surface temperature, and ocean heat content over the NP and NWP oceans for the period 1979–2014 are shown in Fig. 7. Their long-term trends are also summarized in Table 6. With respect to the regional averages, all three variables in reference gradually increased over the 36-year period in the NP and NWP oceans (see the thick black line in Fig. 7), and these warming trends (i.e., 0.47–0.65 W m−2, 1.66–3.88 °C per century) were statistically significant (Table 6). Stronger warming trends in the near-surface air temperature (i.e., 2.16–3.88 °C per century), compared to the sea surface temperature (i.e., 1.66–2.47 °C per century), are apparent in both the NP and NWP oceans. In addition, the NWP ocean tends to have more robust warming trends (i.e., 2.47–3.88 °C per century) than the NP ocean (i.e., 1.66–2.16 °C per century).
Overall, the CMIP6 multi-model ensemble reasonably captures the regionally averaged physical quantities of the warming trends of the three variables over time compared to the reference (see the thick purple line in Fig. 7). In addition, it reproduces the characteristics of the relative magnitude of the observed warming trends according to the variables and regions analyzed. However, it shows more substantial warming (1.5–2.0 times) than the reference, except for the ocean heat content in the NP ocean (see the second row in Table 6).
Most of the CMIP6 models reproduce the observed warming trends well, although their magnitudes differ. As shown in Fig. 7f, there is significant diversity in the ability of CMIP6 models to simulate the ocean heat content compared to the other variables, and this could be attributable to the model resolution. The NWP region contains complex coastal areas. It may significantly affect simulations of the seawater potential temperature in deep layers because of the prescribed seabed topography which differs depending on the model resolution (e.g., de la Vara et al. 2020).
The relative errors of the long-term trends simulated by the 17 CMIP6 models compared to the reference are further examined in Fig. 8. The blue color indicates that the warming trend of the CMIP6 model is smaller than that in the reference. Note that EC-Earth3-Veg-LR model shows a negative trend in ocean heat content in the NWP ocean (see Table 6). The relative errors, even in a single model, depend on the variable and the region analyzed. For example, most of the CMIP6 models underestimate warming in the ocean heat content in the NP ocean, but overestimate warming in the other variables in this ocean. Some models, such as the MPI-ESM1-2-HR and MRI-ESM2-0 models, show lower warming trends in the ocean heat content in the NP ocean but overestimate warming in the NWP ocean.
When considering the warming levels of the three variables, EC-Earth3 and UKESM1-0-LL models show the most significant deviations from the warming levels in the reference for both the NP and NWP oceans. In contrast, MPI-ESM1-2-HR and MRI-ESM2-0 models show relatively smaller deviations from the warming levels of the reference over these regions. It is of note that the MRI-ESM2-0 model provides a good physical quantity performance in the NWP ocean (see Table 3), and this model could thus be a good choice for use in dynamic downscaling in the NWP ocean. However, the result suggests that providing a better performance in terms of physical quantity does not directly connect to providing a better performance in relation to the trend. For instance, EC-Earth3-Veg and CNRM-CM6 models, which respectively provide the highest-performing physical quantities based on PS values in the NP and NWP oceans (see Table 3), show moderate performance with respect to their warming trends. This result implies that it is necessary to carefully consider various factors when selecting a CMIP6 model as a driving forcing in dynamic downscaling.
4 Summary and Discussion
This study quantitatively evaluates the performance of CMIP6 models as driving forcing for dynamic downscaling in the NP and NWP oceans in terms of their abilities to reproduce physical quantities and long-term trends. The 17 CMIP6 models provide all the surface and lateral boundary variables required for running the RCM from the ESGF website as of 2022. 04. (see Tables 1 and 2), and their performances are evaluated. The various driving variables, i.e., the ocean heat content, sea surface temperature, near-surface air temperature, precipitation, and near-surface eastward and northward components of wind, are compared to the ERA5 and IAP data for the period 1979–2014. A particular emphasis is placed on the model performance for annual mean climatology using the PS value based on normalized RMSD and TSS. Furthermore, in consideration of oceanographic warming, the long-term trends of the regionally averaged near-surface air temperature, sea temperature, and ocean heat content over the NP and NWP oceans are examined.
Compared with the references, the CMIP6 multi-model ensemble reasonably reproduces the spatial distributions of the physical quantities of the surface and lateral boundary variables, with a spatial correlation of 0.85–0.99. However, the performance of a single CMIP6 model significantly depends on the variable and region analyzed, particularly in terms of the physical magnitude. Overall, the RMSDs of the EC-Earth3 and CNRM model series are relatively lower in the NP and NWP oceans compared to the other model series. Of the 17 CMIP6 models, EC-Earth-Veg and CNRM-CM6-1 models show the best performances in terms of the PS values (85.89 and 76.97) for the NP and NWP oceans, respectively. In particular, the EC-Earth3 model series are less sensitive to the driving variables used in computing the PS value for the NP ocean, which suggests that this model series could be a good choice as the driving forcing for dynamic downscaling in this ocean. In the NWP ocean, the model performance shows a significant dependency on the variable considered in computing the PS value. This implies that selecting the appropriate CMIP6 model as the driving forcing for dynamic downscaling in this ocean depends on the research perspective.
In the trend analysis, the CMIP6 multi-model ensemble reasonably captures the regionally averaged warming trends, although its warming is more robust than the reference by 1.5–2.0 times. Both MPI-ESM1-2-HR and MRI-ESM2-0 models provide relatively good performances in the NP and NWP oceans compared to the other models. In particular, MRI-ESM2-0 model shows a high performance for the physical quantity in the NWP ocean (see Table 3), and it could thus be a good choice for use in dynamic downscaling in the NWP ocean.
However, providing a better performance in terms of physical quantity does not directly correlate with providing a better performance with respect to long-term trends. It should be noted that the PS value used in this study could be sensitive to the selection of variables and regions. Therefore, when conducting dynamic downscaling in NP and NWP oceans, users need to make a subjective decision as to which model to employ, based on their specific research needs. Besides, when conducting dynamic downscaling, multi-model experiments forced by independent driving models are needed rather than using a single model. This study is meaningful in that it provides objective information, and thus saves time and computing resources, for researchers to construct a more systematic ensemble experiment and perform dynamic downscaling on the NP and NWP oceans.
Investigating the vertical profile of ocean warming trends along the RCM boundary assists in evaluating global climate models because substantial heat exchange with the surrounding region occurs through the boundary of the RCM, especially for the NWP. Preliminary results for the vertical warming trend profile along the southern boundary are shown in Fig. 9a. Both ACCESS-CM2 and ACCESS-ESM1-5 models show a shallowing of the thermocline and an increase in water temperature in the intermediate layer (500–1,000 m). This vertical structure, which differs from that of the reference, is problematic when driving an RCM. An extensive model spread is also found, especially in the upper 1,000 m along the eastern boundary of the NWP ocean (Fig. 9b). Therefore, further studies are necessary to comprehensively consider the performance of the vertical profiles.
Data availability
The 17 CMIP6 models used in this study are available on the Earth System Grid Federation (ESGF) website at https://esgf-node.llnl.gov/search/cmip6/. The reference dataset, ERA5, and IAP data can be found at “https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/era5 ” and “http://www.ocean.iap.ac.cn/pages/dataService/dataService.html,” respectively. To obtain the analysis codes based on Fortran, Grads, R, and MATLAB, which were used to express the results of this study, please contact S.-G. Oh (seokgeunoh@snu.ac.kr), and B.-G. Kim (bongan@snu.ac.kr).
References
Alexander, M.A., Scott, J.D., Friedland, K.D., Mills, K.E., Nye, J.A., Pershing, A.J., Thomas, A.C.: Projected sea surface temperatures over the 21st century: changes in the mean, variability and extremes for large marine ecosystem regions of Northern Oceans. Elem Sci. Anth. 6(9), (2018). https://doi.org/10.1525/elementa.191
Annan, J.D., Hargreaves, J.C.: Understanding the CMIP3 multi model ensemble. J. Clim. 24(16), 4529–4538 (2011). https://doi.org/10.1175/2011JCLI3873.1
Boyer, T.P., Garcia, H.E., Locarnini, R.A., Zweng, M.M., Mishonov, A.V., Reagan, J.R., Weathers, A.K., Baranova, O.K., Seidov, D., Smolyar I.V.: World Ocean Atlas 2018. NOAA National Centers for Environmental Information (2018)
Cheng, L., Trenberth, K., Fasullo, J., Boyer, T., Abraham, J., Zhu, J.: Improved estimates of ocean heat content from 1960 to 2015. Sci. Adv. 3, e1601545 (2017). https://doi.org/10.1126/sciadv.1601545
De la Vara, A., Cabos, W., Sein, D.V., Sidorenko, D., Koldunov, N.V., Koseki, S., Soares, P.M.M., Danilov, S.: On the impact of atmospheric vs oceanic resolutions on the representation of the sea surface temperature in the South Eastern Tropical Atlantic. Clim. Dyn. 54, 4733–4757 (2020). https://doi.org/10.1007/s00382-020-05256-9
Dee, D.P., Uppala, S.M., Simmons, A.J., Berrisford, P., Poli, P., Kobayashi, S., Andrae, U., Balmaseda, M.A., Balsamo, G., Bauer, P., Bechtold, P., Beljaars, A.C.M., van de Berg, L., Bidlot, J., Bormann, N., Delsol, C., Dragani, R., Fuentes, M., Geer, A.J., Haimberger, L., Healy, S.B., Hersbach, H., Hólm, E.V., Isaksen, L., Kållberg, P., Köhler, M., Matricardi, M., McNally, A.P., Monge-Sanz, B.M., Morcrette, J.-J., Park, B.-K., Peubey, C., de Rosnay, P., Tavolato, C., Thépaut, J.-N., Vitart, F.: The ERA-Interim reanalysis: configuration and performance of the data assimilation system. Quart. J. Roy. Meteor. Soc. 137, 553–597 (2011). https://doi.org/10.1002/qj.828
Dobush, B.J., Gallo, N.D., Guerra, M., Guilloux, B., Holland, E., Seabrook, S., Levin, L.A.: A new way forward for ocean-climate policy as reflected in the UNFCCC ocean and climate change dialogue submissions. Clim. Policy 22(2), 254–271 (2022). https://doi.org/10.1080/14693062.2021.1990004
Eyring, V., Bony, S., Meehl, G.A., Senior, C.A., Stevens, B., Stouffer, R.J., Taylor, K.E.: Overview of the coupled model intercomparison project phase 6 (CMIP6) experimental design and organization. Geosci. Model. Dev. 9, 1937–1958 (2016). https://doi.org/10.5194/gmd-9-1937-2016
Fan, X., Duan, Q., Shen, C., Wu, Y., Xing, C.: Evaluation of historical CMIP6 model simulations and future projections of temperature over the Pan-Third Pole region. Environ. Sci. Pollut. Res. 29, 26241–26229 (2022). https://doi.org/10.1007/s11356-021-17474-7
Fox-Kemper, B., Hewitt, H. T., Xiao, C., Aðalgeirsdóttir, G., Drijfhout, S.S., Edwards, T.L, Golledge, N.R., Hemer, M., Kopp, R.E., Krinner, G., Mix, A., Notz, D., Nowicki, S., Nurhati, I.S., Ruiz, L., Sallée, J.B., Slangen A.B.A., Yu, Y.: Ocean, Cryosphere and Sea Level Change. In Masson-Delmotte, V., Zhai, P., Pirani, A., Connors S. L., Péan, C., Berger, S., Caud, N., Chen, Y., Goldfarb, L., Gomis, M. I., Huang, M., Leitzell, K., Lonnoy, E., Matthews, J. B. R., Maycock, T. K., Waterfield, T., Yelekçi, O., Yu, R., Zhou B., Eds.: Climate Change 2021: The Physical Science Basis. Contribution of Working Group I to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change, 1211–1362, Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA. (2021). https://doi.org/10.1017/9781009157896.011
Garcia-Soto, C., Cheng, L., Caesar, L., Schmidtko, S., Jewett, E.B., Cheripka, A., Rigor, I., Caballero, A., Chiba, S., Báez, J.C., Zielinski, T., Abraham, J.P.: An Overview of Ocean Climate Change Indicators: Sea Surface Temperature, Ocean Heat Content, Ocean pH, Dissolved Oxygen Concentration, Arctic Sea Ice Extent, Thickness and Volume, Sea Level and Strength of the AMOC (Atlantic Meridional Overturning Circulation). Front. Mar. Sci. 8, 642372 (2021). https://doi.org/10.3389/fmars.2021.642372
Giorgi, F., Coppola, E., Solmon, F., Mariotti, L., Sylla, M., Bi, X., Elguindi, N., Diro, G., Nair, V., Giuliani, G., Turuncoglu, U., Cozzini, S., Güttler, I., O’Brien, T., Tawfik, A., Shalaby, A., Zakey, A., Steiner, A., Stordal, F., Sloan, L., Brankovic, C.: RegCM4: Model description and preliminary test over multiple CORDEX domains. Clim. Res. 52, 7–29 (2012). https://doi.org/10.3354/cr01018
Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Horányi, A., Muñoz-Sabater, J., Nicolas, J., Peubey, C., Radu, R., Schepers, D., Simmons, A., Soci, C., Abdalla, S., Abellan, X., Balsamo, G., Bechtold, P., Biavati, G., Bidlot, J., Bonavita, M., Chiara, G., Dahlgren, P., Dee, D., Diamantakis, M., Dragani, R., Flemming, J., Forbes, R., Fuentes, M., Geer, A., Haimberger, L., Healy, S., Hogan, R.J., Hólm, E., Janisková, M., Keeley, S., Laloyaux, P., Lopez, P., Lupu, C., Radnoti, G., Rosnay, P., Rozum, I., Vamborg, F., Villaume, S., Thépaut, J.: The ERA5 global reanalysis. Q. J. r. Meteorol. Soc. (2020). https://doi.org/10.1002/qj.3803
IOC, SCOR, and IAPSO: The International thermodynamic equation of seawater – 2010: calculation and use of thermodynamic properties. [includes corrections up to 31st October 2015]. UNESCO, París, France (2015). https://doi.org/10.25607/OBP-1338
IPCC: In: Core Writing Team, Pachauri, R. K., Meyer L. A. (eds) Climate change 2014: synthesis report. Contribution of Working Groups I, II and III to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change. IPCC, Geneva, p151 (2014)
Kalnay, E., Ebisuzaki, W., Woollen, J., Yang, S.K., Hnilo, J.J., Fiorino, M., Potter, G.L.: NCEP-DOE AMIP-II reanalysis (R-2). Bull. Am. Meteorol. Soc. 83, 1631–1643 (2002)
Kim, Y.H., Min, S., Zhang, X., Sillmann, J., Sandstad, M.: Evaluation of the CMIP6 multi-model ensemble for climate extreme indices. Weather. Clim. Extrem. 29, 100269 (2020). https://doi.org/10.1016/j.wace.2020.100269
Komkoua Mbienda, A.J., Guenang, G.M., Tanessong, R.S., Ashu Ngono, S.V., Zebaze, S., Vondou, D.A.: Possible influence of the convection schemes in regional climate model RegCM4.6 for climate services over Central Africa. Meteorol. Appl. 28(2), e1980 (2021). https://doi.org/10.1002/met.1980
Lee, J., Waliser, D., Lee, H., Loikith, P., Kunkel, K.E.: Evaluation of CMIP5 ability to reproduce twentieth century regional trends in surface air temperature and precipitation over CONUS. Clim. Dyn. 53, 5459–5480 (2019). https://doi.org/10.1007/s00382-019-04875-1
Lee, J., Sperber, K., Gleckler, P., Taylor, K., Bonfils, C.: Benchmarking performance changes in the simulation of extratropical modes of variability across CMIP generations. J. Climate 34, 6945–6969 (2021). https://doi.org/10.1175/JCLI-D-20-0832.1
Li, Y., Chen, Q., Liu, X., Li, J., Xing, N., Xie, F., Feng, J., Zhou, X., Cai, H., Wang, Z.: Long-term trend of the tropical Pacific trade winds under global warming and its causes. J. Geophys. Res. Oceans 124, 2626–2640 (2019). https://doi.org/10.1029/2018JC014603
Li, C., Zwiers, F., Zhang, X., Li, G., Sun, Y., Wehner, M.: Changes in annual extremes of daily temperature and precipitation in CMIP6 model. J. Climate 34, 3441–3460 (2021). https://doi.org/10.1175/JCLI-D-19-1013.1
Lindstedt, D., Lind, P., Kjellstrom, E., Jones, C.: A new regional climate model operating at the meso-gamma scale: performance over Europe. Tellus a: Dyn. Meteorol. and Oceanogr. 67(1), 24138 (2015). https://doi.org/10.3402/tellusa.v67.24138
Maher, N., England, M.H., Gupta, A.S., Spence, P.: Role of Pacific trade winds in driving ocean temperatures during the recent slowdown and projections under a wind trend reversal. Clim. Dyn. 51, 324–336 (2018). https://doi.org/10.1007/s00382-017-3923-3
Meehl, G.A., Boer, G.J., Covey, C., Latif, M., Stouffer, R.J.: The coupled model intercomparison project (CMIP). Bull. Amer. Meteorol. Soc. 81(2), 313–318 (2000)
Nishikawa, S., Wakamatsu, T., Ishizaki, H., Sakamoto, K., Tanaka, Y., Tsujino, H., Yamanaka, G., Kamachi, M., Ishikawa, Y.: Development of high-resolution future ocean regional projection datasets for coastal applications in Japan. Prog. Earth Planet. Sci. 8, 1–22 (2021). https://doi.org/10.1186/s40645-020-00399-z
O’Neill, B.C., Tebaldi, C., van Vuuren, D.P., Eyring, V., Friedlingstein, P., Hurtt, G., Knutti, R., Kriegler, E., Lamarque, J., Lowe, J., Meehl, G.A., Moss, R., Riahi, K., Sanderson, B.M.: The scenario model intercomparison project (ScenarioMIP) for CMIP6. Geosci. Model Dev. 9, 3461–3482 (2016). https://doi.org/10.5194/gmd-9-3461-2016
O’Neill, B.C., Carter, T.R., Ebi, K., Harrison, P.A., Kemp-Benedict, E., Kok, K., Kriegler, E., Preston, B.L., Riahi, K., Sillmann, J., van Ruijven, B.J., van Vuuren, D., Carlisle, D., Conde, C., Fuglestvedt, J., Green, C., Hasegawa, T., Leininger, J., Monteith, S., Pichs-Madruga, R.: Achievements and needs for the climate change scenario framework. Nat. Clim. Change 10(12), 1074–1084 (2020). https://doi.org/10.1038/s41558-020-00952-0
Oh, S.G., Sushama, L.: Urban-climate interactions during summer over eastern North America. Clim. Dyn. 57, 3015–3028 (2021). https://doi.org/10.1007/s00382-021-05852-3
Oh, S.G., Park, J.H., Lee, S.H., Suh, M.S.: Assessment of the RegCM4 over East Asia and future precipitation change adapted to the RCP scenarios. J. Geophys. Res. Atmos. 119, 2913–2927 (2014). https://doi.org/10.1002/2013JD020693
Patro, S.G.K., Sahu, K.K.: Normalization: a processing stage. Comput. Sci. (2015). https://doi.org/10.48550/arXiv.1503.06462
Planton, Y., Guilyardi, E., Wittenberg, A.T., Lee, J., Gleckler, P.J., Bayr, T., McGregor, S., McPhaden, M.J., Power, S., Roehrig, R., Voldoire, A.: Evaluating climate models with the CLIVAR 2020 ENSO metrics package. Bull. Am. Meteorol. Soc. 102, E193–E217 (2021). https://doi.org/10.1175/BAMS-D-19-0337.1
Riahi, K., van Vuuren, D.P., Kriegler, E., Edmonds, J., O’Neill, B.C., Fujimori, S., Bauer, N., Calvin, K., Dellink, R., Fricko, O., Lutz, W., Popp, A., Cuaresma, J.C., Kc, S., Leimbach, M., Jiang, L., Kram, T., Rao, S., Emmerling, J., Ebi, K., Hasegawa, T., Havlik, P., Humpenöder, F., Da Silva, L.A., Smith, S., Stehfest, E., Bosetti, V., Eom, J., Gernaat, D., Masui, T., Rogelj, J., Strefler, J., Drouet, L., Krey, V., Luderer, G., Harmsen, M., Takahashi, K., Baumstark, L., Doelman, J.C., Kainuma, M., Klimont, Z., Marangoni, G., Lotze-Campen, H., Obersteiner, M., Tabeau, A., Tavoni, M.: The Shared Socio-economic Pathways and their energy, land use, and greenhouse gas emissions implications: an overview. Glob. Environ. Change 42, 153–168 (2017). https://doi.org/10.1016/j.gloenvcha.2016.05.009
Rocheta, E., Evans, J.P., Sharma, A.: Correcting lateral boundary biases in regional climate modelling: the effect of the relaxation zone. Clim. Dyn. 55, 2511–2521 (2020). https://doi.org/10.1007/s00382-020-05393-1
Seo, G.H., Cho, Y.K., Choi, B.J., Kim, K., Kim, B., Tak, Y.: Climate change projection in the Nortwest Pacific marginal seas through dynamic downscaling. J. Geophys. Res. Oceans. 119, 3497–3516 (2014). https://doi.org/10.1002/2013JC009646
Suh, M.S., Oh, S.G.: Impacts of boundary conditions on the simulation of atmospheric field using RegCM4 over CORDEX East Asia. Atmos. 6, 783–804 (2015). https://doi.org/10.3390/atmos6060783
Sun, Q., Miao, C., AghaKouchak, A., Mallakpour, I., Ji, D., Duan, Q.: Possible increased frequency of ENSO-related dry and wet conditions over some major watersheds in a warming climate. Bull. Amer. Meteorol. Soc. 101(4), E409–E426 (2020). https://doi.org/10.1175/BAMS-D-18-0258.1
Tang, S., Gleckler, P., Xie, S., Lee, J., Covey, C., Zhang, C., Ahn, M.-S.: Evaluating Diurnal and Semi-Diurnal Cycle of Precipitation in CMIP6 Models Using Satellite- and Ground-Based Observations. J. Climate 34, 3189–3210 (2021). https://doi.org/10.1175/JCLI-D-20-0639.1
Taylor, K.E.: Summarizing multiple aspects of model performance in a single diagram. J. Geophys. Res. 106, 7183–7192 (2001). https://doi.org/10.1029/2000JD900719
Taylor, K.E., Stouffer, R.J., Meehl, G.A.: An overview of CMIP5 and the experiment design. Bull. Amer. Meteorol. Soc. 94(4), 485–498 (2012). https://doi.org/10.1175/BAMS-D-11-00094.1
Teufel, B., Diro, G.T., Whan, K., Milrad, S.M., Jeong, D.I., Ganji, A., Huziy, O., Winger, K., Gyakum, J.R., de Elia, R., Zwiers, F.W., Sushama, L.: Investigation of the 2013 Alberta flood from weather and climate perspectives. Clim. Dyn. 48, 2881–2899 (2017). https://doi.org/10.1007/s00382-016-3239-8
Tian, T., Su, J., Boberg, F., Yang, S., Schmith, T.: Estimating uncertainty caused by ocean heat transport to the North Sea: experiments downscaling EC-Earth. Clim. Dyn. 46, 99–110 (2016). https://doi.org/10.1007/s00382-015-2571-8
Torres-Alavez, J.A., Glazer, R., Giorgi, F., Coppola, E., Gao, X., Hodges, K.I., Das, S., Ashfaq, M., Reale, M., Sines, T.: Future projections in tropical cyclone activity over multiple CORDEX domains from RegCM4 CORDEX-CORE simulations. Clim. Dyn. 57(5), 1507–1531 (2021). https://doi.org/10.1007/s00382-021-05728-6
Vicente-Serrano, S.M., Garcia-Herrera, R., Pena-Angulo, D., Tomas-Burguera, M., Domínguez-Castro, F., Noguera, I., Calvo, N., Murphy, C., Nieto, R., Gimeno, L., Gutierrez, J.M., Azorin-Molina, C., El Kenawy, A.: Do CMIP models capture long-term observed annual precipitation trends? Clim. Dyn. 58, 2825–2842 (2022). https://doi.org/10.1007/s00382-021-06034-x
Xie, W., Zhou, B., Han, Z., Xu, Y.: Substantial increase in daytime-nighttime compound heat waves and associated population exposure in China projected by the CMIP6 multimodel ensemble. Environ. Res. Lett. 17, 045007 (2022). https://doi.org/10.1088/1748-9326/ac592d
Zhang, L., Xu, Y., Meng, C., Li, X., Liu, H.: Comparison of statistical and dynamic downscaling techniques in generating high-resolution temperatures in China from CMIP5 GCMs. J. Appl. Meteorol. Climatol. 59, 207–235 (2020). https://doi.org/10.1175/JAMC-D-19-0048.1
Acknowledgements
This research was supported by Korea Institute of Marine Science & Technology Promotion (KIMST) funded by the Ministry of Oceans and Fisheries (20220033).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they have no known competing financial interests or personal relationships that could have influenced the work reported in this study.
Additional information
Communicated by Seon Tae Kim.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Oh, SG., Kim, BG., Cho, YK. et al. Quantification of The Performance of CMIP6 Models for Dynamic Downscaling in The North Pacific and Northwest Pacific Oceans. Asia-Pac J Atmos Sci 59, 367–383 (2023). https://doi.org/10.1007/s13143-023-00320-w
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13143-023-00320-w