Skip to main content
SpringerLink
Log in

An inter-comparison performance assessment of a Brazilian global sub-seasonal prediction model against four sub-seasonal to seasonal (S2S) prediction project models

  • Published:
Climate Dynamics Aims and scope Submit manuscript

Abstract

This paper presents an inter-comparison performance assessment of the newly developed Centre for Weather Forecast and Climate Studies (CPTEC) model (the Brazilian Atmospheric Model version 1.2, BAM-1.2) against four sub-seasonal to seasonal (S2S) prediction project models from: Japan Meteorological Agency (JMA), Environmental and Climate Change Canada (ECCC), European Centre for Medium-range Weather Forecasts (ECMWF) and Australian Bureau of Meteorology (BoM). The inter-comparison was performed using hindcasts of weekly precipitation anomalies and the daily evolution of Madden–Julian Oscillation (MJO) for 12 extended austral summers (November–March, 1999/2000–2010/2011), leading to a verification sample of 120 hindcasts. The deterministic assessment of the prediction of precipitation anomalies revealed ECMWF as the model presenting the highest (smallest) correlation (root mean squared error, RMSE) values among all examined models. JMA ranked as the second best performing model, followed by ECCC, CPTEC and BoM. The probabilistic assessment for the event “positive precipitation anomaly” revealed that ECMWF presented better discrimination, reliability and resolution when compared to CPTEC and BoM. However, these three models produced overconfident probabilistic predictions. For MJO predictions, CPTEC crosses the 0.5 bivariate correlation threshold at around 19 days when using the mean of 4 ensemble members, presenting similar performance to BoM, JMA and ECCC. Overall, CPTEC proved to be competitive compared to the S2S models investigated, but with respect to ECMWF there is scope to improve the prediction system, likely by a combination of including coupling to an interactive ocean, improving resolution and model parameterization schemes, and better methods for ensemble generation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Coelho CAS, Firpo MA, de Andrade FM (2018) A verification framework for South American sub-seasonal precipitation predictions. Meteorol Z 27(6):503–520

    Google Scholar 

  • Coelho CAS, Brown B, Wilson L, Mittermaier M, Casati B (2019) Forecast verification for S2S time scales. In: Robertson AW, Vitart F (eds) Sub-seasonal to seasonal prediction: the gap between weather and climate forecasting, Book Chap. 17, 1st edn. Elsevier, Amsterdam, pp 337–361. Accessed 24th Oct 2018 (ISBN: 9780128117149. eBook ISBN: 9780128117156)

  • de Andrade FM, Coelho CAS, Cavalcanti IF (2019) Global precipitation hindcast quality assessment of the Subseasonal to Seasonal (S2S) prediction project models. Clim Dyn 52:5451–5475

    Google Scholar 

  • Dee DP, Uppala SM, Simmons AJ et al (2011) The ERA-Interim reanalysis: configuration and performance of the data assimilation system. Q J R Meteorol Soc 137:553–597

    Google Scholar 

  • Gonzalez PL, Vera CS (2014) Summer precipitation variability over South America on long and short intraseasonal timescales. Clim Dyn 43(7–8):1993–2007

    Google Scholar 

  • Gottschalck J et al (2010) A framework for assessing operational Madden–Julian oscillation forecasts: a CLIVAR MJO working group project. Bull Am Meteor Soc 91:1247–1258

    Google Scholar 

  • Guimarães BS et al (2020) Configuration and hindcast quality assessment of a brazilian global sub-seasonal prediction system. Q J R Meteorol Soc 146(728):1067–1084

    Google Scholar 

  • Grimm AM (2019) Madden–Julian Oscillation impacts on South American summer monsoon season: precipitation anomalies, extreme events, teleconnections, and role in the MJO cycle. Clim Dyn 53(1–2):907–932

    Google Scholar 

  • Hudson D, Alves O, Hendon HH, Marshall AG (2011a) Bridging the gap between weather and seasonal forecasting: intraseasonal forecasting for Australia. Q J R Meteorol Soc 137:673–689

    Google Scholar 

  • Hudson D, Alves O, Hendon HH, Wang G (2011b) The impact of atmospheric initialisation on seasonal prediction of tropical Pacific SST. Clim Dyn 36:1155–1171

    Google Scholar 

  • Huffman GJ, Adler RF, Morrissey MM, Bolvin DT, Curtis S, Joyce R, McGavock B, Susskind J (2001) Global precipitation at one-degree daily resolution from multisatellite observations. J Hydrometeorol 2:36–50

    Google Scholar 

  • Hurrell JW, Kushnir Y, Ottersen G, Visbeck M (2003) An overview of the North Atlantic oscillation. Geophys Monogr Am Geophys Union 134:1–36

    Google Scholar 

  • Japan Meteorological Agency (2019) Outline of the operational numerical weather prediction at the Japan Meteorological Agency. Appendix to WMO Technical Progress Report on The Global Data-Processing and Forecasting SYSTEM (GDPFS) and Numerical Weather Prediction (NWP) Research, Japan Meteorological Agency, Tokyo, Japan, 229 pp. https://www.jma.go.jp/jma/jma-eng/jma-center/nwp/outline2019-nwp/pdf/outline2019_all.pdf. Accessed 10 Jan 2020

  • Karpechko AY, Charlton-Perez A, Balmaseda M, Tyrrell N, Vitart F (2018) Predicting sudden stratospheric warming 2018 and its climate impacts with a multimodel ensemble. Geophys Res Lett 45:13–538

    Google Scholar 

  • Kidston J et al (2015) Stratospheric influence on tropospheric jet streams, storm tracks and surface weather. Nat Geosci 8.6:433–440

    Google Scholar 

  • Klingaman NP, Young M, Chevuturi A, Guimarães BS, Guo L, Woolnough SJ, Coelho CAS, Kubota PY, Holloway CE (2020) Subseasonal prediction performance for austral summer South American rainfall. Weather Forecast. https://doi.org/10.1175/WAF-D-19-0203.1

    Article  Google Scholar 

  • Kobayashi C, Iwasaki T (2016) Brewer–Dobson circulation diagnosed from JRA‐55. J Geophys Res Atmos 121(4):1493–1510

    Google Scholar 

  • Li S, Robertson AW (2015) Evaluation of submonthly precipitation forecast skill from global ensemble prediction systems. Mon Weather Rev 143:2871–2889

    Google Scholar 

  • Liang P, Lin H (2018) Sub-seasonal prediction over East Asia during boreal summer using the ECCC monthly forecasting system. Clim Dyn 50:1007–1022

    Google Scholar 

  • Liebmann B, Smith CA (1996) Description of a complete (interpolated) outgoing longwave radiation dataset. Bull Am Meteor Soc 77(6):1275–1277

    Google Scholar 

  • Lim Y, Son SW, Kim D (2018) MJO prediction skill of the subseasonal-to‐seasonal prediction models. J Clim 31(10):4075–4094

    Google Scholar 

  • Lin H, Brunet G, Derome J (2008) Forecast skill of the Madden–Julian Oscillation in two Canadian atmospheric models. Mon Weather Rev 136:4130–4149

    Google Scholar 

  • Liu X, Yang S, Li Q, Kumar A, Weaver S, Liu S (2014) Subseasonal forecast skills and biases of global summer monsoons in the NCEP Climate Forecast System version 2. Clim Dyn 42:1487–1508

    Google Scholar 

  • Liu X, Wu T, Yang S, Li T, Jie W, Zhang L, Wang Z, Liang X, Li Q, Cheng Y, Ren H (2017) MJO prediction using the sub-seasonal to seasonal forecast model of Beijing Climate Center. Clim Dyn 48:3283–3307

    Google Scholar 

  • Madden RA, Julian PR (1971) Detection of a 40–50 day oscillation in the zonal wind in the tropical Pacific. J Atmos Sci 28:702–708

    Google Scholar 

  • Madden RA, Julian PR (1972) Description of global-scale circulation cells in the tropics with a 40–50 day period. J Atmos Sci 29:1109–1123

    Google Scholar 

  • Mastrangelo D, Malguzzi P, Rendina C, Drofa O, Buzzi A (2012) First outcomes from the CNR-ISAC monthly forecasting system. Adv Sci Res 8:77–82

    Google Scholar 

  • Matthews AJ, Hoskins BJ, Masutani M (2004) The global response to tropical heating in the Madden–Julian oscillation during the northern winter. Q J R Meteorol Soc 130(601):1991–2011

    Google Scholar 

  • Mendonça AM, Bonatti J (2009) Experiments with EOF-based perturbation methods and their impact on the CPTEC/INPE ensemble prediction system. Mon Weather Rev 137:1438–1459

    Google Scholar 

  • Newman M, Sardeshmukh PD, Winkler CR, Whitaker JS (2003) A study of subseasonal predictability. Mon Weather Rev 131(8):1715

    Google Scholar 

  • Nogués-Paegle J, Mo KC (1997) Alternating wet and dry conditions over South America during summer. Mon Weather Rev 125(2):279–291

    Google Scholar 

  • Pegion K et al (2019) The subseasonal experiment (SubX): a multimodel subseasonal prediction experiment. Bull Am Meteor Soc 100(10):2043–2060

    Google Scholar 

  • Rashid HA, Hendon HH, Wheeler MC, Alves O (2011) Prediction of the Madden–Julian Oscillation with the POAMA dynamical prediction system. Clim Dyn 36:649–661

    Google Scholar 

  • Sun S, Green BW, Bleck R, Benjamin SG (2018) Subseasonal forecasting with an icosahedral, vertically quasi-Lagrangian coupled model. Part II: Probabilistic and deterministic forecast skill. Mon Weather Rev 146(5):1619–1639

    Google Scholar 

  • Tyrlis E, Hoskins BJ (2008) Aspects of a Northern Hemisphere atmospheric blocking climatology. J Atmos Sci 65:1638–1652

    Google Scholar 

  • Vitart F (2004) Monthly forecasting at ECMWF. Mon Weather Rev 132:2761–2779

    Google Scholar 

  • Vitart F (2014) Evolution of ECMWF sub-seasonal forecast skill scores. Q J R Meteorol Soc 140:1889–1899

    Google Scholar 

  • Vitart F (2017) Madden–Julian Oscillation prediction and teleconnections in the S2S database. Q J R Meteorol Soc 143(706):2210–2220

    Google Scholar 

  • Vitart F. Ardilouze C, Bonet A, Brookshaw A, Chen M, Codorean C, Déqué M, Ferranti L, Fucile E, Fuentes M, Hendon H (2017) The subseasonal to seasonal (S2S) prediction project database. Bull Am Meteor Soc 98:163–173

    Google Scholar 

  • Vitart F et al (2008) The new VAREPS-monthly forecasting system: a first step towards seamless prediction. Q J R Meteorol Soc 134(636):1789–1799

    Google Scholar 

  • Weber NJ, Mass CF (2017) Evaluating CFSv2 subseasonal forecast skill with an emphasis on tropical convection. Mon Weather Rev 145:3795–3815

    Google Scholar 

  • Wheeler MC, Zhu H, Sobel AH, Hudson D, Vitart F (2017) Seamless precipitation prediction skill comparison between two global models. Q J R Meteorol Soc 143(702):374–383

    Google Scholar 

  • Wheeler MC, Hendon HH (2004) An all-season real‐time multivariate MJO index: development of an index for monitoring and prediction. Mon Weather Rev 132:1917–1932

    Google Scholar 

  • White CJ et al (2017) Potential applications of subseasonal-to‐seasonal (S2S) predictions. Meteorol Appl 24.3:315–325

    Google Scholar 

  • Woolnough SJ (2019) The Madden-Julian Oscillation. In: Robertson AW, Vitart F (eds) Sub-seasonal to seasonal prediction: the gap between weather and climate forecasting. Elsevier, Amsterdam, pp 93–117

    Google Scholar 

  • Yin Y, Alves O, Oke PR (2011) An ensemble ocean data assimilation system for seasonal prediction. Mon Weather Rev 139:786–808

    Google Scholar 

  • Zhu H, Wheeler MC, Sobel AH, Hudson D (2014) Seamless precipitation prediction skill in the tropics and extratropics from a global model. Mon Weather Rev 142:1556–1569

    Google Scholar 

  • Zuo H, Balmaseda MA, Tietsche S, Mogensen K, Mayer M (2019) The ECMWF operational ensemble reanalysis–analysis system for ocean and sea ice: a description of the system and assessment. Ocean Sci 15(3):779–808

    Google Scholar 

Download references

Acknowledgements

We thank Matthew Wheeler and an anonymous reviewer for providing valuable comments and suggestions that contributed for improving the quality of this manuscript. The first author was supported by Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq), Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES), process 88881.189179/2018-01, and University of Reading (ref GS18-179). CASC thanks CNPq, process 305206/2019-2, and Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP), process 2015/50687-8 (CLIMAX Project) for the support received. SJW was supported by the National Centre for Atmospheric Science ODA national capability programme ACREW (NE/R000034/1), which is supported by NERC and the GCRF. This research was partially supported by the Climate Science for Services Partnership Brazil project (CSSP-Brazil) funded by the Newton Fund. DCS was supported by CNPq (process 167804/2018-9) and CAPES.

Author information

Authors and Affiliations

  1. Center for Weather Forecast and Climate Studies, National Institute for Space Research, Rodovia presidente Dutra, km 39, Cachoeira Paulista, SP, 12630-000, Brazil

    Bruno dos Santos Guimarães, Caio Augusto dos Santos Coelho, Paulo Yoshio Kubota, Carlos Frederico Bastarz, Silvio Nilo Figueroa, José Paulo Bonatti & Dayana Castilho de Souza

  2. Department of Meteorology, National Centre for Atmospheric Science, University of Reading, Reading, Berkshire, UK

    Steven James Woolnough

Authors
  1. Bruno dos Santos Guimarães
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Caio Augusto dos Santos Coelho
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Steven James Woolnough
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Paulo Yoshio Kubota
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Carlos Frederico Bastarz
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Silvio Nilo Figueroa
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. José Paulo Bonatti
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Dayana Castilho de Souza
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bruno dos Santos Guimarães.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guimarães, B.S., Coelho, C.A.S., Woolnough, S.J. et al. An inter-comparison performance assessment of a Brazilian global sub-seasonal prediction model against four sub-seasonal to seasonal (S2S) prediction project models. Clim Dyn 56, 2359–2375 (2021). https://doi.org/10.1007/s00382-020-05589-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00382-020-05589-5

Keywords

Search

Navigation