Abstract
A statistical learning method called random forests is applied to the prediction of transitions between weather regimes of wintertime Northern Hemisphere (NH) atmospheric low-frequency variability. A dataset composed of 55 winters of NH 700-mb geopotential height anomalies is used in the present study. A mixture model finds that the three Gaussian components that were statistically significant in earlier work are robust; they are the Pacific–North American (PNA) regime, its approximate reverse (the reverse PNA, or RNA), and the blocked phase of the North Atlantic Oscillation (BNAO). The most significant and robust transitions in the Markov chain generated by these regimes are PNA → BNAO, PNA → RNA and BNAO → PNA. The break of a regime and subsequent onset of another one is forecast for these three transitions. Taking the relative costs of false positives and false negatives into account, the random-forests method shows useful forecasting skill. The calculations are carried out in the phase space spanned by a few leading empirical orthogonal functions of dataset variability. Plots of estimated response functions to a given predictor confirm the crucial influence of the exit angle on a preferred transition path. This result points to the dynamic origin of the transitions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Breiman L (2001) Random forests. Mach Learn 45:5–32
Breiman L, Friedman J, Olshen R, Stone C (1984) Classification and regression trees. Wadsworth Press, Monterey, 368 pp
Chang KI, Ghil M, Ide K, Lai CCA (2001) Transition to aperiodic variability in a wind-driven double-gyre circulation model. J Phys Oceanogr 31:1260–1286
Cheng XH, Wallace JM (1993) Cluster-analysis of the Northern-Hemisphere wintertime 500-hPa height field spatial patterns. J Atmos Sci 50: 2674–2696
Crommelin DT (2002) Homoclinic dynamics: a scenario for atmospheric ultra-low-frequency variability. J Atmos Sci 59:1533–1549
Crommelin DT (2003) Regime transitions and heteroclinic connections in a barotropic atmosphere. J Atmos Sci 60:229–246
Crommelin DT (2004) Observed nondiffusive dynamics in large-scale atmospheric flow. J Atmos Sci 61:2384–2396
D’Andrea F, Vautard R (2001) Extratropical low-frequency variability as a low-dimensional problem. Part I: A simplified model. Q J R Meteorol Soc 127:1357–1374
D’Andrea F, Vautard R (2002) Extratropical low-frequency variability as a low-dimensional problem. Part II: Stationarity and stability of large-scale equilibria. Q J R Meteorol Soc 128:1059–1073
D’Andrea F, Tibaldi S, Blackburn M, et al. (1998) Northern Hemisphere atmospheric blocking as simulated by 15 atmospheric general circulation models in the period 1979–1988. Clim Dyn 14:385–407
Deloncle A, Berk R, D’Andrea F, Ghil M (2007) Weather regime prediction using statistical learning. J Atmos Sci 64:1619–1635
Fraedrich K (1988) El Niño-Southern Oscillation predictability. Mon Wea Rev 116:1001–1012
Franzke C, Majda AJ, Vanden-Eijnden E (2005) Low-order stochastic mode reduction for a realistic barotropic model climate. J Atmos Sci 62:1722–1745
Ghil M (1987) Dynamics, statistics and predictability of planetary flow regimes. In: Nicolis C, Nicolis G (eds) Irreversible phenomena and dynamical systems analysis in the geosciences. D. Reidel, Dordrecht, pp 241–283
Ghil M, Childress S (1987) Topics in geophysical fluid dynamics: atmospheric dynamics, dynamo theory and climate dynamics. Springer, New York, 485 pp
Ghil M, Robertson AW (2002) “Waves” vs. “particles” in the atmosphere’s phase space: a pathway to long-range forecasting? Proc Natl Acad Sci 99(Suppl. 1):2493–2500
Ghil, M, Kimoto M, Neelin JD (1991) Nonlinear dynamics and predictability in the atmospheric sciences. Rev Geophys, Supplement (U.S. Nat’l Rept. to Int’l Union of Geodesy & Geophys 1987–1990), 29(S):46–55, 10.1029/91RG0071
Guckenheimer J, Holmes P (1983) Nonlinear oscillations, dynamical systems, and bifurcations of vector fields. Springer, New York, 453 pp
Hastie T, Tibshirani R, Friedman J (2001) The elements of statistical learning. Springer, New York, 552 pp
Kalnay E et al (1996) The NCEP/NCAR 40-year reanalysis project. Bull Am Meteorol Soc 77:437–471
Kimoto M, Ghil M (1993a) Multiple flow regimes in the Northern Hemisphere winter. Part I: Methodology and hemispheric regimes. J Atmos Sci 50:2625–2643
Kimoto M, Ghil M (1993b) Multiple flow regimes in the Northern Hemisphere winter. Part II: Sectorial regimes and preferred transitions. J Atmos Sci 50:2645–2673
Kimoto M, Mukougawa H, Yoden S (1992) Medium-range forecast skill variation and blocking transition: a case study. Mon Weather Rev 120:1616–1627
Kondrashov D, Ide K, Ghil M (2004) Weather regimes and preferred transition paths in a three-level quasi-gesotrophic model. J Atmos Sci 61:568–587
Kondrashov D, Kravtsov S, Ghil M (2005) A hierarchy of data-based ENSO models. J Clim 18:4425–4444
Kondrashov D, Kravtsov S, Ghil M (2006) Empirical mode reduction in a model of extratropical low-frequency variability. J Atmos Sci 63:1859–1877
Kravtsov S, Kondrashov D, Ghil M (2005) Multiple regression modeling of nonlinear processes: derivation and applications to climatic variability. J Clim 18:4404–4424
Legras B, Ghil M (1985) Persistent anomalies, blocking and variations in atmospheric predictability. J Atmos Sci 42:433–471
Lorenz EN (1963) Deterministic nonperiodic flow. J Atmos Sci 20:130–141
Majda AJ, Timofeyev I, Vanden-Eijnden E (2003) Systematic strategies for stochastic mode reduction in climate. J Atmos Sci 60:1705–1722
Marshall J, Molteni F (1993) Towards a dynamical understanding of planetary-scale flow regimes. J Atmos Sci 50:1792–1818
Meacham SP (2000) Low-frequency variability in the wind-driven circulation. J Phys Oceanogr 30:269–293
Mo K, Ghil M (1987) Statistics and dynamics of persistent anomalies. J Atmos Sci 44:877–901
Mo K, Ghil M (1988) Cluster analysis of multiple planetary flow regimes. J Geophys Res 93D:10927–10952
Molteni F, Tibaldi S, Palmer TN (1990) Regimes in the wintertime circulation over northern extratropics. 1. observational evidence. Q J R Meteorol Soc 116:31–67
Monahan AH, Pandolfo L, Fyfe JC (2001) The preferred structure of variability of the Northern Hemisphere atmospheric circulation. Geophys Res Lett 28:1019–1022
Nadiga BT, Luce BP (2001) Global bifurcation of Shilnikov type in a double-gyre ocean model. J Phys Oceanogr 31:2669–2690
Namias J (1953) Thirty-day forecasting: A review of a ten-year experiment. Meteorol Monogr 2(6):83
Namias J (1968) Long-range weather forecasting – history, current status and outlook. Bull Am Meteorol Soc 49:438–470
Palmer TN, Doblas-Reyes FJ, Weisheimer A, Rodwell M (2007) Towards seamless prediction: calibration of climate-change projections using seasonal forecasts. Bull Am Meteorol Soc (submitted)
Pasmanter RA, Timmermann A (2003) Cyclic Markov chains with an application to an intermediate ENSO model. Nonlinear Proc Geophys 10:197–210
Pelly JL, Hoskins BJ (2003) How well does the ECMWF Ensemble Prediction System predict blocking. Q J R Meteorol Soc 129:1683–1702
Roulston MS, Smith LA (2004) The boy who cried wolf revisited: the impact of false alarm intolerance on cost-loss scenarios. Weather Forecast 19(2):391–397
Selten FM, Branstator G (2004) Preferred regime transition routes and evidence for an unstable periodic orbit in a baroclinic model. J Atmos Sci 61:2267–2282
Silverman BW (1986) Density estimation for statistics and data analysis. Chapman and Hall, London, 175 pp
Simonnet E, Ghil M, Ide K, Temam R, Wang S (2003a) Low-frequency variability in shallow-water models of the wind-driven ocean circulation. Part I: Steady-state solutions. J Phys Oceanogr 33:712–728
Simonnet E, Ghil M, Ide K, Temam R, Wang S (2003b) Low-frequency variability in shallow-water models of the wind-driven ocean circulation. Part II: Time-dependent solutions. J Phys Oceanogr 33:729–752
Simonnet E, Ghil M, Dijkstra HA (2005) Homoclinic bifurcations in the quasi-geostrophic double-gyre circulation. J Mar Res 63:931–956
Smyth P, Ide K, Ghil M (1999) Multiple regimes in Northern Hemisphere height fields via mixture model clustering. J Atmos Sci 56:3704–3723
Strong CM, Jin Ff, Ghil M (1995) Intraseasonal oscillations in a barotropic model with annual cycle, and their predictability. J Atmos Sci 52:2627–2642
Trevisan A (1995) Statistical properties of predictability from atmospheric analogs and the existence of multiple flow regimes. J Atmos Sci 52:3577–3592
Vannitsem S (2001) Toward a phase-space cartography of the short- and medium-range predictability of weather regimes. Tellus 53A:56–73
Vautard R, Mo K, Ghil M (1990) Statistical significance test for transition matrices of atmospheric Markov chains. J Atmos Sci 47:1926–1931
Von Storch H, Zwiers S (1999) Statistical analysis in climate research. Cambridge University Press, London, 484 pp
Weeks ER, Tian Y, Urbach JS, Ide K, Swinney HL, Ghil M (1997) Transitions between blocked and zonal flows in a rotating annulus with topography. Science 278:1598–1601
Winkler CR, Newman M, Sardeshmukh P (2001) A linear model of wintertime low-frequency variability. Part I: formulation and forecast skill. J Clim 14:4474–4494
Acknowledgments
We are grateful to Padhraic J. Smyth, who provided the code for the Gaussian mixture model, and to four anonymous reviewers, whose constructive comments greatly improved the presentation. Ann Henderson-Sellers attracted our attention to the work of Palmer et al. (2007), and Tim Palmer provided a preprint thereof. Our study was supported by DOE grant ER63251 (MG and DK) and by NSF grant SES04-37169 (RB and JS).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kondrashov, D., Shen, J., Berk, R. et al. Predicting weather regime transitions in Northern Hemisphere datasets. Clim Dyn 29, 535–551 (2007). https://doi.org/10.1007/s00382-007-0293-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00382-007-0293-2