Improving the long-lead predictability of El Niño using a novel forecasting scheme based on a dynamic components model

Abstract

El Niño (EN) is a dominant feature of climate variability on inter-annual time scales driving changes in the climate throughout the globe, and having wide-spread natural and socio-economic consequences. In this sense, its forecast is an important task, and predictions are issued on a regular basis by a wide array of prediction schemes and climate centres around the world. This study explores a novel method for EN forecasting. In the state-of-the-art the advantageous statistical technique of unobserved components time series modeling, also known as structural time series modeling, has not been applied. Therefore, we have developed such a model where the statistical analysis, including parameter estimation and forecasting, is based on state space methods, and includes the celebrated Kalman filter. The distinguishing feature of this dynamic model is the decomposition of a time series into a range of stochastically time-varying components such as level (or trend), seasonal, cycles of different frequencies, irregular, and regression effects incorporated as explanatory covariates. These components are modeled separately and ultimately combined in a single forecasting scheme. Customary statistical models for EN prediction essentially use SST and wind stress in the equatorial Pacific. In addition to these, we introduce a new domain of regression variables accounting for the state of the subsurface ocean temperature in the western and central equatorial Pacific, motivated by our analysis, as well as by recent and classical research, showing that subsurface processes and heat accumulation there are fundamental for the genesis of EN. An important feature of the scheme is that different regression predictors are used at different lead months, thus capturing the dynamical evolution of the system and rendering more efficient forecasts. The new model has been tested with the prediction of all warm events that occurred in the period 1996–2015. Retrospective forecasts of these events were made for long lead times of at least two and a half years. Hence, the present study demonstrates that the theoretical limit of ENSO prediction should be sought much longer than the commonly accepted “Spring Barrier”. The high correspondence between the forecasts and observations indicates that the proposed model outperforms all current operational statistical models, and behaves comparably to the best dynamical models used for EN prediction. Thus, the novel way in which the modeling scheme has been structured could also be used for improving other statistical and dynamical modeling systems.

This is a preview of subscription content, log in to check access.

References

  1. An SI, Choi J (2009) Seasonal locking of the ENSO asymmetry and its influence on the seasonal cycle of the tropical eastern Pacific sea surface temperature. Atmos Res 94:3–9

    Article  Google Scholar 

  2. An SI, Jin FF (2004) Nonlinearity and asymmetry of ENSO. J Clim 17:2399–2412

    Article  Google Scholar 

  3. Ballester J, Bordoni S, Petrova D, Rodo X (2015) On the dynamical mechanism explaining the western pacific subsurface temperature buildup leading to ENSO events. Geophys Res Lett 42:2961–2967

    Article  Google Scholar 

  4. Ballester J, Bordoni S, Petrova D, Rodo X (2016) Heat advection processes leading to El Niño events as depicted by an ensemble of ocean assmiliation products. Submitted for review

  5. Ballester J, Rodriguez-Arias MA, Rodo X (2011) A new extratropical tracer describing the role of the Western Pacific in the onset of El Nñio: Implications for ENSO understanding and forecasting. J Clim 24:1425–1437

    Article  Google Scholar 

  6. Barnston A, Chelliah M, Goldenberg S (1997) Documentation of a highly ENSO-related SST region in the equatorial Pacific. Atmos Ocean 35:367–383

    Article  Google Scholar 

  7. Barnston A, Ropelewski C (1992) Prediction of ENSO episodes using canonical correlation analysis. J Clim 5:1316–1345

    Article  Google Scholar 

  8. Barnston A, Tippett M, L’Heureux M, Li S, DeWitt D (2012) Skill of real-time seasonal ENSO model predictions during 2002–11. Is our capability increasing? Am Meteorol Soc 93:631–651

    Article  Google Scholar 

  9. Bjerknes J (1969) Atmospheric teleconnections from the equatorial Pacific. Mon Weather Rev 97:163–172

    Article  Google Scholar 

  10. Brown J, Fedorov A (2010) How much energy is transferred from the winds to the thermocline on ENSO time scales? J Clim 23:1563–1580

    Article  Google Scholar 

  11. Chen D, Cane M, Kaplan A, Zebiak S, Huang D (2004) Predictability of El Niño over the past 148 years. Nature 428:15

    Article  Google Scholar 

  12. Chen D, Lian T, Fu C, Cane M, Tang Y, Murtugudde R, Song X, Wu Q, Zhou L (2015) Strong influence of westerly wind bursts on El Niño diversity. Nat Geosci 8:339–345

    Article  Google Scholar 

  13. CPC (2015) Cold and warm episodes by season. http://www.cpc.ncep.noaa.gov/products/analysis_monitoring/ensostuff/ensoyears.shtml

  14. de Jong P (1991) The diffuse Kalman Filter. Ann Stat 19:1073–1083

    Article  Google Scholar 

  15. Doornik JA (2013) Object-Oriented Matrix Programming using Ox 7.0. Timberlake Consultants Ltd, London. See http://www.doornik.com

  16. Durbin J, Koopman SJ (2012) Time series analysis by state space methods, 2nd edn. Oxford University Press, Oxford

    Google Scholar 

  17. Eisenman I, Yu L, Tziperman E (2005) Westerly wind bursts: ENSO’s tail rather than the dog? J Clim 18:5224–5238

    Article  Google Scholar 

  18. Fedorov A, Harper S, Philander SG, Winter B, Wittenberg A (2003) How predictable is El Niño? Bull Am Meteorol Soc 84:911–919

    Article  Google Scholar 

  19. Fedorov A, Philander SG (2001) A stability analysis of tropical ocean-atmosphere interactions: bridging measurements and theory for El Niño. J Clim 14:3086–3101

    Article  Google Scholar 

  20. Gebbie G, Tziperman E (2009) Predictability of SST-modulated westerly wind bursts. J Clim 22:3894–3909

    Article  Google Scholar 

  21. Ghil M, Allen MR, Dettinger MD, Ide K, Kondrashov D, Mann ME, Robertson AW, Saunders A, Tian Y, Varadi F, Yiou P (2002) Advanced spectral methods for climatic time series. Rev Geophys 40:3.1–3.41

    Article  Google Scholar 

  22. Gill A (1985) Elements of coupled ocean-atmosphere models for the tropics. Elsevier Oceanogr Ser 40:303–327

    Article  Google Scholar 

  23. Glantz MH (2015) Shades of chaos: lessons learned about lessons learned about forecasting El Niño and its impacts. Int J Disaster Risk Sci 6:94–103

    Article  Google Scholar 

  24. Good SA, Martin MJ, Rayner NA (2013) EN4: quality controlled ocean temperature and salinity profiles and monthly objective analyses with uncertainty estimates. J Geophys Res Oceans 118:6704–6716

    Article  Google Scholar 

  25. Goswami BN, Shukla J (1991) Predictability of a coupled ocean-atmosphere model. J Clim 4:3–22

    Article  Google Scholar 

  26. Hare S, Mantua N (2000) Empirical evidence for North Pacific regime shifts in 1977 and 1989. Prog Oceanogr 47:103–145

    Article  Google Scholar 

  27. Harvey A (1989) Forecasting, structural time series models and the Kalman filter. Cambridge University Press, Cambridge

    Google Scholar 

  28. Harvey A, Koopman SJ (2000) Signal extraction and the formulation of unobserved components models. Econom J 3:84–107

    Article  Google Scholar 

  29. Harvey A, Koopman SJ, Penzer J (1998) Messy time series. In: Fomby T, Hill RC (eds) Advances in econometrics, vol 13. JAI Press, New York

    Google Scholar 

  30. Harvey A, Shephard N (1993) Structural time series models. Handbook of statistics, 11. Elsevier, Amsterdam

    Google Scholar 

  31. Ishii M, Shouji A, Sugimoto S, Matsumoto T (2005) Objective analyses of SST and marine meteorological variables for the 20th century using COADS and the Kobe collection. Int J Climatol 25:865–879

    Article  Google Scholar 

  32. Izumo T, Lengaigne M, Vialard J, Luo J, Yamagata T, Madec G (2014) Influence of Indian Ocean Dipole and Pacific recharge on following year’s El Niño: interdecadal robustness. J Clim Dyn 42:291–310

    Article  Google Scholar 

  33. Jiang N, Neelin D, Ghill M (1993) Quasi-quadrennial and quasibiennial variability in COADS equatorial Pacific sea surface temperature and zonal wind. In: Proceedings of the 17th Climate Diagnostics Workshop. Climate Analysis Center, NOAA, pp 348–353

  34. Jiang N, Neelin D, Ghill M (1995) Quasi-quadrennial and quasi-biennial variability in the equatorial Pacific. J Clim Dyn 12:291–310

    Google Scholar 

  35. Jin FF (1997a) An equatorial ocean recharge paradigm for ENSO. Part I: conceptual model. J Atmos Sci 54:811–829

    Article  Google Scholar 

  36. Jin FF (1997b) An equatorial ocean recharge paradigm for ENSO. Part II: a stripped down coupled model. J Atmos Sci 54:830–847

    Article  Google Scholar 

  37. Jin FF, Kug JS, An S, Kang IS (2003) A near-annual coupled ocean-atmosphere mode in the equatorial Pacific. Geophys Res Lett 30

  38. Jin FF, Neelin JD, Ghil M (1994) El Niño on the devil’s staircase—annual subharmonic steps to chaos. Science 264:70–72

    Article  Google Scholar 

  39. Kalman RE (1960) A new approach to linear filtering and prediction problems. J Basic Eng Trans ASMA Ser D 82:35–45

    Article  Google Scholar 

  40. Kalnay E, Kanamitsu M, Kistler R, Collins W, Deaven D, Gandin L, Iredell M, Saha S, White G, Woollen J, Zhu Y, Leetmaa A, Reynolds R, Chelliah M, Ebisuzaki W, Higgins W, Janowiak J, Mo KC, Ropelewski C, Wang J, Jenne R, Joseph D (1996) The NCEP/NCAR 40-year reanalysis project. Bull Am Meteorol Soc 77:437–471

    Article  Google Scholar 

  41. Kang IS, Kug JS (2002) El Niño and La Niña sea surface temperature anomalies: asymmetry characteristics associated with their wind stress anomalies. J Geophys Res 107:4372

    Article  Google Scholar 

  42. Kleeman R, McCreary J, Klinger B (1999) A mechanism for generating ENSO decadal variability. Geophys Res Lett 26:1743–1746

    Article  Google Scholar 

  43. Kondrashov D, Kravtsov S, Robertson A, Ghil M (2005) A hierarchy of data-based ENSO models. J Clim 18:4425–4444

    Article  Google Scholar 

  44. Koopman SJ, Harvey AC, Doornik JA, Shephard N (2010) STAMP 8.3: structural time series analyser, modeller and predictor. Timberlake Consultants, London

    Google Scholar 

  45. Koopman SJ, Shephard N, Doornik JA (2008) Statistical algorithms for models in state space form: SsfPack 3.0. Timberlake Consultants Press, London

    Google Scholar 

  46. Krishnamurthy L, Vecchi G, Msadek R, Wittenberg A, Delworth T, Zeng F (2015) The seasonality of the great plains low-level jet and ENSO relationship. J Clim 28:4525–4544

    Article  Google Scholar 

  47. Lau K, Shen P (1988) Annual cycle, quasi-biennial oscillation and Southern Oscillation in global precipitation. J Geophys Res 93:10975–10988

    Article  Google Scholar 

  48. Mantua N, Battisti D (1995) Aperiodic variability in the Zebiak-Cane coupled ocean-atmosphere model: air-sea interactions in the western equatorial Pacific. J Clim 8:2897–2927

    Article  Google Scholar 

  49. McGregor S, Timmermann A, Shneider N, Stuecker M, England M (2012) The effect of the South Pacific Convergence Zone on the termination of El Niño events and the meridional asymmetry of ENSO. J Clim 25:5566–5586

    Article  Google Scholar 

  50. McPhaden M (2004) Evolution of the 2002/2003 El Niño. Am Meteorol Soc 85:677–695

    Article  Google Scholar 

  51. McPhaden M, Timmermann A, Widlansky M, Balmaseda M, Stockdale T (2014) The curious case of the El Niño that never happened: a perspective from 40 years of progress in climate research and forecasting. Bull Am Meteorol Soc

  52. McPhaden M, Yu X (1999) Equatorial waves and the 1997/98 El Niño. Geophys Res Lett 26:2961–2964

    Article  Google Scholar 

  53. Moron V, Plaut G (2003) The impact of El Niño-Southern Oscillation upon weather regimes over Europe and the North Atlantic during boreal winter. Int J Climatol 23:363–379

    Article  Google Scholar 

  54. Neelin J (1990) A hybrid coupled general circulation model for El Niño studies. J Atmos Sci 47:677–695

    Article  Google Scholar 

  55. Neelin J (1991) The slow surface temperature mode and the fast-wave limit: analytic theory for tropical interannual oscillations and experiments in a hybrid coupled model. J Atmos Sci 48:584–606

    Article  Google Scholar 

  56. Penland C (1996) A stochastic model of Indo-Pacific sea surface temperature anomalies. Phys D 98:534–558

    Article  Google Scholar 

  57. Penland C, Magorian T (1993) Prediction of Niño3 sea surface temperatures using linear inverse modeling. J Clim 6:1067–1076

    Article  Google Scholar 

  58. Philander S (1989) El Niño and La Niña. J Atmos Sci 77:451–459

    Google Scholar 

  59. Philander S, Yamagata T, Pacanowski R (1984) Unstable air-sea interactions in the tropics. J Atmos Sci 41:604–613

    Article  Google Scholar 

  60. Ramesh N, Murtugudde R (2013) All flavours of El Niño have similar early subsurface origins. Nat Clim Change 3:42–46

    Article  Google Scholar 

  61. Rasmusson E, Carpenter T (1982) Variations in tropical sea surface temperature and surface wind fields associated with the Southern Oscillation/El Niño. Mon Weather Rev 110:354–384

    Article  Google Scholar 

  62. Rasmusson E, Wang X, Ropelewski C (1990) The biennial component of ENSO variability. J Mar Syst 1:71–96

    Article  Google Scholar 

  63. Sarachik E, Cane M (2010) The El Niño Southern Oscillation Phenomenon. Cambridge University Press, Cambridge

    Google Scholar 

  64. Stein K, Timmermann A, Shcneider N (2011) Phase synchronization of El Niño–Southern Oscillation with the annual cycle. Phys Rev Lett 107

  65. Thompson CJ, Battisti DS (2000) A linear stochastic dynamical model of ENSO. Part I: model development. J Clim 13:2818–2832

    Article  Google Scholar 

  66. Thompson CJ, Battisti DS (2001) A linear stochastic dynamical model of ENSO. Part II: analysis. J Clim 14:445–466

    Article  Google Scholar 

  67. Tong H (1990) Non-linear time series. Oxford University Press, Oxford

    Google Scholar 

  68. Torrence C, Webster PJ (1998) The annual cycle of persistence in the El Niño–Southern Oscillation. Q J R Meteorol Soc 124:1985–2004

    Google Scholar 

  69. Trenberth K (1976) Spatial and temporal variations in the Southern Oscillation. Q J R Meteorol Soc 102:639–653

    Article  Google Scholar 

  70. Tziperman E, Stone L, Cane M, Jarosh H (1994) El Niño chaos: overlapping of resonances between the seasonal cycle and the Pacific ocean-atmosphere oscillator. Science 264:72–74

    Article  Google Scholar 

  71. Tziperman E, Yu L (2007) Quantifying the dependence of westerly wind bursts on the large-scale tropical Pacific SST. J Clim 20:2760–2768

    Article  Google Scholar 

  72. Tziperman E, Zebiak S, Cane M (1997) Mechanisms of seasonal—ENSO interaction. J Atmos Sci 54:61–71

    Article  Google Scholar 

  73. Wyrtki K (1975) El Niño—the dynamic response of the equatorial Pacific Ocean to atmospheric forcing. J Phys Oceanogr 5:572–584

    Article  Google Scholar 

  74. Wyrtki K (1985) Water displacements in the Pacific and the genesis of El Niño cycles. J Geophys Res 90:7129–7132

    Article  Google Scholar 

  75. Xue Y, Cane M, Zebiak S, Blumenthal M (1994) On the prediction of ENSO: a study with a low-order Markov model. Tellus A 46:512–528

    Article  Google Scholar 

  76. Yasunari T (1989) A possible link of the QBO’s between the stratosphere, troposphere and the surface temperature in the tropics. J Meteorol Soc Jpn 67

  77. Yeh S, Kirtman B (2004) The decadal ENSO variability in a hybrid coupled. J Am Meteorol Soc 17:1225–1238

    Google Scholar 

  78. Yeh S, Kirtman B (2005) Tropical Pacific decadal variability and ENSO amplitude modulation. Geophys Res Lett 32

  79. Yu J, Kao H, Lee T (2011) Subsurface ocean temperature indices for central-Pacific and eastern-Pacific types of El Niño and La Niña events. Theor Appl Climatol 103:337–344

    Article  Google Scholar 

  80. Yu J, Mechoso C (2001) A coupled atmosphere-ocean GCM study of the ENSO cycle. J Clim 14:2329–2350

    Article  Google Scholar 

  81. Yu X, McPhaden M (1999) Seasonal variability in the equatorial Pacific. J Phys Oceanogr 29:925–947

    Article  Google Scholar 

  82. Zebiak S (1985) Tropical atmosphere-ocean interaction and the El Niño/Southern Oscillation phenomenon. PhD Thesis, Massachusetts Institute of Technology

  83. Zebiak S (1989) Oceanic heat content variability and El Niño cycles. J Phys Oceanogr 19:475–486

    Article  Google Scholar 

  84. Zebiak SE, Cane MA (1987) A model El Niño-Southern Oscillation. Mon Weather Rev 115:2262–2278

    Article  Google Scholar 

Download references

Acknowledgments

J.B. gratefully acknowledges funding from the European Commission through a Marie Curie International Outgoing Fellowship (Project MEMENTO from the FP7-PEOPLE-2011-IOF call), and from the European Commission and the Catalan Government through a Marie Curie—Beatriu de Pinós Fellowship (Project 00068 from the BP-DGR-2014-B call). X.R. gratefully acknowledges funding from the Ministry of Science and Innovation, Spain (Project PANDORA CGL 2007-63053).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Desislava Petrova.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 251 KB)

Appendices

Appendix 1

The linear Gaussian state space model form that we have used is as in de Jong (1991), that is:

$$\begin{aligned} y_t =Z_t\alpha _t+G_t\epsilon _t,\quad \alpha _{t+1} = T_t\alpha _t+H_t\epsilon _t,\quad \epsilon _t\sim {\text{NID}}\left( 0,I\right) , \end{aligned}$$
(6)

for \(t=1,\ldots ,n,\) and where \(\epsilon _t\) is a vector of serially independent disturbance series. The \(m\times 1\) state vector \(\alpha _t\) contains the unobserved components and their associated variables.

$$\begin{aligned} & \alpha _t = \left( \mu _t , \gamma _{t}, \gamma _{t-1},\ldots ,\gamma _{t-10},\psi _{1t} , \psi ^+ _{1t}, \psi _{2t} , \psi ^+ _{2t}, \psi _{3t} , \psi ^+ _{3t} , \delta '\right) ^{\prime }, \\ & \epsilon _t = \left( \varepsilon _{t} , \eta _t , \omega _{t}, \kappa _{1t} , \kappa _{1t} ^+ , \kappa _{2t} , \kappa _{2t} ^+ , \kappa _{3t} , \kappa _{3t} ^+\right) ^{\prime }, \end{aligned}$$

The measurement equation is the first equation in (6) and it relates the observation \(y_t\) to the state vector \(\alpha _t\) through the signal \(Z_t \alpha _t\). The transition equation is the second equation in (6) and it is used to formulate the dynamic processes of the unobserved components in a companion form. The deterministic matrices \(T_t\), \(Z_t\), \(H_t\) and \(G_t\), are time-invariant except the matrix \(Z_t\), and referred to as system matrices that are sparse and known:

$$\begin{aligned} T_t=\, & {} \left[ {\begin{array}{ccccccccccccccccccc} 1 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0\\ 0 &{} p(1) &{} q(1) &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0\\ 0 &{} -q(1) &{} p(1) &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} 0 &{} p(2) &{} q(2) &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} 0 &{} -q(2) &{} p(2) &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} p(3) &{} q(3) &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} -q(3) &{} p(3) &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} p(4) &{} q(4) &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} -q(4) &{} p(4) &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} p(5) &{} q(5) &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} -q(5) &{} p(5) &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} -1 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} c(1) &{} s(1) &{} 0 &{} 0 &{} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} -s(1) &{} c(1) &{} 0 &{} 0 &{} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} c(2) &{} s(2) &{} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} -s(2) &{} c(2) &{} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} c(3) &{} s(3) &{} 0\\ 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} -s(3) &{} c(3) &{} 0\\ 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} I_k \end{array}} \right] ,\\ H_t=\, & {} \left[ { \begin{array}{ccccccccc} 0 &{} \sigma _{\eta } &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} \sigma _{\omega } &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} 0 &{} \sigma _{\kappa ,1} &{} 0 &{} 0 &{} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} 0 &{} 0 &{} \sigma _{\kappa ,1} &{} 0 &{} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} \sigma _{\kappa ,2} &{} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} \sigma _{\kappa ,2} &{} 0 &{} 0\\ 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} \sigma _{\kappa ,3} &{} 0\\ 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} \sigma _{\kappa ,3}\\ 0_k &{} 0_k &{} 0_k &{} 0_k &{} 0_k &{} 0_k &{} 0_k &{} 0_k &{} 0_k \end{array}} \right] , \\ Z_t=\, & {} \left( { \begin{array}{ccccccccccccccccccc} 1 &{} 1 &{} 0 &{} 1 &{} 0 &{} 1 &{} 0 &{} 1 &{} 0 &{} 1 &{} 0 &{} 1 &{} 1 &{} 0 &{} 1 &{} 0 &{} 1 &{} 0 &{} x_t' \\ \end{array}} \right) ,\\ G_t=\, & {} \left( { \begin{array}{ccccccccc} \sigma _{\varepsilon } &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0 &{} 0\\ \end{array}} \right) , \end{aligned}$$

where \(0_k\) is a \(k\times 1\) vector of zeros, \(I_k\) is a \(k\times 1\) vector of ones, \(p(i) = \cos \lambda _{i}\) and \(q(i) = \sin \lambda _{i}\) for \(\lambda _{i}=\frac{2\pi i}{S}\), \(\hbox {i} = 1,2,\ldots ,\lfloor \frac{S}{2} \rfloor\); \(c(j) = \varphi _{\psi ,j} \cos \lambda _{c,j}\) and \(s(j) = \varphi _{\psi ,j} \sin \lambda _{c,j}\) for \(j=1,2,3\).

Appendix 2

Every explanatory regression variable that has been used in the analysis (surface temperature, subsurface temperature at different depth levels and regions, and zonal wind stress at different regions) has been tested separately with the model described in Sect. 3 during the fitting procedure. In this way each variable was fitted at every lag time between 0 and 35 months. Based on the in-sample estimations for each of these fittings (the sample spanned the data from January 1982 to December 2012), the p value (an indicator of statistical significance, showing the probability that the coefficient of the predictor is equal to zero, and thus that the predictor does not add value), the significance level (SL) based on it (\(\hbox {SL}=90\,\%\) for \(p\le 0.10\), \(\hbox {SL}=95\,\%\) for \(p\le 0.05\), \(\hbox {SL}=99\,\%\) for \(p\le 0.01\)), and the \(Rs^2\) value (modified coefficient of determination based on seasonal means—the ratio between the variance explained by the model and the variance of the seasonally differenced time series), were used to determine significance of the regression variable at the respective lag time. The significant values obtained in this way are summarized in the following Tables 3, 4, 5 and 6.

Table 3 Diagnostics of wind stress predictor regression variables
Table 4 Diagnostics of surface and subsurface temperature predictor regression variables
Table 5 Diagnostics of subsurface temperature predictor regression variables
Table 6 Diagnostics of subsurface temperature predictor regression variables
Table 7 Predictor regression variables added to the model at lead times between 17 and 36 months, based on the criteria and results shown in Tables 36
Table 8 Predictor regression variables added to the model at lead times between 0 and 15 months, based on the criteria and results shown in Tables 36
Table 9 Predictions of the January target month for all EN events shown in Fig. 9
Table 10 Correlation between forecasts and observations, and root mean square error (RMSE) as functions of lead month
Fig. 1
figure1

Components graphics of the model. Shown are temperature (\(^\circ\)C) time series of the a level and regression components together with the N3.4 index, b the seasonal component, and the three cycle components of periods: c 1.5, d 2.5, e 4.5 years

Fig. 2
figure2

a Multi Taper Method (MTM) power spectra for the observed N3.4 time series. The solid line indicates the power density and dashed lines the respective confidence level (CL) based on a red noise null hypothesis. The red indicators correspond to the near-annual, biannual, and quasi-quadrennial ENSO modes of variability. Reconstructed components from the multitaper decomposition in a, corresponding to the b seasonal, c near-annual, d biannual, e quasi-quadrennial modes

Fig. 3
figure3

Pearson correlations between the temporal scores of the first CEOF modes of filtered SSTs and surface wind stress anomalies, and filtered spatio-temporal SST anomalies and wind stress anomalies in the equatorial Pacific region. A Butterworth filter has been applied to the SST and wind stress data sets, so that only frequencies corresponding to periods between 14 and 18 months (associated with the near-annual mode of variability) have been kept. Panels correspond to the respective phases of the CEOF shown on the figure. Shaded areas indicate significant anomalies

Fig. 4
figure4

Same as Fig. 3, but the Butterworth filter has been applied so that only frequencies corresponding to periods between 24 and 28 months (associated with the biannual mode of variability) have been kept

Fig. 5
figure5

Same as Fig. 3, but the Butterworth filter has been applied so that only frequencies corresponding to periods between 46 and 63 months (associated with the low-frequency mode of variability) have been kept

Fig. 6
figure6

Composites of surface zonal wind stress (Nm\(^{-2}\), arrows) anomalies with respect to all EN events in the period 1978–2012. Shown are anomalies a 24, b 13, c 7 months before the winter peak of EN. The red boxes indicate the three zonal wind stress regions from Table 1a Region I, b Region II, c Region III

Fig. 7
figure7

Composites of subsurface temperature (\(^{\circ }\)C, shading) anomalies at a 150, b 250, c 400 m depth with respect to all EN events in the period 1978–2012 in Region I (see Table 2). Data is filtered using a low-pass Butterworth filter (cut-off frequency 18, order 10)

Fig. 8
figure8

Same as Fig. 7, but at a surface, b 100, c 200 m depth in Region II (see Table 2)

Fig. 9
figure9

Time series of area-averaged sea surface temperature (\(^\circ\)C) anomalies in the Niño 3.4 region. Shown are forecasts of the a, f, k 1997/98, b, g, l 2002/03, c, h, m 2006/07, d, i, n 2009/10, and e, j, o 2014/15 EN events, starting 29–34 (magenta in a and d), 27–28 (light blue in be), 24–26 (dark green in ac and e), 21–22 (beige in ae), 17–19 (red in fj), 13–16 (blue in fj), 11–12 (green in fj), 8–9 (velvet in ko), 6 (dark blue in ko), and 3–5 (dark yellow in ko) months before the peak of El Niño, respectively. Vertical dotted lines indicate the month in which the respective forecasts are started. Observations are in black

Fig. 10
figure10

a Retrospective forecast of the EN3.4 time series in the period 1983–2014. The EN3.4 observation is in red and the model prediction at 6 months lead time is in blue. Scatterplots of the EN3.4 time series observation against forecast at b 3, c 6, d 18 months lead time. The respective regression coefficients are 0.70, 0.45 and 0.30

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Petrova, D., Koopman, S.J., Ballester, J. et al. Improving the long-lead predictability of El Niño using a novel forecasting scheme based on a dynamic components model. Clim Dyn 48, 1249–1276 (2017). https://doi.org/10.1007/s00382-016-3139-y

Download citation

Keywords

  • El Niño Southern Oscillation
  • Prediction
  • Predictability
  • Subsurface dynamics
  • Time series