Abstract
While neural network models have provided nonlinear generalizations of classical linear multivariate models (e.g. regression, principal component and canonical correlation analyses), their applications to the analysis and prediction of real environmental and climate data are not always successful as many of the datasets are very noisy and/or contain relatively few independent observations. We review recent efforts directed towards making the nonlinear models more robust – the development of (1) an information criterion to alleviate overfitting in nonlinear principal component analysis, and (2) a robust version of nonlinear canonical correlation analysis. We also discuss two common causes undermining nonlinear models relative to linear models: (1) Time-averaging of data (e.g. from daily data to seasonal data) linearizes the relation between predictor and predictand due to the central limit theorem. (2) When new predictor data lies outside the training range, the nonlinear model may extrapolate poorly, thereby decreasing its forecast skills.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bishop, C.: Neural Networks for Pattern Recognition. Clarendon Press, Oxford (1995)
Hsieh, W.: Nonlinear multivariate and time series analysis by neural network methods. Reviews of Geophysics 42 (2004) RG1003, doi:10.1029/2002RG000112
Kramer, M.: Nonlinear principal component analysis using autoassociative neural networks. AIChE Journal 37 (1991) 233–243
Hsieh, W.: Nonlinear principal component analysis by neural networks. Tellus 53A (2001) 599–615
Schölkopf, B., Smola, A., Muller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10 (1998) 1299–1319
Hastie, T., Stuetzle, W.: Principal curves. Journal of the American Statistical Association 84 (1989) 502–516
Roweis, S., Saul, L.: Nonlinear dimensionality reduction by locally linear embedding}. Science 290 (2000) 2323–2326
Tenenbaum, J., de Silva, V., Langford, J.: A global geometric framework for nonlinear dimensionality reduction. Science 290 (2000) 2319–2323
Kohonen, T.: Self-organzing formation of topologically correct feature maps. Biological Cybernetics 43 (1982) 59–69
Kwok, J.T.Y., Tsang, I.W.H.: The pre-image problem in kernel methods. IEEE Transactions on Neural Networks 15 (2004) 1517–1525
Lai, P., Fyfe, C.: A neural implementation of canonical correlation analysis. Neural Networks 12 (1999) 1391–1397
Hsieh, W.: Nonlinear canonical correlation analysis by neural networks. Neural Networks 13 (2000) 1095–1105
Lai, P., Fyfe, F.: Kernel and non-linear canonical correlation analysis. International Journal of Neural Systems 10 (2000) 365–377
Suykens, J., Van Gestel, T., De Brabanter, J., De Moor, B., Vandewalle, J.: Least Squares Support Vector machines. World Scientific, New Jersey (2002)
Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)
MacKay, D.: Bayesian interpolation. Neural Computation 4 (1992) 415–447
Foresee, F., Hagan, M.: Gauss-Newton approximation to Bayesian regularization. In: Proceedings of the 1997 International Joint Conference on Neural Networks 3 (1997) 1930–1936
Hsieh, W.: Nonlinear principal component analysis of noisy data. Neural Networks 20 (2007) 434–443
Christiansen, B.: The shortcomings of nonlinear principal component analysis in identifying circulation regimes. Journal of Climate 18(22) (2005) 4814–4823.
Christiansen, B.: Reply to Monahan and Fyfe’s comment on “The shortcomings of nonlinear principal component analysis in identifying circulation regimes". Journal of Climate 20 (2007) 378–379
Malthouse, E.: Limitations of nonlinear PCA as performed with generic neural networks. IEEE Transactions on Neural Networks 9 (1998) 165–173
Akaike, H.: A new look at the statistical model identification. IEEE Transactions on Automatic Control AC-19 (1974) 716–723
Schwarz, G.: Estimating the dimension of a model. Annals of Statistics 6 (1978) 461–464
Monahan, A.: Nonlinear principal component analysis: Tropical Indo-Pacific sea surface temperature and sea level pressure. Journal of Climate 14 (2001) 219–233
Kirby, M., Miranda, R.: Circular nodes in neural networks. Neural Computation 8 (1996) 390–402
Hsieh, W., Wu, A.: Nonlinear multichannel singular spectrum analysis of the tropical Pacific climate variability using a neural network approach. Journal of Geophysical Research 107 (2002) DOI:10.1029/2001JC000957
Hsieh, W.: Nonlinear canonical correlation analysis of the tropical Pacific climate variability using a neural network approach. Journal of Climate 14 (2001) 2528–2539
Wu, A., Hsieh, W.: Nonlinear canonical correlation analysis of the tropical Pacific wind stress and sea surface temperature. Climate Dynamics 19 (2002) 713–722. DOI:10.1007/s00382–002–0262–8
Wu, A., Hsieh, W., Zwiers, F.: Nonlinear modes of North American winter climate variability detected from a general circulation model. Journal of Climate 16 (2003) 2325–2339
Wilcox, R.: Robust Estimation and Hypothesis Testing. Elsevier, Amsterdam (2004)
Cannon, A., Hsieh, W.: Robust nonlinear canonical correlation analysis: Application to seasonal climate forecasting. Nonlinear Processes in Geophysics 12 (2008) 221–232
Kalnay, E., et al.: The NCEP/NCAR 40-year reanalysis project. Bulletin of the American Meteorological Society 77 (1996) 437–471
Yuval, Hsieh, W.: The impact of time-averaging on the detectability of nonlinear empirical relations. Quarterly Journal of the Royal Meterological Society 128 (2002) 1609–1622
Bickel, P., Doksum, K.: Mathematical Statistics: Basic Ideas and Selected Topics. Holden-Day, Oakland (1977)
Wu, A., Hsieh, W., Cannon, A., Shabbar, A.: Improving neural network predictions of North American seasonal climate by outlier correction. (Nonlinear Processes in Geophysics (submitted))
von Storch, H., Zwiers, F.: Statistical Analysis in Climate Research. Cambridge University Press, Cambridge (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Hsieh, W.W., Cannon, A.J. (2008). Towards Robust Nonlinear Multivariate Analysis by Neural Network Methods. In: Donner, R.V., Barbosa, S.M. (eds) Nonlinear Time Series Analysis in the Geosciences. Lecture Notes in Earth Sciences, vol 112. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-78938-3_6
Download citation
DOI: https://doi.org/10.1007/978-3-540-78938-3_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-78937-6
Online ISBN: 978-3-540-78938-3
eBook Packages: Earth and Environmental ScienceEarth and Environmental Science (R0)