Abstract
One of the major limitations of the classical ensemble Kalman filter (EnKF) is the assumption of a linear relationship between the state vector and the observed data. Thus, the classical EnKF algorithm can suffer from poor performance when considering highly non-linear and non-Gaussian likelihood models. In this paper, we have formulated the EnKF based on kernel-shrinkage regression techniques. This approach makes it possible to handle highly non-linear likelihood models efficiently. Moreover, a solution to the pre-image problem, essential in previously suggested EnKF schemes based on kernel methods, is not required. Testing the suggested procedure on a simple, illustrative problem with a non-linear likelihood model, we were able to obtain good results when the classical EnKF failed.
Article PDF
Similar content being viewed by others
Explore related subjects
Find the latest articles, discoveries, and news in related topics.Avoid common mistakes on your manuscript.
References
Aanonsen, S.I., Nævdal, G., Oliver, D.S., Reynolds, A.C., Vallès, B.: Ensemble Kalman filter in reservoir engineering—a review. SPE J. 14(3), 393–412 (2009)
Anderson, J.L.: A local least squares framework for ensemble filtering. Mon. Weather Rev. 131(4), 634–642 (2003)
Caers, J., Park, K.: Distance-based representation of reservoir uncertainty: the metric EnKF. In: Proceeding to the ECMOR XI, Sept 8–11. Bergen, Norway (2008)
Casella, G., Berger, R.L.: Statistical Inference. Duxbury (2002)
Chiles, J.-P., Delfiner, P.: Geostatistics, Modeling Spatial uncertainty. Wiley, New York (1999)
Doucet, A., Godsill, S., Andrieu, C.: On sequential Monte Carlo sampling methods for Bayesian filtering. Stat. Comput. 10(3), 197–208 (2000)
Evensen, G.: Sequential data assimilation with nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. J. Geophys. Res. 99, 10143–10162 (1994)
Evensen, G.: Data Assimilation. The ensemble Kalman filter. Springer, New York (2007)
Farrer, D.E., Glauber, R.R.: Multicollinearity in regression analysis: the problem revisited. Rev. Econ. Stat. 49(1), 92–107 (1967)
Furrer, R., Bengtsson, T.: Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants. J Multivariate Anal. 98(2), 227–255 (2007)
Gibbs, M.N.: Bayesian Gaussian Processes for Regression and Classification. Ph.D. thesis, University of Cambridge (1997)
Hastie, T., Tibshirani, R., Freidman, J.: The Elements of Statistical Learning; Data Mining, Inference, and Prediction, 2nd edn. Springer, New York (2009)
Helland, I.S.: Some theoretical aspects of partial least squares regression. Chemometr. Intell. Lab. Syst. 58(2), 97–107 (2001)
Hoerl, A.E., Kennard, R.W.: Ridge Regression: Biased estimation for nonorthogonal problems. Technometrics 12(3), 55–67 (1970)
Hofmann, T., Schölkopf, B., Smola, A.J.: Kernel methods in machine learning. Ann. Stat. 36(3), 1171–1220 (2008)
Hotelling, H.: Analysis of a complex of statistical variables into Principal Components. J. Educ. Psychol. 24(6), 417–441 and 498–520 (1933)
Houtekamer, P.L., Mitchell, H.L.: Data assimilation using an ensemble Kalman filter technique. Mon. Weather Rev. 126, 796–811 (1998)
Huang, T.M., Kecman, V., Kopriva, I.: Kernel based algorithms for mining huge data sets, supervised, semi-supervised, and unsupervised learning. Springer, New York (2006)
Journel, A.G., Huijbregts, C.J.: Mining Geostatistics. Academic, London (1978)
Kalivas, J.H.: Cyclic subspace regression with analysis of the hat matrix. Chemometr. Intell. Lab. Syst. 45, 215–224 (1999)
Kalman, R.E.: A new approach to linear filtering and prediction problems. Trans ASME J Basic Eng 82(Series D), 35–45 (1960)
Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks, Piscataway, NJ, pp. 1942–1948 (1995)
Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220(4598), 671–680 (1983)
Kwok, J.T., Tsang, I.W.: The pre-image problem in kernel methods. IEEE Trans. Neural Netw. 15(6), 1517–1525 (2004)
Mardia, K.V., Kent, J.T., Bibby, J.M.: Multivariate analysis. Academic, London (1979)
Mika, S., Schölkopf, B., Smola, A., Müller, R., Scholz, M., Rätch, G.: Advances in neural information processing systems 11, chapter kernel PCA and de-noising in feature spaces. Morgan Kaufmann, San Mateo (1998)
Myrseth, I.B., Omre, H.: Large-scale inverse problems and quantification of uncertainty, chapter the ensemble kalman filter and related filters. John Wiley and Sons (2010)
Myrseth, I.B., Sætrom, J., Omre, H.: Resampling the ensemble Kalman filter. Paper submitted for publication (2010)
Nocedal, J., Wright, S.: Numerical optimization. Springer, New York (2006)
O’Hagan, A.: Curve fitting and optimal design for prediction. J. R. Stat. Soc. B (Met.) 40(1), 1–42 (1978)
Oliver, D.: On conditional simulation to inaccurate data. Journal of Mathematical Geology 28(6), 811–817 (1996)
Rao, C.R.: Linear Statistical Inference and Its Applications, 2nd edn. Wiley, New York (1973)
Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press (2006)
Ränner, S., Lindgren, F., Gelandi, P., Wold, S.: A PLS kernel algorithm for data sets with many variables and fewer objects. Part 1: Theory and Algorithm. J. Chemometr. 8(2), 111–125 (1994)
Rosipal, R., Girolami, M., Trejo, L.J., Cichocki, A.: Kernel PCA for feature extraction and de-noising in non-Linear regression. Neural Comput Appl 10(3), 231–243 (2001)
Rosipal, R., Trejo, L.J.: Kernel Partial Least Squares regression in reproducing kernel Hilbert space. J Mach Learn Res 2, 97–123 (2002)
Sacher, W., Bartello, P.: Sampling errors in Ensemble Kalman filtering. Part I: Theory. Mon. Weather Rev. 136(8), 3035–3049 (2008)
Sarma, P., Chen, W.H.: Generalization of the Ensemble Kalman Filter using kernels for nongaussian Random Fields. In: Proceedings to the Reservoir Simulation Symposium, SPE, The Woodlands, Texas, 2–4 February 2009 (2009)
Sarma, P., Durlofsky, L., Aziz, K.: Kernel principal component analysis for an efficient, differentiable parametrization of multipoint geostatistics. Math. Geol. 40(1), 3–32 (2008)
Scales, J.A., Snieder, R.: To Bayes or not to Bayes? Geophysics 62(4), 1045–1046 (1997)
Schölkopf, B., Smola, A., Muller, K.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 10(5), 1299–1319 (1998)
Seber, G.A.F., Lee, A.J.: Linear Regression Analysis. Wiley, New York (2003)
Smola, A.: Learning with Kernels. Ph.D. thesis, TU Berlin (1998)
Smola, A., Schölkopf, B.: Learning with Kernels. Springer, New York (2002)
Smola, A., Schölkopf, B., Müller, K.: The connection between regularization operators and support vector kernels. Neural Networks 11(4), 637–649 (1998)
Sætrom, J., Omre, H.: Ensemble Kalman filtering with shrinkage regression techniques. Comput. Geosci. 15(2), 271–292 (2011)
Taylor, J.S., Cristianini, N.: Kernel methods for pattern analysis. Cambridge University Press, Cambridge (2004)
van Leeuwen, P.J.: Comments on “Data assimilation using an ensemble Kalman filter technique”. Mon. Weather Rev. 127, 1374–1377 (1999)
Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)
Williams, C.K.I.: Learning and inference in graphical models, chapter prediction with Gaussian processes: from linear regression to linear prediction and beyond. Kluwer, Norwell (1998)
Wold, H.: Quantitative sociology: international perspectives on mathematical and statistical model building, chapter path models with latent variables: the NiPALS Approach. pp. 307–357. Academic, London (1975)
Young, N.: An introduction to Hilbert Space. Cambridge University Press, Cambridge (1988)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Sætrom, J., Omre, H. Ensemble Kalman filtering for non-linear likelihood models using kernel-shrinkage regression techniques. Comput Geosci 15, 529–544 (2011). https://doi.org/10.1007/s10596-010-9222-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10596-010-9222-2