Linear Operators and Stochastic Partial Differential Equations in Gaussian Process Regression

  • Simo Särkkä
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6792)


In this paper we shall discuss an extension to Gaussian process (GP) regression models, where the measurements are modeled as linear functionals of the underlying GP and the estimation objective is a general linear operator of the process. We shall show how this framework can be used for modeling physical processes involved in measurement of the GP and for encoding physical prior information into regression models in form of stochastic partial differential equations (SPDE). We shall also illustrate the practical applicability of the theory in a simulated application.


Gaussian process regression linear operator stochastic partial differential equation inverse problem 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    O’Hagan, A.: Curve fitting and optimal design for prediction (with discussion). Journal of the Royal Statistical Society B 40(1), 1–42 (1978)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)zbMATHGoogle Scholar
  3. 3.
    Akhiezer, N.I., Glazman, I.M.: Theory of Linear Operators in Hilbert Space. Dover (1993)Google Scholar
  4. 4.
    Carlson, A.B.: Communication Systems: An Introduction to Signals and Noise in Electrical Communication, 3rd edn. McGraw-Hill, New York (1986)Google Scholar
  5. 5.
    Hayes, M.H.: Statistical Digital Signal Processing and Modeling. John Wiley & Sons, Chichester (1996)Google Scholar
  6. 6.
    Gonzalez, R.C., Woods, R.E.: Digital Image Processing. Prentice-Hall, Englewood Cliffs (2002)Google Scholar
  7. 7.
    Guenther, R.B., Lee, J.W.: Partial Differential Equations of Mathematical Physics and Integral Equations. Dover, New York (1988)Google Scholar
  8. 8.
    Griffiths, D.J.: Introduction to Electrodynamics, 3rd edn. Pearson, London (2008)Google Scholar
  9. 9.
    Holden, H., Øksendal, B., Ubøe, J., Zhang, T. (eds.): Stochastic Partial Differential Equations: A Modeling, White Noise Functional Approach. Birkhäuser, Basel (1996)zbMATHGoogle Scholar
  10. 10.
    Whittle, P.: On stationary processes in the plane. Biometrica 41(3/4), 434–449 (1954)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Matérn, B.: Spatial variation. Technical report, Meddelanden från Statens Skogforskningsinstitut, Band 49 - Nr 5 (1960)Google Scholar
  12. 12.
    Christakos, G.: Random Field Models in Earth Sciences. Academic Press, London (1992)zbMATHGoogle Scholar
  13. 13.
    Cressie, N.A.C.: Statistics for Spatial Data. Wiley, Chichester (1993)zbMATHGoogle Scholar
  14. 14.
    Gelfand, A.E., Diggle, P.J., Fuentes, M., Guttorp, P.: Handbook of Spatial Statistics. Chapman & Hall/CRC (2010)Google Scholar
  15. 15.
    Jain, A.K.: Partial differential equations and finite-difference methods in image processing, part 1: Image representation. Journal of Optimization Theory and Applications 23(1) (1977)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Jain, A.K., Jain, J.R.: Partial differential equations and finite difference methods in image processing – part II: Image restoration. IEEE Transactions on Automatic Control 23(5) (1978)CrossRefGoogle Scholar
  17. 17.
    Curtain, R.: A survey of infinite-dimensional filtering. SIAM Review 17(3), 395–411 (1975)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Ray, W.H., Lainiotis, D.G.: Distributed Parameter Systems. Marcel Dekker, New York (1978)zbMATHGoogle Scholar
  19. 19.
    Tarantola, A.: Inverse Problem Theory and Methods for Model Parameter Estimation. SIAM, Philadelphia (2004)zbMATHGoogle Scholar
  20. 20.
    Kaipio, J., Somersalo, E.: Statistical and Computational Inverse Problems. Applied mathematical Sciences, vol. 160. Springer, Heidelberg (2005)zbMATHGoogle Scholar
  21. 21.
    Hiltunen, P., Särkkä, S., Nissila, I., Lajunen, A., Lampinen, J.: State space regularization in the nonstationary inverse problem for diffuse optical tomography. Inverse Problems 27(2) (2011)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Alvarez, M., Lawrence, N.D.: Sparse convolved Gaussian processes for multi-output regression. In: Koller, D., Schuurmans, D., Bengio, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems, vol. 21, pp. 57–64. The MIT Press, Cambridge (2009)Google Scholar
  23. 23.
    Alvarez, M., Luengo, D., Titsias, M.K., Lawrence, N.D.: Efficient multioutput Gaussian processes through variational inducing kernels. In: Teh, Y.W., Titterington, M. (eds.) Proceedings of the 13th International Workshop on Artificial Intelligence and Statistics, pp. 25–32 (2010)Google Scholar
  24. 24.
    Alvarez, M., Lawrence, N.D.: Latent force models. In: van Dyk, D., Welling, M. (eds.) Proceedings of the 12th International Workshop on Artificial Intelligence and Statistics, pp. 9–16 (2009)Google Scholar
  25. 25.
    Vapnik, V.: Statistical Learning Theory. Wiley, Chichester (1998)zbMATHGoogle Scholar
  26. 26.
    Graepel, T.: Solving noisy linear operator equations by Gaussian processes: Application to ordinary and partial differential equations. In: Proceedings of 20th International Conference on Machine Learning (2003)Google Scholar
  27. 27.
    Solak, E., Murray-Smith, R., Leithead, W.E., Leith, D., Rasmussen, C.E.: Derivative observations in Gaussian process models of dynamic systems. In: Advances in Neural Information Processing Systems, vol. 15, pp. 1033–1040. MIT Press, Cambridge (2003)Google Scholar
  28. 28.
    O’Hagan, A.: Bayes-Hermite quadrature. Journal of Statistical Planning and Inference 29, 245–260 (1991)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Papoulis, A.: Probability, Random Variables, and Stochastic Processes. McGraw-Hill, New York (1984)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Simo Särkkä
    • 1
  1. 1.Department of Biomedical Engineering and Computational ScienceAalto UniversityFinland

Personalised recommendations