DSMML 2004: Deterministic and Statistical Methods in Machine Learning pp 211-228 | Cite as
Understanding Gaussian Process Regression Using the Equivalent Kernel
Abstract
The equivalent kernel [1] is a way of understanding how Gaussian process regression works for large sample sizes based on a continuum limit. In this paper we show how to approximate the equivalent kernel of the widely-used squared exponential (or Gaussian) kernel and related kernels. This is easiest for uniform input densities, but we also discuss the generalization to the non-uniform case. We show further that the equivalent kernel can be used to understand the learning curves for Gaussian processes, and investigate how kernel smoothing using the equivalent kernel compares to full Gaussian process regression.
Keywords
Power Spectrum Mean Square Error Covariance Function Gaussian Process Target FunctionPreview
Unable to display preview. Download preview PDF.
References
- 1.Silverman, B.W.: Annals of Statistics, 12, 898–916 (1984)Google Scholar
- 2.Williams, C.K.I.: In: Jordan, M.I. (ed.) Learning in Graphical Models, pp. 599–621. Kluwer Academic, Dordrecht (1998)Google Scholar
- 3.Hastie, T.J., Tibshirani, R.J.: Generalized Additive Models. Chapman and Hall, Boca Raton (1990)MATHGoogle Scholar
- 4.Sollich, P., Williams, C.K.I.: NIPS 17 (2005) (to appear)Google Scholar
- 5.Girosi, F., Jones, M., Poggio, T.: Neural Computation 7(2), 219–269 (1995)Google Scholar
- 6.Papoulis, A.: Probability, Random Variables, and Stochastic Processes, 3rd edn. McGraw-Hill, New York (1991)Google Scholar
- 7.Thomas-Agnan, C.: Numerical Algorithms. 13, 21–32 (1996)Google Scholar
- 8.Poggio, T., Voorhees, H., Yuille, A.: Tech. Report AI Memo 833, MIT AI Laboratory (1985)Google Scholar
- 9.Schölkopf, B., Smola, A.: Learning with Kernels. MIT Press, Cambridge (2002)Google Scholar
- 10.Stein, M.L.: Interpolation of Spatial Data. Springer, New York (1999)MATHGoogle Scholar
- 11.Zhu, H., Williams, C.K.I., Rohwer, R.J., Morciniec, M.: Gaussian Regeression and Optimal Finite Dimensional LinearModels. In: Bishop, C.M. (ed.) Neural Networks and Machine Learning. Springer, Heidelberg (1998)Google Scholar
- 12.Abramowitz, M., Stegun, I.A.: Handbook of Mathematical Functions, Dover, New York (1965)Google Scholar
- 13.Sollich, P., Halees, A.: Neural Computation. 14, 1393–1428 (2002)Google Scholar
- 14.Opper, M., Vivarelli, F.: NIPS 11, pp. 302–308 (1999)Google Scholar
- 15.Sollich, P.: NIPS 14, pp. 519–526 (2002)Google Scholar