Filtered Gaussian Processes for Learning with Large Data-Sets

  • Jian Qing Shi
  • Roderick Murray-Smith
  • D. Mike Titterington
  • Barak A. Pearlmutter
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3355)


Kernel-based non-parametric models have been applied widely over recent years. However, the associated computational complexity imposes limitations on the applicability of those methods to problems with large data-sets. In this paper we develop a filtering approach based on a Gaussian process regression model. The idea is to generate a small-dimensional set of filtered data that keeps a high proportion of the information contained in the original large data-set. Model learning and prediction are based on the filtered data, thereby decreasing the computational burden dramatically.


Filtering transformation Gaussian process regression model Karhunen-Loeve expansion Kernel-based non-parametric models Principal component analysis 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Gibbs, M.N., Mark, G., MacKay, D.J.C.: Efficient implementation of Gaussian processes (1997)Google Scholar
  2. 2.
    Luo, Z., Wahba, G.: Hybrid adaptive splines. J. Amer. Statist. Assoc. 92, 107–116 (1997)zbMATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    Murray-Smith, R., Pearlmutter, B.A.: Transformations of Gaussian process priors. TR-2003-149, Department of Computing Science, University of Glasgow, Scotland (June 2003)Google Scholar
  4. 4.
    O’Hagan, A.: On curve fitting and optimal design for regression (with discussion). Journal of the Royal Statistical Society B 40, 1–42 (1978)zbMATHMathSciNetGoogle Scholar
  5. 5.
    Poggio, T., Girosi, F.: Networks for approximation and learning. Proceedings of IEEE 78, 1481–1497 (1990)CrossRefGoogle Scholar
  6. 6.
    Seeger, M., Willians, C.K.I., Lawrence, N.D.: Fast forward selection to speed up sparse Gaussian process regression. In: Bishop, C.M., Frey, B.J. (eds.) Proceedings of the Ninth International Workshop on AI and Statistics (2003)Google Scholar
  7. 7.
    Rasmussen, C.E.: Evaluation of Gaussian Processes and Other Methods for Non-linear Regression. PhD Thesis. University of Toronto (1996), Available from
  8. 8.
    Shi, J.Q., Murray-Smith, R., Titterington, D.M.: Hierarchical Gaussian Process Mixtures for Regression, DCS Technical Report TR-2002-107/Dept. Statistics Tech. Report 02-7, University of Glasgow, Scotland (2002)Google Scholar
  9. 9.
    Tresp, V.: The Bayesian committee machine. Neural Computation 12, 2719–2741 (2000)CrossRefGoogle Scholar
  10. 10.
    Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1995)zbMATHGoogle Scholar
  11. 11.
    Wahba, G.: Spline Models for Observational Data. SIAM, Philadelphia, PA; CBMS NSF Regional Conference series in applied mathematics(1990)Google Scholar
  12. 12.
    Williams, C.K.I., Rasmussen, C.E.: Gaussian process for regression. In: Touretzky, D.S., et al. (eds.) Advances in Neural Information Processing Systems, vol. 8. MIT Press, Cambridge (1996)Google Scholar
  13. 13.
    Williams, C.K.I., Seeger, M.: Using the Nyström method to speed up kernel machines. In: Leen, T.K., Diettrich, T.G., Tresp, V. (eds.) Advances in Neural Information Processing Systems, vol. 13. MIT Press, Cambridge (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Jian Qing Shi
    • 1
  • Roderick Murray-Smith
    • 2
    • 3
  • D. Mike Titterington
    • 4
  • Barak A. Pearlmutter
    • 3
  1. 1.School of Mathematics and StatisticsUniversity of NewcastleUK
  2. 2.Department of Computing ScienceUniversity of GlasgowScotland
  3. 3.Hamilton InstituteNUI Maynooth, Co. KildareIreland
  4. 4.Department of StatisticsUniversity of GlasgowScotland

Personalised recommendations