Journal of Mathematical Sciences

, Volume 203, Issue 6, pp 789–798 | Cite as

Properties of the Bayesian Parameter Estimation of a Regression Based on Gaussian Processes

  • A. A. Zaytsev
  • E. V. Burnaev
  • V. G. Spokoiny


We consider the regression approach based on Gaussian processes and outline our theoretical results about the properties of the posterior distribution of the corresponding covariance function’s parameter vector. We perform statistical experiments confirming that the obtained theoretical propositions are valid for a wide class of covariance functions commonly used in applied problems.


Posterior Distribution Covariance Function Gaussian Process Kernel Density Estimate Hellinger Distance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    E. V. Burnaev, A. A. Zaytsev, and V. G. Spokoiny, “Bernstein–von Mises theorem for regression on the basis of Gaussian processes,” Usp. Mat. Nauk, 68, No. 5 (413), 179–180 (2013).CrossRefGoogle Scholar
  2. 2.
    A. Ya. Chervonenkis, S. S. Chernova, and T. V. Zykova, “Applications of kernel ridge estimation to the problem of computing the aerodynamical characteristics of a passenger plane (in comparison with results obtained with artificial neural networks),” Autom. Remote Control, 72, No. 5, 1061–1067 (2011).CrossRefzbMATHGoogle Scholar
  3. 3.
    J. Eidsvik, A. O. Finley, S. Banerjee, and H. Rue, “Approximate Bayesian inference for large spatial datasets using predictive process models,” Comput. Stat. Data Anal., 56, No. 6, 1362–1380 (2011).CrossRefMathSciNetGoogle Scholar
  4. 4.
    A. Forrester, A. Sobester, and A. Keane, Engineering Design via Surrogate Modelling: A Practical Guide, Wiley (2008).Google Scholar
  5. 5.
    I. A. Ibragimov and R. Z. Has’minskii, Statistical Estimation, Asymptotic Theory, Appl. Math., Vol. 16, Springer, New York (1981).CrossRefzbMATHGoogle Scholar
  6. 6.
    C. G. Kaufman, M. J. Schervish, and D. W. Nychka, “Covariance tapering for likelihood-based estimation in large spatial data sets,” J. Am. Stat. Assoc., 103 (484), 1545–1555 (2008).CrossRefzbMATHMathSciNetGoogle Scholar
  7. 7.
    S. Kok, “The asymptotic behaviour of the maximum likelihood function of Kriging approximations using the Gaussian correlation function,” in: EngOpt 2012 — Intern. Conf. on Engineering Optimization, Rio de Janeiro, Brazil, 1–5 July 2012 (2012).Google Scholar
  8. 8.
    K. V. Mardia and R. J. Marshall, “Maximum likelihood estimation of models for residual covariance in spatial regression,” Biometrika, 71, No. 1, 135–146 (1984).CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning, Vol. 1, MIT Press, Cambridge, MA (2006).zbMATHGoogle Scholar
  10. 10.
    B. Shaby and D. Ruppert, “Tapered covariance: Bayesian estimation and asymptotics,” J. Comput. Graph. Stat., 21, No. 2, 433–452 (2012).CrossRefMathSciNetGoogle Scholar
  11. 11.
    A. N. Shiryaev, Probability, Grad. Texts Math., Vol. 95, Springer, Berlin (1995).zbMATHGoogle Scholar
  12. 12.
    V. Spokoiny, “Parametric estimation. Finite sample theory,” Ann. Stat., 6, 2877–2909 (2012).CrossRefMathSciNetGoogle Scholar
  13. 13.
    V. Spokoiny, Bernstein–von Mises Theorem for Growing Parameter Dimension, arXiv:1302.3430 [math.ST] (2013).Google Scholar
  14. 14.
    L. Wasserman, All of Statistics. A Concise Course in Statistical Inference, Springer, Berlin (2004).zbMATHGoogle Scholar
  15. 15.
    A. A. Zaitsev, E. V. Burnaev, and V. G. Spokoiny, “Properties of the posterior distribution of a regression model based on Gaussian random fields,” Autom. Remote Control, 74, No. 10, 1645–1655 (2013).CrossRefzbMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • A. A. Zaytsev
    • 1
    • 2
  • E. V. Burnaev
    • 3
  • V. G. Spokoiny
    • 3
    • 4
    • 5
  1. 1.DatadvanceMoscowRussia
  2. 2.Institute for Information Transmission ProblemsRussian Academy of SciencesMoscowRussia
  3. 3.Premolab, Moscow Institute of Physics and TechnologyMoscowRussia
  4. 4.Weierstrass Institute (WIAS)BerlinGermany
  5. 5.Humboldt University of BerlinBerlinGermany

Personalised recommendations