Skip to main content
SpringerLink
Log in

Kernel Ridge Regression

  • Chapter
  • First Online:
Empirical Inference

Abstract

This chapter discusses the method of Kernel Ridge Regression, which is a very simple special case of Support Vector Regression. The main formula of the method is identical to a formula in Bayesian statistics, but Kernel Ridge Regression has performance guarantees that have nothing to do with Bayesian assumptions. I will discuss two kinds of such performance guarantees: those not requiring any assumptions whatsoever, and those depending on the assumption of randomness.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Azoury, K.S., Warmuth, M.K.: Relative loss bounds for on-line density estimation with the exponential family of distributions. Mach. Learn. 43, 211–246 (2001)

    Article  MATH  Google Scholar 

  2. Beckenbach, E.F., Bellman, R.: Inequalities. Springer, Berlin (1965)

    Google Scholar 

  3. Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge University Press, Cambridge (2006)

    Book  MATH  Google Scholar 

  4. Cressie, N.: The origins of kriging. Math. Geol. 22, 239–252 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  5. Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines and Other Kernel-Based Methods. Cambridge University Press, Cambridge (2000)

    Book  Google Scholar 

  6. Diaconis, P., Freedman, D.: On the consistency of Bayes estimates (with discussion). Ann. Stat. 14, 1–67 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  7. Gammerman, A., Vovk, V., Vapnik, V.: Learning by transduction. In: Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, Madison, pp. 148–155. Morgan Kaufmann, San Francisco (1998)

    Google Scholar 

  8. Kakade, S.M., Ng, A.Y.: Online bounds for Bayesian algorithms. In: Proceedings of the Eighteenth Annual Conference on Neural Information Processing Systems, Vancouver (2004)

    Google Scholar 

  9. Kumon, M., Takemura, A., Takeuchi, K.: Sequential optimizing strategy in multi-dimensional bounded forecasting games. Stoch. Process. Appl. 121, 155–183 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  10. Lei, J., Wasserman, L.: Distribution free prediction bands. Tech. Rep. arXiv:1203.5422 [stat.ME], arXiv.org e-Print archive (2012). To appear in the Journal of the Royal Statistical Society B

    Google Scholar 

  11. Nouretdinov, I., Melluish, T., Vovk, V.: Ridge regression confidence machine. In: Proceedings of the Eighteenth International Conference on Machine Learning, Williamstown, pp. 385–392. Morgan Kaufmann, San Francisco (2001)

    Google Scholar 

  12. Saunders, C., Gammerman, A., Vovk, V.: Ridge regression learning algorithm in dual variables. In: Shavlik, J.W. (ed.) Proceedings of the Fifteenth International Conference on Machine Learning, Madison, pp. 515–521. Morgan Kaufmann, San Francisco (1998)

    Google Scholar 

  13. Shafer, G., Vovk, V.: A tutorial on conformal prediction. J. Mach. Learn. Res. 9, 371–421 (2008)

    MathSciNet  MATH  Google Scholar 

  14. Steinwart, I.: On the influence of the kernel on the consistency of support vector machines. J. Mach. Learn. Res. 2, 67–93 (2001)

    MathSciNet  Google Scholar 

  15. Vapnik, V.N.: Statistical Learning Theory. Wiley, New York (1998)

    MATH  Google Scholar 

  16. Vovk, V.: Competitive on-line statistics. Int. Stat. Rev. 69, 213–248 (2001)

    MATH  Google Scholar 

  17. Vovk, V., Gammerman, A., Shafer, G.: Algorithmic Learning in a Random World. Springer, New York (2005)

    MATH  Google Scholar 

  18. Wasserman, L.: Frequentist Bayes is objective (comment on articles by Berger and by Goldstein). Bayesian Anal. 1, 451–456 (2006)

    Article  MathSciNet  Google Scholar 

  19. Wasserman, L.: Frasian inference. Stat. Sci. 26, 322–325 (2011)

    MathSciNet  MATH  Google Scholar 

  20. Zhdanov, F., Kalnishkan, Y.: An identity for kernel ridge regression. Theor. Comput. Sci. 473, 157–178 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  21. Zhdanov, F., Vovk, V.: Competing with Gaussian linear experts. Tech. Rep. arXiv:0910.4683 [cs.LG], arXiv.org e-Print archive (2009). Revised in May 2010

    Google Scholar 

Download references

Acknowledgements

I am deeply grateful to Vladimir Vapnik for numerous discussions and support over the years, starting from our first meetings in the summer of 1996. Many thanks to Alexey Chervonenkis, Alex Gammerman, Valya Fedorova, and Ilia Nouretdinov for their advice and help. This work has been supported in part by the Cyprus Research Promotion Foundation (TPE/ORIZO/0609(BIE)/24) and EPSRC (EP/K033344/1).

Author information

Authors and Affiliations

  1. Dept. of Computer Science, Royal Holloway, University of London, Egham, Surrey, UK

    Vladimir Vovk

Authors
  1. Vladimir Vovk
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vladimir Vovk .

Editor information

Editors and Affiliations

  1. Max Planck Institute for Intelligent Systems, Tübingen, Germany

    Bernhard Schölkopf

  2. Dept. of Computer Science, Royal Holloway, University of London, Egham, Surrey, United Kingdom

    Zhiyuan Luo

  3. Department of Computer Science, Royal Holloway, University of London, Egham, Surrey, United Kingdom

    Vladimir Vovk

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Vovk, V. (2013). Kernel Ridge Regression. In: Schölkopf, B., Luo, Z., Vovk, V. (eds) Empirical Inference. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41136-6_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-41136-6_11

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-41135-9

  • Online ISBN: 978-3-642-41136-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation