Kernels for the Relevance Vector Machine - An Empirical Study

  • David Ben-Shimon
  • Armin Shmilovici
Part of the Studies in Computational Intelligence book series (SCI, volume 23)


The Relevance Vector Machine (RVM) is a generalized linear model that can use kernel functions as basis functions. Experiments with the Matérn kernel indicate that the kernel choice has a significant impact on the sparsity of the solution. Furthermore, not every kernel is suitable for the RVM. Our experiments indicate that the Matérn kernel of order 3 is a good initial choice for many types of data.


Machine Learning Relevance Vector Machine Kernel Regression Matérn Kernel 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    A. Juditsky, H. Hjalmarsson, A. Beneviste, B. Delyon, L. Ljung, J. Sjoberg, Q. Zhang (1995), Nonlinear Black-box Models in System Identification: Mathematical Foundations, Automatica, 31(12), pp. 1725–1750.MATHMathSciNetCrossRefGoogle Scholar
  2. 2.
    M.G. Genton (2001). Classes of Kernels for Machine Learning: A Statistics Perspective. Journal of Machine Learning Research 2, pages 299–312.CrossRefGoogle Scholar
  3. 3.
    B. Matérn (1960). Spatial Variation. New York, Springer.MATHGoogle Scholar
  4. 4.
    M.E. Tipping (2001). Sparse Bayesian Learning and the Relevance Vector Machine. Journal of Machine Learning Research 1, 211–244.MATHMathSciNetCrossRefGoogle Scholar
  5. 5.
    A. Shmilovici (2005). Support Vector Machines. In O. Maimon and L. Rokach (editors), Data Mining and Knowledge Discovery Handbook: A Complete Guide for Practitioners and Researchers, Springer.Google Scholar
  6. 6.
    C.E. Rasmussen, J. Quinonero-Candela, (2005). Healing the Relevance Vector Machine through Augmentation. Proceedings of the 22nd International Conference on Machine Learning, August 7–11, Bonn, Germany.Google Scholar
  7. 7.
    D. Wipf, J. Palmer, B. Rao (2004). Perspectives on Sparse Bayesian Learning. Advances in Neural Information Processing systems, 16. Cambridge, Massachussettes, MIT Press.Google Scholar
  8. 8.
    D. Ben-Shimon, A. Shmilovici (2006). Accelerating the Relevance Vector Machine via Data Partitioning, Journal of Computing and Decision Sciences, forthcoming.Google Scholar
  9. 9.
    J.R. Gilbert, C. Moler, R. Schreiber (1992). Sparse matrices in MATLAB design and implementation. SIAM Journal on Matrix Analysis, 13(1), pages 333–356.MATHMathSciNetCrossRefGoogle Scholar
  10. 10.
    N. Cristianini, J. Shawe-Taylor (2003). An Introduction to Support Vector Machines and other kernel-based learning methods. Cambridge University Press.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • David Ben-Shimon
    • 1
  • Armin Shmilovici
    • 1
  1. 1.Dept. of Information Systems EngineeringBen-Gurion UniversityBeer-ShevaIsrael

Personalised recommendations