Advances in Machine Learning I pp 227-246

Part of the Studies in Computational Intelligence book series (SCI, volume 262)

| Cite as

An Analysis of Relevance Vector Machine Regression

  • Matti Saarela
  • Tapio Elomaa
  • Keijo Ruohonen

Abstract

The relevance vector machine (RVM) is a Bayesian framework for learning sparse regression models and classifiers. Despite of its popularity and practical success, no thorough analysis of its functionality exists. In this paper we consider the RVM in the case of regression models and present two kinds of analysis results: we derive a full characterization of the behavior of the RVM analytically when the columns of the regression matrix are orthogonal and give some results concerning scale and rotation invariance of the RVM. We also consider the practical implications of our results and present a scenario in which our results can be used to detect potential weakness in the RVM framework.

Keywords

Relevance vector machine regression sparse Bayesian learning 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Agarwal, A., Triggs, B.: 3D human pose from silhouettes by relevance vector regression. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 882–888. IEEE Computer Society, Los Alamitos (2004)Google Scholar
  2. 2.
    Brookes, M.: The Matrix Reference Manual (2005), http://www.ee.ic.ac.uk/hp/staff/dmb/matrix/intro.html
  3. 3.
    Chen, S., Gunn, S.R., Harris, C.J.: The relevance vector machine technique for channel equalization application. IEEE Transactions on Neural Networks 12(6), 1529–1532 (2004)CrossRefGoogle Scholar
  4. 4.
    Faul, A.C., Tipping, M.E.: Analysis of sparse Bayesian learning. In: Dietterich, T.G., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems, vol. 14, pp. 383–389. MIT Press, Cambridge (2001)Google Scholar
  5. 5.
    Li, Y., Campbell, C., Tipping, M.: Bayesian automatic relevance determination algorithms for classifying gene expression data. Bioinformatics 18(10), 1332–1339 (2002)CrossRefGoogle Scholar
  6. 6.
    Candela, J.Q., Hansen, L.K.: Time series prediction based on the relevance vector machine with adaptive kernels. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Piscataway, NJ, pp. 985–988. IEEE Signal Processing Society, Los Alamitos (2002)Google Scholar
  7. 7.
    Tipping, M.: The relevance vector machine. In: Leen, T.K., Dietterich, T.G., Tresp, V. (eds.) Advances in Neural Information Processing Systems, vol. 12, pp. 652–658. MIT Press, Cambridge (2000)Google Scholar
  8. 8.
    Tipping, M.: Sparse Bayesian learning and the relevance vector machine. Journal of Machine Learning Research 1, 211–244 (2001)MATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Tipping, M.E.: Sparse Bayesian Learning and the Relevance Vector Machine, http://research.microsoft.com/mlp/RVM
  10. 10.
    Vapnik, V.N.: Statistical Learning Theory. John Wiley & Sons, New York (1998)MATHGoogle Scholar
  11. 11.
    Vapnik, V.N., Golowich, S.E., Smola, A.J.: Support vector method for function approximation, regression estimation and signal processing. In: Mozer, M., Jordan, M.I., Petsche, T. (eds.) Advances in Neural Information Processing Systems, vol. 9, pp. 281–287. MIT Press, Cambridge (1997)Google Scholar
  12. 12.
    Wei, L., Yang, Y., Nishikawa, R.M., Wernick, M.N., Edwards, A.: Relevance vector machine for automatic detection of clustered microcalcifications. IEEE Transactions on Medical Imaging 24(10), 1278–1285 (2005)CrossRefGoogle Scholar
  13. 13.
    Weston, J., Elisseeff, A., Schölkopf, B., Tipping, M.E.: Use of the zero-norm with linear models and kernel methods. Journal of Machine Learning Research 3, 1439–1461 (2003)MATHCrossRefGoogle Scholar
  14. 14.
    Williams, O., Blake, A., Cipolla, R.: A sparse probabilistic learning algorithm for real-time tracking. In: Proceedings of the Ninth IEEE International Conference on Computer Vision, pp. 353–361. IEEE Computer Society, Los Alamitos (2003)CrossRefGoogle Scholar
  15. 15.
    Wipf, D.P., Nagarajan, S.: A new view of automatic relevance determination. In: McCallum, A. (ed.) Advances in Neural Information Processing Systems, vol. 20. MIT Press, Cambridge (2008)Google Scholar
  16. 16.
    Wipf, D.P., Palmer, J.A., Rao, B.D.: Perspectives on sparse Bayesian learning. In: Thrun, S., Saul, L.K., Schölkopf, B. (eds.) Advances in Neural Information Processing Systems, vol. 16. MIT Press, Cambridge (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Matti Saarela
    • 1
  • Tapio Elomaa
    • 1
  • Keijo Ruohonen
    • 2
  1. 1.Department of Software SystemsTampere University of Technology 
  2. 2.Department of MathematicsTampere University of TechnologyTampereFinland

Personalised recommendations