Abstract
The relevance vector machine (RVM) is a Bayesian framework for learning sparse regression models and classifiers. Despite of its popularity and practical success, no thorough analysis of its functionality exists. In this paper we consider the RVM in the case of regression models and present two kinds of analysis results: we derive a full characterization of the behavior of the RVM analytically when the columns of the regression matrix are orthogonal and give some results concerning scale and rotation invariance of the RVM. We also consider the practical implications of our results and present a scenario in which our results can be used to detect potential weakness in the RVM framework.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Agarwal, A., Triggs, B.: 3D human pose from silhouettes by relevance vector regression. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 882–888. IEEE Computer Society, Los Alamitos (2004)
Brookes, M.: The Matrix Reference Manual (2005), http://www.ee.ic.ac.uk/hp/staff/dmb/matrix/intro.html
Chen, S., Gunn, S.R., Harris, C.J.: The relevance vector machine technique for channel equalization application. IEEE Transactions on Neural Networks 12(6), 1529–1532 (2004)
Faul, A.C., Tipping, M.E.: Analysis of sparse Bayesian learning. In: Dietterich, T.G., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems, vol. 14, pp. 383–389. MIT Press, Cambridge (2001)
Li, Y., Campbell, C., Tipping, M.: Bayesian automatic relevance determination algorithms for classifying gene expression data. Bioinformatics 18(10), 1332–1339 (2002)
Candela, J.Q., Hansen, L.K.: Time series prediction based on the relevance vector machine with adaptive kernels. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Piscataway, NJ, pp. 985–988. IEEE Signal Processing Society, Los Alamitos (2002)
Tipping, M.: The relevance vector machine. In: Leen, T.K., Dietterich, T.G., Tresp, V. (eds.) Advances in Neural Information Processing Systems, vol. 12, pp. 652–658. MIT Press, Cambridge (2000)
Tipping, M.: Sparse Bayesian learning and the relevance vector machine. Journal of Machine Learning Research 1, 211–244 (2001)
Tipping, M.E.: Sparse Bayesian Learning and the Relevance Vector Machine, http://research.microsoft.com/mlp/RVM
Vapnik, V.N.: Statistical Learning Theory. John Wiley & Sons, New York (1998)
Vapnik, V.N., Golowich, S.E., Smola, A.J.: Support vector method for function approximation, regression estimation and signal processing. In: Mozer, M., Jordan, M.I., Petsche, T. (eds.) Advances in Neural Information Processing Systems, vol. 9, pp. 281–287. MIT Press, Cambridge (1997)
Wei, L., Yang, Y., Nishikawa, R.M., Wernick, M.N., Edwards, A.: Relevance vector machine for automatic detection of clustered microcalcifications. IEEE Transactions on Medical Imaging 24(10), 1278–1285 (2005)
Weston, J., Elisseeff, A., Schölkopf, B., Tipping, M.E.: Use of the zero-norm with linear models and kernel methods. Journal of Machine Learning Research 3, 1439–1461 (2003)
Williams, O., Blake, A., Cipolla, R.: A sparse probabilistic learning algorithm for real-time tracking. In: Proceedings of the Ninth IEEE International Conference on Computer Vision, pp. 353–361. IEEE Computer Society, Los Alamitos (2003)
Wipf, D.P., Nagarajan, S.: A new view of automatic relevance determination. In: McCallum, A. (ed.) Advances in Neural Information Processing Systems, vol. 20. MIT Press, Cambridge (2008)
Wipf, D.P., Palmer, J.A., Rao, B.D.: Perspectives on sparse Bayesian learning. In: Thrun, S., Saul, L.K., Schölkopf, B. (eds.) Advances in Neural Information Processing Systems, vol. 16. MIT Press, Cambridge (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Saarela, M., Elomaa, T., Ruohonen, K. (2010). An Analysis of Relevance Vector Machine Regression. In: Koronacki, J., Raś, Z.W., Wierzchoń, S.T., Kacprzyk, J. (eds) Advances in Machine Learning I. Studies in Computational Intelligence, vol 262. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-05177-7_11
Download citation
DOI: https://doi.org/10.1007/978-3-642-05177-7_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-05176-0
Online ISBN: 978-3-642-05177-7
eBook Packages: EngineeringEngineering (R0)