Incremental Relevance Vector Machine with Kernel Learning
Recently, sparse kernel methods such as the Relevance Vector Machine (RVM) have become very popular for solving regression problems. The sparsity and performance of these methods depend on selecting an appropriate kernel function, which is typically achieved using a cross-validation procedure. In this paper we propose a modification to the incremental RVM learning method, that also learns the location and scale parameters of Gaussian kernels during model training. More specifically, in order to effectively model signals with different characteristics at various locations, we learn different parameter values for each kernel, resulting in a very flexible model. In order to avoid overfitting we use a sparsity enforcing prior that controls the effective number of parameters of the model. Finally, we apply the proposed method to one-dimensional and two-dimensional artificial signals, and evaluate its performance on two real-world datasets.
KeywordsBasis Function Mean Square Error Marginal Likelihood Incremental Algorithm Relevance Vector Machine
Unable to display preview. Download preview PDF.
- 3.Lanckriet, G.R.G., Cristianini, N., Bartlett, P., Ghaoui, L.E., Jordan, M.I.: Learning the kernel matrix with semidefinite programming. J. Mach. Learn. Res. 5, 27–72 (2004)Google Scholar
- 6.Quiñonero-Candela, J., Hansen, L.K.: Time series prediction based on the relevance vector machine with adaptive kernels. In: Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, Piscataway, New Jersey, vol. 1, pp. 985–988. IEEE, Los Alamitos (2002)Google Scholar
- 9.Tipping, M., Faul, A.: Fast marginal likelihood maximisation for sparse Bayesian models. In: Proc. of the Ninth International Workshop on Artificial Intelligence and Statistics (2003)Google Scholar
- 10.Faul, A.C., Tipping, M.E.: Analysis of sparse Bayesian learning. In: Advances in Neural Information Processing Systems, pp. 383–389. MIT Press, Cambridge (2001)Google Scholar
- 11.Holmes, C.C., Denison, D.G.T.: Bayesian wavelet analysis with a model complexity prior. In: Bernardo, J.M., Berger, J.O., Dawid, A.P., Smith, A.F.M. (eds.) Bayesian Statistics 6: Proceedings of the Sixth Valencia International Meeting. Oxford University Press, Oxford (1999)Google Scholar