Robust relevance vector machine for classification with variational inference
- 159 Downloads
The relevance vector machine (RVM) is a widely employed statistical method for classification, which provides probability outputs and a sparse solution. However, the RVM can be very sensitive to outliers far from the decision boundary which discriminates between two classes. In this paper, we propose the robust RVM based on a weighting scheme, which is insensitive to outliers and simultaneously maintains the advantages of the original RVM. Given a prior distribution of weights, weight values are determined in a probabilistic way and computed automatically during training. Our theoretical result indicates that the influences of outliers are bounded through the probabilistic weights. Also, a guideline for determining hyperparameters governing a prior is discussed. The experimental results from synthetic and real data sets show that the proposed method performs consistently better than the RVM if a training data set is contaminated by outliers.
KeywordsRelevance vector machine Outlier Robust classification Sparsity
The authors thank the anonymous reviewers and editors for their helpful and constructive comments that greatly contributed to improving the paper.
- Bishop, C. M., & Tipping, M. E. (2000), Variational relevance vector machine. In Proceedings of the 16th conference on uncertainty in artificial intelligence (pp. 46–53).Google Scholar
- Caruana, R., & Niculescu-Mizil, A. (2004). Data mining in metric space: An empirical analysis of supervised learning performance criteria. In Proceedings of the 10th international conference on knowledge discovery and data mining (pp. 69–78).Google Scholar
- Christmann, A., & Steinwart, I. (2004). On robustness properties of convex risk minimization methods for pattern recognition. Journal of Machine Learning Research, 5, 1007–1034.Google Scholar
- Demsar, J. (2006). Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7, 1–30.Google Scholar
- Frank, A., & Asuncion, A. (2010). UCI machine learning repository. Irvine, CA: University of California, School of Information and Computer Science.Google Scholar
- Hwang, S., Kim, N., Jeong, M. K., & Yum, B. (2015). Robust kernel based regression with bounded influence for outliers. Journal of Operations Research Society (to appear).Google Scholar
- Jaakkola, T. S. (2000). Tutorial on variational approximation methods. Technical Report, MIT Artificial Intelligence Lab.Google Scholar
- Ling, C. X., Huang, J., & Zhang, H. (2003). AUC: A better measure than accuracy in comparing learning algorithms. In Proceedings of the 2003 Canadian artificial intelligence conference (pp. 329–341).Google Scholar
- Tipping, M. E. (2001). Sparse Bayesian learning and the relevance vector machine. Journal of Machine Learning Research, 1, 211–244.Google Scholar