Robust chance-constrained support vector machines with second-order moment information
- 664 Downloads
Support vector machines (SVM) is one of the well known supervised classes of learning algorithms. Basic SVM models are dealing with the situation where the exact values of the data points are known. This paper studies SVM when the data points are uncertain. With some properties known for the distributions, chance-constrained SVM is used to ensure the small probability of misclassification for the uncertain data. As infinite number of distributions could have the known properties, the robust chance-constrained SVM requires efficient transformations of the chance constraints to make the problem solvable. In this paper, robust chance-constrained SVM with second-order moment information is studied and we obtain equivalent semidefinite programming and second order cone programming reformulations. The geometric interpretation is presented and numerical experiments are conducted. Three types of estimation errors for mean and covariance information are studied in this paper and the corresponding formulations and techniques to handle these types of errors are presented.
KeywordsSupport vector machines Robust chance constraints Semidefinite programming Second order cone programming Second-order moment information Estimation errors
We are grateful to Danial Kuhn and Berç Rustem for their valuable discussions. We would like to thank the anonymous reviewers for their helpful comments. Research was conducted at National Research University, Higher School of Economics, and supported by RSF grant 14-41-00039.
- Bi, J., & Zhang, T. (2005). Support vector classification with input data uncertainty. In L. K. Saul, Y. Weiss, & L. Bottou (Eds.), Advances in neural information processing systems 17: Proceedings of the 2004 conference. Cambridge: MIT Press.Google Scholar
- Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297.Google Scholar
- Fan, N., Sadeghi, E., & Pardalos, P. M. (2014). Robust support vector machines with polyhedral uncertainty of the input data. In P. M. Pardalos, M. G. C. Resende, C. Vogiatzis, & J. L. Walteros (Eds.), Learning and intelligent optimization (pp. 291–305). Berlin: Springer.Google Scholar
- Ghaoui, L. E., Lanckriet, G. R., & Natsoulis, G. (2003). Robust classification with interval data. Technical report UCB/CSD-03-1279, Computer Science Division, University of California, Berkeley.Google Scholar
- Lanckriet, G. R., Ghaoui, L. E., Bhattacharyya, C., & Jordan, M. I. (2002). A robust minimax approach to classification. Journal of Machine Learning Research, 3, 555–582.Google Scholar
- Pant, R., Trafalis, T. B., & Barker, K. (2011). Support vector machine classification of uncertain and imbalanced data using robust optimization. In Proceedings of the 15th WSEAS international conference on computers (pp. 369–374). World Scientific and Engineering Academy and Society (WSEAS).Google Scholar
- Shivaswamy, P. K., Bhattacharyya, C., & Smola, A. J. (2006). Second order cone programming approaches for handling missing and uncertain data. Journal of Machine Learning Research, 7, 1283–1314.Google Scholar
- Vapnik, V. N. (1998). Statistical learning theory. New York: Wiley.Google Scholar
- Xanthopoulos, P., Pardalos, P. M., & Trafalis, T. B. (2012). Robust data mining. Berlin: Springer.Google Scholar
- Yakubovich, V. A. (1971). S-procedure in nonlinear control theory. Vestnik Leningrad University, 1, 62–77.Google Scholar