Advertisement

Annals of Operations Research

, Volume 263, Issue 1–2, pp 45–68 | Cite as

Robust chance-constrained support vector machines with second-order moment information

  • Ximing Wang
  • Neng Fan
  • Panos M. Pardalos
Data Mining and Analytics

Abstract

Support vector machines (SVM) is one of the well known supervised classes of learning algorithms. Basic SVM models are dealing with the situation where the exact values of the data points are known. This paper studies SVM when the data points are uncertain. With some properties known for the distributions, chance-constrained SVM is used to ensure the small probability of misclassification for the uncertain data. As infinite number of distributions could have the known properties, the robust chance-constrained SVM requires efficient transformations of the chance constraints to make the problem solvable. In this paper, robust chance-constrained SVM with second-order moment information is studied and we obtain equivalent semidefinite programming and second order cone programming reformulations. The geometric interpretation is presented and numerical experiments are conducted. Three types of estimation errors for mean and covariance information are studied in this paper and the corresponding formulations and techniques to handle these types of errors are presented.

Keywords

Support vector machines Robust chance constraints  Semidefinite programming Second order cone programming  Second-order moment information Estimation errors 

Notes

Acknowledgments

We are grateful to Danial Kuhn and Berç Rustem for their valuable discussions. We would like to thank the anonymous reviewers for their helpful comments. Research was conducted at National Research University, Higher School of Economics, and supported by RSF grant 14-41-00039.

References

  1. Abe, S. (2010). Support vector machines for pattern classification. Berlin: Springer.CrossRefGoogle Scholar
  2. Ben-Hur, A., & Weston, J. (2010). A users guide to support vector machines. In O. Carugo & F. Eisenhaber (Eds.), Data mining techniques for the life sciences (pp. 223–239). Berlin: Springer.CrossRefGoogle Scholar
  3. Ben-Tal, A., Bhadra, S., Bhattacharyya, C., & Nath, J. S. (2011). Chance constrained uncertain classification via robust optimization. Mathematical Programming, 127(1), 145–173.CrossRefGoogle Scholar
  4. Bertsimas, D., & Popescu, I. (2005). Optimal inequalities in probability theory: A convex optimization approach. Siam Journal on Optimization, 15(3), 780–804.CrossRefGoogle Scholar
  5. Bhattacharyya, C., Grate, L. R., Jordan, M. I., El Ghaoui, L., & Mian, I. S. (2004). Robust sparse hyperplane classifiers: Application to uncertain molecular profiling data. Journal of Computational Biology, 11(6), 1073–1089.CrossRefGoogle Scholar
  6. Bi, J., & Zhang, T. (2005). Support vector classification with input data uncertainty. In L. K. Saul, Y. Weiss, & L. Bottou (Eds.), Advances in neural information processing systems 17: Proceedings of the 2004 conference. Cambridge: MIT Press.Google Scholar
  7. Burges, C. J. (1998). A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2(2), 121–167.CrossRefGoogle Scholar
  8. Chang, C. C., & Lin, C. J. (2011). Libsvm: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(3), 27.CrossRefGoogle Scholar
  9. Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297.Google Scholar
  10. Fan, N., Sadeghi, E., & Pardalos, P. M. (2014). Robust support vector machines with polyhedral uncertainty of the input data. In P. M. Pardalos, M. G. C. Resende, C. Vogiatzis, & J. L. Walteros (Eds.), Learning and intelligent optimization (pp. 291–305). Berlin: Springer.Google Scholar
  11. Ghaoui, L. E., Lanckriet, G. R., & Natsoulis, G. (2003). Robust classification with interval data. Technical report UCB/CSD-03-1279, Computer Science Division, University of California, Berkeley.Google Scholar
  12. Ghaoui, L. E., Oks, M., & Oustry, F. (2003). Worst-case value-at-risk and robust portfolio optimization: A conic programming approach. Operations Research, 51(4), 543–556.CrossRefGoogle Scholar
  13. Isii, K. (1960). The extrema of probability determined by generalized moments (i) bounded random variables. Annals of the Institute of Statistical Mathematics, 12(2), 119–134.CrossRefGoogle Scholar
  14. Lanckriet, G. R., Ghaoui, L. E., Bhattacharyya, C., & Jordan, M. I. (2002). A robust minimax approach to classification. Journal of Machine Learning Research, 3, 555–582.Google Scholar
  15. Marshall, A. W., & Olkin, I. (1960). Multivariate chebyshev inequalities. The Annals of Mathematical Statistics, 31(4), 1001–1014.CrossRefGoogle Scholar
  16. Pant, R., Trafalis, T. B., & Barker, K. (2011). Support vector machine classification of uncertain and imbalanced data using robust optimization. In Proceedings of the 15th WSEAS international conference on computers (pp. 369–374). World Scientific and Engineering Academy and Society (WSEAS).Google Scholar
  17. Pólik, I., & Terlaky, T. (2007). A survey of the s-lemma. SIAM Review, 49(3), 371–418.CrossRefGoogle Scholar
  18. Shivaswamy, P. K., Bhattacharyya, C., & Smola, A. J. (2006). Second order cone programming approaches for handling missing and uncertain data. Journal of Machine Learning Research, 7, 1283–1314.Google Scholar
  19. Tian, Y., Shi, Y., & Liu, X. (2012). Recent advances on support vector machines research. Technological and Economic Development of Economy, 18(1), 5–33.CrossRefGoogle Scholar
  20. Trafalis, T. B., & Alwazzi, S. A. (2010). Support vector machine classification with noisy data: A second order cone programming approach. International Journal of General Systems, 39(7), 757–781.CrossRefGoogle Scholar
  21. Trafalis, T. B., & Gilbert, R. C. (2006). Robust classification and regression using support vector machines. European Journal of Operational Research, 173(3), 893–909.CrossRefGoogle Scholar
  22. Trafalis, T. B., & Gilbert, R. C. (2007). Robust support vector machines for classification and computational issues. Optimization Methods and Software, 22(1), 187–198.CrossRefGoogle Scholar
  23. Vapnik, V. N. (1998). Statistical learning theory. New York: Wiley.Google Scholar
  24. Vapnik, V. N. (1999). An overview of statistical learning theory. IEEE Transactions on Neural Networks, 10(5), 988–999.CrossRefGoogle Scholar
  25. Wang, X., & Pardalos, P. M. (2014). A survey of support vector machines with uncertainties. Annals of Data Science, 1(3–4), 293–309.CrossRefGoogle Scholar
  26. Xanthopoulos, P., Guarracino, M. R., & Pardalos, P. M. (2014). Robust generalized eigenvalue classifier with ellipsoidal uncertainty. Annals of Operations Research, 216(1), 327–342.CrossRefGoogle Scholar
  27. Xanthopoulos, P., Pardalos, P. M., & Trafalis, T. B. (2012). Robust data mining. Berlin: Springer.Google Scholar
  28. Yakubovich, V. A. (1971). S-procedure in nonlinear control theory. Vestnik Leningrad University, 1, 62–77.Google Scholar
  29. Zymler, S., Kuhn, D., & Rustem, B. (2013). Distributionally robust joint chance constraints with second-order moment information. Mathematical Programming, 137(1–2), 167–198.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Department of Industrial and Systems EngineeringUniversity of FloridaGainesvilleUSA
  2. 2.Department of Systems and Industrial EngineeringUniversity of ArizonaTucsonUSA

Personalised recommendations