Abstract
In this research, a robust optimization approach applied to multiclass support vector machines (SVMs) is investigated. Two new kernel based-methods are developed to address data with input uncertainty where each data point is inside a sphere of uncertainty. The models are called robust SVM and robust feasibility approach model (Robust-FA) respectively. The two models are compared in terms of robustness and generalization error. The models are compared to robust Minimax Probability Machine (MPM) in terms of generalization behavior for several data sets. It is shown that the Robust-SVM performs better than robust MPM.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bennett, K., Mangasarian, O.: Robust linear programming discrimination of two linearly inseparable sets. Optim. Methods Softw. 1, 23–34 (1992)
Ben-Tal, A., Nemirovski, A.: Robust solutions to uncertain linear program via convex programming. Oper. Res. Lett. 25(1), 1–17 (1996)
Ben-Tal, A., Nemirovski, A.: Robust convex optimization. Math. Oper. Res. 23(4), 769–805 (1998)
Bertsimas, D., Pachamanova, D., Sim, M.: Robust linear optimization under general norms, Oper. Res. Lett. (April), 510–516 (2004)
Birge, J.: The value of statistic solution in stochastic linear programming with fixed resources. Math. Program. 24, 314–325 (1982)
Blake, C., Merz, C.: UCI Repository of machine learning databases. http://www.ics.uci.edu/~mlearn/MLRepository.html (1998)
Boyd, S., Lobo, M., Vandenberghe, L.: Application of second-order cone programming. Linear Algebra Appl. 284, 193–226 (1998)
Chvatal, V.: Linear Programming. Freeman, New York (1983)
Fung, G.M., Mangasarian, O., Shavlik, J.: Knowledge-based support vector machines classifiers. Technical report 01-09, Data Mining Institute, Computer Sciences Department, University of Wisconsin (2001)
Ghaoui, L.E., Lanckriet, G., Natsoulis, G.: Robust classification with interval data. Technical report CSD-03-1279, Division of Computer Science, University of California, Berkeley (2003) http://robotics.eecs.berkeley.edu/~gert/
Ghaoui, L.E., Lebret, H.: Robust solutions to least square problems to uncertain data matrices. SIAM J. Matrix Anal. Appl. 18, 1035–1064 (1997)
Ghaoui, L.E., Oustry, F., Lebret, H.: Robust solutions to uncertain semidefinite programs. SIAM J. Optim. 18, 1035–1064 (1997)
Gill, P., Murray, W., Saunders, M.: Snopt: An SQP algorithm for large-scale constrained optimization. SIAM J. Optim. 12(4), 979–1006 (2002)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, New York (2001)
Haykin, S.: Neural Network: a Comprehensive Foundation. Prentice-Hall, New Jersey (1999)
Hsu, C.-W., Lin, C.-J.: A comparison of methods for multi-class support vector machines. IEEE Trans. Neural Networks 13, 415–425 (2002)
Lanckriet, G., Ghaoui, L.E., Bhattacharyya, C., Jordan, M.: A robust minimax approach to classification. J. Mach. Learn. Res. 3, 555–582 (2002)
Popescue, I., Bertsimas, D.: Optimal inequalities in probability theory: a convex optimization approach, Technical report TM62, INSEAD (2001)
Schölkopf, B., Smola, A.: Learning with Kernels. MIT, Cambridge (2002)
Street, W., Mangasarian, O.: Improved generalization error via tolerant training. J. Optim. Theory Appl. 96(2), 259–279 (1998)
Trafalis, T., Alwazzi, S.: Robust optimization in support vector machine training with bounded errors. In: Proceedings of the International Joint Conference on Neural Networks, Portland, OR, pp. 2039–2042. IEEE, New York (2003)
Trafalis, T., Alwazzi, S.: Robust support vector regression and applications. In: Dagli, C., Buczak, A., Ghosh, J., Embrechts, M., Ersoy, O., Kerc, S. (eds.) Intelligent Engineering Systems through Artificial Neural Networks, vol. 13, pp. 181–186. ASME, New York (2003)
Trafalis, T.B., Oladunni, O., Papavassiliou, D.V.: Two-phase flow regime identification with a multi-classification SVM model. Ind. Eng. Chem. Res. 44, 4414–4426 (2005)
Trafalis, T., Santosa, B., Richman, M.: Tornado detection with kernel-based methods. In: Dagli, C., Buczak, A., Ghosh, J., Embrechts, M., Ersoy, O., Kerc, S. (eds.) Intelligent Engineering Systems through Artificial Neural Networks 13, pp. 677–682. ASME, New York (2003)
Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)
Weston, J., Watkins, C.: Support vector machines for multi-class pattern recognition. In: Proceeding of the Seventh European Symposium on Artificial Neural Networks, pp. 219–224 (1999)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Santosa, B., Trafalis, T.B. Robust multiclass kernel-based classifiers. Comput Optim Appl 38, 261–279 (2007). https://doi.org/10.1007/s10589-007-9042-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10589-007-9042-z