Abstract
This paper studies the problem of constructing robust classifiers when the training is plagued with uncertainty. The problem is posed as a Chance-Constrained Program (CCP) which ensures that the uncertain data points are classified correctly with high probability. Unfortunately such a CCP turns out to be intractable. The key novelty is in employing Bernstein bounding schemes to relax the CCP as a convex second order cone program whose solution is guaranteed to satisfy the probabilistic constraint. Prior to this work, only the Chebyshev based relaxations were exploited in learning algorithms. Bernstein bounds employ richer partial information and hence can be far less conservative than Chebyshev bounds. Due to this efficient modeling of uncertainty, the resulting classifiers achieve higher classification margins and hence better generalization. Methodologies for classifying uncertain test data points and error measures for evaluating classifiers robust to uncertain data are discussed. Experimental results on synthetic and real-world datasets show that the proposed classifiers are better equipped to handle data uncertainty and outperform state-of-the-art in many cases.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ben-Tal A., El Ghaoui L., Nemirovski A.: Robust Optimization. Princeton Series in Applied Mathematics, Englewood Cliffs (2009)
Ben-Tal A., Nemirovski A.: Selected topics in Robust convex optimization. Math. Programm. 112(1), 125–158 (2007)
Bhattacharyya C., Grate L.R., Jordan M.I., Ghaoui L.E.L., Mian I.S.: Robust sparse hyperplane classifiers: application to uncertain molecular profiling data. J. Comput. Biol. 11(6), 1073–1089 (2004)
Bi, J., Zhang, T.: Support vector classification with input data uncertainty. In: Advances in Neural Information Processing Systems (2004)
Chen, W., Sim, M.: Goal driven optimization. Oper. Res. (to appear)
Chen X., Sim M., Sun P.: A robust optimization perspective on stochastic programming. Oper. Res. 55(6), 1058–1071 (2005)
Chen, X., Sim, M., Sun, P., Teo, C.P.: From CVaR to uncertainty set: implications in joint chance constrained optimization. Oper. Res. (to appear)
Demichelis, F., Magni, P., Piergiorgi, P., Rubin, M.A., Bellazzi R.: A hierarchical Nave Bayes model for handling sample heterogeneity in classification problems: an application to tissue microarrays. BMC Bioinform. 7, 514 (2006)
Ghaoui, L.E., Lanckriet, G.R.G., Natsoulis, G.: Robust classification with interval data. Technical report UCB/CSD-03-1279, Computer Science Division, University of California, Berkeley (2003)
Johnson R.A., Wichern D.W.: Applied Multivariate Statistical Analysis, 5th edn. Prentice Hall, Englewood Cliffs (2002)
Lanckriet G.R., El Ghaoui L., Bhattacharyya C., Jordan M.I.: A Robust minimax approach to classification. J. Mach. Learn. Res. 3, 555–582 (2003)
Natsoulis G., Laurent El G., Lanckriet G.R.G., Tolley A.M., Leroy F., Dunlea S., Eynon B.P., Pearson C.I., Tugendreich S., Jarnagin K.: Classification of a large microarray data set: algorithm comparison and analysis of drug signatures. Genome Res. 15, 724–736 (2005)
Nemirovski A., Shapiro A.: Convex approximations of chance constrained programs. SIAM J. Optim. 17(4), 969–996 (2006)
Nesterov, Y., Nemirovskii, A.: Interior Point Polynomial Algorithms in Convex Programming. Number 13. Studies in Applied and Numerical Mathematics, SIAM books, Philadelphia (1993)
Rockafellar R.T.: Convex Analysis. Princeton University Press, Princeton (1970)
Saketha Nath, J., Bhattacharyya, C., Murty, M.N.: Clustering based large margin classification: a scalable approach using SOCP formulation. In: Proceedings of the 12th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 674–679. ACM Press, New York (2006)
Scheffé H.: The Analysis of Variance. Wiley, London (1959)
Shivaswamy P.K., Bhattacharyya C., Smola A.J.: Second order cone programming approaches for handling missing and uncertain data. J. Mach. Learn. Res. 7, 1283–1314 (2006)
Vapnik V.: Statistical Learning Theory. Wiley, New York (1998)
Author information
Authors and Affiliations
Corresponding author
Additional information
J. Saketha Nath—Part of this work was done when the author was visiting MINERVA Optimization Center, Faculty of Industrial Engineering and Management, Technion, Haifa 32000, ISRAEL.
Rights and permissions
About this article
Cite this article
Ben-Tal, A., Bhadra, S., Bhattacharyya, C. et al. Chance constrained uncertain classification via robust optimization. Math. Program. 127, 145–173 (2011). https://doi.org/10.1007/s10107-010-0415-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-010-0415-1