Robust generalized eigenvalue classifier with ellipsoidal uncertainty
- 467 Downloads
Uncertainty is a concept associated with data acquisition and analysis, usually appearing in the form of noise or measure error, often due to some technological constraint. In supervised learning, uncertainty affects classification accuracy and yields low quality solutions. For this reason, it is essential to develop machine learning algorithms able to handle efficiently data with imprecision. In this paper we study this problem from a robust optimization perspective. We consider a supervised learning algorithm based on generalized eigenvalues and we provide a robust counterpart formulation and solution in case of ellipsoidal uncertainty sets. We demonstrate the performance of the proposed robust scheme on artificial and benchmark datasets from University of California Irvine (UCI) machine learning repository and we compare results against a robust implementation of Support Vector Machines.
KeywordsRobust optimization Generalized eigenvalue classification Uncertainty
This project was partially funded by National Science Foundation (N.S.F.) grants and Italian Flagship Project Interomics funded by MIUR and CNR.
- Aeberhard, S., Coomans, D., & De Vel, O. (1992a). Comparison of classifiers in high dimensional settings. Tech. Rep. Dept. Math. Statist., James Cook Univ. North Queensland, Australia Google Scholar
- Aeberhard, S., Coomans, D., & De Vel, O. (1992b). The classification performance of RDA, Cambridge. Tech. Rep. (pp. 92–01). Dept. of Computer Science/Dept. of Mathematics and Statistics, James Cook University of North Queensland Google Scholar
- Andersen, M. S., Dahl, J., Liu, Z., & Vandenberghe, L. (2011). Interior-point methods for large-scale cone programming. Optimization for machine learning. Cambridge: MIT Press. Google Scholar
- Ben-Tal, A., El Ghaoui, L., & Nemirovski, A. S. (2009). Robust optimization. Princeton: Princeton University Press. Google Scholar
- Caramanis, C., Mannor, S., & Xu, H. (2011). Robust optimization in machine learning. In S. Sra, S. Nowozin, & S. J. Wright (Eds.), Optimization for machine learning (pp. 369–402). Cambridge: MIT Press. Google Scholar
- Cortes, C., & Vapnik, V. N. (1995). Support-vector networks. Machine Learning, 20(3), 273–297. Google Scholar
- Frank, A., & Asuncion, A. (2010). UCI machine learning repository. http://archive.ics.uci.edu/ml.
- Irpino, A., Guarracino, M. R., & Verde, R. (2010). Multiclass generalized eigenvalue proximal support vector machines. In 4th IEEE conference on complex, intelligent and software intensive systems (CISIS 2010). (pp. 25–32). Los Alamitos: IEEE Computer Society. Google Scholar
- Kim, S. J., Magnani, A., & Boyd, S. (2006). Robust fisher discriminant analysis. Advances in Neural Information Processing Systems, 18, 659. Google Scholar
- Musicant, D. R. (1998). NDC: normally distributed clustered datasets. http://www.cs.wisc.edu/dmi/svm/ndc/.
- Pothin, J., & Richard, C. (2006). Incorporating prior information into support vector machines in the form of ellipsoidal knowledge sets. Citeseer. Google Scholar
- Shahbazpanahi, S., Gershman, A., Luo, Z., & Wong, K. (2003). Robust adaptive beamforming using worst-case SINR optimization: a new diagonal loading-type solution for general-rank signal models. In 2003 IEEE international conference on acoustics, speech, and signal processing. Proceedings (ICASSP’03) (Vol. 5). New York: IEEE Press. Google Scholar
- Shivaswamy, P., Bhattacharyya, C., & Smola, A. (2006). Second order cone programming approaches for handling missing and uncertain data. The Journal of Machine Learning Research, 7, 1283–1314. Google Scholar
- Smith, J. W., Everhart, J. E., Dickson, W. C., Knowler, W. C., & Johannes, R. S. (1988). Using the adap learning algorithm to forecast the onset of diabetes mellitus. Johns Hopkins APL Technical Digest, 10, 262–266. Google Scholar
- Vandenberghe, L. (2010). The CVXOPT linear and quadratic cone program solvers. Google Scholar
- Vapnik, V. N. (1999). The nature of statistical learning theory. Information science and statistics. Berlin: Springer. Google Scholar
- Xanthopoulos, P., Pardalos, P. M., & Trafalis, T. B. (2012). Robust data mining. New York: Springer. Google Scholar
- Xu, H., Caramanis, C., & Mannor, S. (2009). Robustness and regularization of support vector machines. Journal of Machine Learning Research, 10, 1485–1510. Google Scholar