Supervised Subspace Learning with Multi-class Lagrangian SVM on the Grassmann Manifold
Learning robust subspaces to maximize class discrimination is challenging, and most current works consider a weak connection between dimensionality reduction and classifier design. We propose an alternate framework wherein these two steps are combined in a joint formulation to exploit the direct connection between dimensionality reduction and classification. Specifically, we learn an optimal subspace on the Grassmann manifold jointly minimizing the classification error of an SVM classifier. We minimize the regularized empirical risk over both the hypothesis space of functions that underlies this new generalized multi-class Lagrangian SVM and the Grassmann manifold such that a linear projection is to be found. We propose an iterative algorithm to meet the dual goal of optimizing both the classifier and projection. Extensive numerical studies on challenging datasets show robust performance of the proposed scheme over other alternatives in contexts wherein limited training data is used, verifying the advantage of the joint formulation.
KeywordsDimensionality Reduction Reproduce Kernel Hilbert Space Grassmann Manifold Locality Preserve Projection Joint Formulation
Unable to display preview. Download preview PDF.
- 2.Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press (2004)Google Scholar
- 3.Cai, D.: Codes and datasets for subspace learning, http://www.cs.uiuc.edu/homes/dengcai2/Data/data.html (as retrieved on October 2007)
- 5.Cai, D., He, X., Hu, Y., Han, J., Huang, T.: Learning a spatially smooth subspace for face recognition. In: Proceedings CVPR (2007)Google Scholar
- 8.He, X., Niyogi, P.: Locality preserving projection. In: Proceedings NIPS (2003)Google Scholar
- 9.Ji, S., Ye, J.: Linear dimensionality reduction for multi-label classification. In: Proc. IJCAI (2009)Google Scholar
- 10.Lee, Y., Lin, Y., Wahba, G.: Multicategory support vector machines. Technical report, Department of Statistics, University of Wisconsin-Madison (2001)Google Scholar
- 11.Lin, D., Yan, S., Tang, X.: Pursuing informative projection on Grassman manifold. In: Proceedings CVPR (2006)Google Scholar
- 14.Pham, D.S., Venkatesh, S.: Robust learning of discriminative projection for multicategory classification on the stiefel manifold. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR 2008), Anchorage, Alaska, June 24-26 (2008)Google Scholar