Abstract
In this paper, we mainly focus on two issues (1) SVM is very sensitive to noise. (2) The solution of SVM does not take into consideration of the intrinsic structure and the discriminant information of the data. To address these two problems, we first propose an integration model to integrate both the local manifold structure and the local discriminant information into ℓ1 graph embedding. Then we add the integration model into the objection function of υ-support vector machine. Therefore, a discriminant sparse neighborhood preserving embedding υ-support vector machine (υ-DSNPESVM) method is proposed. The theoretical analysis demonstrates that υ-DSNPESVM is a reasonable maximum margin classifier and can obtain a very lower generalization error upper bound by minimizing the integration model and the upper bound of margin error. Moreover, in the nonlinear case, we construct the kernel sparse representation-based ℓ1 graph for υ-DSNPESVM, which is more conducive to improve the classification accuracy than ℓ1 graph constructed in the original space. Experimental results on real datasets show the effectiveness of the proposed υ-DSNPESVM method.
Similar content being viewed by others
References
Vapnik V (1995) The nature of statistical learning theory. Springer, New York
Scholkopf B, Mika S, Burges CJC, Knirsch P, Muller KR, Ratsch G, Smola AJ (1999) Input space vs. feature space in kernel-based methods. IEEE Trans Neural Netw 10(5):1000–1017
Tao WJ, Wang ST, Hu WJ, Ying WH (2010) ρ-margin kernel learning machine with magnetic field effect for both binary classification and novelty detection. Int J Softw Inform 4(3):305–324
Wang YQ, Wang SY, Lai KK (2005) A new fuzzy support vector machine to evaluate credit risk. IEEE Trans Fuzzy Sys 13(6):820–831
Qian MJ, Nie FP, Zhang CS (2009) Probabilistic labeled Semi-supervised SVM. 2009 IEEE International Conference on Data Mining Workshops, pp. 394–399
Qian MJ, Nie FP, Zhang CS (2009) Efficient multi-class unlabeled constrained semi-supervised SVM. Proceedings of the 18th ACM conference on Information and knowledge management, pp 1665–1668
Tao DC, Tang XO, Li XL, Rui Y (2006) Direct kernel biased discriminant analysis: a new content-based image retrieval relevance feedback algorithm. IEEE Trans Multimedia 8:716–727
Keerthi SS, Shevade SK, Bhattacharyya C, Murthy KPK (2000) A fast iterative nearest point algorithm for support vector machine classifier design. IEEE Trans Neural Netw 11(1):200–211
Flake GW, Lawrence S (2002) Efficient SVM regression training with SMO. Mach Learn 46(1–3):271–290
Suykens JA, Van Gestel T, De Brabanter J, De Moor B, Vandewalle J (2002) Least squares support vectors machines. World Scintific, Singapore
Schölkopf B, Smola AJ, Williamson RC, Bartlett PL (2000) New support vector algorithms. Neural Comput 12(5):1207–1245
Chang MW, Lin CJ, Weng RCH (2004) Analysis of switching dynamics with competing support vector machines. IEEE Trans Neural Netw 15(3):720–727
Pang SN, Kim D, Bang SY (2005) Face membership authentication using SVM classification tree generated by membership-based LLE data partition. IEEE Trans Neural Netw 16(2):436–446
Li S, Kwok JTY, Tsang IWH, Wang Y (2004) Fusing images with different focuses using support vector machines. IEEE Trans Neural Netw 15(6):1555–1561
Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from example. J Mach Learn Res 7:2399–2434
Nie FP, Xiang SM, Yang YQ, Zhang CS (2009) Orthogonal locality minimizing globality maximizing projections for feature extraction. Optical Eng 48(1):017202–017205
Mu Y, Ding W, Tao DC (2013) Local discriminative distance metrics ensemble learning. Pattern Recogn 46:2337–2349
Mu Y, Lo HZ, Ding W, Amaral K, Crouter SE (2014) Bipart: learning block structure for activity detection. IEEE Trans Knowl Data Eng 26(10):2397–2409
Mu Y, Lo HZ, Ding W, Tao DC (2014) Face recognition from multiple images per subject. Proceedings of the 22nd ACM International Conference on Multimedia, pp 889–892
Wang XM, Chung FL, Wang ST (2010) On minimum class locality preserving variance support vector machine. Pattern Recogn 43(8):2753–2762
Zafeiriou S, Tefas A, Pitas I (2007) Minimum class variance support vector machines. IEEE Trans Image Process 16(10):2551–2564
Wang HX, Chen S, Hu LZ, Zheng WM (2008) Locality-preserved maximum information projection. IEEE Trans Neural Netw 19(4):571–585
Nie FP, Xu D, Tsang IWH, Zhang CS (2010) Flexible manifold embedding: a framework for semi-supervised and unsupervised dimension reduction. IEEE Trans Image Process 19(7):1921–1932
Nie FP, Xiang SM, Zhang CS (2007) Neighborhood minmax projections. Proceedings of the 20th International Joint Conference on Artifical intelligence, pp 993–998
Ni BB, Yan SC, Kassim AA (2012) Learning a propagable graph for semisupervised learning: classification and regression. IEEE Trans Knowl Data Eng 24(1):114–126
Lu GF, Lin Z, Jin Z (2010) Face recognition using discriminant locality preserving projection based on maximum margin criterion. Pattern Recogn 43(10):3572–3579
Yan SC, Wang H (SDM09) Semi-supervised learning by sparse representation. SIAM International Conference on Data Mining
Xu Y, Zhang D, Yang J, Yang JY (2011) A two-phase test sample sparse representation method for use with face recognition. IEEE Trans Circuits Syst Video Technol 21(9):1255–1262
Yang J, Zhang L, Xu Y, Yang JY (2012) Beyond sparsity: the role of L1-optimizer in pattern classification. Pattern Recogn 45(3):1104–1118
Belkin M, Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput 15(6):1373–1396
Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227
Rao R, Olshausen B, Lewicki M (2002) Probabilistic model of the brain: preception and neural function. MIT Press, Cambridge
Zhang X (NNSP’99) Using class-center vectors to build support vector machines, in proc. IEEE Workshop Neural Netw. Signal Process. Madison, pp 3–11
Cai D, He XF, Zhou K, Hang JW, Bao HJ (2007) Locality sensitive discriminant analysis. In: Proceeding of the 20th International Joint Conference on Artificial Intelligence. IJCAI press, Hyderabad, India, 708–713
Zhang CS, Nie FP (2010) A general kernalization framework for learning algorithms based on kernel PCA. Neurocomputing 73:959–967
Zhang L, Zhou WD, Chang PC, Liu J, Yang Z, Wang T, Li FZ (2012) Kernel sparse representation-based classifier. IEEE Trans Signal Process 60(4):1684–1695
Gao SH, Tsang IWH, Chia LT (2010) Kernel sparse representation for image classification and face recognition. In: Proceedings 11th Eur. Conference Comput. Vis.: Part IV
Xu Y, Zhang D, Song FX, Yang JY, Jing Z, Li M (2007) A method for speeding up feature extraction based on KPCA. Neurocomputing 70(4):1056–1061
Li HF, Jiang T, Zhang KS (2006) Efficient and robust feature extraction by maximun margin criterion. IEEE Transaction on Neural Netw 17(1):157–165
Liu J, Chen SC, Tan XY, Zhang DQ (2007) Comments on efficient and robust feature extraction by maximum margin criterion. IEEE Trans Neural Netw 18(6):1862–1864
Wu MR, Yie JP (2009) A small sphere and large margin approach for novelty detection using training data with outliers. IEEE Trans Pattern Anal Mach Intell 31(11):2088–2092
Blake C, Merz C, UCI Repository of machine learning databases. http://www.ics.uci.edu/~mlearn/MLRepository.html. Accessed 20 Mar 2015
Yang AY, Sastry SS, Ganesh A, Ma Y (2010) Fast ℓ1-minimization algorithms and an application in robust face recognition: A review. In: 17th IEEE International Conference on Image Processing, IEEE, Hong Kong, pp 1849–1852
Yale Univ, Face Database. http://www.cvc.yale.edu/projects/yalefaces/yalefaces.html. Accessed 20 Mar 2015
Yang J, Zhang D, Frangi AF, Yang JY (2004) Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Trans Pattern Anal Mach Intell 26(1):131–137
Jiang ZL, Lin Z, Davis LS (2013) Label consistent K-SVD: learning a discriminative dictionary for recognition. IEEE Trans Pattern Anal Mach Intell 35(11):2651–2664
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Fang, B., Huang, Z., Li, Y. et al. υ-Support vector machine based on discriminant sparse neighborhood preserving embedding. Pattern Anal Applic 20, 1077–1089 (2017). https://doi.org/10.1007/s10044-016-0547-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10044-016-0547-x