Skip to main content
SpringerLink
Log in

υ-Support vector machine based on discriminant sparse neighborhood preserving embedding

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

In this paper, we mainly focus on two issues (1) SVM is very sensitive to noise. (2) The solution of SVM does not take into consideration of the intrinsic structure and the discriminant information of the data. To address these two problems, we first propose an integration model to integrate both the local manifold structure and the local discriminant information into ℓ1 graph embedding. Then we add the integration model into the objection function of υ-support vector machine. Therefore, a discriminant sparse neighborhood preserving embedding υ-support vector machine (υ-DSNPESVM) method is proposed. The theoretical analysis demonstrates that υ-DSNPESVM is a reasonable maximum margin classifier and can obtain a very lower generalization error upper bound by minimizing the integration model and the upper bound of margin error. Moreover, in the nonlinear case, we construct the kernel sparse representation-based ℓ1 graph for υ-DSNPESVM, which is more conducive to improve the classification accuracy than ℓ1 graph constructed in the original space. Experimental results on real datasets show the effectiveness of the proposed υ-DSNPESVM method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Vapnik V (1995) The nature of statistical learning theory. Springer, New York

    Book  MATH  Google Scholar 

  2. Scholkopf B, Mika S, Burges CJC, Knirsch P, Muller KR, Ratsch G, Smola AJ (1999) Input space vs. feature space in kernel-based methods. IEEE Trans Neural Netw 10(5):1000–1017

    Article  Google Scholar 

  3. Tao WJ, Wang ST, Hu WJ, Ying WH (2010) ρ-margin kernel learning machine with magnetic field effect for both binary classification and novelty detection. Int J Softw Inform 4(3):305–324

    Google Scholar 

  4. Wang YQ, Wang SY, Lai KK (2005) A new fuzzy support vector machine to evaluate credit risk. IEEE Trans Fuzzy Sys 13(6):820–831

    Article  Google Scholar 

  5. Qian MJ, Nie FP, Zhang CS (2009) Probabilistic labeled Semi-supervised SVM. 2009 IEEE International Conference on Data Mining Workshops, pp. 394–399

  6. Qian MJ, Nie FP, Zhang CS (2009) Efficient multi-class unlabeled constrained semi-supervised SVM. Proceedings of the 18th ACM conference on Information and knowledge management, pp 1665–1668

  7. Tao DC, Tang XO, Li XL, Rui Y (2006) Direct kernel biased discriminant analysis: a new content-based image retrieval relevance feedback algorithm. IEEE Trans Multimedia 8:716–727

    Article  Google Scholar 

  8. Keerthi SS, Shevade SK, Bhattacharyya C, Murthy KPK (2000) A fast iterative nearest point algorithm for support vector machine classifier design. IEEE Trans Neural Netw 11(1):200–211

    Article  Google Scholar 

  9. Flake GW, Lawrence S (2002) Efficient SVM regression training with SMO. Mach Learn 46(1–3):271–290

    Article  MATH  Google Scholar 

  10. Suykens JA, Van Gestel T, De Brabanter J, De Moor B, Vandewalle J (2002) Least squares support vectors machines. World Scintific, Singapore

    Book  MATH  Google Scholar 

  11. Schölkopf B, Smola AJ, Williamson RC, Bartlett PL (2000) New support vector algorithms. Neural Comput 12(5):1207–1245

    Article  Google Scholar 

  12. Chang MW, Lin CJ, Weng RCH (2004) Analysis of switching dynamics with competing support vector machines. IEEE Trans Neural Netw 15(3):720–727

    Article  Google Scholar 

  13. Pang SN, Kim D, Bang SY (2005) Face membership authentication using SVM classification tree generated by membership-based LLE data partition. IEEE Trans Neural Netw 16(2):436–446

    Article  Google Scholar 

  14. Li S, Kwok JTY, Tsang IWH, Wang Y (2004) Fusing images with different focuses using support vector machines. IEEE Trans Neural Netw 15(6):1555–1561

    Article  Google Scholar 

  15. Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from example. J Mach Learn Res 7:2399–2434

    MATH  MathSciNet  Google Scholar 

  16. Nie FP, Xiang SM, Yang YQ, Zhang CS (2009) Orthogonal locality minimizing globality maximizing projections for feature extraction. Optical Eng 48(1):017202–017205

    Article  Google Scholar 

  17. Mu Y, Ding W, Tao DC (2013) Local discriminative distance metrics ensemble learning. Pattern Recogn 46:2337–2349

    Article  MATH  Google Scholar 

  18. Mu Y, Lo HZ, Ding W, Amaral K, Crouter SE (2014) Bipart: learning block structure for activity detection. IEEE Trans Knowl Data Eng 26(10):2397–2409

    Article  Google Scholar 

  19. Mu Y, Lo HZ, Ding W, Tao DC (2014) Face recognition from multiple images per subject. Proceedings of the 22nd ACM International Conference on Multimedia, pp 889–892

  20. Wang XM, Chung FL, Wang ST (2010) On minimum class locality preserving variance support vector machine. Pattern Recogn 43(8):2753–2762

    Article  MATH  Google Scholar 

  21. Zafeiriou S, Tefas A, Pitas I (2007) Minimum class variance support vector machines. IEEE Trans Image Process 16(10):2551–2564

    Article  MathSciNet  Google Scholar 

  22. Wang HX, Chen S, Hu LZ, Zheng WM (2008) Locality-preserved maximum information projection. IEEE Trans Neural Netw 19(4):571–585

    Article  Google Scholar 

  23. Nie FP, Xu D, Tsang IWH, Zhang CS (2010) Flexible manifold embedding: a framework for semi-supervised and unsupervised dimension reduction. IEEE Trans Image Process 19(7):1921–1932

    Article  MATH  MathSciNet  Google Scholar 

  24. Nie FP, Xiang SM, Zhang CS (2007) Neighborhood minmax projections. Proceedings of the 20th International Joint Conference on Artifical intelligence, pp 993–998

  25. Ni BB, Yan SC, Kassim AA (2012) Learning a propagable graph for semisupervised learning: classification and regression. IEEE Trans Knowl Data Eng 24(1):114–126

    Article  Google Scholar 

  26. Lu GF, Lin Z, Jin Z (2010) Face recognition using discriminant locality preserving projection based on maximum margin criterion. Pattern Recogn 43(10):3572–3579

    Article  MATH  Google Scholar 

  27. Yan SC, Wang H (SDM09) Semi-supervised learning by sparse representation. SIAM International Conference on Data Mining

  28. Xu Y, Zhang D, Yang J, Yang JY (2011) A two-phase test sample sparse representation method for use with face recognition. IEEE Trans Circuits Syst Video Technol 21(9):1255–1262

    Article  MathSciNet  Google Scholar 

  29. Yang J, Zhang L, Xu Y, Yang JY (2012) Beyond sparsity: the role of L1-optimizer in pattern classification. Pattern Recogn 45(3):1104–1118

    Article  MATH  Google Scholar 

  30. Belkin M, Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput 15(6):1373–1396

    Article  MATH  Google Scholar 

  31. Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227

    Article  Google Scholar 

  32. Rao R, Olshausen B, Lewicki M (2002) Probabilistic model of the brain: preception and neural function. MIT Press, Cambridge

    Google Scholar 

  33. Zhang X (NNSP’99) Using class-center vectors to build support vector machines, in proc. IEEE Workshop Neural Netw. Signal Process. Madison, pp 3–11

  34. Cai D, He XF, Zhou K, Hang JW, Bao HJ (2007) Locality sensitive discriminant analysis. In: Proceeding of the 20th International Joint Conference on Artificial Intelligence. IJCAI press, Hyderabad, India, 708–713

  35. Zhang CS, Nie FP (2010) A general kernalization framework for learning algorithms based on kernel PCA. Neurocomputing 73:959–967

    Article  Google Scholar 

  36. Zhang L, Zhou WD, Chang PC, Liu J, Yang Z, Wang T, Li FZ (2012) Kernel sparse representation-based classifier. IEEE Trans Signal Process 60(4):1684–1695

    Article  MathSciNet  Google Scholar 

  37. Gao SH, Tsang IWH, Chia LT (2010) Kernel sparse representation for image classification and face recognition. In: Proceedings 11th Eur. Conference Comput. Vis.: Part IV

  38. Xu Y, Zhang D, Song FX, Yang JY, Jing Z, Li M (2007) A method for speeding up feature extraction based on KPCA. Neurocomputing 70(4):1056–1061

    Article  Google Scholar 

  39. Li HF, Jiang T, Zhang KS (2006) Efficient and robust feature extraction by maximun margin criterion. IEEE Transaction on Neural Netw 17(1):157–165

    Article  Google Scholar 

  40. Liu J, Chen SC, Tan XY, Zhang DQ (2007) Comments on efficient and robust feature extraction by maximum margin criterion. IEEE Trans Neural Netw 18(6):1862–1864

    Article  Google Scholar 

  41. Wu MR, Yie JP (2009) A small sphere and large margin approach for novelty detection using training data with outliers. IEEE Trans Pattern Anal Mach Intell 31(11):2088–2092

    Article  Google Scholar 

  42. Blake C, Merz C, UCI Repository of machine learning databases. http://www.ics.uci.edu/~mlearn/MLRepository.html. Accessed 20 Mar 2015

  43. Yang AY, Sastry SS, Ganesh A, Ma Y (2010) Fast ℓ1-minimization algorithms and an application in robust face recognition: A review. In: 17th IEEE International Conference on Image Processing, IEEE, Hong Kong, pp 1849–1852

  44. Yale Univ, Face Database. http://www.cvc.yale.edu/projects/yalefaces/yalefaces.html. Accessed 20 Mar 2015

  45. Yang J, Zhang D, Frangi AF, Yang JY (2004) Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Trans Pattern Anal Mach Intell 26(1):131–137

    Article  Google Scholar 

  46. Jiang ZL, Lin Z, Davis LS (2013) Label consistent K-SVD: learning a discriminative dictionary for recognition. IEEE Trans Pattern Anal Mach Intell 35(11):2651–2664

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, No. 29, Street Yudao, Nanjing, 210016, China

    Bingwu Fang, Zhiqiu Huang, Yong Li & Yong Wang

  2. Department of Electronics and Information, Anhui Vocational College of Finance and Trade, Hefei, 230601, China

    Bingwu Fang

Authors
  1. Bingwu Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhiqiu Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bingwu Fang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fang, B., Huang, Z., Li, Y. et al. υ-Support vector machine based on discriminant sparse neighborhood preserving embedding. Pattern Anal Applic 20, 1077–1089 (2017). https://doi.org/10.1007/s10044-016-0547-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-016-0547-x

Keywords

Search

Navigation