Soft Computing

, Volume 21, Issue 21, pp 6407–6420 | Cite as

Incorporating neighbors’ distribution knowledge into support vector machines

  • Fa Zhu
  • Jian Yang
  • Sheng Xu
  • Cong Gao
  • Ning Ye
  • Tongming Yin
Methodologies and Application


The prior knowledge plays an important role in increasing the performance of the support vector machines (SVMs). Traditional SVMs do not consider any prior knowledge of the training set. In this paper, the neighbors’ distribution knowledge is incorporated into SVMs. The neighbors’ distribution can be measured by the sum of the cosine value of the angle, which is between the difference between the sample and its corresponding neighbor, and the difference between the sample and the mean of corresponding neighbors. The neighbors’ distribution knowledge reflects the sample’s importance in the training processing. It can be explained as the relative margin or instance weight. In this paper, the neighbors’ distribution knowledge is regarded as the relative margin and incorporated into the framework of density-induced margin support vector machines whose relative margin is measured by relative density degree. The results of the experiments, performed on both artificial synthetic datasets and real-world benchmark datasets, demonstrate that SVMs performs better after incorporating neighbors’ distribution. Furthermore, experimental results also show that neighbors’ distribution are more suitable than relative density degree to represent the relative margin.


Prior knowledge Support vector machine Neighbors’ distribution Relative margin 



The authors would like to thank the editor and the anonymous reviewers for their critical and constructive comments and suggestions. This work was partially supported by the National Science Fund for Distinguished Young Scholars under Grant Nos. 61125305, 61472187, 61233011 and 61373063, the Key Project of Chinese Ministry of Education under Grant No. 313030, the 973 Program (No. 2014CB349303), Fundamental Research Funds for the Central Universities (No. 30920140121005), Program for Changjiang Scholars and Innovative Research Team in University No. IRT13072, National Basic Research Program of China (973 Program) (2012CB114505), China National Funds for Distinguished Young Scientists (31125008).

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest to this work.


  1. Alcala-Fdez J, Sanchez L, Garcia S, del Jesus MJ, Ventura S, Garrell JM, Otero CR, Bacardit J, Rivas VM, Fernández JC, Herrera F (2009) KEEL: a software tool to assess evolutionary algorithms for data mining problems. Soft Comput 13(3):307–318CrossRefGoogle Scholar
  2. Alcalá J, Fernández A, Luengo J, Derrac J, García S, Sánchez L, Herrera F (2010) Keel data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework. J Multiple-Valued Logic Soft Comput 17(2–3):255–287Google Scholar
  3. Bertelli L, Yu T, Vu D, Gokturk, B (2011) Kernelized structural SVMlearning for supervised object segmentation. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, Milpitas, pp 2153–2160Google Scholar
  4. Bottou L (2010) Large-scale machine learning with stochastic gradient descent. In: Proceedings of COMPSTAT’2010. Physica-Verlag HD, pp 177–186Google Scholar
  5. Chang CC, Lin CJ (2011) LIBSVM: A library for support vector machines. ACM Trans Intell Syst Technol 2(3):27CrossRefGoogle Scholar
  6. Chapelle O, Schölkopf B (2001) Incorporating invariances in non-linear support vector machines. In: Advances in neural information processing systems. MIT Press, Vancouver, pp 609–616Google Scholar
  7. Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, CambridgeCrossRefzbMATHGoogle Scholar
  8. Friedman JH, Bentley JL, Finkel RA (1977) An algorithm for finding best matches in logarithmic expected time. ACM Trans Math Softw 3(3):209–226CrossRefzbMATHGoogle Scholar
  9. García S, Fernández A, Luengo J, Herrera F (2010) Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power. Inf Sci 180(10):2044–2064CrossRefGoogle Scholar
  10. Gumus E, Kilic N, Sertbas A, Ucan ON (2010) Evaluation of face recognition techniques using PCA, wavelets and SVM. Expert Syst Appl 37(9):6404–6408CrossRefGoogle Scholar
  11. Hsieh CJ, Chang KW, Lin CJ, Keerthi SS, Sundararajan S (2008) A dual coordinate descent method for large-scale linear SVM. In: Proceedings of the 25th international conference on machine learning. ACM, Helsinki, pp 408–415Google Scholar
  12. Hua X, Ding S (2015) Weighted least squares projection twin support vector machines with local information. Neurocomputing 160:228–237CrossRefGoogle Scholar
  13. Hwang JP, Park S, Kim E (2011) A new weighted approach to imbalanced data classification problem via support vector machine with quadratic cost function. Expert Syst Appl 38(7):8580–8585CrossRefGoogle Scholar
  14. Karasuyama M, Harada N, Sugiyama M, Takeuchi I (2012) Multi-parametric solution-path algorithm for instance-weighted support vector machines. Mach Learn 88(3):297–330MathSciNetCrossRefzbMATHGoogle Scholar
  15. Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910CrossRefzbMATHGoogle Scholar
  16. Kruskal WH, Wallis WA (1952) Use of ranks in one-criterion variance analysis. J Am Stat Assoc 47(260):583–621CrossRefzbMATHGoogle Scholar
  17. Lauer F, Bloch G (2008) Incorporating prior knowledge in support vector machines for classification: a review. Neurocomputing 71(7):1578–1594CrossRefGoogle Scholar
  18. Lee K, Kim DW, Lee D, Lee KH (2005) Improving support vector data description using local density degree. Pattern Recogn 38(10):1768–1771CrossRefzbMATHGoogle Scholar
  19. Lichman M (2013) UCI machine learning repository. University of California, School of Information and Computer Science, Irvine, CA
  20. Mangasarian OL, Wild EW (2001) Proximal support vector machine classifiers. In: Proceedings KDD-2001: knowledge discovery and data mining. ACM, San FranciscoGoogle Scholar
  21. Mangasarian OL, Wild EW (2006) Multisurface proximal support vector machine classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28(1):69–74CrossRefGoogle Scholar
  22. Niu XX, Suen CY (2012) A novel hybrid CNN-SVM classifier for recognizing handwritten digits. Pattern Recogn 45(4):1318–1325Google Scholar
  23. Palmieri F, Fiore U, Castiglione A, De Santis A (2013) On the detection of card-sharing traffic through wavelet analysis and support vector machines. Appl Soft Comput 13(1):615–627CrossRefGoogle Scholar
  24. Palmieri F, Fiore U, Castiglione A (2014) A distributed approach to network anomaly detection based on independent component analysis. Concurr Comput Pract Exp 26(5):1113–1129CrossRefGoogle Scholar
  25. Shao YH, Chen WJ, Zhang JJ, Wang Z, Deng NY (2014) An efficient weighted Lagrangian twin support vector machine for imbalanced data classification. Pattern Recogn 47(9):3158–3167CrossRefzbMATHGoogle Scholar
  26. Tian J, Gu H, Liu W, Gao C (2011) Robust prediction of protein subcellular localization combining PCA and WSVMs. Comput Biol Med 41(8):648–652CrossRefGoogle Scholar
  27. Vapnik V (2013) The nature of statistical learning theory. Springer, BerlinzbMATHGoogle Scholar
  28. Vapnik V, Vashist A (2009) A new learning paradigm: learning using privileged information. Neural Netw 22(5):544–557CrossRefzbMATHGoogle Scholar
  29. Wang F, Zhang D (2013) A new locality-preserving canonical correlation analysis algorithm for multi-view dimensionality reduction. Neural Process Lett 37(2):135–146CrossRefGoogle Scholar
  30. Wang J, Shen HT, Song J, Ji J (2014) Hashing for similarity search: a survey. arXiv preprint arXiv:1408.2927
  31. Wu X, Srihari R (2004) Incorporating prior knowledge with weighted margIn support vector machines. In: Proceedings of the tenth ACM SIGKDD international conference on knowledge discovery and data mining. ACM, Seattle, pp 326–333Google Scholar
  32. Wu Y, Wei B, Liu H, Li T, Rayner S (2011) MiRPara: a SVM-based software tool for prediction of most probable microRNA coding regions in genome scale sequences. BMC Bioinform 12(1):107CrossRefGoogle Scholar
  33. Xiong T, Cherkassky V (2005) A combined SVM and LDA approach for classification. In: 2005 IEEE international joint conference on neural networks, 2005. IJCNN’05. Proceedings, vol 3. IEEE, Montreal, pp 1455–1459Google Scholar
  34. Yang X, Song Q, Wang Y (2007) A weighted support vector machine for data classification. Int J Pattern Recogn Artif Intell 21(05):961–976CrossRefGoogle Scholar
  35. Yang X, Chen S, Chen B, Pan Z (2009) Proximal support vector machine using local information. Neurocomputing 73(1):357–365CrossRefGoogle Scholar
  36. Zafeiriou S, Tefas A, Pitas I (2007) Minimum class variance support vector machines. IEEE Trans Image Process 16(10):2551–2564MathSciNetCrossRefGoogle Scholar
  37. Zhang L, Zhou WD (2011) Density-induced margin support vector machines. Pattern Recogn 44(7):1448–1460CrossRefzbMATHGoogle Scholar
  38. Zhang H, Cao L, Gao S (2014) A locality correlation preserving support vector machine. Pattern Recogn 47(9):3168–3178CrossRefzbMATHGoogle Scholar
  39. Zhu F, Ye N, Yu W, Xu S, Li G (2014a) Boundary detection and sample reduction for one-class support vector machines. Neurocomputing 123:166–173CrossRefGoogle Scholar
  40. Zhu F, Yang J, Ye N, Gao C, Li G, Yin T (2014b) Neighbors’ distribution property and sample reduction for support vector machines. Appl Soft Comput 16:201–209CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Fa Zhu
    • 1
  • Jian Yang
    • 1
  • Sheng Xu
    • 2
  • Cong Gao
    • 3
  • Ning Ye
    • 4
  • Tongming Yin
    • 5
  1. 1.School of Computer Science and EngineeringNanjing University of Science and TechnologyNanjingPeople’s Republic of China
  2. 2.Department of Geomatics EngineeringUniversity of CalgaryCalgaryCanada
  3. 3.Department of Computer ScienceUniversity of ReginaReginaCanada
  4. 4.College of Information Science and TechnologyNanjing Forestry UniversityNanjingPeople’s Republic of China
  5. 5.College of Forest Resources and EnvironmentNanjing Forestry UniversityNanjingPeople’s Republic of China

Personalised recommendations