Incorporating neighbors’ distribution knowledge into support vector machines
- 201 Downloads
The prior knowledge plays an important role in increasing the performance of the support vector machines (SVMs). Traditional SVMs do not consider any prior knowledge of the training set. In this paper, the neighbors’ distribution knowledge is incorporated into SVMs. The neighbors’ distribution can be measured by the sum of the cosine value of the angle, which is between the difference between the sample and its corresponding neighbor, and the difference between the sample and the mean of corresponding neighbors. The neighbors’ distribution knowledge reflects the sample’s importance in the training processing. It can be explained as the relative margin or instance weight. In this paper, the neighbors’ distribution knowledge is regarded as the relative margin and incorporated into the framework of density-induced margin support vector machines whose relative margin is measured by relative density degree. The results of the experiments, performed on both artificial synthetic datasets and real-world benchmark datasets, demonstrate that SVMs performs better after incorporating neighbors’ distribution. Furthermore, experimental results also show that neighbors’ distribution are more suitable than relative density degree to represent the relative margin.
KeywordsPrior knowledge Support vector machine Neighbors’ distribution Relative margin
The authors would like to thank the editor and the anonymous reviewers for their critical and constructive comments and suggestions. This work was partially supported by the National Science Fund for Distinguished Young Scholars under Grant Nos. 61125305, 61472187, 61233011 and 61373063, the Key Project of Chinese Ministry of Education under Grant No. 313030, the 973 Program (No. 2014CB349303), Fundamental Research Funds for the Central Universities (No. 30920140121005), Program for Changjiang Scholars and Innovative Research Team in University No. IRT13072, National Basic Research Program of China (973 Program) (2012CB114505), China National Funds for Distinguished Young Scientists (31125008).
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest to this work.
- Alcalá J, Fernández A, Luengo J, Derrac J, García S, Sánchez L, Herrera F (2010) Keel data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework. J Multiple-Valued Logic Soft Comput 17(2–3):255–287Google Scholar
- Bertelli L, Yu T, Vu D, Gokturk, B (2011) Kernelized structural SVMlearning for supervised object segmentation. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, Milpitas, pp 2153–2160Google Scholar
- Bottou L (2010) Large-scale machine learning with stochastic gradient descent. In: Proceedings of COMPSTAT’2010. Physica-Verlag HD, pp 177–186Google Scholar
- Chapelle O, Schölkopf B (2001) Incorporating invariances in non-linear support vector machines. In: Advances in neural information processing systems. MIT Press, Vancouver, pp 609–616Google Scholar
- Hsieh CJ, Chang KW, Lin CJ, Keerthi SS, Sundararajan S (2008) A dual coordinate descent method for large-scale linear SVM. In: Proceedings of the 25th international conference on machine learning. ACM, Helsinki, pp 408–415Google Scholar
- Lichman M (2013) UCI machine learning repository. http://archive.ics.uci.edu/ml. University of California, School of Information and Computer Science, Irvine, CA
- Mangasarian OL, Wild EW (2001) Proximal support vector machine classifiers. In: Proceedings KDD-2001: knowledge discovery and data mining. ACM, San FranciscoGoogle Scholar
- Niu XX, Suen CY (2012) A novel hybrid CNN-SVM classifier for recognizing handwritten digits. Pattern Recogn 45(4):1318–1325Google Scholar
- Wang J, Shen HT, Song J, Ji J (2014) Hashing for similarity search: a survey. arXiv preprint arXiv:1408.2927
- Wu X, Srihari R (2004) Incorporating prior knowledge with weighted margIn support vector machines. In: Proceedings of the tenth ACM SIGKDD international conference on knowledge discovery and data mining. ACM, Seattle, pp 326–333Google Scholar
- Xiong T, Cherkassky V (2005) A combined SVM and LDA approach for classification. In: 2005 IEEE international joint conference on neural networks, 2005. IJCNN’05. Proceedings, vol 3. IEEE, Montreal, pp 1455–1459Google Scholar