Identification of uncertainty and decision boundary for SVM classification training using belief function

  • Javad HamidzadehEmail author
  • Somaye Moslemnejad


The existence of noisy samples increases the inefficiency of SVM training. In SVM training, the classification hyperplane is determined by the support vectors, therefore, the other samples do not affect the SVM classifier. This paper presents a novel method that does not use all samples for SVM training. The basic idea is a novel method in which, at the first step, using the belief function theory and fuzzy rough set theory, the boundary samples are identified. And at the second step, the boundary samples uncertainty such as noisy samples are identified and discarded. Finally, at the last step, using the obtained boundary samples, the training of the SVM classifier is done. To show the performance of the proposed method, BFFR-BS (Belief Function and Fuzzy Rough Set-Boundary Samples), several experiments have been conducted on various real-world data sets from UCI repository. Experimental results reveal that the proposed method is superior to the state-of-the-art competing methods regarding accuracy, precision, time, and AUC metrics.


Support vector machine SVM classifier Belief function theory Fuzzy rough set theory Boundary samples Noisy samples 


  1. 1.
    Vapnik V (1995) The nature of statistical learning theory. IEEE Trans Neural Netw 10(5):988–999CrossRefGoogle Scholar
  2. 2.
    Yang L, Xu Z (2017) Feature extraction by PCA and diagnosis of breast tumors using SVM with DE-based parameter tuning. Int J Mach Learn & Cyber:1–11Google Scholar
  3. 3.
    Mao WT, Xu JC, Wang C et al (2014) A fast and robust model selection algorithm for multi-input multi-output support vector machine. Neurocomputing 130:10–19CrossRefGoogle Scholar
  4. 4.
    Santhanama V, Morariua VI, Harwooda D, Davisa LS (2016) A non-parametric approach to extending generic binary classifiers for multi-classification. Pattern Recogn 58:149–158CrossRefGoogle Scholar
  5. 5.
    Vanir V (1999) An overview of statistical learning theory. IEEE Trans Neural Netw 10(5):988–999CrossRefGoogle Scholar
  6. 6.
    Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297zbMATHGoogle Scholar
  7. 7.
    Xue Y, Zhang L, Wang B, Zhang Z, Li F (2018) Nonlinear feature selection using Gaussian kernel SVM-RFE for fault diagnosis. Appl Intell:1–26Google Scholar
  8. 8.
    Moghaddam VH, Hamidzadeh J (2016) New Hermite orthogonal polynomial kernel and combined kernels in support vector machine classifier. Pattern Recogn 60:921–935CrossRefGoogle Scholar
  9. 9.
    Hamidzadeh J, Moradi M (2018) Improved one-class classification using filled function. Appl Intell:1–17Google Scholar
  10. 10.
    Hamidzadeh J, Sadeghi R, Namaei N (2017) Weighted support vector data description based on chaotic bat algorithm. Appl Soft Comput 60:540–551CrossRefGoogle Scholar
  11. 11.
    Hamidzadeh J, Namaei N (2018) Belief-based chaotic algorithm for support vector data description. Soft Comput:1–26Google Scholar
  12. 12.
    Hsu HT, Lee PL, Shyu KK (2017) Improvement of classification accuracy in a phase-tagged steady-state visual evoked potential-based brain–computer Interface using adaptive neuron-fuzzy classifier. International Journal of Fuzzy Systems 19:542–552CrossRefGoogle Scholar
  13. 13.
    Onan A (2015) A fuzzy-rough nearest neighbor classifier combined with consistency-based subset evaluation and instance selection for automated diagnosis of breast cancer. Expert Syst Appl 42:6844–6852CrossRefGoogle Scholar
  14. 14.
    Zhou Q, Chao F, Lin CM (2018) A functional-link-based fuzzy brain emotional learning network for breast tumor classification and chaotic system synchronization. International Journal of Fuzzy Systems 20:349–365MathSciNetCrossRefGoogle Scholar
  15. 15.
    Yue X, Chen Y, Miao D, Qian J (2017) Tri-partition neighborhood covering reduction for robust classification. Int J Approx Reason 83:371–384MathSciNetCrossRefGoogle Scholar
  16. 16.
    Chen Y, Xue Y, Ma Y, Xu F (2017) Measures of uncertainty for neighborhood rough sets. Knowl-Based Syst 120:1–10CrossRefGoogle Scholar
  17. 17.
    Kar S, Majumder DD (2016) An investigative study on early diagnosis of breast Cancer using a new approach of mathematical shape theory and neuro-fuzzy classification system. International Journal of Fuzzy Systems 18:349–366MathSciNetCrossRefGoogle Scholar
  18. 18.
    Du SQ, Wei W, May D, Younan NH (2010) Noise-adjusted principal component analysis for buried radioactive target detection and classification. IEEE Trans Nucl Sci 57:349–366Google Scholar
  19. 19.
    Han D, Liu W, Dezert J, Yang Y (2016) A novel approach to pre-extracting support vectors based on the theory of belief functions. Knowl-Based Syst 110:210–223CrossRefGoogle Scholar
  20. 20.
    Han DQ, Han CZ, Yang Y (2009) Approach for pre-extracting support vectors based on K-NN. Control Decis 24(4):494–498zbMATHGoogle Scholar
  21. 21.
    Zhou C, Lu X, Huang M (2016) Dempster–Shafer theory-based robust least squares support vector machine for stochastic modelling. Neurocomputing 182:145–153CrossRefGoogle Scholar
  22. 22.
    Yang X, Song Q, Cao A (2005) Weighted support vector machine for data classification. IEEE International Joint Conference on Neural Networks 2:859–864Google Scholar
  23. 23.
    Jayadeva R, Khemchandani S, Chandra HZ (2004) Fast and robust learning through fuzzy linear proximal support vector machines. Neurocomputing 61:401–411CrossRefGoogle Scholar
  24. 24.
    Lin CF, Wang SD (2002) Fuzzy support vector machines. IEEE Trans Neural Netw 13:464–471CrossRefGoogle Scholar
  25. 25.
    Lu X, Liu W, Zhou C, Huang M (2017) Probabilistic weighted support vector machine for robust modeling with application to hydraulic actuator. IEEE Trans Industrial Informatics 13(4):1723–1733CrossRefGoogle Scholar
  26. 26.
    Chau AL, Li X, Yu W (2013) Convex and concave hulls for classification with support vector machine. Neurocomputing 122:198–209CrossRefGoogle Scholar
  27. 27.
    Xiaa S y, Xiong Z y, Luo Y g, Dong L m (2015) A method to improve support vector machine based on distance to hyperplane. Optik - International Journal for Light and Electron Optics 126:2405–2410CrossRefGoogle Scholar
  28. 28.
    Triguero I, Peralta D, Bacardit J, García S, Herrera F (2015) MRPR: a MapReduce solution for prototype reduction in big data classification. Neurocomputing 150 (331–345CrossRefGoogle Scholar
  29. 29.
    Hamidzadeh J, Monsefi R, Yazdi HS (2015) IRAHC: Instance reduction algorithm using hyperrectangle clustering. Pattern Recogn 48:1878–1889CrossRefGoogle Scholar
  30. 30.
    Shafer G (1976) A mathematical theory of evidence. Princeton University Press, PrincetonzbMATHGoogle Scholar
  31. 31.
    Dubois D, Prade H (1990) Rough fuzzy sets and fuzzy rough sets. Int J Gen Syst 17:191–209CrossRefGoogle Scholar
  32. 32.
    Xu P, Davoine F, Zha H, Denœux T (2016) Evidential calibration of binary SVM classifiers. Int J Approx Reason 72:55–70MathSciNetCrossRefGoogle Scholar
  33. 33.
    Liu Z g, Pan Q, Dezert J, Mercier G (2014) Credal classification rule for uncertain data based on belief functions. Pattern Recognit 47:2532–2541CrossRefGoogle Scholar
  34. 34.
    Djelloul M, Sari Z, Latreche K (2018) Uncertain fault diagnosis problem using neuro-fuzzy approach and probabilistic model for manufacturing systems. Appl Intell:1–18Google Scholar
  35. 35.
    Reineking T, Denœux T (2016) Active classification using belief functions and information gain maximization. Int J Approx Reason 72:43–54MathSciNetCrossRefGoogle Scholar
  36. 36.
    Liu ZG, Pan Q, Mercier G, Dezert J (2015) A new incomplete pattern classification method based on evidential reasoning. IEEE Transactions on Cybernetics 45:635–646CrossRefGoogle Scholar
  37. 37.
    Zhu F, Ye N, Yu W, Xu S, Li G (2014) Boundary detection and sample reduction for one-class support vector machines. Neurocomputing 123:166–173CrossRefGoogle Scholar
  38. 38.
    Wang L, Sui M, Li Q, Xiao H (2012) A New Method of Sample Reduction for Support Vector Classification, 2012 IEEE Asia-Pacific Services Computing Conference 301–304Google Scholar
  39. 39.
    Xia S, Xiong Z, Luo Y, Dong L, Xing C (2015) Relative density based support vector machine. Neurocomputing 149 (1424–1432CrossRefGoogle Scholar
  40. 40.
    Wang S, Li Z, Liu C, Zhang X, Zhang H (2014) Training data reduction to speed up SVM training. Appl Intell 41:405–420CrossRefGoogle Scholar
  41. 41.
    Han DQ, Dezert J, Duan ZS (2016) Evaluation of probability transformations of belief functions for decision making. IEEE Trans. Syst. Man Cybern. 46(1):93–108CrossRefGoogle Scholar
  42. 42.
    Liu ZG, Pan Q, Dezert J (2013) Evidential classifier for imprecise data based on belief functions. Knowl-Based Syst 52:246–257CrossRefGoogle Scholar
  43. 43.
    Liu ZG, Pan Q, Dezert J, Mercier G (2015) Credal c-means clustering method based on belief functions. Knowl-Based Syst 74:119–132CrossRefGoogle Scholar
  44. 44.
    Jousselme AL, Liu CS, Grenier D (2006) Measuring ambiguity in the evidence theory. IEEE Trans Syst Man Cybern 36(5):890–903CrossRefGoogle Scholar
  45. 45.
    Yager RR (2007) Entropy and specificity in a mathematical theory of evidence. Int J General Syst 9(4):249–260MathSciNetCrossRefGoogle Scholar
  46. 46.
    Smets P, Kennes R (1994) The transferable belief model. Artif Intell 66(2):191–234MathSciNetCrossRefGoogle Scholar
  47. 47.
    Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30MathSciNetzbMATHGoogle Scholar
  48. 48.
    Pawlak Z (1982) Rough sets. Int J Comput Inf Sci 11:341–356CrossRefGoogle Scholar
  49. 49.
    Chang CC, Lin CJ (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol (TIST) 2(3):27Google Scholar
  50. 50.
    M. Lichman, UCI machine learning repository, 2013 Google Scholar
  51. 51.
    Musicant JDR (1998) Ndc:normally distributed clustered datasetsGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Faculty of Computer Engineering and Information TechnologySadjad University of TechnologyMashhadIran
  2. 2.Department of Computer EngineeringSalman Institute of Higher EducationMashhadIran

Personalised recommendations