Applied Intelligence

, Volume 48, Issue 11, pp 4212–4231 | Cite as

Entropy based fuzzy least squares twin support vector machine for class imbalance learning

  • Deepak Gupta
  • Bharat Richhariya


In classification problems, the data samples belonging to different classes have different number of samples. Sometimes, the imbalance in the number of samples of each class is very high and the interest is to classify the samples belonging to the minority class. Support vector machine (SVM) is one of the widely used techniques for classification problems which have been applied for solving this problem by using fuzzy based approach. In this paper, motivated by the work of Fan et al. (Knowledge-Based Systems 115: 87–99 2017), we have proposed two efficient variants of entropy based fuzzy SVM (EFSVM). By considering the fuzzy membership value for each sample, we have proposed an entropy based fuzzy least squares support vector machine (EFLSSVM-CIL) and entropy based fuzzy least squares twin support vector machine (EFLSTWSVM-CIL) for class imbalanced datasets where fuzzy membership values are assigned based on entropy values of samples. It solves a system of linear equations as compared to the quadratic programming problem (QPP) as in EFSVM. The least square versions of the entropy based SVM are faster than EFSVM and give higher generalization performance which shows its applicability and efficiency. Experiments are performed on various real world class imbalanced datasets and compared the results of proposed methods with new fuzzy twin support vector machine for pattern classification (NFTWSVM), entropy based fuzzy support vector machine (EFSVM), fuzzy twin support vector machine (FTWSVM) and twin support vector machine (TWSVM) which clearly illustrate the superiority of the proposed EFLSTWSVM-CIL.


Information entropy Class imbalance Fuzzy membership Least squares support vector machine (LSSVM) K-nearest neighbour (K-NN) 


  1. 1.
    Chaudhuri, De K (2010) Fuzzy support vector machine for bankruptcy prediction. Appl Soft Comput 11 (1):2472–2486Google Scholar
  2. 2.
    Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(2):273–297zbMATHGoogle Scholar
  3. 3.
    Lin C-F, Wang S-D (2002) Fuzzy support vector machines. IEEE Trans Neural Netw 13(1):464–471Google Scholar
  4. 4.
    Burges CJC (1998) Geometry and invariance in kernel based methods. In: Scholkopf B, Burges CJC, Smola AJ (eds) Advances in kernel methods-support vector learning. MIT, CambridgeGoogle Scholar
  5. 5.
    Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30MathSciNetzbMATHGoogle Scholar
  6. 6.
    Tomar D, Ojha D, Agarwal S (2014) An emotion detection system based on multi least squares twin support vector machine. Adv Artif Intell Article ID 282659:11Google Scholar
  7. 7.
    Tomar D, Agarwal S (2015) Hybrid feature selection based weighted least squares twin support vector machine approach for diagnosing breast cancer, hepatitis, and diabetes. Adv Artif Neural Syst. (Article ID 265637), 10Google Scholar
  8. 8.
    Tsujinishi D, Abe S (2003) Fuzzy least squares support vector machines. In: Proceedings of the international joint conference on neural networks. Portland, pp 1599–1604Google Scholar
  9. 9.
    Tian D-Z, Peng G-B, Ha M-H (2012) Fuzzy support vector machine based on non-equilibrium data. In: International conference on machine learning and cybernetics. Xi’an, pp 15–17Google Scholar
  10. 10.
    Borovikov E (2005) An evaluation of support vector machines as a pattern recognition tool. University of Maryland at College Park.
  11. 11.
    Osuna E, Freund R, Girosi F (1997) Training support vector machines: an application to face detection. In: Proceedings of 1997 IEEE computer society conference on computer vision and pattern recognition. IEEE, pp 130–136Google Scholar
  12. 12.
    Golub GH, Van Loan C (1996) F, Matrix computations, 3rd edn. The John Hopkins University PressGoogle Scholar
  13. 13.
    Alcalá-Fdez J, Fernandez A, Luengo J, Derrac J, García S, Sánchez L, Herrera F (2011) KEEL data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. J Multiple-Valued Logic Soft Comput 17(2–3):255–287Google Scholar
  14. 14.
    Jayadeva RK, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell (TPAMI) 29:905–910CrossRefGoogle Scholar
  15. 15.
    Suykens JAK, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9:293–300CrossRefGoogle Scholar
  16. 16.
    Suykens JAK, De Brabanter J, Lukas L, Vandewalle J (2002) Weighted least squares support vector machines: robustness and sparse approximation. Neurocomputing 48(1):85–105CrossRefGoogle Scholar
  17. 17.
    Keller J, Hunt D (1985) Incorporating fuzzy membership functions into the perceptron algorithm. IEEE Trans Pattern Anal Mach Intell 6:693–699CrossRefGoogle Scholar
  18. 18.
    Sartakhti JS, Ghadiri N, Afrabandpey H, Yousefnezhad N (2016) Fuzzy Least squares twin support vector machines. arXiv:1505.05451
  19. 19.
    Zhang J, Liu Y (2004) Cervical cancer detection using SVM-based feature screening. In: Proceedings of the seventh international conference on medical image computing and computer aided intervention, pp 873–880Google Scholar
  20. 20.
    Khan L, Awad M, Thuraisingham B (2007) A new intrusion detection system using support vector machines and hierarchical clustering. Int J Very Large Data Bases 16(3):507–521CrossRefGoogle Scholar
  21. 21.
    Kumar MA, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(3):7535–7543CrossRefGoogle Scholar
  22. 22.
    Mehrkanoon S, Suykens JAK (2015) Learning solutions to partial differential equations using LS-SVM. Neurocomputing 159:105–116CrossRefGoogle Scholar
  23. 23.
    Schmidt M, Gish H (1996) Speaker identification via support vector classifiers. In: Conference proceedings of 1996 IEEE international conference on acoustics, speech, and signa processing, 1996, ICASSP-96, vol 1. Atlanta, pp 105–108Google Scholar
  24. 24.
    Tanveer M, Khan MA, Ho S-S (2016) Robust energy-based least squares twin support vector machines. Appl Intell, CrossRefGoogle Scholar
  25. 25.
    Cristianini N, Taylor JS (1999) An introduction to support vector machines: and other kernel-based learning methods. Cambridge University Press, New YorkzbMATHGoogle Scholar
  26. 26.
    Mangasarian OL (1994) Nonlinear programming. SIAMGoogle Scholar
  27. 27.
    Phillips PJ (1998) Support vector machines applied to face recognition. In: Proceedings conference advances in neural information processing systems, vol 11, pp 803–809Google Scholar
  28. 28.
    Murphy PM, Aha DW (1992) UCI repository of machine learning databases. University of California, Irvine.
  29. 29.
    Michel P, el Kaliouby R (2003) Real time facial expression recognition in video using support vector machines. In: Proceedings of the 5th international conference on multimodal interfaces, pp 258–264, ISBN: 1-58113-621-8Google Scholar
  30. 30.
    Fan Q, Wang Z, Li D, Gao D, Zha H (2017) Entropy-based fuzzy support vector machine for imbalanced datasets. Knowl-Based Syst 115:87–99CrossRefGoogle Scholar
  31. 31.
    Tong Q, Zheng H, Wang X (2005) Gene prediction algorithm based on the statistical combination and the classification in terms of gene characteristics. Int Conf Neural Netw Brain 2:673–677Google Scholar
  32. 32.
    Batuwita R, Palade V (2010) FSVM-CIL: fuzzy support vector machines for class imbalance learning. IEEE Trans Fuzzy Syst 18(2):558–571CrossRefGoogle Scholar
  33. 33.
    Malhotra R, Malhotra DK (2003) Evaluating consumer loans using neural networks. Omega 31:83–96CrossRefGoogle Scholar
  34. 34.
    Rastogi R, Saigal P (2017) Tree-based localized fuzzy twin support vector clustering with square loss function. Applied Intelligence. CrossRefGoogle Scholar
  35. 35.
    Balasundaram S, Gupta D, Prasad SC (2017) A new approach for training Lagrangian twin support vector machine via unconstrained convex minimization. Appl Intell 46(1):124–134CrossRefGoogle Scholar
  36. 36.
    Gunn SR (1998) Support vector machines for classification and regression. ISIS technical report 14, University of SouthamptonGoogle Scholar
  37. 37.
    Zhang S, Zhao S, Sui Y, Zhang L (2015) Single object tracking with fuzzy least squares support vector machine. IEEE Trans Image Process 24:5723–5738MathSciNetCrossRefGoogle Scholar
  38. 38.
    Phu VN, Dat ND, Tran VTN, Chau VTN, Nguyen TA (2017) Fuzzy C-means for english sentiment classification in a distributed system. Appl Intell 46(2):717–738CrossRefGoogle Scholar
  39. 39.
    Vapnik VN (1998) Statistical learning theory. Wiley, New YorkzbMATHGoogle Scholar
  40. 40.
    Chen S, Wu X (2017) A new fuzzy support vector machine for pattern classification. Int J Mach Learn Cybern. CrossRefGoogle Scholar
  41. 41.
    Shao Y, Chen W, Zhang J, Wang Z, Deng N (2014) An efficient weighted Lagrangian twin support vector machine for imbalanced data classification. Pattern Recogn 47(9):3158–3167CrossRefGoogle Scholar
  42. 42.
    Shao YH, Deng NY, Yang ZM (2012) Least squares recursive projection twin support vector machine for classification. Pattern Recogn 45(6):2299–2307CrossRefGoogle Scholar
  43. 43.
    Shao YH, Chen WJ, Wang Z, Li CN, Deng NY (2015) Weighted linear loss twin support vector machine for large-scale classification. Knowl-Based Syst 73:276–288CrossRefGoogle Scholar
  44. 44.
    Bao Y-K, Liu Z-T, Guo L, Wang W (2005) Forecasting stock composite index by fuzzy support vector machines regression. In: Proceeding of international conference on machine learning and cybernetics, vol 6, pp 3535–3540Google Scholar
  45. 45.
    Wang Y, Wang S, Lai KK (2005) A new fuzzy support vector machine to evaluate credit risk. IEEE Trans Fuzzy Syst 13(6):820–831CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Computer Science & EngineeringNational Institute of TechnologyArunachal PradeshIndia

Personalised recommendations