Skip to main content
SpringerLink
Log in

Entropy based fuzzy least squares twin support vector machine for class imbalance learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In classification problems, the data samples belonging to different classes have different number of samples. Sometimes, the imbalance in the number of samples of each class is very high and the interest is to classify the samples belonging to the minority class. Support vector machine (SVM) is one of the widely used techniques for classification problems which have been applied for solving this problem by using fuzzy based approach. In this paper, motivated by the work of Fan et al. (Knowledge-Based Systems 115: 87–99 2017), we have proposed two efficient variants of entropy based fuzzy SVM (EFSVM). By considering the fuzzy membership value for each sample, we have proposed an entropy based fuzzy least squares support vector machine (EFLSSVM-CIL) and entropy based fuzzy least squares twin support vector machine (EFLSTWSVM-CIL) for class imbalanced datasets where fuzzy membership values are assigned based on entropy values of samples. It solves a system of linear equations as compared to the quadratic programming problem (QPP) as in EFSVM. The least square versions of the entropy based SVM are faster than EFSVM and give higher generalization performance which shows its applicability and efficiency. Experiments are performed on various real world class imbalanced datasets and compared the results of proposed methods with new fuzzy twin support vector machine for pattern classification (NFTWSVM), entropy based fuzzy support vector machine (EFSVM), fuzzy twin support vector machine (FTWSVM) and twin support vector machine (TWSVM) which clearly illustrate the superiority of the proposed EFLSTWSVM-CIL.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Chaudhuri, De K (2010) Fuzzy support vector machine for bankruptcy prediction. Appl Soft Comput 11 (1):2472–2486

    Google Scholar 

  2. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(2):273–297

    MATH  Google Scholar 

  3. Lin C-F, Wang S-D (2002) Fuzzy support vector machines. IEEE Trans Neural Netw 13(1):464–471

    Google Scholar 

  4. Burges CJC (1998) Geometry and invariance in kernel based methods. In: Scholkopf B, Burges CJC, Smola AJ (eds) Advances in kernel methods-support vector learning. MIT, Cambridge

  5. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  6. Tomar D, Ojha D, Agarwal S (2014) An emotion detection system based on multi least squares twin support vector machine. Adv Artif Intell Article ID 282659:11

    Google Scholar 

  7. Tomar D, Agarwal S (2015) Hybrid feature selection based weighted least squares twin support vector machine approach for diagnosing breast cancer, hepatitis, and diabetes. Adv Artif Neural Syst. (Article ID 265637), 10

  8. Tsujinishi D, Abe S (2003) Fuzzy least squares support vector machines. In: Proceedings of the international joint conference on neural networks. Portland, pp 1599–1604

  9. Tian D-Z, Peng G-B, Ha M-H (2012) Fuzzy support vector machine based on non-equilibrium data. In: International conference on machine learning and cybernetics. Xi’an, pp 15–17

  10. Borovikov E (2005) An evaluation of support vector machines as a pattern recognition tool. University of Maryland at College Park. http://www.umiacs.umd.edu/users/yab/SVMForPatternRecognition/report.pdf

  11. Osuna E, Freund R, Girosi F (1997) Training support vector machines: an application to face detection. In: Proceedings of 1997 IEEE computer society conference on computer vision and pattern recognition. IEEE, pp 130–136

  12. Golub GH, Van Loan C (1996) F, Matrix computations, 3rd edn. The John Hopkins University Press

  13. Alcalá-Fdez J, Fernandez A, Luengo J, Derrac J, García S, Sánchez L, Herrera F (2011) KEEL data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. J Multiple-Valued Logic Soft Comput 17(2–3):255–287

    Google Scholar 

  14. Jayadeva RK, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell (TPAMI) 29:905–910

    Article  Google Scholar 

  15. Suykens JAK, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9:293–300

    Article  Google Scholar 

  16. Suykens JAK, De Brabanter J, Lukas L, Vandewalle J (2002) Weighted least squares support vector machines: robustness and sparse approximation. Neurocomputing 48(1):85–105

    Article  Google Scholar 

  17. Keller J, Hunt D (1985) Incorporating fuzzy membership functions into the perceptron algorithm. IEEE Trans Pattern Anal Mach Intell 6:693–699

    Article  Google Scholar 

  18. Sartakhti JS, Ghadiri N, Afrabandpey H, Yousefnezhad N (2016) Fuzzy Least squares twin support vector machines. arXiv:1505.05451

  19. Zhang J, Liu Y (2004) Cervical cancer detection using SVM-based feature screening. In: Proceedings of the seventh international conference on medical image computing and computer aided intervention, pp 873–880

  20. Khan L, Awad M, Thuraisingham B (2007) A new intrusion detection system using support vector machines and hierarchical clustering. Int J Very Large Data Bases 16(3):507–521

    Article  Google Scholar 

  21. Kumar MA, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(3):7535–7543

    Article  Google Scholar 

  22. Mehrkanoon S, Suykens JAK (2015) Learning solutions to partial differential equations using LS-SVM. Neurocomputing 159:105–116

    Article  Google Scholar 

  23. Schmidt M, Gish H (1996) Speaker identification via support vector classifiers. In: Conference proceedings of 1996 IEEE international conference on acoustics, speech, and signa processing, 1996, ICASSP-96, vol 1. Atlanta, pp 105–108

  24. Tanveer M, Khan MA, Ho S-S (2016) Robust energy-based least squares twin support vector machines. Appl Intell, https://doi.org/10.1007/s10489-015-0751-1

    Article  Google Scholar 

  25. Cristianini N, Taylor JS (1999) An introduction to support vector machines: and other kernel-based learning methods. Cambridge University Press, New York

    MATH  Google Scholar 

  26. Mangasarian OL (1994) Nonlinear programming. SIAM

  27. Phillips PJ (1998) Support vector machines applied to face recognition. In: Proceedings conference advances in neural information processing systems, vol 11, pp 803–809

  28. Murphy PM, Aha DW (1992) UCI repository of machine learning databases. University of California, Irvine. http://www.ics.uci.edu/~mlearn

  29. Michel P, el Kaliouby R (2003) Real time facial expression recognition in video using support vector machines. In: Proceedings of the 5th international conference on multimodal interfaces, pp 258–264, ISBN: 1-58113-621-8

  30. Fan Q, Wang Z, Li D, Gao D, Zha H (2017) Entropy-based fuzzy support vector machine for imbalanced datasets. Knowl-Based Syst 115:87–99

    Article  Google Scholar 

  31. Tong Q, Zheng H, Wang X (2005) Gene prediction algorithm based on the statistical combination and the classification in terms of gene characteristics. Int Conf Neural Netw Brain 2:673–677

    Google Scholar 

  32. Batuwita R, Palade V (2010) FSVM-CIL: fuzzy support vector machines for class imbalance learning. IEEE Trans Fuzzy Syst 18(2):558–571

    Article  Google Scholar 

  33. Malhotra R, Malhotra DK (2003) Evaluating consumer loans using neural networks. Omega 31:83–96

    Article  Google Scholar 

  34. Rastogi R, Saigal P (2017) Tree-based localized fuzzy twin support vector clustering with square loss function. Applied Intelligence. https://doi.org/10.1007/s10489-016-0886-8

    Article  Google Scholar 

  35. Balasundaram S, Gupta D, Prasad SC (2017) A new approach for training Lagrangian twin support vector machine via unconstrained convex minimization. Appl Intell 46(1):124–134

    Article  Google Scholar 

  36. Gunn SR (1998) Support vector machines for classification and regression. ISIS technical report 14, University of Southampton

  37. Zhang S, Zhao S, Sui Y, Zhang L (2015) Single object tracking with fuzzy least squares support vector machine. IEEE Trans Image Process 24:5723–5738

    Article  MathSciNet  Google Scholar 

  38. Phu VN, Dat ND, Tran VTN, Chau VTN, Nguyen TA (2017) Fuzzy C-means for english sentiment classification in a distributed system. Appl Intell 46(2):717–738

    Article  Google Scholar 

  39. Vapnik VN (1998) Statistical learning theory. Wiley, New York

    MATH  Google Scholar 

  40. Chen S, Wu X (2017) A new fuzzy support vector machine for pattern classification. Int J Mach Learn Cybern. https://doi.org/10.1007/s13042-017-0664-x

    Article  Google Scholar 

  41. Shao Y, Chen W, Zhang J, Wang Z, Deng N (2014) An efficient weighted Lagrangian twin support vector machine for imbalanced data classification. Pattern Recogn 47(9):3158–3167

    Article  Google Scholar 

  42. Shao YH, Deng NY, Yang ZM (2012) Least squares recursive projection twin support vector machine for classification. Pattern Recogn 45(6):2299–2307

    Article  Google Scholar 

  43. Shao YH, Chen WJ, Wang Z, Li CN, Deng NY (2015) Weighted linear loss twin support vector machine for large-scale classification. Knowl-Based Syst 73:276–288

    Article  Google Scholar 

  44. Bao Y-K, Liu Z-T, Guo L, Wang W (2005) Forecasting stock composite index by fuzzy support vector machines regression. In: Proceeding of international conference on machine learning and cybernetics, vol 6, pp 3535–3540

  45. Wang Y, Wang S, Lai KK (2005) A new fuzzy support vector machine to evaluate credit risk. IEEE Trans Fuzzy Syst 13(6):820–831

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science & Engineering, National Institute of Technology, Arunachal Pradesh, India

    Deepak Gupta & Bharat Richhariya

Authors
  1. Deepak Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bharat Richhariya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Deepak Gupta.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gupta, D., Richhariya, B. Entropy based fuzzy least squares twin support vector machine for class imbalance learning. Appl Intell 48, 4212–4231 (2018). https://doi.org/10.1007/s10489-018-1204-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-018-1204-4

Keywords

Search

Navigation