Neural Computing and Applications

, Volume 25, Issue 3–4, pp 927–935 | Cite as

Imbalanced data classification based on scaling kernel-based support vector machine

Original Article
First Online:
Received:
Accepted:

Abstract

In many classification problems, the class distribution is imbalanced. Learning from the imbalance data is a remarkable challenge in the knowledge discovery and data mining field. In this paper, we propose a scaling kernel-based support vector machine (SVM) approach to deal with the multi-class imbalanced data classification problem. We first use standard SVM algorithm to gain an approximate hyperplane. Then, we present a scaling kernel function and calculate its parameters using the chi-square test and weighting factors. Experimental results on KEEL data sets show the proposed algorithm can resolve the classifier performance degradation problem due to data skewed distribution and has a good generalization.

Keywords

Imbalance data Scaling kernel Chi-square test Support vector machine Classification 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgments

This work is partly supported by National Natural Science Foundation of China (No. 61373127), the China Postdoctoral Science Foundation (No. 20110491530), and the University Scientific Research Project of Liaoning Education Department of China (No. 2011186).

References

  1. 1.
    Menardi G, Torelli N (2014) Training and assessing classification rules with imbalanced data. Data Min Knowl Discov 28(1):92–122MATHMathSciNetCrossRefGoogle Scholar
  2. 2.
    Chawla NV, Japkowicz N, Kolcz A (2004) Editorial: special issue on learning from imbalanced data sets. SIGKDD Explor 6(1):1–6CrossRefGoogle Scholar
  3. 3.
    Mazurowski MA, Habas PA, Zurada JM, Lo JY, Baker JA, Tourassi GD (2008) Training neural network classifiers for medical decision making: the effects of imbalanced data sets on classification performance. Neural Netw 21(2–3):427–436CrossRefGoogle Scholar
  4. 4.
    Kubat M, Holte RC, Matwin S (1998) Machine learning for the detection of oil spills in satellite radar images. Mach Learn 6:195–215CrossRefGoogle Scholar
  5. 5.
    Daskalaki S, Kopanas I, Avouris N (2006) Evaluation of classifiers for an uneven class distribution problem. Appl Artif Intell 20(5):381–417CrossRefGoogle Scholar
  6. 6.
    He H, Garcia EA (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21(9):1263–1284CrossRefGoogle Scholar
  7. 7.
    Gao M, Hong X, Chen S, Harris CJ (2011) A combined SMOTE and PSO based RBF classifier for two-class imbalanced problems. Neurocomputing 74(17):3456–3466CrossRefGoogle Scholar
  8. 8.
    Kubat M, Matwin S (1997) Addressing the curse of imbalanced training sets: one-sided selection. In: Proceedings of the international conference on machine learning, 1997, pp. 179–186Google Scholar
  9. 9.
    Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 16(1):231–357Google Scholar
  10. 10.
    Liu XY, Wu JX, Zhou ZH (2009) Exploratory undersampling for class-imbalance learning. IEEE Trans Syst Man Cybern Part B Cybern 39(2):539–550CrossRefGoogle Scholar
  11. 11.
    Wang S, Li Z, Chao W, Cao Q (2012) Applying adaptive over-sampling technique based on data density and cost-sensitive SVM to imbalanced learning. The 2012 international joint conference on neural networks (IJCNN 2012). doi: 10.1109/IJCNN.2012.6252696
  12. 12.
    Gao M, Hong X, Chen S, Harris CJ (2012) Probability density function estimation based over-sampling for imbalanced two-class problems. The 2012 international joint conference on neural networks (IJCNN 2012). doi: 10.1109/IJCNN.2012.6252384
  13. 13.
    Gu Q, Cai Z, Zhu L, Huang B (2008) Data mining on imbalanced data sets. In: Proceedings of IEEE international conference on advanced computer theory and engineering (ICACTE 2008), Cairo, Egypt, 2008, pp 1020–1024Google Scholar
  14. 14.
    Sun Y, Kamel MS, Wong AKC, Wang Y (2007) Cost-sensitive boosting for classification of imbalanced data. Pattern Recognit 40(12):3358–3378MATHCrossRefGoogle Scholar
  15. 15.
    Wang BX, Japkowicz N (2010) Boosting support vector machines for imbalanced data sets. Knowl Inf Syst 25(1):1–20CrossRefGoogle Scholar
  16. 16.
    Zhang Y, Wang D (2013) A cost-sensitive ensemble method for class-imbalanced data sets. Abstract and applied analysis, 2013, Vol 2013, Article ID 196256, 6 pagesGoogle Scholar
  17. 17.
    Batuwita R, Palade V (2010) FSVM-CIL: fuzzy support vector machines for class imbalance learning. IEEE Trans Fuzzy Syst 18(3):558–571CrossRefGoogle Scholar
  18. 18.
    Cano A, Zafra A, Ventura S (2013) Weighted data gravitation classification for standard and imbalanced data. IEEE Trans Cybern 43(6):1672–1687CrossRefGoogle Scholar
  19. 19.
    Wu G, Chang EY (2003) Class-boundary alignment for imbalanced data set learning. In: ICML 2003 workshop on learning from imbalanced data sets, 2003, pp 49–56Google Scholar
  20. 20.
    Ertekin S, Huang J, Giles CL (2007) Active learning for class imbalance problem. In: Proceedings of ACM SIGIR’07, pp 823–824Google Scholar
  21. 21.
    Fu JH, Lee SL (2013) Certainty-based active learning for sampling imbalanced data sets. Neurocomputing 119:350–358CrossRefGoogle Scholar
  22. 22.
    Galar M, Fernández A, Barrenechea E, Bustince H (2012) A review on ensembles for the class imbalance problem bagging-, boosting-, and hybrid-based approaches. IEEE Trans Syst Man Cybern Part C Appl Rev 42(2):463–484CrossRefGoogle Scholar
  23. 23.
    Oh S, Lee MS, Zhang BT (2011) Ensemble learning with active example selection for imbalanced biomedical data classification. IEEE/ACM Trans Comput Biol Bioinf 8(2):316–325CrossRefGoogle Scholar
  24. 24.
    Liu Y, Yu XH, Huang JX, An AJ (2011) Combining integrated sampling with SVM ensembles for learning from imbalanced data sets. Inf Process Manage 47(4):617–631CrossRefGoogle Scholar
  25. 25.
    Yang X, Song Q, Wang Y (2007) A weighted support vector machine for data classification. Int J Pattern Recognit Artif Intell 21(5):961–976CrossRefGoogle Scholar
  26. 26.
    Hwang JP, Park S, Kim E (2011) A new weighted approach to imbalanced data classification problem via support vector machine with quadratic cost function. Expert Syst Appl 38(7):8580–8585CrossRefGoogle Scholar
  27. 27.
    Vapnik V (1995) The nature of statistical learning theory. Springer, New YorkMATHCrossRefGoogle Scholar
  28. 28.
    Hsu CW, Lin CJ (2002) A comparison of methods for multi-class support vector machines. IEEE Trans Neural Netw 13(2):415–425CrossRefGoogle Scholar
  29. 29.
    Maratea A, Petrosino A, Manzo M (2014) Adjusted F-measure and kernel scaling for imbalanced data learning. Inf Sci 257:331–341CrossRefGoogle Scholar
  30. 30.
    G. Wu, E.Y. Chang. Adaptive feature-space conformal transformation for imbalanced-data learning, In: Proceedings of the Twentieth International Conference on Machine Learning (ICML-2003), Washington, DC, 2003, pp 816–823Google Scholar
  31. 31.
    Amari S, Wu S (1999) Improving SVM classifiers by modifying kernel functions. Neural Netw 12(6):783–789CrossRefGoogle Scholar
  32. 32.
    Williams P, Li S, Feng J, Wu S (2005) Scaling the kernel function to improve performance of the support vector machine. Lect Notes Comput Sci 3496:251–312Google Scholar
  33. 33.
    Alcalá-Fdez J, Fernandez A, Luengo J, Derrac J, García S, Sánchez L, Herrera F (2011) KEEL data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. J Mult Valued Logic Soft Comput 17(2–3):255–287Google Scholar
  34. 34.
    Fawcett T (2003) ROC graphs: notes and practical considerations for researchers. HP Labs, Palo Alto, CA, Tech. Rep. HPL-2003-4Google Scholar
  35. 35.
    Lewis D, Gale W (1998) Training text classifiers by uncertainty sampling. In: Proceedings of the seventeenth annual international ACM SIGIR conference on research and development in information, New York, NY, 1998, pp 73–79Google Scholar
  36. 36.
    Wu G, Chang EY (2005) KBA: kernel boundary alignment considering imbalanced data distribution. IEEE Trans Knowl Data Eng 17(6):786–795CrossRefGoogle Scholar
  37. 37.
    Wu S, Amari S (2001) Conformal transformation of kernel functions: a data-dependent way to improve support vector machine classifiers. Neural Process Lett 15:59–67CrossRefGoogle Scholar
  38. 38.
    Specht DF (1991) A general regression neural network. IEEE Trans Neural Netw 2(6):568–576CrossRefGoogle Scholar
  39. 39.
    Specht DF (1990) Probabilistic neural networks and the polynomial adaline as complementary techniques for classification. IEEE Trans Neural Netw 1(1):112–116CrossRefGoogle Scholar
  40. 40.
    Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London 2014

Authors and Affiliations

  • Yong Zhang
    • 1
  • Panpan Fu
    • 1
    • 2
  • Wenzhe Liu
    • 1
  • Guolong Chen
    • 2
  1. 1.School of Computer and Information TechnologyLiaoning Normal UniversityDalianChina
  2. 2.College of Information EngineeringSuzhou UniversitySuzhouChina

Personalised recommendations