KNN-based maximum margin and minimum volume hyper-sphere machine for imbalanced data classification

  • Yitian XuEmail author
  • Yuqun Zhang
  • Jiang Zhao
  • Zhiji Yang
  • Xianli Pan
Original Article


Imbalanced data classification is often met in our real life. In this paper, a novel k-nearest neighbor (KNN)-based maximum margin and minimum volume hyper-sphere machine (KNN-M3VHM) is presented for the imbalanced data classification. The basic idea is to construct two hyper-spheres with different centres and radiuses. The first one contains majority examples and the second one covers minority examples. When constructing the first hyper-sphere, we remove some redundant majority samples using k-nearest neighbor (KNN)-based strategy to balance two classes of samples. Meanwhile, we maximize the margin between two hyper-spheres and minimize their volumes, which can result in two tight boundaries around each class. Similar to the twin hyper-sphere support vector machine (THSVM), KNN-M3VHM solves two related SVM-type problems and avoids the matrix inverse operation when solving the convex optimization problems. KNN-M3VHM considers not only the within-class information but also the between-class margin, then it achieves better performance in comparison with other state-of-the-art algorithms. Experimental results on twenty-five datasets validate the significant advantages of our proposed algorithm.


Maximum margin Minimum volume K-nearest neighbor Hyper-sphere KNN-M3VHM 



The authors gratefully acknowledge the helpful comments and suggestions of the reviewers, which have improved the presentation. This work was supported in part by the Beijing Natural Science Foundation (No. 4172035) and National Natural Science Foundation of China (No. 11671010).


  1. 1.
    Vapnik V (1995) The nature of statistical learning theory. Springer, New YorkzbMATHGoogle Scholar
  2. 2.
    Wang X, Aamir R, Fu A (2015) Fuzziness based sample categorization for classifier performance improvement. J Intell Fuzzy Syst 29:1185–1196MathSciNetGoogle Scholar
  3. 3.
    Manevitz LM, Yousef M (2001) One-class SVMs for document classification. J Mach Learn Res 2(1):139–154zbMATHGoogle Scholar
  4. 4.
    Zhang W, Yoshida T, Tang X (2008) Text classification based on multi-word with support vector machine. Knowl Based Syst 21(8):879–886Google Scholar
  5. 5.
    Kaper M, Meinicke P, Grossekathoefer U (2004) BCI competition 2003-data set IIb: support vector machines for the P300 speller paradigm. IEEE Trans Biomed Eng 51:1073–1076Google Scholar
  6. 6.
    Xu Y, Wang L (2005) Fault diagnosis system based on rough set theory and support vector machine. Lecture Notes Comput Sci 3614:981–988Google Scholar
  7. 7.
    Liu Z, Wu QH, Zhang Y et al (2011) Adaptive least squares support vector machines filter for hand tremor canceling in microsurgery. Int J Mach Learn Cybern 2(1):37–47Google Scholar
  8. 8.
    Jayadeva Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29:905–910zbMATHGoogle Scholar
  9. 9.
    Fung G, Mangasarian O (2001) Proximal support vector machine classifiers. In: Provost F, Srikant R (eds) KDD '01 proceedings of the seventh ACM SIGKDD international conference on knowledge discovery and data mining. Asscociation for Computing Machinery, New York, pp 77–86Google Scholar
  10. 10.
    Ghorai S, Mukherjee A, Dutta P (2009) Nonparallel plane proximal classifier. Signal Process 89:510–522zbMATHGoogle Scholar
  11. 11.
    Fung G, Mangasarian O (2005) Multicategory proximal support vector machine classifiers. Mach Learn 59:77–97zbMATHGoogle Scholar
  12. 12.
    Peng X (2010) A \(\nu\)-twin support vector machine (\(\nu\)-TSVM) classifier and its geometric algorithms. Inf Sci 180:3863–3875MathSciNetzbMATHGoogle Scholar
  13. 13.
    Xu Y, Wang L, Zhong P (2012) A rough margin-based \(\nu\)-twin support vector machine. Neural Comput Appl 21:1307–1317Google Scholar
  14. 14.
    Kumar M, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36:7535–7543Google Scholar
  15. 15.
    Peng X (2010) TSVR: an efficient twin support vector machine for regression. Neural Netw 23:365–372zbMATHGoogle Scholar
  16. 16.
    Xu Y, Wang L (2012) A weighted twin support vector regression. Knowl Based Syst 33:92–101MathSciNetGoogle Scholar
  17. 17.
    Xu Y, Guo R (2013) A twin multi-class classification support vector machine. Cognit Comput 5(4):580–588Google Scholar
  18. 18.
    Wang X, He Q, Chen D, Yeung D (2005) A genetic algorithm for solving the inverse problem of support vector machines. Neurocomputing 68:225–238Google Scholar
  19. 19.
    Peng X, Xu D (2013) A twin hypersphere support vector machine classifier and the fast learning algorithm. Inf Sci 221:12–27MathSciNetzbMATHGoogle Scholar
  20. 20.
    He HB, Garcia EA (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21(9):1263–1284Google Scholar
  21. 21.
    Japkowicz N, Stephen S (2002) The class imbalance problem: a systematic study. Intell Data Anal 6(5):429–449zbMATHGoogle Scholar
  22. 22.
    Wei W, Li J, Cao L et al (2013) Effective detection of sophisticated online banking fraud on extremely imbalanced data. World Wide Web 16:449–475Google Scholar
  23. 23.
    Thomas C (2013) Improving intrusion detection for imbalanced network traffic. Secur Commun Netw 6:309–324Google Scholar
  24. 24.
    Khalilia M, Chakraborty S, Popescu M (2011) Predicting disease risks from highly imbalanced data using random forest. BMC Med Inform Decis Mak 11(1):51Google Scholar
  25. 25.
    Pedrajas NG, Rodriguez JP, Pedrajas MG et al (2012) Class imbalance methods for translation initiation site recognition in DNA sequences. Knowl Based Syst 25:22–34Google Scholar
  26. 26.
    Mao W, Wang J, Xue Z (2017) An ELM-based model with sparse-weighting strategy for sequential data imbalance problem. Int J Mach Learn Cybern 8(4):1333–1345Google Scholar
  27. 27.
    Vong CM, Ip WF, Wong PK, Chiu CC (2014) Predicting minority class for suspended particulate matters level by extreme learning machine. Neurocomputing 128:136–144Google Scholar
  28. 28.
    Galar M, Fernandez A, Barrenechea E, Bustince H, Herrera F (2012) A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Trans Syst Man Cybern Part C Appl Rev 42(4):463–484Google Scholar
  29. 29.
    Sun YM, Wong AKC, Kamel MS (2009) Classification of imbalanced data: a review. Int J Pattern Recognit Artif Intell 23(4):687–719Google Scholar
  30. 30.
    Zhai JH, Zhang SF, Wang CX (2017) The classification of imbalanced large data sets based on mapreduce and ensemble of ELM classifiers. J Mach Learn Cybern 8(3):1009–1017Google Scholar
  31. 31.
    Zhai JH, Wang XZ, Pang XH (2016) Voting-based instance selection from large data sets with mapreduce and random weight networks. Inf Sci 367:1066–1077Google Scholar
  32. 32.
    Zhai JH, Li T, Wang XZ (2016) A cross-selection instance algorithm. J Intell Fuzzy Syst 30(2):717–728Google Scholar
  33. 33.
    Wang X, Xing H, Li Y et al (2015) A study on relationship between generalization abilities and fuzziness of base classifiers in ensemble learning. IEEE Trans Fuzzy Syst 23(5):1638–1654Google Scholar
  34. 34.
    Tax D, Duin R (2004) Support vector data description. Mach Learn 54:45–66zbMATHGoogle Scholar
  35. 35.
    Wu M, Ye J (2009) A small sphere and large margin approach for novelty detection using training data with outliers. IEEE Trans Pattern Anal Mach Intell 31(11):2088–2092Google Scholar
  36. 36.
    Akbani R, Kwek S, Japkowicz N (2004) Applying support vector machines to imbalanced data sets. In: Boulicaut JF, Esposito F, Giannotti F, Pedreschi D (eds) Proceedings of 15th ECML, vol 3201. Springer, Berlin, Heidelberg, pp 39–50Google Scholar
  37. 37.
    Cover TM, Hart PE (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13:21–27zbMATHGoogle Scholar
  38. 38.
    Ye Q, Zhao C, Gao S, Zheng H (2012) Weighted twin support vector machines with local information and its application. Neural Netw 35:31–39zbMATHGoogle Scholar
  39. 39.
    Xu Y, Yu J, Zhang Y (2014) KNN-based weighted rough v-twin support vector machine. Knowl Based Syst 71:303–313Google Scholar
  40. 40.
    Shao Y, Chen W, Zhang J, Wang Z, Deng N (2014) An efficient weighted Lagrangian twin support vector machine for imbalanced data classification. Pattern Recognit 47:3158–3167zbMATHGoogle Scholar
  41. 41.
    Xu Y, Yang Z, Zhang Y, Pan X, Wang L (2016) A maximum margin and minimum volume hyper-spheres machine with pinball loss for imbalanced data classification. Knowl Based Syst 95:75–85Google Scholar
  42. 42.
    Demsar J (2006) Statistical comparisons of classification over multiple data sets. J Mach Learn Res 7:1–30MathSciNetzbMATHGoogle Scholar
  43. 43.
    Garca S, Fernndez A, Luengo J, Herrera F (2010) Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power. Inf Sci 180:2044–2064Google Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  • Yitian Xu
    • 1
    Email author
  • Yuqun Zhang
    • 1
  • Jiang Zhao
    • 1
  • Zhiji Yang
    • 1
  • Xianli Pan
    • 1
  1. 1.College of ScienceChina Agricultural UniversityBeijingChina

Personalised recommendations