Advertisement

Cluster Computing

, Volume 22, Supplement 3, pp 6937–6952 | Cite as

Artificial bee colony optimization-based weighted extreme learning machine for imbalanced data learning

  • Xiaofen TangEmail author
  • Li Chen
Article

Abstract

The imbalanced datasets are common in real-world application and the problem of imbalanced dataset affect classification performance of many standard learning approaches. To address imbalanced datasets, a weighted extreme learning machine (WELM) solving the L\(_{2}\)-regularized weighted least squares problem is presented to avoid the generation of an over-fitting model and obtain better generalization ability compared with ELM. However, the weight generated according to class distribution of training data leads to lack of finding optimal weight with good generalization performance and the randomness of input weight and hidden biases of network makes the algorithm produce suboptimal classification model. In this paper, a weighted extreme learning machine based on hybrid artificial bee colony (HABC) is proposed to obtain better performance than WELM, in which input weights and hidden bias of WELM and the weight assigned to training samples are optimized by the hybrid artificial bee colony algorithm. HABC combines the diversities of the perturbed parameter vectors of differential evolution with the best solution information of the artificial bee colony effectively. In the empirical study, different class imbalance data handling methods including four WELM-based methods, weighted support vector machine, four ensemble methods which combine data sampling and the Bagging or Boosting are compared with our method. The experimental results on 15 imbalanced datasets show that the proposed method outperforms most methods, which indicates its superiority.

Keywords

Artificial bee colony algorithm Weighted extreme learning machine Imbalanced data learning Single hidden layer feed-forward networks 

Notes

Acknowledgements

This study was funded by National Key Technology Science and Technique Support Program (No. 2013BAH49f03).

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Research Involving Human Participants or Animals

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

References

  1. 1.
    Huang, G.B., Zhu, Q.Y., & Siew, C.K.: Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings of 2004 IEEE International Joint Conference on Neural Networks, Vol. 2, pp. 985–990. IEEE (2004)Google Scholar
  2. 2.
    Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: theory and applications. Neurocomputing 70(1), 489–501 (2006)Google Scholar
  3. 3.
    Huang, G.B., Zhou, H., Ding, X., Zhang, R.: Extreme learning machine for regression and multiclass classification. IEEE Trans. Syst. Man Cybern. Part B 42(2), 513–529 (2012)Google Scholar
  4. 4.
    Huang, G., Huang, G.B., Song, S., You, K.: Trends in extreme learning machines: a review. Neural Netw. 61, 32–48 (2015)zbMATHGoogle Scholar
  5. 5.
    Savojardo, C., Fariselli, P., Casadio, R.: BETAWARE: a machine-learning tool to detect and predict transmembrane beta-barrel proteins in prokaryotes. Bioinformatics 29(4), 504–505 (2013)Google Scholar
  6. 6.
    Savojardo, C., Fariselli, P., Casadio, R.: Improving the detection of transmembrane \(\beta \)-barrel chains with N-to-1 extreme learning machines. Bioinformatics 27(22), 3123–3128 (2011)Google Scholar
  7. 7.
    Fan, Y.X., Shen, H.B.: Predicting pupylation sites in prokaryotic proteins using pseudo-amino acid composition and extreme learning machine. Neurocomputing 128, 267–272 (2014)Google Scholar
  8. 8.
    Lan, Y., Soh, Y.C., & Huang, G.B.: Extreme learning machine based bacterial protein subcellular localization prediction. In: IJCNN 2008, IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), pp. 1859–1863. IEEE (2008)Google Scholar
  9. 9.
    Czarnecki, W.M.: Weighted tanimoto extreme learning machine with case study in drug discovery. IEEE Comput. Intell. Mag. 10(3), 19–29 (2015)Google Scholar
  10. 10.
    Vong, C.M., Ip, W.F., Wong, P.K., Chiu, C.C.: Predicting minority class for suspended particulate matters level by extreme learning machine. Neurocomputing 128, 136–144 (2014)Google Scholar
  11. 11.
    Saraswathi, S., Sundaram, S., Sundararajan, N., Zimmermann, M., Nilsen-Hamilton, M.: ICGA-PSO-ELM approach for accurate multiclass cancer classification resulting in reduced gene sets in which genes encoding secreted proteins are highly represented. IEEE/ACM Trans. Comput. Biol. Bioinf. 8(2), 452–463 (2011)Google Scholar
  12. 12.
    Li, L.N., Ouyang, J.H., Chen, H.L., Liu, D.Y.: A computer aided diagnosis system for thyroid disease using extreme learning machine. J. Med. Syst. 36(5), 3327–3337 (2012)Google Scholar
  13. 13.
    Lin, S.J., Chang, C., Hsu, M.F.: Multiple extreme learning machines for a two-class imbalance corporate life cycle prediction. Knowl.-Based Syst. 39, 214–223 (2013)Google Scholar
  14. 14.
    Baradarani, A., Wu, Q.J., Ahmadi, M.: An efficient illumination invariant face recognition framework via illumination enhancement and DD-DTCWT filtering. Pattern Recognit. 46(1), 57–72 (2013)Google Scholar
  15. 15.
    Mohammed, A.A., Minhas, R., Wu, Q.J., Sid-Ahmed, M.A.: Human face recognition based on multidimensional PCA and extreme learning machine. Pattern Recognit. 44(10), 2588–2597 (2011)zbMATHGoogle Scholar
  16. 16.
    Gao, X., Chen, Z., Tang, S., Zhang, Y., Li, J.: Adaptive weighted imbalance learning with application to abnormal activity recognition. Neurocomputing 173, 1927–1935 (2016)Google Scholar
  17. 17.
    Mirza, B., Lin, Z., Liu, N.: Ensemble of subset online sequential extreme learning machine for class imbalance and concept drift. Neurocomputing 149, 316–329 (2015)Google Scholar
  18. 18.
    Xia, S.X., Meng, F.R., Liu, B., Zhou, Y.: A kernel clustering-based possibilistic fuzzy extreme learning machine for class imbalance learning. Cognit. Comput. 7(1), 74–85 (2015)Google Scholar
  19. 19.
    Zong, W., Huang, G.B., Chen, Y.: Weighted extreme learning machine for imbalance learning. Neurocomputing 101, 229–242 (2013)Google Scholar
  20. 20.
    Liu, N., Wang, H.: Ensemble based extreme learning machine. IEEE Signal Process. Lett. 17(8), 754–757 (2010)Google Scholar
  21. 21.
    Cao, J., Lin, Z., Huang, G.B., Liu, N.: Voting based extreme learning machine. Inf. Sci. 185(1), 66–77 (2012)MathSciNetGoogle Scholar
  22. 22.
    Sharma, R., Bist, A.S.: Genetic algorithm based weighted extreme learning machine for binary imbalance learning. In: 2015 International Conference on Cognitive Computing and Information Processing (CCIP), pp. 1–6. IEEE (2015)Google Scholar
  23. 23.
    Dudek, G.: Extreme learning machine for function approximation-interval problem of input weights and biases. In: 2015 IEEE 2nd International Conference on Cybernetics (CYBCONF), pp. 62–67. IEEE (2015)Google Scholar
  24. 24.
    Zhang, N., Qu, Y., Deng, A.: Evolutionary extreme learning machine based weighted nearest-neighbor equality classification. In: 2015 7th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC), vol. 2, pp. 274–279. IEEE (2015)Google Scholar
  25. 25.
    Hu, K., Zhou, Z., Weng, L., Liu, J., Wang, L., Su, Y., Yang, Y.: An optimization strategy for weighted extreme learning machine based on PSO. Int. J. Pattern Recognit. Artif. Intell. 31(01), 1751001 (2017)Google Scholar
  26. 26.
    Zhu, Q.Y., Qin, A.K., Suganthan, P.N., Huang, G.B.: Evolutionary extreme learning machine. Pattern Recognit. 38(10), 1759–1763 (2005)zbMATHGoogle Scholar
  27. 27.
    Batista, G.E., Prati, R.C., Monard, M.C.: A study of the behavior of several methods for balancing machine learning training data. ACM Sigkdd Explor. Newsl. 6(1), 20–29 (2004)Google Scholar
  28. 28.
    Estabrooks, A., Jo, T., Japkowicz, N.: A multiple resampling method for learning from imbalanced data sets. Comput. Intell. 20(1), 18–36 (2004)MathSciNetGoogle Scholar
  29. 29.
    Mazurowski, M.A., Habas, P.A., Zurada, J.M., Lo, J.Y., Baker, J.A., Tourassi, G.D.: Training neural network classifiers for medical decision making: the effects of imbalanced datasets on classification performance. Neural Netw. 21(2), 427–436 (2008)Google Scholar
  30. 30.
    Seiffert, C., Khoshgoftaar, T.M., Van Hulse, J., Napolitano, A.: RUSBoost: a hybrid approach to alleviating class imbalance. IEEE Trans. Syst. Man Cybern.-Part A 40(1), 185–197 (2010)Google Scholar
  31. 31.
    Liu, X.Y., Wu, J., Zhou, Z.H.: Exploratory undersampling for class-imbalance learning. IEEE Trans. Syst. Man Cybern. Part B (Cybernetics) 39(2), 539–550 (2009)Google Scholar
  32. 32.
    Galar, M., Fernandez, A., Barrenechea, E., Bustince, H., Herrera, F.: A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 42(4), 463–484 (2012)Google Scholar
  33. 33.
    Barandela, R., Valdovinos, R.M., Sánchez, J.S.: New applications of ensembles of classifiers. Pattern Anal. Appl. 6(3), 245–256 (2003)MathSciNetGoogle Scholar
  34. 34.
    Walsh, I., Pollastri, G., Tosatto, S.C.: Correct machine learning on protein sequences: a peer-reviewing perspective. Briefings Bioinf. 17(5), 831–840 (2015)Google Scholar
  35. 35.
    Arunkumar, N., Ram Kumar, K., Venkataraman, V.: Automatic detection of epileptic seizures using permutation entropy, Tsallis entropy and Kolmogorov complexity. J. Med. Imaging Health Inf. 6(2), 526–531 (2016)Google Scholar
  36. 36.
    Basturk, B., Karaboga, D.: An artificial bee colony (ABC) algorithm for numeric function optimization. IEEE Swarm Intell. Symp. 8(1), 687–697 (2006)zbMATHGoogle Scholar
  37. 37.
    Karaboga, D., Gorkemli, B., Ozturk, C., Karaboga, N.: A comprehensive survey: artificial bee colony (ABC) algorithm and applications. Artif. Intell. Rev. 42(1), 21–57 (2014)Google Scholar
  38. 38.
    Jadon, S.S., Bansal, J.C., Tiwari, R., Sharma, H.: Accelerating artificial bee colony algorithm with adaptive local search. Memet. Comput. 7(3), 215–230 (2015)Google Scholar
  39. 39.
    Ozturk, C., Karaboga, D.: Hybrid artificial bee colony algorithm for neural network training. In: 2011 IEEE Congress on Evolutionary Computation (CEC), pp. 84–88. IEEE (2011) [34] Differential evolution-a simple and efficient heuristic for global optimization over continuous spacesGoogle Scholar
  40. 40.
    Storn, R., Price, K.: Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J. Glob. Optim. 11(4), 341–359 (1997)MathSciNetzbMATHGoogle Scholar
  41. 41.
    Stephygraph, L.R., Arunkumar, N.: Brain-actuated wireless mobile robot control through an adaptive human-machine interface. Adv. Intell. Syst. Comput. 397, 537–549 (2016)Google Scholar
  42. 42.
    Fernandes, S.L., Gurupur, V.P., Sunder, N.R., Arunkumar, N., Kadry, S.: A novel nonintrusive decision support approach for heart rate measurement. Pattern Recognit. Lett.  https://doi.org/10.1016/j.patrec.2017.07.002 (2017)
  43. 43.
    Yu, T., Jan, T., Simoff, S., Debenham, J.: A hierarchical VQSVM for imbalanced data sets. In: 2007 International Joint Conference on Neural Networks, pp. 518–523. IEEE (2007)Google Scholar
  44. 44.
    Zhang, Z., Ou, J., Li, D., Zhang, S.: Optimization design of coupling beam metal damper in shear wall structures. Appl. Sci. 7(2), 137 (2017)Google Scholar
  45. 45.
    Pan, W., Chen, S., Feng, Z.: Automatic clustering of social tag using community detection. Appl. Math. Inf. Sci. 7(2), 675–681 (2013)Google Scholar
  46. 46.
    Liu, C., Li, Y., Zhang, Y., Yang, C., Hongbin, W., Qin, J., Cao, Y.: Solution-processed. undoped, deep-blue organic light-emitting diodes based on starburst oligofluorenes with a planar triphenylamine core. Chem. A Eur. J. 18(22), 6928–6934 (2012)Google Scholar
  47. 47.
    Pacifico, L.D., Ludermir, T.B.: Evolutionary extreme learning machine based on particle swarm optimization and clustering strategies. In: The 2013 International Joint Conference on Neural Networks (IJCNN), pp. 1–6. IEEE (2013)Google Scholar
  48. 48.
    López, V., Fernández, A., García, S., Palade, V., Herrera, F.: An insight into classification with imbalanced data: empirical results and current trends on using data intrinsic characteristics. Inf. Sci. 250, 113–141 (2013)Google Scholar
  49. 49.
    Bhowan, U., Johnston, M., Zhang, M.: Developing new fitness functions in genetic programming for classification with unbalanced data. IEEE Trans. Syst. Man Cybern. Part B 42(2), 406–421 (2012)Google Scholar
  50. 50.
    Bartlett, P.L.: The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network. IEEE Trans. Inf. Theory 44(2), 525–536 (1998)MathSciNetzbMATHGoogle Scholar
  51. 51.
    Barandela, R., Sánchez, J.S., Garcıa, V., Rangel, E.: Strategies for learning in class imbalance problems. Pattern Recognit. 36(3), 849–851 (2003)Google Scholar
  52. 52.
    Shao, Y.H., Chen, W.J., Zhang, J.J., Wang, Z., Deng, N.Y.: An efficient weighted Lagrangian twin support vector machine for imbalanced data classification. Pattern Recognit. 47(9), 3158–3167 (2014)zbMATHGoogle Scholar
  53. 53.
    Frank, A., Asuncion, A.: UCI Machine Learning Repository http://archive.ics.uci.edu/ml. School of Information and Computer Science, p. 213. University of California, Irvine, CA (2010)
  54. 54.
    Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)MathSciNetzbMATHGoogle Scholar
  55. 55.
    Sun, Z., Song, Q., Zhu, X., Sun, H., Xu, B., Zhou, Y.: A novel ensemble method for classifying imbalanced data. Pattern Recognit. 48(5), 1623–1637 (2015)Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Information Science and TechnologyNorthwest UniversityXi’anChina
  2. 2.School of Information EngineeringNingxia UniversityYinchuanChina

Personalised recommendations