Ensemble Learning Based on Multimodal Multiobjective Optimization

Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1159)


In ensemble learning, the accuracy and diversity are two conflicting objectives. As the number of base learners increases, the prediction speed of ensemble learning machines drops significantly and the required storage space also increases rapidly. How to balance these two goals for selective ensemble learning is an extremely essential problem. In this paper, ensemble learning based on multimodal multiobjective optimization is studied in detail. The great significance and importance of multimodal multiobjective optimization algorithm is to find these different classifiers ensemble by considering the balance between accuracy and diversity, and different classifiers ensemble correspond to the same accuracy and diversity. Experimental results show that multimodal multiobjective optimization algorithm can find more ensemble combinations than unimodal optimization algorithms.


Selective ensemble learning Ensemble learning Multimodal multiobjective optimization Learner diversity 



This work is supported by the National Natural Science Foundation of China (61976237,61922072, 61876169, 61673404).


  1. 1.
    Li, W., Ding, S., Wang, H., Chen, Y., Yang, S.: Heterogeneous ensemble learning with feature engineering for default prediction in peer-to-peer lending in China. World Wide Web 23, 23–45 (2020). Scholar
  2. 2.
    Barushka, A., Hajek, P.: Spam filtering in social networks using regularized deep neural networks with ensemble learning. In: Iliadis, L., Maglogiannis, I., Plagianakos, V. (eds.) AIAI 2018. IAICT, vol. 519, pp. 38–49. Springer, Cham (2018). Scholar
  3. 3.
    Bekiroglu, K., Duru, O., Gulay, E., Su, R., Lagoa, C.: Predictive analytics of crude oil prices by utilizing the intelligent model search engine. Appl. Energy 228, 2387–2397 (2018)CrossRefGoogle Scholar
  4. 4.
    Lemaître, G., Nogueira, F., Aridas, C.K.: Imbalanced-learn: a python toolbox to tackle the curse of imbalanced datasets in machine learning. J. Mach. Learn. Res. 18(1), 559–563 (2017)zbMATHGoogle Scholar
  5. 5.
    Oh, S., Lee, M.S., Zhang, B.T.: Ensemble learning with active example selection for imbalanced biomedical data classification. IEEE/ACM Trans. Comput. Biol. Bioinf. 8(2), 316–325 (2010)Google Scholar
  6. 6.
    Zhang, C., Ma, Y.: Ensemble Machine Learning: Methods and Applications. Springer, Boston (2012). Scholar
  7. 7.
    Zhou, Z.H., Wu, J., Wei, T.: Ensembling neural networks: many could be better than all. Artif. Intell. 137(1), 239–263 (2002)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Islam, M.M., Xin, Y.: Evolving artificial neural network ensembles. IEEE Comput. Intell. Mag. 3(1), 31–42 (2008)CrossRefGoogle Scholar
  9. 9.
    Pan, L., He, C., Tian, Y., Wang, H., Zhang, X., Jin, Y.: A classification-based surrogate-assisted evolutionary algorithm for expensive many-objective optimization. IEEE Trans. Evol. Comput. 23(1), 74–88 (2018)CrossRefGoogle Scholar
  10. 10.
    Pan, L., He, C., Tian, Y., Su, Y., Zhang, X.: A region division based diversity maintaining approach for many-objective optimization. Integr. Comput. Aided Eng. 24(3), 279–296 (2017)CrossRefGoogle Scholar
  11. 11.
    He, C., Tian, Y., Jin, Y., Zhang, X., Pan, L.: A radial space division based evolutionary algorithm for many-objective optimization. Appl. Soft Comput. 61, 603–621 (2017)CrossRefGoogle Scholar
  12. 12.
    Pan, L., Li, L., He, C., Tan, K.C.: A subregion division-based evolutionary algorithm with effective mating selection for many-objective optimization. IEEE Trans. 2019, 1–14 (2019)Google Scholar
  13. 13.
    Bui, L.T., Dinh, T.T.H., et al.: A novel evolutionary multi-objective ensemble learning approach for forecasting currency exchange rates. Data Knowl. Eng. 114, 40–66 (2018)CrossRefGoogle Scholar
  14. 14.
    Ojha, V.K., Abraham, A., Snášel, V.: Ensemble of heterogeneous flexible neural trees using multiobjective genetic programming. Appl. Soft Comput. 52, 909–924 (2017)CrossRefGoogle Scholar
  15. 15.
    Mocanu, D.C., Mocanu, E., Stone, P., Nguyen, P.H., Gibescu, M., Liotta, A.: Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science. Nat. Commun. 9(1), 2383 (2018). Scholar
  16. 16.
    Hu, J., Li, T., Luo, C., Fujita, H., Yang, Y.: Incremental fuzzy cluster ensemble learning based on rough set theory. Knowl.-Based Syst. 132, 144–155 (2017)CrossRefGoogle Scholar
  17. 17.
    Yang, D., Liu, Y., Li, S., Li, X., Ma, L.: Gear fault diagnosis based on support vector machine optimized by artificial bee colony algorithm. Mech. Mach. Theory 90, 219–229 (2015)CrossRefGoogle Scholar
  18. 18.
    Ni, Z., Zhang, C., Ni, L.: Haze forecast method of selective ensemble based on glowworm swarm optimization algorithm. Int. J. Pattern Recognit. Artif. Intell. 29(2), 143–153 (2016)MathSciNetGoogle Scholar
  19. 19.
    Yong, Z., Bo, L., Fan, Y.: Differential evolution based selective ensemble of extreme learning machine. In: Trustcom/BigDataSE/ISPA, pp.1327–1333 (2017)Google Scholar
  20. 20.
    Liu, Y., Yao, X., Higuchi, T.: Evolutionary ensembles with negative correlation learning. IEEE Trans. Evol. Comput. 4(4), 380–387 (2000)CrossRefGoogle Scholar
  21. 21.
    Sheng, W., Shan, P., Chen, S., Liu, Y., Alsaadi, F.E.: A niching evolutionary algorithm with adaptive negative correlation learning for neural network ensemble. Neurocomputing 247, 173–182 (2017)CrossRefGoogle Scholar
  22. 22.
    Kottathra, K., Attikiouzel, Y.: A novel multicriteria optimization algorithm for the structure determination of multilayer feedforward neural networks. J. Netw. Comput. Appl. 19(2), 135–147 (1996)CrossRefGoogle Scholar
  23. 23.
    Kupinski, M.A., Anastasio, M.A.: Multiobjective genetic optimization of diagnostic classifiers with implications for generating receiver operating characteristic curves. IEEE Trans. Med. Imaging 18(8), 675–685 (1999)CrossRefGoogle Scholar
  24. 24.
    Chandra, A., Yao, X.: Ensemble learning using multi-objective evolutionary algorithms. J. Math. Model. Algorithms 5(4), 417–445 (2006). Scholar
  25. 25.
    Thompson, S.: Pruning boosted classifiers with a real valued genetic algorithm. In: Miles, R., Moulton, M., Bramer, M. (eds.) Research and Development in Expert Systems XV, pp. 133–146. Springer, London (1999). Scholar
  26. 26.
    Zhou, Z.-H., Tang, W.: Selective ensemble of decision trees. In: Wang, G., Liu, Q., Yao, Y., Skowron, A. (eds.) RSFDGrC 2003. LNCS (LNAI), vol. 2639, pp. 476–483. Springer, Heidelberg (2003). Scholar
  27. 27.
    Mao, W., Tian, M., Cao, X., Xu, J.: Model selection of extreme learning machine based on multi-objective optimization. Neural Comput. Appl. 22(3–4), 521–529 (2013). Scholar
  28. 28.
    Pavelski, L.M., Delgado, M.R., Almeida, C.P., Gonçalves, R.A., Venske, S.M.: Extreme learning surrogate models in multi-objective optimization based on decomposition. Neurocomputing 180, 55–67 (2016)CrossRefGoogle Scholar
  29. 29.
    Deb, K.: Multi-objective genetic algorithms: problem difficulties and construction of test problems. Evol. Comput. 7(3), 205–230 (1999)CrossRefGoogle Scholar
  30. 30.
    Liang, J., Yue, C., Qu, B.: Multimodal multi-objective optimization: a preliminary study. In: 2016 IEEE Congress on Evolutionary Computation (CEC), pp. 2454–2461. IEEE (2016)Google Scholar
  31. 31.
    Yue, C., Qu, B., Liang, J.: A multiobjective particle swarm optimizer using ring topology for solving multimodal multiobjective problems. IEEE Trans. Evol. Comput. 22(5), 805–817 (2017)CrossRefGoogle Scholar
  32. 32.
    Liang, J., Guo, Q., Yue, C., Qu, B., Yu, K.: A self-organizing multi-objective particle swarm optimization algorithm for multimodal multi-objective problems. In: Tan, Y., Shi, Y., Tang, Q. (eds.) ICSI 2018. LNCS, vol. 10941, pp. 550–560. Springer, Cham (2018). Scholar
  33. 33.
    Yue, C., Qu, B., Yu, K., Liang, J., Li, X.: A novel scalable test problem suite for multimodal multiobjective optimization. Swarm Evol. Comput. 48, 62–71 (2019)CrossRefGoogle Scholar
  34. 34.
    Dua, D., Graff, C.: UCI machine learning repository (2017).
  35. 35.
    Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  1. 1.School of Electrical EngineeringZhengzhou UniversityZhengzhouChina
  2. 2.School of Electronic and Information EngineeringZhongyuan University of TechnologyZhengzhouChina

Personalised recommendations