Abstract
In this study, we introduce an ensemble selection method for deep ensemble systems called VEGAS. The deep ensemble models include multiple layers of the ensemble of classifiers (EoC). At each layer, we train the EoC and generates training data for the next layer by concatenating the predictions for training observations and the original training data. The predictions of the classifiers in the last layer are combined by a combining method to obtain the final collaborated prediction. We further improve the prediction accuracy of a deep ensemble model by searching for its optimal configuration, i.e., the optimal set of classifiers in each layer. The optimal configuration is obtained using the Variable-Length Genetic Algorithm (VLGA) to maximize the prediction accuracy of the deep ensemble model on the validation set. We developed three operators of VLGA: roulette wheel selection for breeding, a chunk-based crossover based on the number of classifiers to generate new offsprings, and multiple random points-based mutation on each offspring. The experiments on 20 datasets show that VEGAS outperforms selected benchmark algorithms, including two well-known ensemble methods (Random Forest and XgBoost) and three deep learning methods (Multiple Layer Perceptron, gcForest, and MULES).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
Caruana, R., Niculescu-Mizil, A., Crew, G., Ksikes, A.: Ensemble selection from libraries of models. In: Proceedings of the 21st ICML (2004)
Chen, T., Guestrin, C.: Xgboost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD, pp. 785–794 (2016)
Dai, Q., Liu, Z.: Modenpbt: a modified backtracking ensemble pruning algorithm. Appl. Soft Comput. 13(11), 4292–4302 (2013)
Demsar, J.: Statistical comparisons of classifers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)
Lamini, C., Benhlima, S., Elbekri, A.: Genetic algorithm based approach for autonomous mobile robot path planning. Procedia Comput. Sci. 127, 180–189 (2018)
Luong, V., Nguyen, T., Liew, A.: Streaming active deep forest for evolving data stream classification. arXiv preprint arXiv:2002.11816 (2020)
Mitchell, M.: An Introduction to Genetic Algorithms. MIT press, New York (1998)
Nazarahari, M., Khanmirza, E., Doostie, S.: Multi-objective multi-robot path planning in continuous environment using an enhanced genetic algorithm. Expert Syst. Appl. 115, 106–120 (2019)
Nguyen, T., Dang, T., Baghel, V., Luong, V., McCall, J., Liew, A.: Evolving interval-based representation for multiple classifier fusion. Knowledge-based systems, p. 106034 (2020)
Nguyen, T., Dang, T., Liew, A., Bezdek, J.C.: A weighted multiple classifier framework based on random projection. Inf. Sci. 490, 36–58 (2019)
Nguyen, T., et al.: Deep heterogeneous ensemble. Australian J. Intell. Inf. Process. Syst. 16(1), (2019)
Nguyen, T., Luong, V., Liew, A., McCall, J., Van Nguyen, M., Ha, S.: Simultaneous meta-data and meta-classifier selection in multiple classifier system. In: Proceedings of the GECCO, pp. 39–46 (2019)
Nguyen, T., Pham, C., Liew, A., Pedrycz, W.: Aggregation of classifiers: a justifiable information granularity approach. IEEE Trans. Cybern. 49(6), 2168–2177 (2018)
Nguyen, T., Van, N., Dang, T., Luong, V., McCall, J., Liew, A.: Multi-layer heterogeneous ensemble with classifier and feature selection. In: Proceedings of the GECCO, pp. 725–733 (2020)
Partalas, I., Tsoumakas, G., Vlahavas, I.: An ensemble uncertainty aware measure for directed hill climbing ensemble pruning. Mach. Learn. 81(3), 257–282 (2010)
Qi, Z., Wang, B., Tian, Y., Zhang, P.: When ensemble learning meets deep learning: a new deep support vector machine for classification. Knowl.-Based Syst. 107, 54–60 (2016)
Qiongbing, Z., Lixin, D.: A new crossover mechanism for genetic algorithms with variable-length chromosomes for path optimization problems. Expert Syst. Appl. 60, 183–189 (2016)
Sun, Y., Xue, B., Zhang, M., Yen, G.G.: Evolving deep convolutional neural networks for image classification. IEEE Trans. Evol. Comput, 24(2), 394–407 (2019)
Sun, Y., Xue, B., Zhang, M., Yen, G.G., Lv, J.: Automatically designing CNN architectures using the genetic algorithm for image classification. IEEE Trans. Cybern. (2020)
Yang, L., Wu, X.Z., Jiang, Y., Zhou, Z.H.: Multi-label learning with deep forest. arXiv preprint arXiv:1911.06557 (2019)
Zhou, Z.H., Feng, J.: Deep forest. arXiv preprint arXiv:1702.08835 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Han, K., Pham, T., Vu, T.H., Dang, T., McCall, J., Nguyen, T.T. (2021). VEGAS: A Variable Length-Based Genetic Algorithm for Ensemble Selection in Deep Ensemble Learning. In: Nguyen, N.T., Chittayasothorn, S., Niyato, D., Trawiński, B. (eds) Intelligent Information and Database Systems. ACIIDS 2021. Lecture Notes in Computer Science(), vol 12672. Springer, Cham. https://doi.org/10.1007/978-3-030-73280-6_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-73280-6_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-73279-0
Online ISBN: 978-3-030-73280-6
eBook Packages: Computer ScienceComputer Science (R0)