Abstract
Convolutional Neural Networks (CNNs) have proven to be an effective tool in many real-world applications. The main problem of CNNs is the lack of a well-defined and largely shared set of criteria for the choice of architecture for a given problem. This lack represents a drawback for this approach since the choice of architecture plays a crucial role in CNNs’performance. Usually, these architectures are manually designed by experts. However, such a design process is computationally intensive because of the trial-and-error process and also not easy to realize due to the high level of expertise required. Recently, to try to overcome those drawbacks, many techniques that automize the task of designing the architecture neural networks have been proposed. To denote these techniques has been defined the term “Neural Architecture Search” (NAS). Among the many methods available for NAS, Evolutionary Computation (EC) methods have recently gained much attention and success. In this paper, we present a novel approach based on evolutionary computation to optimize CNNs. The proposed approach is based on a newly devised structure which encodes both hyperparameters and the architecture of a CNN. The experimental results show that the proposed approach allows us to achieve better performance than that achieved by state-of-the-art CNNs on a real-world problem. Furthermore, the proposed approach can generate smaller networks than the state-of-the-art CNNs used for the comparison.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
An, S., Lee, M., Park, S., Yang, H., So, J.: An ensemble of simple convolutional neural network models for mnist digit recognition. arXiv preprint arXiv:2008.10400 (2020)
Baker, B., Gupta, O., Naik, N., Raskar, R.: Designing neural network architectures using reinforcement learning. In: International Conference on Learning Representations (2017)
Bruno, A., Moroni, D., Martinelli, M.: Efficient adaptive ensembling for image classification. arXiv preprint arXiv:2206.07394 (2022)
Chu, X., Zhang, B., Ma, H., Xu, R., Li, Q.: Fast, accurate and lightweight super-resolution with neural architecture search. In: 2020 25th International Conference on Pattern Recognition (ICPR), pp. 59–64. IEEE Computer Society, Los Alamitos (2021)
Cilia, N., De Stefano, C., Fontanella, F., Scotto Di Freca, A.: An experimental protocol to support cognitive impairment diagnosis by using handwriting analysis. In: Procedia Computer Science, Proceeding of the 8th International Conference on Current and Future Trends of Information and Communication Technologies in Healthcare (ICTH), pp. 1–9. Elsevier (2019)
Cilia, N.D., D’Alessandro, T., De Stefano, C., Fontanella, F., Molinara, M.: From online handwriting to synthetic images for alzheimer’s disease detection using a deep transfer learning approach. IEEE J. Biomed. Health Inform. 25(12), 4243–4254 (2021)
Cilia, N.D., D’Alessandro, T., Stefano, C.D., Fontanella, F.: Deep transfer learning algorithms applied to synthetic drawing images as a tool for supporting alzheimer’s disease prediction. Mach. Vis. Appl. 33(3), 49 (2022)
Hussain, A., Muhammad, Y.S., Nawaz, A.: Optimization through genetic algorithm with a new and efficient crossover operator. Int. J. Adv. Math. 2018, 14 (2018)
Krizhevsky, A.: Learning multiple layers of features from tiny images (2009). https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf
Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Liu, Y., Sun, Y., Xue, B., Zhang, M., Yen, G.G., Tan, K.C.: A survey on evolutionary neural architecture search. IEEE Trans. Neural Netw. Learn. Syst. 34(2), 550–570 (2023)
Pham, H., Guan, M., Zoph, B., Le, Q., Dean, J.: Efficient neural architecture search via parameters sharing. In: Dy, J., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 80, pp. 4095–4104. PMLR (2018). https://proceedings.mlr.press/v80/pham18a.html
Real, E., et al.: Large-scale evolution of image classifiers. In: Proceedings of the 34th International Conference on Machine Learning, vol. 70, pp. 2902–2911. ICML’17, JMLR.org (2017)
Ren, P., et al.: A comprehensive survey of neural architecture search: challenges and solutions. ACM Comput. Surv. (CSUR) 54(4), 1–34 (2021)
Sun, Y., Xue, B., Zhang, M., Yen, G.G.: Completely automated CNN architecture design based on blocks. IEEE Trans. Neural Netw. Learn. Syst. 31(4), 1242–1254 (2020)
Xie, L., Yuille, A.: Genetic CNN. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2017)
Zoph, B., Vasudevan, V., Shlens, J., Le, Q.V.: Learning transferable architectures for scalable image recognition. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8697–8710. IEEE Computer Society, Los Alamitos (2018)
Zoph, B., Le, Q.: Neural architecture search with reinforcement learning. In: International Conference on Learning Representations (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Bria, A., De Ciccio, P., D’Alessandro, T., Fontanella, F. (2023). A Novel Evolutionary Approach for Neural Architecture Search. In: De Stefano, C., Fontanella, F., Vanneschi, L. (eds) Artificial Life and Evolutionary Computation. WIVACE 2022. Communications in Computer and Information Science, vol 1780. Springer, Cham. https://doi.org/10.1007/978-3-031-31183-3_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-31183-3_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-31182-6
Online ISBN: 978-3-031-31183-3
eBook Packages: Computer ScienceComputer Science (R0)