Abstract
The choice of an architecture is crucial for the performance of the neural network, and thus automatic methods for architecture search have been proposed to provide a data-dependent solution to this problem. In this paper, we deal with an automatic neural architecture search for convolutional neural networks. We propose a novel approach for architecture selection based on multi-objective evolutionary optimisation. Our algorithm optimises not only the performance of the network, but it controls also the size of the network, in terms of the number of network parameters. The proposed algorithm is evaluated on experiments, including MNIST and fashionMNIST classification problems. Our approach outperforms both the considered baseline architectures and the standard genetic algorithm.
This work was partially supported by the Czech Grant Agency grant 18-23827S and the long-term strategic development financing of the Institute of Computer Science RVO 67985807.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Keras + Hyperopt: A very simple wrapper for convenient hyperparameter optimization. http://maxpumperla.com/hyperas/
Autonomio talos [computer software] (2019). http://github.com/autonomio/talos
Bergstra, J., Yamins, D., Cox, D.D.: Making a science of model search: hyperparameter optimization in hundreds of dimensions for vision architectures. In: Proceedings of the 30th International Conference on Machine Learning (ICML 2013) (2013)
Chollet, F.: Keras (2015). https://github.com/fchollet/keras
Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002). https://doi.org/10.1109/4235.996017
Elsken, T., Metzen, J.H., Hutter, F.: Efficient multi-objective neural architecture search via Lamarckian evolution. In: International Conference on Learning Representations (2019). https://openreview.net/forum?id=ByME42AqK7
Elsken, T., Metzen, J.H., Hutter, F.: Neural architecture search: a survey. J. Mach. Learn. Res. 20(55), 1–21 (2019). http://jmlr.org/papers/v20/18-598.html
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016). http://www.deeplearningbook.org
Goodfellow, I., et al.: TensorFlow: large-scale machine learning on heterogeneous systems (2015). Software available from tensorflow.org. https://www.tensorflow.org/
Jin, H., Song, Q., Hu, X.: Auto-keras: an efficient neural architecture search system. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1946–1956. ACM (2019)
Lecun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015). https://doi.org/10.1038/nature14539
Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998). https://doi.org/10.1109/5.726791
LeCun, Y., Cortes, C.: The MNIST database of handwritten digits (2012). http://research.microsoft.com/apps/pubs/default.aspx?id=204699
Miikkulainen, R., et al.: Evolving deep neural networks. CoRR abs/1703.00548 (2017). http://arxiv.org/abs/1703.00548
Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002). http://nn.cs.utexas.edu/?stanley:ec02
Vidnerová, P., Neruda, R.: Evolution strategies for deep neural network models design. In: Proceedings ITAT 2017: Information Technologies - Applications and Theory. Aachen & Charleston: Technical University & CreateSpace Independent Publishing Platform, 2017 - (Hlaváčová, J.), CEUR Workshop Proceedings, V-1885, pp. 159–166 (2017)
Vidnerova, P., Neruda, R.: Evolving keras architectures for sensor data analysis. In: 2017 Federated Conference on Computer Science and Information Systems (FedCSIS), pp. 109–112, September 2017. https://doi.org/10.15439/2017F241
Vidnerová, P., Neruda, R.: Asynchronous evolution of convolutional networks. In: Proceedings ITAT 2018 (2018)
Vidnerová, P., Procházka, Š.: NSGA-keras: neural architecture search for keras sequential models. https://github.com/PetraVidnerova/nsga-keras
Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Vidnerová, P., Procházka, Š., Neruda, R. (2020). Multiobjective Evolution for Convolutional Neural Network Architecture Search. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds) Artificial Intelligence and Soft Computing. ICAISC 2020. Lecture Notes in Computer Science(), vol 12415. Springer, Cham. https://doi.org/10.1007/978-3-030-61401-0_25
Download citation
DOI: https://doi.org/10.1007/978-3-030-61401-0_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-61400-3
Online ISBN: 978-3-030-61401-0
eBook Packages: Computer ScienceComputer Science (R0)