Abstract
Convolutional neural network has demonstrated high performance in many real-world problems in recent years. However, the results and accuracy of a CNN that are applied for a specific problem are highly controlled by the architecture and its hyperparameters. The process of finding the right set of hyperparameters for the network’s architecture is a very time-consuming process and requires expertise. To address this problem, we present a powerful method that does automatic hyperparameter search for designing CNN architecture. The hyperparameter optimization is performed by the artificial flora optimization algorithm. The proposed framework has the ability to explore different architectures and optimize the values of hyperparameters for a given task. In this research, the proposed framework is validated on MNIST image classification task and it can be concluded from the experimental results that the suggested search method accomplishes promising achievement in this domain.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Farabet, C., Couprie, C., Najman, L., LeCun, Y.: Learning hierarchical features for scene labeling. IEEE Trans. Pattern Anal. Mach. Intell. 35, 1915–1929 (2013)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, pp. 91–99 (2015)
Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Deepface: closing the gap to human-level performance in face verification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1701–1708 (2014)
Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 25, pp. 1097–1105, Curran Associates, Inc., (2012)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: 3rd International Conference on Learning Representations (ICLR) (2015)
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9 (2015)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Huang, G., Liu, Z., Weinberger, K.Q.: Densely connected convolutional networks. CoRR, vol. abs/1608.06993 (2016)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)
Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13(1), 281–305 (2012)
Dolicanin, E., Fetahovic, I., Tuba, E., Capor-Hrosik, R., Tuba, M.: Unmanned combat aerial vehicle path planning by brain storm optimization algorithm. Stud. Inform. Control 27(1), 15–24 (2018)
Strumberger, I., Tuba, E., Bacanin, N., Beko, M., Tuba, M.: Wireless sensor network localization problem by hybridized moth search algorithm. In: 2018 14th International Wireless Communications Mobile Computing Conference (IWCMC), pp. 316–321 (2018)
Young, S.R., Rose, D.C., Karnowski, T.P., Lim, S.-H., Patton, R.M.: Optimizing deep learning hyper-parameters through an evolutionary algorithm. In: Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments, pp. 4:1–4:5, ACM (2015)
Bochinski, E., Senst, T., Sikora, T.: Hyper-parameter optimization for convolutional neural network committees based on evolutionary algorithms. In: 2017 IEEE International Conference on Image Processing, pp. 3924–3928 (2017)
Cheng, L., Wu, X.-h., Wang, Y.: Artificial flora (AF) optimization algorithm. Appl. Sci. 8, 329:1–22 (2018)
Tuba, E., Tuba, M., Dolicanin, E.: Adjusted fireworks algorithm applied to retinal image registration. Stud. Inform. Control 26(1), 33–42 (2017)
Tuba, E., Strumberger, I., Bacanin, N., Tuba, M.: Bare bones fireworks algorithm for capacitated p-median problem. In: LNCS: Advances in Swarm Intelligence, (Cham), pp. 283–291, Springer, Berlin (2018)
Alihodzic, A., Tuba, E., Simian, D., Tuba, V., Tuba, M.: Extreme learning machines for data classification tuning by improved bat algorithm. In: International Joint Conference on Neural Networks (IJCNN), pp. 1–8, IEEE (2018)
Tuba, E., Strumberger, I., Zivkovic, D., Bacanin, N., Tuba, M.: Mobile robot path planning by improved brain storm optimization algorithm. In: 2018 IEEE Congress on Evolutionary Computation (CEC), pp. 1–8 (2018)
Strumberger, I., Tuba, E., Bacanin, N., Beko, M., Tuba, M.: Modified monarch butterfly optimization algorithm for RFID network planning. In: 6th International Conference on Multimedia Computing and Systems (ICMCS), pp. 1–6 (2018)
LeCun, Y., Cortes, C., Burges, C.: MNIST handwritten digit database. AT&T Labs. http://yann.lecun.com/exdb/mnist, vol. 2, p. 18 (2010)
Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML’10, (USA), pp. 807–814. Omnipress (2010)
Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Lee, C.-Y., Xie, S., Gallagher, P., Zhang, Z., Tu, Z.: Deeply-supervised nets. In: Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics 38, pp. 562–570 (2015)
Mcdonnell, M., Vladusich, T.: Enhanced image classification with a fast-learning shallow convolutional neural network. In: 2015 International Joint Conference on Neural Networks (IJCNN), pp. 1–7 (2015)
Liang, M., Hu, X.: Recurrent convolutional neural network for object recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3367–3375 (2015)
Lee, C.-Y., Gallagher, P.W., Tu, Z.: Generalizing pooling functions in convolutional neural networks: mixed, gated, and tree. In: Artificial Intelligence and Statistics, pp. 464–472 (2015)
Acknowledgements
Ministry of Education and Science of Republic of Serbia, Grant No. III-44006 supported this research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Bezdan, T., Tuba, E., Strumberger, I., Bacanin, N., Tuba, M. (2020). Automatically Designing Convolutional Neural Network Architecture with Artificial Flora Algorithm. In: Tuba, M., Akashe, S., Joshi, A. (eds) ICT Systems and Sustainability. Advances in Intelligent Systems and Computing, vol 1077. Springer, Singapore. https://doi.org/10.1007/978-981-15-0936-0_39
Download citation
DOI: https://doi.org/10.1007/978-981-15-0936-0_39
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-0935-3
Online ISBN: 978-981-15-0936-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)