Abstract
This paper provides a solution to select a suitable architecture of convolutional neural network (CNN). A hybrid evolutionary gradient descent (HyEGD) approach is proposed to automatically evolve the architecture of CNN. The evolution of the structure is done using compact genetic algorithm (cGA) by optimizing the number of filters in each layer, and simultaneously, the associated weight parameters are tuned by stochastic gradient descent (SGD). This brings forth an effective way to search the solution space seamlessly integrating both exploration, spearheaded by cGA, and the exploitation, naturally done by SGD. Moreover, using HyEGD approach, the user specified architecture can also be evolved trading-off between two objectives—network performance on one side and network size on the other side. Experiments to illustrate the salient features of the HyEGD approach are performed on two benchmark problems: COIL-20 dataset and MNIST dataset. The results clearly highlight the powerful capability of generating architectures based on the required performance and size of network.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abraham, A., Nath, B.: Evolutionary design of neuro-fuzzy systems—a generic framework. In: Proceedings of the 4th Japan-Australia Joint Workshop on Intelligent and Evolutionary Systems, pp. 106–113 (2000)
Ahn, C.W., Ramakrishna, R.S.: Elitism-based compact genetic algorithms. IEEE Trans. Evol. Comput. 7(4), 367–385 (2003)
Cordón, O.: A historical review of evolutionary learning methods for Mamdani-type fuzzy rule-based systems: designing interpretable genetic fuzzy systems. Int. J. Approx. Reason. 52(6), 894–913 (2011)
Das, S., Suganthan, P.N.: Differential evolution: a survey of the state-of-the-art. IEEE Trans. Evol. Comput. 15(1), 4–31 (2011)
Frenzel, J.F.: Genetic algorithm. IEEE Potentials 12(3), 21–24 (1993)
Harik, G.R., Lobo, F.G., Goldberg, D.E.: The compact genetic algorithm. IEEE Trans. Evol. Comput. 3(4), 287–297 (1999)
Huang, P., Huang, Y., Wang, W., Wang, L.: Deep embedding network for clustering. In: 22nd International Conference on Pattern Recognition, Sweden, 24–28 Aug, pp. 1532–1537 (2014)
Karaboga, D., Basturk, B.: A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm. J. Glob. Optim. 39(3), 459–471 (2007)
LeCun, Y., Bengio, Y., Hinton, G.E.: Deep learning. Nature 521(7553), 436–444 (2015)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. In: Haykin, S., Kosko, B. (eds.) Intelligent Signal Processing, vol. 86, pp. 306–351. IEEE Press (2001)
LeCun, Y., Cortes, C., Burges, C.J.C.: The MNIST database for handwritten digits. http://yann.lecun.com/exdb/mnist/
Li, F., Qiao, H., Zhang, B., Xi, X.: Discriminatively boosted image clustering with fully convolutional auto-encoders (2017) (Online). https://arxiv.org/abs/1703.07980
Mininno, E., Cupertino, F., Naso, D.: Real-valued compact genetic algorithms for embedded microcontroller optimization. IEEE Trans. Evol. Comput. 12(2), 203–219 (2008)
Mininno, E., Neri, F., Cupertino, F., Naso, D.: Compact differential evolution. IEEE Trans. Evol. Comput. 15(1), 32–54 (2011)
Nene, S.A., Nayar, S.K., Murase, H.: Columbia object image library (Coil-20). Technical Report No. CUCS-006-96, Department of Computer Science, Columbia University, New York, NY, United States (1996)
Paul, S., Kumar, S., Singh, L.: Novel hybrid compact genetic algorithm for simultaneous structure and parameter learning of neural networks. In: Proceedings of the IEEE Congress on Evolutionary Computation, CEC 2012, Brisbane, Australia, 10–15 June, 2012. pp. 1–6 (2012)
Poli, R., Kennedy, J., Blackwell, T.: Particle swarm optimization. Swarm Intell. 1(1), 33–57 (2007)
Real, E., Moore, S., Selle, A., Saxena, S., Suematsu, Y.L., Le, Q.V., Kurakin, A.: Large-scale evolution of image classifiers (2017) (Online). https://arxiv.org/abs/1703.01041
Rere, L.M.R., Fanany, M.I., Arymurthy, A.M.: Metaheuristic algorithms for convolution neural network (2016) (Online). https://arxiv.org/abs/1610.01925
Salimans, T., Ho, J., Chen, X., Sutskever, I.: Evolution strategies as a scalable alternative to reinforcement learning (2017) (Online). https://arxiv.org/abs/1703.03864
Singh, L., Kumar, S., Paul, S.: Automatic simultaneous architecture and parameter search in fuzzy neural network learning using novel variable length crossover differential evolution. In: Proceedings of FUZZ-IEEE 2008, IEEE International Conference on Fuzzy Systems, pp. 1795–1802 (2008)
Wang, Y., Xu, C., Qiu, J., Xu, C., Tao, D.: Towards evolutional compression (2017) (Online). https://arxiv.org/abs/1707.08005
Xie, L., Yuille, A.: Genetic CNN (2017) (Online). https://arxiv.org/abs/1703.01513
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Soniya, Paul, S., Singh, L. (2019). Simultaneous Structure and Parameter Learning of Convolutional Neural Network. In: Verma, N., Ghosh, A. (eds) Computational Intelligence: Theories, Applications and Future Directions - Volume II. Advances in Intelligent Systems and Computing, vol 799. Springer, Singapore. https://doi.org/10.1007/978-981-13-1135-2_8
Download citation
DOI: https://doi.org/10.1007/978-981-13-1135-2_8
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-1134-5
Online ISBN: 978-981-13-1135-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)