Abstract
Genetic Algorithm (GA) is a meta-heuristics search and optimization approach which we have utilized for fine-tuning the standard deep learning models for reducing the storage space with improvement in inference time. The pre-trained models have been widely acclaimed as the best choice of CNN for various image classification problems in different domains, but they require huge storage space and it causes a problem for deploying these models on mobile or edge devices. As these devices are constrained with limited memory and computational power. In this paper, a novel GA based method has been proposed which compresses and accelerates the CNN models so that these models can be easily deployed on edge devices. Extensive computer simulations have been conducted on widely used models such as AlexNet, VGG16, SqueezeNet, and ResNet50. We used benchmark datasets such as MNIST, CIFAR-10, and CIFAR-100 to determine the performance. Results reveal that storage space of the AlexNet model was reduced by 87.5%, 86.55% and 86.16% on MNIST, CIFAR-10 and CIFAR- 100 datasets respectively, while VGG16, ResNet50, and SqueezeNet have been compressed by nearly 91%, 78%, 38% respectively. From the results, it has been noticed that there is a significant improvement in inference time of around 35% in AlexNet, 9% in SqueezeNet, 73% in ResNet50, and 80% in VGG16. This improvement is noticed mainly because of the fine-tuning of the deep learning models using the GA. Overall, the proposed GA-based method showed outstanding performance and it motivates research and practitioners to explore it further.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Agarwal M, Gupta S, Biswas K (2020a) A new conv2d model with modified Relu activation function for identification of disease type and severity in cucumber plant. Sustain Comput Inf Syst 30:100473
Agarwal M, Gupta SK, Biswas K (2020b) Development of efficient cnn model for tomato crop disease identification. Sustain Comput Inf Syst 28:100407
Agarwal M, Gupta S.K, Biswas K (2021a) A compressed and accelerated SegNet for plant leaf disease segmentation: a differential evolution based approach. In: Pacific-Asia Conference on knowledge discovery and data mining, pp 272–284. Springer
Agarwal M, Gupta S.K, Biswas K (2021b) Plant leaf disease segmentation using compressed UNet architecture. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 9–14. Springer
Anwar S, Hwang K, Sung W (2017) Structured pruning of deep convolutional neural networks. ACM J Emerg Technol Comput Syst (JETC) 13(3):1–18
Arkadan A, Sareen T, Subramaniam S (1994) Genetic algorithms for nondestructive testing in crack identification. IEEE Trans Magn 30(6):4320–4322
Bac FQ, Perov V (1993) New evolutionary genetic algorithms for NP-complete combinatorial optimization problems. Biol Cybern 69(3):229–234
Beale HD, Demuth HB, Hagan M (1996) Neural network design. PWS, Boston
Chen CJ, Chen K.C, Martin-Kuo MC (2018) Acceleration of neural network model execution on embedded systems. In: 2018 International Symposium on VLSI design, automation and test (VLSI-DAT), pp 1–3. IEEE
Cheng Y, Wang D, Zhou P, Zhang T (2017) A survey of model compression and acceleration for deep neural networks. arXiv preprint arXiv:1710.09282
Cheng Y, Wang D, Zhou P, Zhang T (2018) Model compression and acceleration for deep neural networks: The principles, progress, and challenges. IEEE Signal Process Mag 35(1):126–136
Choudhary T, Mishra V, Goswami A, Sarangapani J (2020) A comprehensive survey on model compression and acceleration. Artif Intell Rev 53:1–43
Devarakonda A, Naumov M, Garland M (2017) Adabatch: adaptive batch sizes for training deep neural networks. arXiv preprint arXiv:1712.02029
Gong Y, Liu L, Yang M, Bourdev L (2014) Compressing deep convolutional networks using vector quantization. arXiv preprint arXiv:1412.6115
Gui S, Wang HN, Yang H, Yu C, Wang Z, Liu J (2019) Model compression with adversarial robustness: A unified optimization framework. Adv Neural Inf Process Syst 32:1285–1296
Han S, Mao H, Dally WJ (2015a) Deep compression: compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149
Han S, Pool J, Tran J, Dally W (2015b) Learning both weights and connections for efficient neural network. In: Advances in neural information processing systems, pp 1135–1143
He Y, Zhang X, Sun J (2017) Channel pruning for accelerating very deep neural networks. In: Proceedings of the IEEE International Conference on computer vision, pp 1389–1397
Howard A.G, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861
Hughes D, Salathé M et al (2015) An open access repository of images on plant health to enable the development of mobile disease diagnostics. arXiv preprint arXiv:1511.08060
Hu Y, Sun S, Li J, Wang X, Gu Q (2018) A novel channel pruning method for deep neural network compression. arXiv preprint arXiv:1805.11394
Keesing R, Stork DG (1991) Evolution and learning in neural networks: the number and distribution of learning trials affect the rate of evolution. In: Advances in neural information processing systems, pp 804–810
Khan S, Yairi T (2018) A review on the application of deep learning in system health management. Mech Syst Signal Process 107:241–265
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105
Li H, Kadav A, Durdanovic I, Samet H, Graf HP (2016) Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710
Li H, Ota K, Dong M (2018) Learning iot in edge: Deep learning for the internet of things with edge computing. IEEE Netw 32(1):96–101
Li T, Wu B, Yang Y, Fan Y, Zhang Y, Liu W (2019) Compressing convolutional neural networks via factorized convolutional filters. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 3977–3986
Liu Z, Li J, Shen Z, Huang G, Yan S, Zhang C (2017) Learning efficient convolutional networks through network slimming. In: Proceedings of the IEEE International Conference on computer vision, pp 2736–2744
Luo JH, Wu J, Lin W (2017) Thinet: a filter level pruning method for deep neural network compression. In: Proceedings of the IEEE International Conference on computer vision, pp 5058–5066
Minarro-Giménez JA, Marin-Alonso O, Samwald M (2014) Exploring the application of deep learning techniques on medical text corpora. Stud Health Technol Inf 205:584–588
Mohanty SP, Hughes DP, Salathé M (2016) Using deep learning for image-based plant disease detection. Front Plant Sci 7:1419
Panchal G, Panchal D (2015) Solving np hard problems using genetic algorithm. Transportation 106:6–2
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556
Singh P, Verma V.K, Rai P, Namboodiri V (2020) Leveraging filter correlations for deep model compression. In: Proceedings of the IEEE/CVF Winter Conference on applications of computer vision, pp 835–844
Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 2818–2826
Ullah I, Petrosino A (2016) About pyramid structure in convolutional neural networks. In: 2016 International Joint Conference on neural networks (IJCNN), pp 1318–1324. IEEE
Wang X, Han Y, Wang C, Zhao Q, Chen X, Chen M (2019) In-edge ai: Intelligentizing mobile edge computing, caching and communication by federated learning. IEEE Netw 33(5):156–165
Wang N, Zhou W, Song Y, Ma C, Li H (2020) Real-time correlation tracking via joint model compression and transfer. IEEE Trans Image Process 29:6123–6135
Yu R, Li A, Chen CF, Lai JH, Morariu VI, Han X, Gao M, Lin CY, Davis L.S (2018) Nisp: pruning networks using neuron importance score propagation. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 9194–9203
Zhang Q, Zhang M, Chen T, Sun Z, Ma Y, Yu B (2019) Recent advances in convolutional neural network acceleration. Neurocomputing 323:37–51
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Agarwal, M., Gupta, S.K., Biswas, M. et al. Compression and acceleration of convolution neural network: a Genetic Algorithm based approach. J Ambient Intell Human Comput 14, 13387–13397 (2023). https://doi.org/10.1007/s12652-022-03793-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-022-03793-1