Skip to main content

Advertisement

SpringerLink
Log in

Automatic filter pruning algorithm for image classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Network pruning is an essential technique for compressing and accelerating convolutional neural networks (CNNs). Existing pruning algorithms primarily evaluate filter importance or similarity, and then remove unimportant filters or keep only one similar filter at each convolutional layer based on a global pruning ratio. These methods, ignoring the sensitivity of pruning among different convolutional layers, rely on a lot of manual experience and multiple experiments to obtain the optimal convolutional neural network structure. To this end, we propose an automatic filter pruning algorithm via feature map average similarity and reverse search genetic algorithm(RSGA), dubbed as AFPruner, which automatically searches for the optimal combination of pruning ratio for all convolutional layers, evaluates filter similarity by feature map average similarity and then prunes similarity filter. Our method is evaluated against several state-of-the-art CNNs on three different classification datasets, and the experimental results show that our algorithm outperforms most current network pruning algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data Availability Statements

The datasets generated during and/or analysed during the current study are available in http://www.cs.toronto.edu/~kriz/cifar.html and https://image-net.org/.

References

  1. Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90

    Article  Google Scholar 

  2. Ren S, He K, Girshick R, Sun J (2017) Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  3. Noh H, Hong S, Han B (2015) Learning deconvolution network for semantic segmentation. In: Proceedings of the IEEE International conference on computer vision, pp 1520–1528

  4. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International conference on learning representations (ICLR), pp 1–14

  5. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  6. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  7. Li H, Kadav A, Durdanovic I, Samet H, Graf HP (2016) Pruning filters for efficient convnets. In: International conference on learning representations

  8. Liu Z, Li J, Shen Z, Huang G, Yan S, Zhang C (2017) Learning efficient convolutional networks through network slimming. In: Proceedings of the IEEE international conference on computer vision, pp 2736–2744

  9. Lin M, Ji R, Wang Y, Zhang Y, Zhang B, Tian Y, Shao L (2020) Hrank: Filter pruning using high-rank feature map. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1529–1538

  10. He Y, Kang G, Dong X, Fu Y, Yang Y (2018) Soft filter pruning for accelerating deep convolutional neural networks. In: International joint conference on artificial intelligence

  11. He Y, Dong X, Kang G, Fu Y, Yan C, Yang Y (2020) Asymptotic soft filter pruning for deep convolutional neural networks. IEEE Trans Cybern 50(8):3594–3604

    Article  Google Scholar 

  12. Famili A, Lao Y (2022) Genetic-based joint dynamic pruning and learning algorithm to boost dnn performance. In: 2022 26th International conference on pattern recognition (ICPR), pp 2100–2106

  13. Yu R, Li A, Chen C-F, Lai J-H, Morariu VI, Han X, Gao M, Lin C-Y, Davis LS (2018) Nisp: Pruning networks using neuron importance score propagation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 9194–9203

  14. Hu J, Shen L, Albanie S, Sun G, Wu E (2020) Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell 42(8):2011–2023

    Article  Google Scholar 

  15. Cheng Y, Wang X, Xie X, Li W, Peng S (2022) Channel pruning guided by global channel relation. Applied Intelligence, pp 1–12

  16. He Y, Liu P, Wang Z, Hu Z, Yang Y (2019) Filter pruning via geometric median for deep convolutional neural networks acceleration. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4340–4349

  17. Sawant SS, Bauer J, Erick FX, Ingaleshwar S, Holzer N, Ramming A, Lang E, Götz T (2022) An optimal-score-based filter pruning for deep convolutional neural networks. Appl Intell pp 1–23

  18. Li J, Shao H, Zhai S, Jiang Y, Deng X (2023) A graphical approach for filter pruning by exploring the similarity relation between feature maps. Pattern Recogn Lett 166:69–75

    Article  Google Scholar 

  19. Lin M, Cao L, Zhang Y, Shao L, Lin C-W, Ji R (2022) Pruning networks with cross-layer ranking & k-reciprocal nearest filters. IEEE Transactions on neural networks and learning systems, pp 1–10

  20. Ayinde BO, Inanc T, Zurada JM (2019) Redundant feature pruning for accelerated inference in deep neural networks. Neural Netw 118:148–158

    Article  Google Scholar 

  21. Shao M, Dai J, Wang R, Kuang J, Zuo W (2021) Cshe: network pruning by using cluster similarity and matrix eigenvalues. Int J Mach Learn Cybern 13:371–382

    Article  Google Scholar 

  22. Song K, Yao W, Zhu X (2022) Filter pruning via similarity clustering for deep convolutional neural networks. In: International conference on neural information processing. Springer, pp 88–99

  23. Orseau L, Hutter M, Rivasplata O (2020) Logarithmic pruning is all you need. Adv Neural Inf Process Syst 33:2925–2934

    Google Scholar 

  24. Yang C, Liu H (2022) Channel pruning based on convolutional neural network sensitivity. Neurocomputing 507:97–106

    Article  Google Scholar 

  25. He Y, Lin J, Liu Z, Wang H, Li L-J, Han S (2018) Amc: Automl for model compression and acceleration on mobile devices. In: Proceedings of the European conference on computer vision (ECCV), pp 784–800

  26. Ning X, Zhao T, Li W, Lei P, Wang Y, Yang H (2020) Dsa: More efficient budgeted pruning via differentiable sparsity allocation. In: Computer vision–ECCV 2020: 16th European conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III, Springer, pp 592–607

  27. Gao S, Huang F, Pei J, Huang H (2020) Discrete model compression with resource constraint for deep neural networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1899–1908

  28. Liu Z, Sun M, Zhou T, Huang G, Darrell T (2018) Rethinking the value of network pruning. In: International conference on learning representations

  29. Lin M, Ji R, Zhang Y, Zhang B, Wu Y, Tian Y (2020) Channel pruning via automatic structure search. In: Proceedings of the international joint conference on artificial intelligence (IJCAI), pp 673–679

  30. Liu Z, Mu H, Zhang X, Guo Z, Yang X, Cheng K-T, Sun J (2019) Metapruning: Meta learning for automatic neural network channel pruning. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 3296–3305

  31. Chang J, Lu Y, Xue P, Xu Y, Wei Z (2022) Automatic channel pruning via clustering and swarm intelligence optimization for cnn. Appl Intell pp 1–21

  32. Lian Y, Peng P, Xu W (2021) Filter pruning via separation of sparsity search and model training. Neurocomputing 462:185–194

    Article  Google Scholar 

  33. Tmamna J, Ayed EB, Ayed MB (2021) Neural network pruning based on improved constrained particle swarm optimization. In: Neural information processing: 28th international conference, ICONIP 2021, Sanur, Bali, Indonesia, December 8–12, 2021, Proceedings, Part VI 28, Springer, pp 315–322

  34. Lin L, Chen S, Yang Y, Guo Z (2022) Aacp: Model compression by accurate and automatic channel pruning. In: 2022 26th International conference on pattern recognition (ICPR), IEEE, pp 2049–2055

  35. Wang Z, Li F, Shi G, Xie X, Wang F (2020) Network pruning using sparse learning and genetic algorithm. Neurocomputing 404:247–256

    Article  Google Scholar 

  36. He Y, Ding Y, Liu P, Zhu L, Zhang H, Yang Y (2020) Learning filter pruning criteria for deep convolutional neural networks acceleration. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2009–2018

  37. Huang K, Wu S, Li F, Yang C, Gui W (2022) Fault diagnosis of hydraulic systems based on deep learning model with multirate data samples. IEEE Transactions on Neural Networks and Learning Systems 33(11):6789–6801

    Article  Google Scholar 

  38. Li H, Ma C, Xu W, Liu X (2020) Feature statistics guided efficient filter pruning. In: Proceedings of the international joint conference on artificial intelligence (IJCAI)

Download references

Acknowledgements

This work was partially supported by Priority Academic Program Development of Jiangsu Higher Education Institutions(PAPD), Collaborative Innovation Center of Novel Software Technology and Industrialization.

Author information

Authors and Affiliations

  1. School of computer science and technology, Soochow University, Suzhou, 21500, China

    Yifan Xue, Wangshu Yao, Siyuan Peng & Shiyou Yao

  2. School of software, Soochow University, Suzhou, 21500, China

    Wangshu Yao

  3. Collaborative Innovation Center of Novel Software Technology and Industrialization, Jiangsu, 21500, China

    Wangshu Yao

Authors
  1. Yifan Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wangshu Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Siyuan Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shiyou Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Yifan Xue: Conceptualization, Methodology, Software, Investigation, Writing - original draft. Wangshu Yao: Conceptualization, Writing - review & editing, Project administration, Funding acquisition. Siyuan Peng: Validation, Resources, Investigation. Shiyou Yao: Visualization.

Corresponding author

Correspondence to Wangshu Yao.

Ethics declarations

Competing Interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xue, Y., Yao, W., Peng, S. et al. Automatic filter pruning algorithm for image classification. Appl Intell 54, 216–230 (2024). https://doi.org/10.1007/s10489-023-05207-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-05207-x

Keywords

Search

Navigation