Skip to main content
SpringerLink
Log in

Efficient channel expansion and pyramid depthwise-pointwise-depthwise neural networks

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In popular lightweight convolutional neural networks (CNNs), pointwise convolution (PWC) layers for combining information occupy approximately 70% weights and computation, but depthwise convolution (DWC) layers for extracting spatial information only occupy less than 2% weights and computation. The weights and computation for extracting spatial information are not enough in lightweight CNNs. In this paper, we proposed expanding the number of channels and improving the extraction of spatial information by more efficient DWC instead of PWC. Firstly, the results of the proposed PSDNet demonstrate that DWC is more efficient than PWC for channel expansion and it can improve the accuracy of the network. Then, the efficient Depthwise-Pointwise-Depthwise (DPD) block is proposed by using DWC to expand channels. Different from the general bottleneck block, the DPD block consists of one PWC layer and two DWC layers. Four kinds of efficient lightweight DPDNets (DPDNet-G, DPDNet-A, DPDNet-C, DPDNet-D) are proposed by stacking different DPD blocks. To extract multi-scale features and achieve high accuracy, the pyramid DWC layer is used when channel expansion in DPDNet. Compared with common lightweight CNNs, DPDNets use more weights and computation in the DWC layer for extracting spatial information. Four competitive benchmark datasets (CIFAR-10, CIFAR-100, ImageNet, and PASCAL VOC) were used to verify the superiority of DPDNet. Experiments demonstrate that the proposed DPDNet has higher accuracy than MobileNet with a similar number of weights and computations. Furthermore, compared DPDNet with MobileNet, it can be found that improving the ratio of DWC to PWC can improve accuracy, which helps researchers to design better lightweight CNNs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Chen T, Duan B, Sun Q, Zhang M, Li G, Geng H, Zhang Q, Yu B (2021) An efficient sharing grouped convolution via bayesian learning. IEEE Trans Neural Netw Learn Syst :1–13

  2. Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 1800–1807

  3. Dong Y, Ni R, Li J, Chen Y, Su H, Zhu J (2019) Stochastic quantization for learning accurate low-bit deep neural networks. Int J Comp Vision 127(11–12):1629–1642

    Article  Google Scholar 

  4. Everingham M, Gool LV, Williams CKI, Winn JM, Zisserman A (2010) The pascal visual object classes (VOC) challenge. Int J Comp Vision 88(2):303–338

    Article  Google Scholar 

  5. He K, Zhang X, Ren S, Sun J (2016a) Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 770–778

  6. He K, Zhang X, Ren S, Sun J (2016b) Identity mappings in deep residual networks. In: European Conference on Computer Vision, pp 630–645

  7. Howard A, Pang R, Adam H, Le QV, Sandler M, Chen B, Wang W, Chen L, Tan M, Chu G, Vasudevan V, Zhu Y (2019) Searching for mobilenetv3. In: IEEE International Conference on Computer Vision, pp 1314–1324

  8. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: Efficient convolutional neural networks for mobile vision applications. CoRR arXiv:abs/1704.04861

  9. Huang C, Liu P, Fang L (2021) Mxqn:mixed quantization for reducing bit-width of weights and activations in deep convolutional neural networks. Appl Intell

  10. Huang G, Liu Z, van der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2261–2269

  11. Hui Z, Gao X, Yang Y, Wang X (2019) Lightweight image super-resolution with information multi-distillation network. In: ACM International Conference on Multimedia, pp 2024–2032

  12. Kim T, Lee J, Choe Y (2020) Bayesian optimization-based global optimal rank selection for compression of convolutional neural networks. IEEE Access 8:17605–17618

    Article  Google Scholar 

  13. Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90

    Article  Google Scholar 

  14. Kumar A, Shaikh AM, Li Y, Bilal H, Yin B (2021) Pruning filters with l1-norm and capped l1-norm for CNN compression. Appl Intell 51(2):1152–1160

    Article  Google Scholar 

  15. Li G, Shen X, Li J, Wang J (2021) Diagonal-kernel convolutional neural networks for image classification. Digit Signal Process 108:102898

    Article  Google Scholar 

  16. Li G, Zhang M, Li J, Lv F, Tong G (2021) Efficient densely connected convolutional neural networks. Pattern Recognit 109:107610

    Article  Google Scholar 

  17. Lin S, Ji R, Li Y, Deng C, Li X (2020) Toward compact convnets via structure-sparsity regularized filter pruning. IEEE Trans Neural Netw Learn Syst 31(2):574–588

    Article  MathSciNet  Google Scholar 

  18. Liu W, Anguelov D, Erhan D, Szegedy C, Reed SE, Fu C, Berg AC (2016) SSD: single shot multibox detector. In: European Conference on Computer Vision, pp 21–37

  19. Liu W, Wang Z, Liu X, Zeng N, Liu Y, Alsaadi FE (2017) A survey of deep neural network architectures and their applications. Neurocomputing 234:11–26

    Article  Google Scholar 

  20. Ma N, Zhang X, Zheng H, Sun J (2018) Shufflenet V2: practical guidelines for efficient CNN architecture design. In: European Conference on Computer Vision, pp 122–138

  21. Ou J, Li Y (2019) Vector-kernel convolutional neural networks. Neurocomputing 330:253–258

    Article  Google Scholar 

  22. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S et al (2015) Imagenet large scale visual recognition challenge. Int J Comp Vision 115(3):211–252

    Article  MathSciNet  Google Scholar 

  23. Sandler M, Howard AG, Zhu M, Zhmoginov A, Chen L (2018) Mobilenetv2: Inverted residuals and linear bottlenecks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 4510–4520

  24. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2020) Grad-cam: Visual explanations from deep networks via gradient-based localization. Int J Comp Vision 128(2):336–359

    Article  Google Scholar 

  25. Shao J, Cheng Q (2021) E-FCNN for tiny facial expression recognition. Appl Intell 51(1):549–559

    Article  Google Scholar 

  26. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations

  27. Wang J, Xiong H, Wang H, Nian X (2020) Adscnet: asymmetric depthwise separable convolution for semantic segmentation in real-time. Appl Intell 50(4):1045–1056

    Article  Google Scholar 

  28. Wang P, Cheng J (2016) Accelerating convolutional neural networks for mobile applications. In: ACM International Conference on Multimedia, pp 541–545

  29. Wang W, Liu Q, Wang W (2021) Pyramid-dilated deep convolutional neural network for crowd counting. Appl Intell

  30. Wen N, Guo R, He B, Fan Y, Ma D (2021) Block-sparse CNN: towards a fast and memory-efficient framework for convolutional neural networks. Appl Intell 51(1):441–452

    Article  Google Scholar 

  31. Wu Q, Lu X, Xue S, Wang C, Wu X, Fan J (2020) Sbnn: Slimming binarized neural network. Neurocomputing 401:113–122

    Article  Google Scholar 

  32. Zeng L, Tian X (2020) Accelerating convolutional neural networks by removing interspatial and interkernel redundancies. IEEE Trans Cybernet 50(2):452–464

    Article  Google Scholar 

  33. Zhang Q, Zhang M, Chen T, Sun Z, Ma Y, Yu B (2019) Recent advances in convolutional neural network acceleration. Neurocomputing 323:37–51

    Article  Google Scholar 

  34. Zhang X, Zhou X, Lin M, Sun J (2018) Shufflenet: An extremely efficient convolutional neural network for mobile devices. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 6848–6856

  35. Zhou D, Hou Q, Chen Y, Feng J, Yan S (2020) Rethinking bottleneck structure for efficient mobile network design. In: European Conference on Computer Vision, pp 680–697

Download references

Acknowledgements

This research work was partly supported by the Key R&D Program of China (Project No. 2018YFB2202703), and the Natural Science Foundation of Jiangsu Province (Project No. BK20201145).

Author information

Authors and Affiliations

  1. National ASIC Engineering Technology Research Center, School of Electronics Science and Engineering, Southeast University, Nanjing, 210096, People’s Republic of China

    Guoqing Li, Meng Zhang, Yu Zhang, Ruixia Wu & Dongpeng Weng

Authors
  1. Guoqing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Meng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ruixia Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dongpeng Weng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meng Zhang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, G., Zhang, M., Zhang, Y. et al. Efficient channel expansion and pyramid depthwise-pointwise-depthwise neural networks. Appl Intell 52, 12860–12872 (2022). https://doi.org/10.1007/s10489-021-03152-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-03152-1

Keywords

Search

Navigation