Fast Depthwise Separable Convolution for Embedded Systems

  • Byeongheon Yoo
  • Yongjun Choi
  • Heeyoul ChoiEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11307)


Convolutional neural networks (CNNs) have achieved outstanding performance in many applications. However, as the total number of layers has increased and the model structure has become compound, the computational cost comes into question. The large models cannot operate in embedded or mobile environments where hardware resources are quite limited. To overcome these problems, there have been several attempts like reducing the depth of networks, pruning, quantization or low rank approximation. Depthwise separable convolution (DSC) was proposed to reduce computation especially in convolutional layers by separating one convolution into a spatial convolution and a pointwise convolution. In this paper, we apply DSC to the YOLO network for object detection and propose a faster version of DSC, FastDSC by replacing the pointwise convolution with general matrix multiplication. Experiments on the NVIDIA Jetson TX2 board show that FastDSC speeds up DSC for object detection.


Network optimization Depthwise separable convolution Pointwise convolution General matrix multiplication 



This work was partly supported by Institute for Information & communications Technology Promotion(IITP) grant funded by the Korea government(MSIT) (No. 2018-0-00749,Development of virtual network management technology based on artificial intelligence) and the National Program for Excellence in Software funded by the Ministry of Science, ICT and Future Planning, Republic of Korea (2017-0-00130).


  1. 1.
    Chollet, F.: Xception: deep learning with depthwise separable convolutions. CoRR abs/1610.02357 (2016).
  2. 2.
    Courbariaux, M., Bengio, Y., David, J.: Binaryconnect: training deep neural networks with binary weights during propagations. CoRR abs/1511.00363 (2015).
  3. 3.
    Dahl, G.E., Sainath, T.N., Hinton, G.E.: Improving deep neural networks for LVCSR using rectified linear units and dropout (2013)Google Scholar
  4. 4.
    Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)CrossRefGoogle Scholar
  5. 5.
    Fukushima, K.: Neocognitron: a self-organizing neural network for a mechanism of pattern recognition unaffected by shift in position. Biol. Cybern. 36(4), 193–202 (1980)CrossRefGoogle Scholar
  6. 6.
    Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS 2010). Society for Artificial Intelligence and Statistics (2010)Google Scholar
  7. 7.
    Hinton, G., et al.: Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Process. Mag. 29, 82–97 (2012)CrossRefGoogle Scholar
  8. 8.
    Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531v1 (2015)
  9. 9.
    Howard, A.G., et al,: Mobilenets: efficient convolutional neural networks for mobile vision applications. CoRR abs/1704.04861 (2017).
  10. 10.
    Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. CoRR abs/1502.03167 (2015).
  11. 11.
    Kim, Y., Park, E., Yoo, S., Choi, T., Yang, L., Shin, D.: Compression of deep convolutional neural networks for fast and low power mobile applications. CoRR abs/1511.06530 (2015).
  12. 12.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  13. 13.
    LeCun, Y., Bengio, Y., Hinton, G.E.: Deep learning. Nature 521(7553), 436–444 (2015)CrossRefGoogle Scholar
  14. 14.
    LeCun, Y., Cortes, C.: MNIST handwritten digit database (2010).
  15. 15.
    Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines (2010)Google Scholar
  16. 16.
    Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. CoRR abs/1612.08242 (2016).
  17. 17.
    Romero, A., Ballas, N., Kahou, S.E., Chassang, A., Gatta, C., Bengio, Y.: Fitnets: hints for thin deep nets. CoRR abs/1412.6550 (2014).
  18. 18.
    Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)CrossRefGoogle Scholar
  19. 19.
    Zhang, X., Zhou, X., Lin, M., Sun, J.: ShuffleNet: an extremely efficient convolutional neural network for mobile devices. CoRR abs/1707.01083 (2017).

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.School of Computer Science and Electrical EngineeringHandong Global UniversityPohangSouth Korea

Personalised recommendations