Abstract
Computational machine learning has attracted a great deal of attention for its ability to analyze large-scale data. In particular, convolutional neural networks (CNNs) have been proposed in the fields of image recognition and object detection in efforts to develop models with improved accuracy as well as more lightweight models that require a smaller number of parameters and a lower computational cost. Octave Convolution (OctConv) is a method used to reduce the memory and computational cost of a model while also improving its accuracy by replacing the conventional convolutional layer with an OctConv layer. However, the number of parameters used in OctConv is almost the same as that in the case of conventional convolutional processing. In this paper, we propose the Pointwise Octave Convolution (Pointwise OctConv) method, which combines the Pointwise Convolution (Pointwise Conv) method with OctConv to reduce the number of parameters used in OctConv and thus create a lighter model. In the proposed method, the number of parameters is reduced by performing Pointwise Conv before and after the convolution process for each path in the OctConv layer. In an evaluation using ResNet-56, the proposed method reduces the number of parameters by about 63.8\(\%\) with a loss of classification accuracy of 3.04\(\%\) when \(\alpha = 0.75\).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv (online). https://arxiv.org/pdf/1409.1556.pdf. Accessed 15 Nov 2021
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition, arXiv (online). https://arxiv.org/pdf/1512.03385.pdf. Accessed 15 Nov 2021
Huang, G., Liu, Z., Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks, arXiv (online). https://arxiv.org/pdf/1608.06993.pdf. Accessed 15 Nov 2021
Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., Keutzer, K.: SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and \(<\) 0.5 MB model size, arXiv (online). https://arxiv.org/pdf/1602.07360.pdf. Accessed 15 Nov 2021
Howard, A.G., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications, arXiv (online). https://arxiv.org/pdf/1704.04861.pdf. Accessed 15 Nov 2021
Zhang, X., Zhou, X., Lin, M., Sun, J.: ShuffleNet: an extremely efficient convolutional neural networks for mobile devices, arXiv (online). https://arxiv.org/pdf/1707.01083.pdf. Accessed 15 Nov 2021
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: MobileNetV2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)
Chen, Y., et al.: Drop an octave: reducing spatial redundancy in convolutional neural networks with octave convolution, arXiv (online). https://arxiv.org/pdf/1904.05049.pdf. Accessed 15 Nov 2021
Ahmad, S., Scheinkman, L.: How can we be so dense? The benefits of using highly sparse representations, arXiv (online). https://arxiv.org/pdf/1903.11257.pdf. Accessed 15 Nov 2021
Han, S., Pool, J., Tran, J., Dally, W.J.: Learning both weights and connections for efficient neural networks, arXiv (online). https://arxiv.org/pdf/1506.02626v3.pdf. Accessed 15 Nov 2021
Han, S., Mao, H., Dally, W.J.: Deep compression: compressing deep neural networks with pruning, trained quantization and Huffman coding, arXiv (online). https://arxiv.org/pdf/1510.00149.pdf. Accessed 15 Nov 2021
Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient ConvNets, arXiv (online). https://arxiv.org/pdf/1608.08710.pdf. Accessed 15 Nov 2021
Kaiser, L., Gomez, A.N., Chollet, F.: Depthwise separable convolutions for neural machine translation, arXiv (online). https://arxiv.org/pdf/1706.03059.pdf. Accessed 15 Nov 2021
Chollet, F.: Xception: deep learning with depthwise separable convolutions, arXiv (online). https://arxiv.org/pdf/1610.02357.pdf. Accessed 15 Nov 2021
He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks, arXiv (online). https://arxiv.org/pdf/1603.05027.pdf. Accessed 15 Nov 2021
The CIFAR-10 and CIFAR-100 datasets (online). https://www.cs.toronto.edu/~kriz/cifar.html. Accessed 15 Nov 2021
Acknowledgement
This work was supported by JSPS KAKENHI Grant Number 18K11265 and 21H03429, and JGC-S Scholarship Foundation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Gotoh, Y., Inoue, Y. (2022). A Method for Reducing Number of Parameters of Octave Convolution in Convolutional Neural Networks. In: Barolli, L., Kulla, E., Ikeda, M. (eds) Advances in Internet, Data & Web Technologies. EIDWT 2022. Lecture Notes on Data Engineering and Communications Technologies, vol 118. Springer, Cham. https://doi.org/10.1007/978-3-030-95903-6_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-95903-6_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-95902-9
Online ISBN: 978-3-030-95903-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)