Abstract
This is an extension from a selected paper from JSAI2019. Convolutional variants of recurrent neural networks, ConvRNNs, are widely used for spatio-temporal modeling, since they are well suited to model sequences of two-dimensional inputs. Similar to conventional RNNs, the introduction of gating architecture, such as ConvLSTM, brings additional parameters and increases the computational complexity. The computation load can be an obstacle in training efficient models and putting ConvRNNs in operation in real-world applications. However, the correspondence between ConvRNN unit complexity and its performance is not well investigated. We propose to reduce the number of parameters and multiplications by substituting some convolutional operations with the Hadamard product. We evaluate our proposal using the task of next video frame prediction and the Moving MNIST dataset. The proposed method requires 38% less multiplications and 21% less parameters compared to the fully convolutional counterpart. In price of the reduced computational complexity, the performance measured by structural similarity index measure (SSIM) decreased about 1.5%. ConvRNNs with reduced computations can be used in a wider range of situations like in web applications or embedded systems.
This paper is an extension of a selected paper from JSAI2019 [10].
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ballas, N., Yao, L., Pal, C., Courville, A.: Delving deeper into convolutional networks for learning video representations. In: International Conference Learning Representations (ICLR) (2016)
Donahue, J., Hendricks, L.A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Saenko, K., Darrell, T.: Long-term recurrent convolutional networks for visual recognition and description. In: IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 2625–2634 (2015)
Elsayed, N., Maida, A.S., Bayoumi, M.: Reduced-gate convolutional LSTM using predictive coding for spatiotemporal prediction. arXiv:1810.07251 (2018)
Li, S., Li, W., Cook, C., Zhu, C., Gao, Y.: Independently recurrent neural network (IndRNN): building a longer and deeper RNN. In: IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 5457–5466 (2018)
Sautermeister, B.: Deep learning approaches to predict future frames in videos. Master’s thesis, Technishe Universität München (2016)
Shi, X., Chen, Z., Wang, H., Yeung, D.Y., Wong, W., Woo, W.: Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In: Advances in Neural Information Processing Systems (NIPS), pp. 802–810 (2015)
Shi, X., Gao, Z., Lausen, L., Wang, H., Yeung, D.Y., Wong, W., Woo, W.: Deep learning for precipitation nowcasting: a benchmark and a new model. In: Advances in Neural Information Processing Systems (NIPS), pp. 5617–5627 (2017)
Srivastava, N., Mansimov, E., Salakhudinov, R.: Unsupervised learning of video representations using LSTMs. In: International Conference Machine Learning (ICML), pp. 843–852 (2015)
van der Westhuizen, J., Lasenby, J.: The unreasonable effectiveness of the forget gate. arXiv:1804.04849 (2018)
Vazhenina, D., Kanemura, A.: Reducing the number of multiplications in convolutional recurrent neural networks (ConvRNNs). In: Annual Conference of the Japanese Society for Artificial Intelligence (JSAI) (2019)
Wang, Y., Gao, Z., Long, M., Wang, J., Yu, P.S.: PredRNN++: towards a resolution of the deep-in-time dilemma in spatiotemporal predictive learning. In: International Conference Machine Learning (ICML) (2018)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Vazhenina, D., Kanemura, A. (2020). Reducing the Number of Multiplications in Convolutional Recurrent Neural Networks (ConvRNNs). In: Ohsawa, Y., et al. Advances in Artificial Intelligence. JSAI 2019. Advances in Intelligent Systems and Computing, vol 1128. Springer, Cham. https://doi.org/10.1007/978-3-030-39878-1_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-39878-1_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-39877-4
Online ISBN: 978-3-030-39878-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)