Abstract
This paper describes the comparison results of two types of recurrent neural network: LSTM and GRU. In the article the two types of RNN architecture are compared with the criterion of time consumed for test problems solving and training. Information about network training is provided in order to explain the differences in the training of LSTM and GRU RNN’s types and the final difference in time. Mathematic models of this neural network types are provided. The article includes description of software implementation of recurrent neural networks. As a result of research the numerical comparison of training and solving time is provided, and practical hints and conclusions are derived.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Yakovlev, V.V., Murphy, E.K., Eves, E.E.: Neural networks for FDTD-backed permittivity reconstruction. COMPEL: Int. J. Comput. Math. Electric. Electron. Eng. 24(1), 291–304 (2005)
Haykin, S.: Neural Networks: A Comprehensive Foundation. 2nd edn. Prentice Hall (1999)
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., LeCun, Y.: OverFeat: integrated recognition, localization and detection using convolutional networks. In: International Conference on Learning Representations (ICLR2014), CBLS, (Arxiv:1312.6229) (2014)
Dolinina, O.: Test Set generation method for debugging of neural network-based expert systems. In: Dolinina, O., Kuzmin, A. (eds.) Proceedings of International Congress on Information Technologies ICIT-12 (Information & Communication Technologies in Education, Manufacturing & Research). 6–9 June 2012, pp. 53–59. Saratov, Russia (2012)
Brovko, A.V., Murphy, E.K., Yakovlev, V.V.: Waveguide microwave imaging: neural network reconstruction of functional 2-D permittivity profiles. IEEE Trans. Microwave Theory Techn. 57(2), 406–414 (2009). https://doi.org/10.1109/TMTT.2008.2011203
Dolinina, O., Kushnikov, V., Kulakova, E.: Analysis of objective trees in security management of distributed computer networks of enterprises and organizations. In: Gaj, P., Kwiecień, A., Stera, P. (eds.) CN 2015. CCIS, vol. 522, pp. 117–126. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-19419-6_11
Kumaran, D., Hassabis, D., McClelland, J.L.: What learning systems do intelligent agents need? Trends in Cognitive Sciences 20(7), 512–534 (2016)
Lampert, C., Nickisch, H., Harmeling, S.: Learning to detect unseen object classes by between-class attribute transfer. In: CVPR 2009. MiamiBeach, Florida (2009)
Lomonaco, V., Maltoni, D.: Core50: A New Dataset and Benchmark for Continuous Object Recognition. In CoRL, Mountain View (2017)
LeCun, Y., Bottou, L., Bengio,Y., Haffner, P.: Gradient-based learning applied to document recognition. In: Proceedings of the IEEE (1998)
Chao, J., Hoshino, M., Kitamura, T., Masuda, T.: A multilayer RBF network and its supervised learning. In: IEEE IJCNN’01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222), vol. 3, pp. 1995–2000 (2001). https://doi.org/10.1109/ijcnn.2001.938470
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Pudikov, A., Brovko, A. (2021). Comparison of LSTM and GRU Recurrent Neural Network Architectures. In: Dolinina, O., et al. Recent Research in Control Engineering and Decision Making. ICIT 2020. Studies in Systems, Decision and Control, vol 337. Springer, Cham. https://doi.org/10.1007/978-3-030-65283-8_10
Download citation
DOI: https://doi.org/10.1007/978-3-030-65283-8_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-65282-1
Online ISBN: 978-3-030-65283-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)