Minimal gated unit for recurrent neural networks

  • Guo-Bing Zhou
  • Jianxin Wu
  • Chen-Lin Zhang
  • Zhi-Hua Zhou
Research Article

DOI: 10.1007/s11633-016-1006-2

Cite this article as:
Zhou, GB., Wu, J., Zhang, CL. et al. Int. J. Autom. Comput. (2016) 13: 226. doi:10.1007/s11633-016-1006-2

Abstract

Recurrent neural networks (RNN) have been very successful in handling sequence data. However, understanding RNN and finding the best practices for RNN learning is a difficult task, partly because there are many competing and complex hidden units, such as the long short-term memory (LSTM) and the gated recurrent unit (GRU). We propose a gated unit for RNN, named as minimal gated unit (MGU), since it only contains one gate, which is a minimal design among all gated hidden units. The design of MGU benefits from evaluation results on LSTM and GRU in the literature. Experiments on various sequence data show that MGU has comparable accuracy with GRU, but has a simpler structure, fewer parameters, and faster training. Hence, MGU is suitable in RNN's applications. Its simple architecture also means that it is easier to evaluate and tune, and in principle it is easier to study MGU's properties theoretically and empirically.

Keywords

Recurrent neural network minimal gated unit (MGU) gated unit gate recurrent unit (GRU) long short-term memory (LSTM) deep learning 

Copyright information

© Institute of Automation, Chinese Academy of Sciences and Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Guo-Bing Zhou
    • 1
  • Jianxin Wu
    • 1
  • Chen-Lin Zhang
    • 1
  • Zhi-Hua Zhou
    • 1
  1. 1.National Key Laboratory for Novel Software TechnologyNanjing UniversityNanjingChina

Personalised recommendations