Abstract
The popular approach for several natural language processing tasks involves deep neural networks, and in particular, recurrent neural networks (RNNs) and convolutional neural networks (CNNs). While RNNs can capture the dependency in a sequence of arbitrary length, CNNs are suitable for extracting position-invariant features. In this study, a state-of-the-art CNN model incorporating a gate mechanism that is typically used in RNNs, is adapted to text classification tasks. The incorporated gate mechanism allows the CNNs to better select which features or words are relevant for predicting the corresponding class. Through experiments on various large datasets, it was found that the introduction of a gate mechanism into CNNs can improve the accuracy of text classification tasks such as sentiment classification, topic classification, and news categorization.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Sigggmonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556
Hinton G, Deng L, Yu D, Dahl GE, Mohamed AR, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Sainath TN, Kingsbury B (2012) Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process Mag 29(6):82–97
Lai S, Xu L, Liu K, Zhao J (2015) Recurrent convolutional neural networks for text classification. In: Twenty-ninth AAAI conference on artificial intelligence
Liu P, Qiu X, Huang X (2016) Recurrent neural network for text classification with multi-task learning. arXiv:1605.05101
Zhang X, Zhao J, LeCun Y (2015) Character-level convolutional networks for text classification. In: Advances in neural information processing systems, pp 649–657
Conneau A, Schwenk H, Barrault L, Lecun Y (2017) Very deep convolutional networks for text classification. In: Proceedings of the 15th conference of the European chapter of the association for computational linguistics: Volume 1, Long papers, vol 1, pp 1107–1116
Dauphin YN, Fan A, Auli M, Grangier D (2016) Language modeling with gated convolutional networks. arXiv:1612.08083
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Grave E, Joulin A, Cissé M, Grangier D, Jégou H (2016) Efficient softmax approximation for GPUs. arXiv:1609.04309
Xiao Y, Cho K (2016) Efficient character-level document classification by combining convolution and recurrent layers. arXiv:1602.00367
Kim Y (2014) Convolutional neural networks for sentence classification. arXiv:1408.5882
Kalchbrenner N, Grefenstette E, Blunsom P (2014) A convolutional neural network for modelling sentences. arXiv:1404.2188
Bengio Y, Ducharme R, Vincent P, Jauvin C (2003) A neural probabilistic language model. J Mach Learn Res 3:1137–1155
Acknowledgements
This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2019R1F1A1058548).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Sun, J., Jin, R., Ma, X., Park, Jy., Sohn, Ka., Chung, Ts. (2021). Gated Convolutional Neural Networks for Text Classification. In: Park, J.J., Fong, S.J., Pan, Y., Sung, Y. (eds) Advances in Computer Science and Ubiquitous Computing. Lecture Notes in Electrical Engineering, vol 715. Springer, Singapore. https://doi.org/10.1007/978-981-15-9343-7_43
Download citation
DOI: https://doi.org/10.1007/978-981-15-9343-7_43
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-9342-0
Online ISBN: 978-981-15-9343-7
eBook Packages: Computer ScienceComputer Science (R0)