Abstract
The purpose of this paper is to make a concise description of the current deep learning methods for natural language processing (NLP) and discusses their advantages and disadvantages. The research further discusses the applicability of each deep learning method in the context of natural language processing. Additionally, a series of significant advances that have driven the processing, understanding, and generation of natural language are also discussed.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Word2vec, in Wikipedia. https://en.wikipedia.org/wiki/Word2vec
McCormick, C.: Word2vec Tutorial - The Skip-Gram Model. http://mccormickml.com/2016/04/19/word2vec-tutorial-the-skip-gram-model/
Olah, C.: Understanding LSTM Networks. https://colah.github.io/posts/2015-08-Understanding-LSTMs/
Olah, C.: Neural Networks, Types, and Functional Programming (2015). http://colah.github.io/posts/2015-09-NN-Types-FP/
Prabhu, R.: Understanding of Convolutional Neural Network (CNN) – Deep Learning (2018). https://medium.com/@RaghavPrabhu/understanding-of-convolutional-neural-network-cnn-deep-learning-99760835f148
Sugomori, Y., Kaluža, B., Soares, F.M., Souza, A.M.F.: Deep Learning - Practical Neural Networks with Java. Packt Publishing, Birmingham (2017). Chapter 4
Britz, D.: Understanding Convolutional Neural Network for NLP. http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/
Jurafsky, D., Martin, J.H.: N-gram Language Models (2019). https://web.stanford.edu/~jurafsky/slp3/3.pdf
Howard, J., Ruder, S.: Introducing state of the art text classification with universal language models. http://nlp.fast.ai/classification/2018/05/15/introducing-ulmfit.html
Eisenschlos, J., Ruder, S., Czapla, P., Kardas, M.: Efficient multi-lingual language model fine-tuning. http://nlp.fast.ai/classification/2019/09/10/multifit.html
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT - Pre-training of Deep Bidirectional Transformers for Language Understanding. Google AI (2019)
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Language Models are Unsupervised Multitask Learners (2019). https://d4mucfpksywv.cloudfront.net/better-language-models/language-models.pdf
Abigail: Taming Recurrent Neural Networks for Better Summarization (2017). http://www.abigailsee.com/2017/04/16/taming-rnns-for-better-summarization.html
Thiruvengadam, A.: Transformer Architecture: Attention Is All You Need (2018). https://medium.com/@adityathiruvengadam/transformer-architecture-attention-is-all-you-need-aeccd9f50d09
Rudes, S.: Tracking Progress in Natural Language Processing (2019). https://nlpprogress.com/
Saravia, E.: Deep Learning for NLP, An Overview of Recent Trends (2018). https://medium.com/dair-ai/deep-learning-for-nlp-an-overview-of-recent-trends-d0d8f40a776d
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Alulema Flores, A.S. (2020). Deep Learning Methods in Natural Language Processing. In: Botto-Tobar, M., Zambrano Vizuete, M., Torres-Carrión, P., Montes León, S., Pizarro Vásquez, G., Durakovic, B. (eds) Applied Technologies. ICAT 2019. Communications in Computer and Information Science, vol 1194. Springer, Cham. https://doi.org/10.1007/978-3-030-42520-3_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-42520-3_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-42519-7
Online ISBN: 978-3-030-42520-3
eBook Packages: Computer ScienceComputer Science (R0)