Advertisement

A Study of Deep Learning in Text Analytics

  • Noopur BallalEmail author
  • Sri Khetwat Saritha
Conference paper
  • 19 Downloads
Part of the Lecture Notes in Networks and Systems book series (LNNS, volume 100)

Abstract

Most of the data present today on Web is in the form of text. This text data is a rich source of information, but it is difficult to process because it is unstructured in nature. Various techniques have been developed in past to process this text data for information retrieval and text mining. Many machine learning techniques have been developed in past to drive valuable information from text data. However, these methods require a lot of preprocessing and feature extraction prior to processing which increases manual task. There are a lot of human interactions which not only consumes time but also makes the whole system where rigid and hard to generalize. Deep learning architectures mitigate these limitations. They require less human intervention and create better solutions than other machine learning techniques. Deep architectures are neural networks with multiple processing layers of neuron with each layer having a specific task. Various deep learning architectures have been developed till date with each having a specific task. These architectures have contributed significantly in the domain of text processing. Word2Vec and GloVe have helped to represent text in vector forms. Vector representation helps to process text easily. Architectures like CNN & autoencoders have helped to automate the task of feature extraction. On the other hand, architectures like LSTM and RNN are used for text processing. Text classification, document summarization, question answering, machine translation, caption generation, and speech recognition are some of the areas where improvements have been found due to deep learning architectures. This paper first introduces the text preprocessing techniques and feature extraction methods. Later on, a study of contributions of deep learning architectures in text processing is presented.

Keywords

Feature extraction Caption generation Text classification Machine translation Text summarization 

References

  1. 1.
    Najafabadi MM et al (2015) Deep learning applications and challenges in big data analytics. J Big Data.  https://doi.org/10.1186/s40537-014-0007-7
  2. 2.
    Mikolov T et al (2013) Efficient estimation of word representations in vector space. arXiv:1301.3781v3 [cs.CL] 7 Sep 2013
  3. 3.
    Shaheen F et al (2016) Impact of automatic feature extraction in deep learning architecture. In: 2016 International conference on digital image computing: techniques and applications (DICTA)Google Scholar
  4. 4.
    Rifai S et al (2011) Contractive auto-encoders: explicit invariance during feature extraction. In: Proceedings of the 28th international conference on machine learning, Bellevue, WA, USAGoogle Scholar
  5. 5.
    Masci J et al (2011) Stacked convolutional auto-encoders for hierarchical feature extraction. In: Conference paper, June 2011.  https://doi.org/10.1007/978-3-642-21735-7_7. Source: DBLPGoogle Scholar
  6. 6.
    Deng L (2014) A tutorial survey of architectures, algorithms, and applications for deep learning. SIP 3(e2):1–29Google Scholar
  7. 7.
  8. 8.
    Reshma U et al (2018) Deep learning for digital text analytics: sentiment analysis. arXiv:1804.03673v1 [cs.CL] 10 Apr 2018
  9. 9.
    Ramadhani AM (2017) Twitter sentiment analysis using deep learning methods. In: 2017 7th international annual engineering seminar (InAES), Yogyakarta, IndonesiaGoogle Scholar
  10. 10.
    Iyyer M et al (2015) Deep unordered composition rivals syntactic methods for text classification. In: Conference paper, Jan 2015.  https://doi.org/10.3115/v1/p15-1162
  11. 11.
    Seth S et al (2017) Multimodal spam classification using deep learning techniques. In: 2017 13th international conference on signal-image technology and internet-based systems (SITIS)Google Scholar
  12. 12.
    Guimaraes RG et al (2017) Age groups classification in social network using deep learning, digital object identifier. https://doi.org/10.1109/access.2017.2706674CrossRefGoogle Scholar
  13. 13.
    Sunitha C et al (2016) Study of abstractive summarization techniques in Indian language. Procedia Comput Sci 87:25–31. https://doi.org/10.1016/j.procs.2016.05.121CrossRefGoogle Scholar
  14. 14.
    Rush AM et al (2015) A neural attention model for abstractive sentence summarization. arXiv:1509.00685v2 [cs.CL] 3 Sep 2015
  15. 15.
    Nallapati R et al (2016) Abstractive text summarization using sequence-to-sequence RNNs and beyond. arXiv:1602.06023v5 [cs.CL] 26 Aug 2016
  16. 16.
    Cheng J et al (2016) Neural summarization by extracting sentences and words. arXiv:1603.07252v3 [cs.CL] 1 Jul 2016
  17. 17.
    Yu L et al (2014) Deep learning for answer sentence selection. arXiv:1412.1632v1 [cs.CL] 4 Dec 2014
  18. 18.
    Dong L et al (2015) Question answering over freebase with multi-column convolutional neural networks. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing, pp 260–269. Beijing, China, July 26–31Google Scholar
  19. 19.
    Mortiz K et al (2015) Teaching machines to read and comprehend. In: Advances in neural information processing systems 28 (NIPS 2015). arXiv:1506.03340
  20. 20.
    Suutskever I (2014) Sequence to sequence learning with neural networks. arXiv:1409.3215v3 [cs.CL] 14 Dec 2014
  21. 21.
    Auli M et al (2013) Joint language and translation modeling with recurrent neural networks. In: Proceedings of the 2013 conference on empirical methods in natural language processing, pp 1044–1054. Seattle, Washington, USA, 18–21 Oct 2013Google Scholar
  22. 22.
    Bahdanau D (2016) Neural machine translation by jointly learning to align and translate. arXiv:1409.0473v7 [cs.CL] 19 May 2016
  23. 23.
    Mathur P et al (2017) Camera2Caption: a real-time image caption generator. In: International conference on computational intelligence in data science (ICCIDS)Google Scholar
  24. 24.
    Vinyals O et al (2015) Show and tell: a neural image caption generator. arXiv:1411.4555v2 [cs.CV] 20 Apr 2015
  25. 25.
    Xu K et al (2016) Show, attend and tell: neural image caption generation with visual attention. arXiv:1502.03044v3 [cs.LG] 19 Apr 2016
  26. 26.
    Venugopalan S et al (2015) Sequence to sequence—video to text, arXiv:1505.00487v3 [cs.CV] 19 Oct 2015
  27. 27.
    Jia X et al (2015) Guiding long-short term memory for image caption generation. In: IEEE international conference on computer visionGoogle Scholar
  28. 28.
    Zhu Z et al (2018) Topic-guided attention for image-captioning. In: International conference on image processing (IEEE)Google Scholar
  29. 29.
    Kinghorn P et al (2018) A region-based image caption generator with refined descriptions. Neurocomputing JGoogle Scholar
  30. 30.
    Donahue J et al (2017) Long-term recurrent convolutional networks for visual recognition and description. IEEE Trans Pattern Anal Mach Intell 39(4)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  1. 1.Department of Computer Science and EngineeringMaulana Azad National Institute of TechnologyBhopalIndia

Personalised recommendations