Skip to main content
SpringerLink
Log in

A Short Review on Deep Learning for Entity Recognition

  • Conference paper
  • First Online:
Future Data and Security Engineering (FDSE 2018)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11251))

Included in the following conference series:

  • 1022 Accesses

Abstract

Deep learning is a kind of representation learning − a subfield of machine learning. While most machine learning methods work well thanks to feature engineering, deep learning automatically learns good feature representations of input data at multiple levels. In this paper, we present distributed representations and deep learning models that automatically learn features for coarse- and fine-grained entity recognition. The former recognizes entities with very few types, whereas the latter identifies entities and classifies them into a large number of types. Until now, most of research on entity recognition has focused on the former. However, the latter is more challenging and has attracted much research attention recently. This paper presents state-of-the-art methods for both coarse- and fine-grained entity recognition until late 2017.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abhishek, A.A., Awekar, A.: Fine-grained entity type classification by jointly learning representations and label embeddings. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL), pp. 797–807 (2017)

    Google Scholar 

  2. Arik, S.O., et al.: Deep Voice: Real-time Neural Text-to-Speech. arXiv preprint arXiv:1702.07825 (2017)

  3. Cui, K.Y., Ren, P.J., Chen, Z.M., Lian, T., Ma, J.: Relation enhanced neural model for type classification of entity mentions with a fine-grained taxonomy. J. Comput. Sci. Technol. 32(4), 814–827 (2017)

    Article  Google Scholar 

  4. Cai, H., Zheng, V.W., Chang, K.C.C.: A Comprehensive Survey of Graph Embedding: Problems, Techniques and Applications. arXiv preprint arXiv:1709.07604 (2017)

  5. Huang, L., May, J., Pan, X., Ji, H., Ren, X., Han, J., Zhao, L., Hendler, J.A.: Liberal entity extraction: rapid construction of fine-grained entity typing systems. Big Data 5(1), 19–31 (2017)

    Article  Google Scholar 

  6. Karn, S.K., Waltinger, U., and Schütze, H.: End-to-end trainable attentive decoder for hierarchical entity classification. In: EACL, pp. 752–758 (2017)

    Google Scholar 

  7. Liu, L., et al.: Empower Sequence Labeling with Task-Aware Neural Language Model. https://arxiv.org/pdf/1709.04109v3.pdf (2017)

  8. Peters, M.E., Ammar, W., Bhagavatula, C., Power, R.: Semi-supervised sequence tagging with bidirectional language models. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 1756–1765 (2017)

    Google Scholar 

  9. Reimers, N., Gurevych, I.: Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks. arXiv preprint arXiv:1707.06799 (2017)

  10. Strubell, E., Verga, P., Belanger, D., McCallum, A.: Fast and accurate sequence labeling with iterated dilated convolutions. In: Proceedings of the Conference on Empirical Methods on Natural Language Processing (EMNLP) (2017)

    Google Scholar 

  11. Shimaoka, S., Stenetorp, P., Inui, K., Riedel, S.: Neural architectures for fine-grained entity type classification. In: EACL, pp. 1271–1280 (2017)

    Google Scholar 

  12. Tran, Q., MacKinlay, A., Yepes, A.J.: Named entity recognition with stack residual LSTM and trainable bias decoding. In: Proceedings of the 8th International Joint Conference on Natural Language Processing (IJCNLP) (2017)

    Google Scholar 

  13. Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. In: Advances in Neural Information Processing Systems (NIPS) (2017)

    Google Scholar 

  14. Wang, Q., Mao, Z., Wang, B., Guo, L.: Knowledge graph embedding: a survey of approaches and applications. IEEE Trans. Knowl. Data Eng. 29, 2724–2743 (2017)

    Article  Google Scholar 

  15. Wang, Y., Kosinski, M.: Deep neural networks are more accurate than humans at detecting sexual orientation from facial images. J. Pers. Soc. Psychol. (2017)

    Google Scholar 

  16. Xu, M., Jiang, H., Watcharawittayakul, S.: A local detection approach for named entity recognition and mention detection. In: ACL, pp. 1237–1247 (2017)

    Google Scholar 

  17. Yang, Z., Salakhutdinov, R., Cohen, W.W.: Transfer learning for sequence tagging with hierarchical recurrent networks. In: ICLR (2017)

    Google Scholar 

  18. Yang, J., Zhang, Y., Dong, F.: Neural reranking for named entity recognition. In: Proceedings of RANLP (2017)

    Google Scholar 

  19. Chiu, J.P., Nichols, E.: Named entity recognition with bidirectional LSTM-CNNs. TACL 4(1), 357–370 (2016)

    Google Scholar 

  20. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)

    Google Scholar 

  21. Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., Dyer, C.: Neural architectures for named entity recognition. In: The 15th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (2016)

    Google Scholar 

  22. Oord, A.V.D. et al.: Wavenet: A Generative Model for Raw Audio. arXiv preprint arXiv:1609.03499 (2016)

  23. Ma, X., Hovy, E.: End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In: ACL, pp. 1064–1074 (2016)

    Google Scholar 

  24. Ma, Y., Cambria, E., Gao, S.: Label embedding for zero-shot fine-grained named entity typing. In: COLING, pp. 171–180 (2016)

    Google Scholar 

  25. Ren, X., He, W., Qu, M., Huang, L., Ji, H., Han, J.: AFET: automatic fine-grained entity typing by hierarchical partial-label embedding. In: EMNLP (2016)

    Google Scholar 

  26. Ren, X., He, W., Qu, M., Voss, C.R., Ji, H., Han, J.: Label noise reduction in entity typing by heterogeneous partial-label embedding. In: Proceedings of the 22nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (2016)

    Google Scholar 

  27. Wu, Y., et al.: Google’s Neural Machine Translation System: Bridging the gap between Human and Machine Translation. arXiv preprint arXiv:1609.08144 (2016)

  28. Xiong, C., Merity, S., Socher, R.: Dynamic memory networks for visual and textual question answering. In: International Conference on Machine Learning (ICML), pp. 2397–2406 (2016)

    Google Scholar 

  29. Corro, L.D., Abujabal, A., Gemulla, R., Weikum, G.: FINET: context-aware fine-grained named entity typing. In: EMNLP, pp. 868–878 (2015)

    Google Scholar 

  30. Dong, L., Wei, F., Sun, H., Zhou, M., Xu, K.: A hybrid neural model for type classification of entity mentions. In: IJCAI, pp. 1243–1249 (2015)

    Google Scholar 

  31. Huang, Z.H., Xu, W., Yu, K.: Bidirectional LSTM-CRF Models for Sequence Tagging. arXiv:1508.01991 (2015)

  32. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  33. Yogatama, D., Gillick, D., Lazic, N.: Embedding methods for fine grained entity type classification. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL-IJCNLP), pp. 291–296 (2015)

    Google Scholar 

  34. Zhang, S., Jiang, H., Xu, M., Hou, J., Dai, L.: The fixed-size ordinally forgetting encoding method for neural network language lodels. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL-IJCNLP) (2015)

    Google Scholar 

  35. Chen, D., Manning, C.: A fast and accurate dependency parser using neural networks. In: EMNLP, pp. 740–750 (2014)

    Google Scholar 

  36. Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: EMNLP, pp. 1724–1734 (2014)

    Google Scholar 

  37. Goodfellow, I., et al.: Generative adversarial nets. In: NIPS, pp. 2672–2680 (2014)

    Google Scholar 

  38. Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: EMNLP, vol. 14, pp. 1532–1543 (2014)

    Google Scholar 

  39. Mikolov, T., Sutskever, I., Chen, K., Corrado, G., and Dean, J.: Distributed representations of words and phrases and their compositionality. In: NIPS (2013)

    Google Scholar 

  40. Ling, X., Weld, D.S.: Fine-grained entity recognition. In: AAAI (2012)

    Google Scholar 

  41. Yosef, M.A., Bauer, S., Hoffart, J., Spaniol, M., Weikum, G.: HYENA: hierarchical type classification for entity names. In: Proceedings of the 24th International Conference on Computational linguistics (COLING), pp. 1361–1370 (2012)

    Google Scholar 

  42. Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)

    MATH  Google Scholar 

  43. Turian, J., Ratinov, L., Bengio, Y.: Word representations: a simple and general method for semi-supervised learning. In: ACL, pp. 384–394 (2010)

    Google Scholar 

  44. Nadeau, D., Sekine, S.: A survey of named entity recognition and classification. Lingvisticae Investig. 30(1), 3–26 (2007)

    Article  Google Scholar 

  45. Etzioni, O., Cafarella, M., Downey, D., Popescu, A.M., Shaked, T., Soderland, S., Weld, D.S., Yates, A.: Unsupervised named-entity extraction from the web: an experimental study. Artif. Intell. 165(1), 91–134 (2005)

    Article  Google Scholar 

  46. Finkel, J.R., Grenager, T., Manning, C.: Incorporating non-local information into information extraction systems by Gibbs sampling. In: ACL, pp. 363–370 (2005)

    Google Scholar 

  47. Tsochantaridis, I., Hofmann, T., Joachims, T., Altun, Y.: Support vector machine learning for interdependent and structured output spaces. In: Proceedings of the Twenty-First International Conference on Machine Learning (ICML) (2004)

    Google Scholar 

  48. Tjong Kim Sang, E.F., De Meulder, F.: Introduction to the CoNLL-2003 shared task: language-independent named entity recognition. In: NAACL-HLT, pp. 142–147 (2003)

    Google Scholar 

  49. Chieu, H.L., Ng, H.T.: Named entity recognition with a maximum entropy approach. In: Conference on Natural Language Learning (CoNLL), pp. 160–163 (2003)

    Google Scholar 

  50. Fleischman, M., Hovy, E.: Fine grained classification of named entities. In: Proceedings of the 19th International Conference on Computational linguistics (COLING) (2002)

    Google Scholar 

  51. Collins, M., Singer, Y.: Unsupervised models for named entity classification. In: Joint Conference on Empirical Methods in Natural Language Processing and Very Large Corpora (1999)

    Google Scholar 

  52. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  53. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  54. Cunningham, H., Wilks, Y., Gaizauskas, R.J.: GATE: a general architecture for text engineering. In: COLING, pp. 1057–1060 (1996)

    Google Scholar 

  55. Hinton, G.E., McClelland, J., Rumelhart, D.: Distributed representations. In: Rumelhart, D., McClelland, J. (eds.) Parallel Distributed Processing, vol. 1, pp. 77–109 (1986)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Artificial Intelligence Laboratory, Faculty of Information Technology, Ton Duc Thang University, 19 Nguyen Huu Tho St., Tan Phong Ward, District 7, Ho Chi Minh City, 700000, Vietnam

    Hien T. Nguyen

  2. Ho Chi Minh City Open University, 97 Vo Van Tan St., Ward 6, District 3, Ho Chi Minh City, 700000, Vietnam

    Thuan Quoc Nguyen

Authors
  1. Hien T. Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thuan Quoc Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hien T. Nguyen .

Editor information

Editors and Affiliations

  1. Ho Chi Minh City University of Technology, Ho Chi Minh, Vietnam

    Tran Khanh Dang

  2. Johannes Kepler University of Linz, Linz, Austria

    Josef Küng

  3. Johannes Kepler University of Linz, Linz, Austria

    Roland Wagner

  4. Ho Chi Minh City University of Technology, Ho Chi Minh, Vietnam

    Nam Thoai

  5. Hosei University, Tokyo, Japan

    Makoto Takizawa

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Nguyen, H.T., Nguyen, T.Q. (2018). A Short Review on Deep Learning for Entity Recognition. In: Dang, T., Küng, J., Wagner, R., Thoai, N., Takizawa, M. (eds) Future Data and Security Engineering. FDSE 2018. Lecture Notes in Computer Science(), vol 11251. Springer, Cham. https://doi.org/10.1007/978-3-030-03192-3_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-03192-3_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-03191-6

  • Online ISBN: 978-3-030-03192-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation