A Deep Learning Based Reasoner for Global Consistency in Named Entity Recognition

  • Xiaoxiao Yin
  • Ruifang LiuEmail author
  • Daqi Zheng
  • Zhengdong Lu
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1074)


Named Entity Recognition (NER) is a basic task of Natural Language Processing (NLP), it’s a challenging task in a variety of special applications. This paper aims to solve the global consistency of NER, and to improve the performance. Inspired by human reading process, we propose a NE-Reasoner model, which combine deep neural networks and memory artificial neural network to identify named entities with global consistency. The advantages of the model are: (1) The multi-layer deep architecture, allowing it to bootstrap the recognized entity set from coarse to fine. (2) The candidate pool memory mechanism, allowing it to exchange identified entity information between layers. (3) The reasoner, combing encoder-decoder and cached information to infer to get global entities. The experimental results show that the NE-Reasoner can identity ambiguous words and named entities that rarely or never met before.


Named Entity Recognition NE-Reasoner Global consistency Deep neural network Memory artificial neural network 


  1. 1.
    Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12(Aug), 2493–2537 (2011)zbMATHGoogle Scholar
  2. 2.
    Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991 (2015)
  3. 3.
    Ma, X., Hovy, E.: End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 1064–1074 (2016)Google Scholar
  4. 4.
    Chiu, J.P., Nichols, E.: Named entity recognition with bidirectional LSTM-CNNs. Trans. Assoc. Comput. Linguist. 4, 357–370 (2016)CrossRefGoogle Scholar
  5. 5.
    Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., Dyer, C.: Neural architectures for named entity recognition. In: Proceedings of NAACL-HLT, pp. 260–270 (2016)Google Scholar
  6. 6.
    Strubell, E., Verga, P., Belanger, D., McCallum, A.: Fast and accurate entity recognition with iterated dilated convolutions. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2670–2680 (2017)Google Scholar
  7. 7.
    Peters, M., Ammar, W., Bhagavatula, C., Power, R.: Semi-supervised sequence tagging with bidirectional language models. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 1756–1765 (2017)Google Scholar
  8. 8.
    Liu, L., Shang, J., Xu, F.F., Ren, X., Gui, H., Peng, J., Han, J.: Empower sequence labeling with task-aware neural language model. In: International Conference on Artificial Intelligence (2018)Google Scholar
  9. 9.
    Zhang, Y., Yang, J.: Chinese NER using lattice LSTM. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1554–1564. Association for Computational Linguistics (2018).
  10. 10.
    Nguyen, T.H., Sil, A., Dinu, G., Florian, R.: Toward mention detection robustness with recurrent neural networks. arXiv preprint arXiv:1602.07749 (2016)
  11. 11.
    Graves, A., Wayne, G., Danihelka, I.: Neural turing machines. arXiv preprint arXiv:1410.5401 (2014)
  12. 12.
    Sukhbaatar, S., Weston, J., Fergus, R., et al.: End-to-end memory networks. In: Advances in Neural Information Processing Systems, pp. 2440–2448 (2015)Google Scholar
  13. 13.
    Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014).

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Xiaoxiao Yin
    • 1
  • Ruifang Liu
    • 1
    Email author
  • Daqi Zheng
    • 2
  • Zhengdong Lu
    • 2
  1. 1.Beijing University of Posts and TelecommunicationsBeijingChina
  2. 2.DeeplyCurious.aiBeijingChina

Personalised recommendations