Advertisement

End-to-End Recurrent Neural Network Models for Vietnamese Named Entity Recognition: Word-Level Vs. Character-Level

Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 781)

Abstract

This paper demonstrates end-to-end neural network architectures for Vietnamese named entity recognition. Our best model is a combination of bidirectional Long Short-Term Memory (Bi-LSTM), Convolutional Neural Network (CNN), Conditional Random Field (CRF), using pre-trained word embeddings as input, which achieves an \(F_{1}\) score of 88.59% on a standard test set. Our system is able to achieve a comparable performance to the first-rank system of the VLSP campaign without using any syntactic or hand-crafted features. We also give an extensive empirical study on using common deep learning models for Vietnamese NER, at both word and character level.

Keywords

Vietnamese Named entity recognition End-to-end Long Short-Term Memory Conditional Random Field Convolutional Neural Network 

Notes

Acknowledgement

The second author is partly funded by the Vietnam National University, Hanoi (VNU) under project number QG.15.04. Any opinions, findings and conclusion expressed in this paper are those of the authors and do not necessarily reflect the view of VNU.

References

  1. 1.
    Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5(2), 157–166 (1994)CrossRefGoogle Scholar
  2. 2.
    Chiu, J.P., Nichols, E.: Named entity recognition with bidirectional LSTM-CNNs. Trans. Assoc. Comput. Linguist. 4, 357–370 (2016)Google Scholar
  3. 3.
    Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)MATHGoogle Scholar
  4. 4.
    Durrett, G., Klein, D.: A joint model for entity analysis: coreference, typing, and linking. Trans. Assoc. Comput. Linguist. 2, 477–490 (2014)Google Scholar
  5. 5.
    Florian, R., Ittycheriah, A., Jing, H., Zhang, T.: Named entity recognition through classifier combination. In: Daelemans, W., Osborne, M. (eds.) Proceedings of CoNLL-2003, pp. 168–171. Edmonton, Canada (2003)Google Scholar
  6. 6.
    Graves, A., Mohamed, A.R., Hinton, G.: Speech recognition with deep recurrent neural networks. In: Proceedings of 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 6645–6649. IEEE, Vancouver (2013)Google Scholar
  7. 7.
    Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional LSTM networks. In: Proceedings of 2005 IEEE International Joint Conference on Neural Networks, vol. 4, pp. 2047–2052. IEEE, Montreal (2005)Google Scholar
  8. 8.
    Hammerton, J.: Named entity recognition with long short-term memory. In: Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL, vol. 4, pp. 172–175. Association for Computational Linguistics (2003)Google Scholar
  9. 9.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  10. 10.
    Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991 (2015)
  11. 11.
    Kuru, O., Can, O.A., Yuret, D.: Charner: character-level named entity recognition. In: Proceedings of the 26th International Conference on Computational Linguistics, pp. 911–921 (2016)Google Scholar
  12. 12.
    Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of the Eighteenth International Conference on Machine Learning, vol. 1, pp. 282–289 (2001)Google Scholar
  13. 13.
    Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., Dyer, C.: Neural architectures for named entity recognition. arXiv preprint arXiv:1603.01360 (2016)
  14. 14.
    Le, T.H., Nguyen, T.T.T., Do, T.H., Nguyen, X.T.: Named entity recognition in Vietnamese text. In: Proceedings of the Fourth International Workshop on Vietnamese Language and Speech Processing. Hanoi, Vietnam (2016)Google Scholar
  15. 15.
    Le-Hong, P.: Vietnamese named entity recognition using token regular expressions and bidirectional inference. In: Proceedings of the Fourth International Workshop on Vietnamese Language and Speech Processing. Hanoi, Vietnam (2016)Google Scholar
  16. 16.
    Hông Phuong, L., Thi Minh Huyên, N., Roussanaly, A., Vinh, H.T.: A hybrid approach to word segmentation of Vietnamese texts. In: Martín-Vide, C., Otto, F., Fernau, H. (eds.) LATA 2008. LNCS, vol. 5196, pp. 240–249. Springer, Heidelberg (2008).  https://doi.org/10.1007/978-3-540-88282-4_23 CrossRefGoogle Scholar
  17. 17.
    Lin, D., Wu, X.: Phrase clustering for discriminative learning. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, vol. 2, pp. 1030–1038. Association for Computational Linguistics (2009)Google Scholar
  18. 18.
    Luo, G., Huang, X., Lin, C.-Y., Nie, Z.: Joint entity recognition and disambiguation. In: Proceedings of the 2015 Conference on Empirical Methods on Natural Language Processing, pp. 879–888. Association for Computational Linguistics (2015)Google Scholar
  19. 19.
    Ma, X., Hovy, E.: End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. arXiv preprint arXiv:1603.01354 (2016)
  20. 20.
    Nguyen, T.C.V., Pham, T.S., Vuong, T.H., Nguyen, N.V., Tran, M.V.: Dsktlabner: nested named entity recognition in Vietnamese text. In: Proceedings of the Fourth International Workshop on Vietnamese Language and Speech Processing. Hanoi, Vietnam (2016)Google Scholar
  21. 21.
    Nguyen, T.S., Nguyen, L.M., Tran, X.C.: Vietnamese named entity recognition at VLSP 2016 evaluation campaign. In: Proceedings of the Fourth International Workshop on Vietnamese Language and Speech Processing. Hanoi, Vietnam (2016)Google Scholar
  22. 22.
    Pascanu, R., Mikolov, T., Bengio, Y.: On the difficulty of training recurrent neural networks. In: The 30th International Conference on Machine Learning, vol. 28, pp. 1310–1318. Atlanta, USA (2013)Google Scholar
  23. 23.
    Petasis, G., Petridis, S., Paliouras, G., Karkaletsis, V., Perantonis, S., Spyropoulos, C.: Symbolic and neural learning for named-entity recognition. In: Symposium on Computational Intelligence and Learning, pp. 58–66. Citeseer, Chios (2000)Google Scholar
  24. 24.
    dos Santos, C., Guimaraes, V., Niterói, R.J.: Boosting named entity recognition with neural character embeddings. In: Proceedings of NEWS 2015 the Fifth Named Entities Workshop, pp. 25–33 (2015)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  1. 1.R&D DepartmentAlt Inc.HanoiVietnam
  2. 2.College of ScienceVietnam National UniversityHanoiVietnam

Personalised recommendations