A Character-Level Deep Lifelong Learning Model for Named Entity Recognition in Vietnamese Text

  • Ngoc-Vu Nguyen
  • Thi-Lan Nguyen
  • Cam-Van Nguyen Thi
  • Mai-Vu Tran
  • Quang-Thuy HaEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11431)


Lifelong Machine Learning (LML) is a continuous learning process, in which the knowledge learned from previous tasks is accumulated in the knowledge base, then the knowledge will be used to support future learning tasks, for which it may be only a few of samples exists. However, there is a little of studies on LML based on deep neural networks for Named Entity Recognition (NER), especial in Vietnamese. We propose DeepLML-NER model, a lifelong learning model based on using deep learning methods with a CRFs layer, for NER in Vietnamese text. DeepLML-NER includes an algorithm to extract the knowledge of “prefix-features” of Named Entities in previous domains. Then the model uses the knowledge stored in the knowledge base to solve a new NER task. The effect of the model was demonstrated by in-domain and cross-domain experiments, achieving promising results.


Deep LML Named Entity Recognition (NER) Deep Lifelong Learning Deep LML for NER in Vietnamese text 



This work was supported in part by Grant TNMT.2017.09.02.


  1. 1.
    Bendale, A., Boult, T.E.: Towards open world recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1893–1902 (2015)Google Scholar
  2. 2.
    Chen, Z., Liu, B.: Lifelong Machine Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 2nd edn. (2018)Google Scholar
  3. 3.
    Chiu, J.P.C., Nichols, E.: Named entity recognition with bidirectional LSTM-CNNs. CoRR, abs/1511.08308 (2015)Google Scholar
  4. 4.
    Dong, N.T.: An investigation of Vietnamese nested entity recognition models. In: VLSP 2018 Workshop, pp. 14–16 (2018)Google Scholar
  5. 5.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  6. 6.
    Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging. CoRR, abs/1508.01991 (2015)Google Scholar
  7. 7.
    Lafferty, J.D., McCallum, A., Pereira, F.C.N.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: ICML 2001, pp. 282–289 (2001)Google Scholar
  8. 8.
    Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., Dyer, C.: Neural architectures for named entity recognition. CoRR, abs/1603.01360 (2016)Google Scholar
  9. 9.
    Luong, V.-T., Phan, L.K.: Zaner: Vietnamese named entity recognition at VLSP 2018 evaluation campaign. In: VLSP 2018 Workshop, pp. 10–13 (2018)Google Scholar
  10. 10.
    Ma, X., Hovy, E.H.: End-to-end sequence labeling via bi-directional LSTM-CNNS-CRFs. CoRR, abs/1603.01354 (2016)Google Scholar
  11. 11.
    Parisi, G.I., Tani, J., Weber, C., Wermter, S.: Lifelong learning of human actions with deep neural network self-organization. Neural Netw. 96, 137–149 (2017)Google Scholar
  12. 12.
    Pham, Q.N.M.: A feature-based model for nested named-entity recognition at VLSP-2018 NER evaluation campaign. In: VLSP 2018 Workshop, pp. 5–9 (2018)Google Scholar
  13. 13.
    Pham, T.-H., Pham, X.-K., Nguyen, T.-A., Le-Hong, P.: NNVLP: A neural network-based Vietnamese language processing toolkit. arXiv preprint arXiv:1708.07241 (2017)
  14. 14.
    Rusu, A.A., et al.: Progressive neural networks. CoRR, abs/1606.04671 (2016)Google Scholar
  15. 15.
    Ramshaw, L.A., Marcus, M.: Text chunking using transformation-based learning. In: VLC@ACL (1995)Google Scholar
  16. 16.
    Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. Trans. Sig. Proc. 45(11), 2673–2681 (1997)Google Scholar
  17. 17.
    Shu, L., Xu, H., Liu, B.: Lifelong learning CRF for supervised aspect extraction. CoRR, abs/1705.00251 (2017)Google Scholar
  18. 18.
    Silver, D.L., Yang, Q., Li, L.: Lifelong machine learning systems beyond learning algorithms. In: AAAI Spring Symposium Lifelong Machine Learning (2013)Google Scholar
  19. 19.
    Thrun, S., Mitchell, T.M.: Lifelong robot learning. Robot. Auton. Syst. 15(1–2), 25–46 (1995)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Ngoc-Vu Nguyen
    • 1
    • 2
  • Thi-Lan Nguyen
    • 1
  • Cam-Van Nguyen Thi
    • 1
  • Mai-Vu Tran
    • 1
  • Quang-Thuy Ha
    • 1
    Email author
  1. 1.University of Engineering and Technology (UET)Vietnam National University, Hanoi (VNU)HanoiVietnam
  2. 2.Department of Information TechnologyMoNREHanoiVietnam

Personalised recommendations