Skip to main content
SpringerLink
Log in

A Multi-Task BERT-BiLSTM-AM-CRF Strategy for Chinese Named Entity Recognition

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Named entity recognition aims to identify and mark entities with specific meanings in text. It is a key technology to further extract entity relationships and mine other potential information in natural language processing. At present, the methods based on machine learning and deep learning have been widely used in the research of named entity recognition, but most learning models use feature extraction based on word and character level. The word preprocessing of this kind of model often ignores the context semantic information of the target word and can not realize polysemy. In addition, the loss of semantic information and limited training data greatly limit the improvement of model performance and generalization ability. In order to solve the above problems and improve the efficiency of named entity recognition technology in Chinese text, this paper constructs a multi-task BERT-BiLSTM-AM-CRF intelligent processing model, uses Bert to extract the dynamic word vector combined with context information, and inputs the results into CRF layer for decoding after further training through BiLSTM module. After attention mechanism network, the model can learn together on two Chinese datasets, Finally, CRF classifies and extracts the observation annotation sequence to get the final result. Compared with many previous single task models, the F1 score of this multi-task model in MASR and people’s daily datasets has been significantly improved (0.55% and 3.41%), which demonstrates the effectiveness of multi-task learning for Chinese named entity recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Xie X, Fu Y, Jin H, Zhao Y, Cao W (2020) A novel text mining approach for scholar information extraction from web content in chinese. Futur Gener Comput Syst 111:859–872

    Article  Google Scholar 

  2. Xiang L, Sun X, Luo G, Xia B (2014) Linguistic steganalysis using the features derived from synonym frequency. Multimed Tools Appl 71(3):1893–1911

    Article  Google Scholar 

  3. Sun C, Yang Z, Wang L, Zhang Y, Lin H, Wang J (2021) Biomedical named entity recognition using bert in the machine reading comprehension framework. J. Biomedical Informatics 118:103799

    Article  Google Scholar 

  4. Zhai Z, Nguyen DQ, Akhondi S, Thorne C, Verspoor K (2019) Improving Chemical Named Entity Recognition in Patents with Contextualized Word Embeddings. Proceedings of the 18th BioNLP Workshop and Shared Task

  5. Ivan L, Nicolas P, Xavier T (2020) Terminologies augmented recurrent neural network model for clinical named entity recognition. J. Biomedical Informatics 102:103356

    Article  Google Scholar 

  6. Zhou S, Tan B (2020) Electrocardiogram soft computing using hybrid deep learning cnn-elm. Appl Soft Comput 86:105778

    Article  Google Scholar 

  7. He S, Li Z, Tang Y, Liao Z, Li F, Lim S (2020) Parameters compressing in deep learning. Cmc-computers Materials & Continua 62(1):321–336

    Article  Google Scholar 

  8. Habibi M, Weber L, Neves M, Wiegandt DL, Leser U (2017) Deep learning with word embeddings improves biomedical named entity recognition. Bioinformatics 33(14):37–48

    Article  Google Scholar 

  9. Lample G, Ballesteros M, Subraman S (2016) Neural architectures for namedentity recognition. Proceedings of NAACL-HLT, 260–270

  10. Ma X, Hovy EK (2016) End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics

  11. Chen Y, Zhou C, Li T, Wu H, Zhao X, Ye K, Liao J (2019) Named entity recognition from chinese adverse drug event reports with lexical feature based bilstm-crf and tri-training. J. Biomedical Informatics 96:103252

    Article  Google Scholar 

  12. Li Z, Li Q, Zou X, Ren J (2021) Causality extraction based on self-attentive bilstm-crf with transferred embeddings. Neurocomputing 423:207–219

    Article  Google Scholar 

  13. Putra FM, Retno K, Adi W (2021) Sentiment analysis using word2vec and long short-term memory (lstm) for indonesian hotel reviews. Procedia Comp Sci 179:728–735

    Article  Google Scholar 

  14. Quang-Thai H, Trinh-Trung-Duong N, Nguyen QKL, Yu-Yen O (2021) Fad-bert: Improved prediction of fad binding sites using pre-training of deep bidirectional transformers. Comput Biol Med 131:104258

    Article  Google Scholar 

  15. Tang X, Cao W, Tang H, Deng T, Mei J, Liu Y, Shi C, Xia M, Zeng Z (2022) Cost-efficient workflow scheduling algorithm for applications with deadline constraint on heterogeneous clouds. IEEE Trans. Parallel Distrib Syst 33(9):2079–2092

    Article  Google Scholar 

  16. Tang X, Shi C, Deng T, Wu Z, Yang L (2021) Parallel random matrix particle swarm optimization scheduling algorithms with budget constraints on cloud computing systems. Appl Soft Comput 113:107914

    Article  Google Scholar 

  17. Fukuda K, Tsunoda T, Tamura A et al (1998) Toward information extraction: identifying protein names from biological papers. Pac Symp Biocomput 707(18):707–718

    Google Scholar 

  18. Hanisch D, Fundel K, Mevissen HT et al (2005) Prominer: rule-based protein and gene entity recognition. BMC bioinformatics 6(1):14

    Article  Google Scholar 

  19. Lee KJ, Hwang YS, Kim S et al (2004) Biomedical named entity recognition using two-phase model based on svms. J. Biomedical Informatics 37(6):436–447

    Article  Google Scholar 

  20. Satyajit N, Justin D (2022) Factored latent-dynamic conditional random fields for single and multi-label sequence modeling. Pattern Recogn 122:108236

    Article  Google Scholar 

  21. Chen H, Sun F, Yuan J, Huan Y (2021) Mirrored conditional random field model for object recognition in indoor environments. Inf Sci 551:291–303

    Article  MathSciNet  MATH  Google Scholar 

  22. Liu X, Zhou Y, Wang Z (2021) Deep neural network-based recognition of entities in chinese online medical inquiry texts. Futur Gener Comput Syst 114:581–604

    Article  Google Scholar 

  23. De Oliveira DM, Laender AHF, Veloso A et al (2013) FS-NER: A lightweight filter-stream approach to named entity recognition on twitter data. Proceedings of the 22nd International Conference on World Wide Web

  24. Liu P, Guo Y, Wang F, Li G (2022) Chinese named entity recognition: The state of the art. Neurocomputing 473:37–53

    Article  Google Scholar 

  25. Jia Y, Xu X (2018) Chinese named entity recognition based on CNN-BiLSTM-CRF. IEEE 9th International Conference on Software Engineering and Service Science (ICSESS)

  26. Zhao S, Cai Z, Chen H, Wang Y, Liu F, Liu A (2019) Adversarial training based lattice lstm for chinese clinical named entity recognition. J. Biomedical Informatics 99:103290

    Article  Google Scholar 

  27. Chang N, Zhong J, Li Q, Zhu J (2020) A mixed semantic features model for chinese ner with characters and words. In: ECIR 2020: Advances in Information Retrieval, pp. 356–368. Springer, Heidelberg

  28. Dai Z, Wang X, Ni P, Li Y, Li G, Bai X (2019) Named entity recognition using BERT BiLSTM CRF for Chinese electronic health records. The 12th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI)

  29. Murugesan G, Abdulkadhar S, Bhasuran B, Natarajan J (2017) Bcc-ner: bidirectional, contextual clues named entity tagger for gene/protein mention recognition. J. Bioinform Sys. Biology 7:1–8

    Google Scholar 

  30. Cheng P, Dai J, Liu J (2022) Catvrnn: Generating category texts via multi-task learning. Knowl-Based Syst 244:108491

    Article  Google Scholar 

  31. Xu K, Zhou Z, Gong T, Hao T, Liu W (2018) Sblc: a hybrid model for disease named entity recognition based on semantic bidirectional lstms and conditional random fields. BMC Med Inform Decis Mak 18:114

    Article  Google Scholar 

  32. Graves A, Mohamed AR, Hinton G (2013) Speech recognition with deep recurrent neural networks. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing

  33. Xu L, Dong Q, Yu C, Tian Y, Liu W, Li L, Zhang X (2020) Cluener2020: Fine-grained name entity recognition for chinese. arXiv preprint arXiv:2001.04351

Download references

Acknowledgements

This research was partially funded by National Natural Science Foundation of China (Grant No. 61972146), Hunan Provincial Natural Science Foundation of China (Grant No. 2020JJ4376).

Author information

Authors and Affiliations

  1. School of Computer and Communications Engineering, Changsha University of Science & Technology, Changsha, 410114, Hunan, China

    Xiaoyong Tang, Yong Huang & Meng Xia

  2. School of Information and Intelligence, Hunan Agricultural University, Changsha, 410128, Hunan, China

    Chengfeng Long

Authors
  1. Xiaoyong Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yong Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Meng Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chengfeng Long
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meng Xia.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tang, X., Huang, Y., Xia, M. et al. A Multi-Task BERT-BiLSTM-AM-CRF Strategy for Chinese Named Entity Recognition. Neural Process Lett 55, 1209–1229 (2023). https://doi.org/10.1007/s11063-022-10933-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-022-10933-3

Keywords

Search

Navigation