Skip to main content

Study on Chinese Named Entity Recognition Based on Dynamic Fusion and Adversarial Training

  • 372 Accesses

Part of the Lecture Notes in Computer Science book series (LNAI,volume 13370)

Abstract

This paper aims at the Chinese Named Entity Recognition task, uses NEZHA Chinese pre-trained language model as the word embedding layer, then adopts the BiLSTM network architecture to encode it, and finally connects the CRF layer to optimize the output sequence. In order to enhance the fusion of semantic features of the NEZHA model in the upper, middle, and lower layers, an attention mechanism has been adopted to integrate the NEZHA coding layers. At first, weight was given to each representation generated by its 12 Transformer coding layers. Secondly, the weight value was dynamically adjusted through supervised training, and then the generated layer representation was weighted average to get the final word embedded representation. Finally, some noise was introduced to the input data, which is used for adversarial training to improve the generalization and robustness of the model. The results show that the F1 Score of the proposed model on Chinese Clinical Named Entity Recognition Dataset and people’s daily corpus are respectively 98.52% and 96.84%, which are 2.36% and 4.21% respectively higher than the benchmark model Bert BiLSTM CRF.

Keywords

  • Natural language processing
  • Chinese named entity recognition
  • NEZHA
  • Dynamic fusion
  • Adversarial training

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-031-10989-8_1
  • Chapter length: 12 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   89.00
Price excludes VAT (USA)
  • ISBN: 978-3-031-10989-8
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   119.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.

References

  1. Liu, W., Yu, B., Zhang, C., et al.: Chinese named entity recognition based on rules and conditional random field. In: Proceedings of the 2018 2nd International Conference on Computer Science and Artificial Intelligence, pp. 268–272 (2018)

    Google Scholar 

  2. Wang, X., Jiang, X., Liu, M., et al.: Bacterial named entity recognition based on dictionary and conditional random field. In: 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 439–444. IEEE (2017)

    Google Scholar 

  3. Wallach, H.M.: Conditional random fields: An introduction. Technical reports (CIS), p. 22 (2004)

    Google Scholar 

  4. Li, X., Wei, X.H., Jia, L., et al.: Recognition of crops, diseases and pesticides named entities in Chinese based in conditional random fields. Trans. Chin. Soc. Agric. Mach. 48, 178–185 (2017)

    Google Scholar 

  5. Malarkodi, C.S., Lex, E., Devi, S.L.: Named entity recognition for the agricultural domain. Res. Comput. Sci. 117, 121–132 (2016)

    CrossRef  Google Scholar 

  6. Zhou, G.D., Su, J.: Named entity recognition using an HMM-based chunk tagger. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 473–480 (2002)

    Google Scholar 

  7. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    CrossRef  Google Scholar 

  8. Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging. arXiv:1508.01991 (2015)

  9. Zhang, Y., Yang, J.: Chinese NER using lattice LSTM. arXiv:1805.02023 (2018)

  10. Devlin, J., Chang, M.W., Lee, K., et al.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805 (2018)

  11. Vaswani, A., Shazeer, N., Parmar, N., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)

    Google Scholar 

  12. Souza, F., Nogueira, R., Lotufo, R.: Portuguese named entity recognition using BERT-CRF. arXiv:1909.10649 (2019)

  13. Straková J., Straka M., Hajič J.: Neural architectures for nested NER through linearization. arXiv:1908.06926 (2019)

  14. Wei, J., Ren, X., Li, X., et al.: NEZHA: neural contextualized representation for Chinese language understanding. arXiv:1909.00204 (2019)

  15. Jawahar, G., Sagot, B., Seddah, D.: What does BERT learn about the structure of language? In: Proceedings of the 57rd Annual Meeting of the ACL (2019)

    Google Scholar 

  16. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. arXiv:1412.6572 (2014)

  17. Madry, A., Makelov, A., Schmidt, L., et al.: Towards deep learning models resistant to adversarial attacks. arXiv:1706.06083 (2017)

  18. Miyato, T., Dai, A.M., Goodfellow, I.: Adversarial training methods for semi-supervised text classification. arXiv:1605.07725 (2016)

  19. Miyato, T., Dai, A.M., Goodfellow, I.: Adversarial training methods for semi-supervised text classification. arXiv preprint arXiv:1605.07725 (2016)

  20. Chen, X., Cardie, C.: Multinomial adversarial networks for multi-domain text classification. arXiv preprint arXiv:1802.05694 (2018)

  21. Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015) 2(7)

Download references

Acknowledgement

This work has been supported by the Major Project of Science and Technology of Yunnan Province under Grant No. 202002AD080002.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Linnan Yang .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Fan, F., Yang, L., Wu, X., Lin, S., Dong, H., Yin, C. (2022). Study on Chinese Named Entity Recognition Based on Dynamic Fusion and Adversarial Training. In: Memmi, G., Yang, B., Kong, L., Zhang, T., Qiu, M. (eds) Knowledge Science, Engineering and Management. KSEM 2022. Lecture Notes in Computer Science(), vol 13370. Springer, Cham. https://doi.org/10.1007/978-3-031-10989-8_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-10989-8_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-10988-1

  • Online ISBN: 978-3-031-10989-8

  • eBook Packages: Computer ScienceComputer Science (R0)