Abstract
Named Entity Recognition (NER) is generally regarded as a sequence labeling task, which faces a serious problem when the named entities are nested. In this paper, we propose a span-based model for nested NER, which enumerates all possible spans as potential entity mentions in a sentence and classifies them with pretrained BERT model. In view of the phenomenon that there are too many negative samples in all spans, we propose a multi-task learning method, which divides NER task into entity identification and entity classification task. In addition, we propose the entity IoU loss function to focus our model on the hard negative samples. We evaluate our model on three standard nested NER datasets: GENIA, ACE2004 and ACE2005, and the results show that our model outperforms other state-of-the-art models with the same pretrained language model, achieving 79.46%, 87.30% and 85.24% respectively in terms of F1 score.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
This model is based on BERT-base and they use different dataset splits. Except for this model, other models are based on BERT-large.
- 2.
This model also uses the fasttext embedding, for the sake of fairness, we re-train this model without the fasttext embedding.
References
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL, pp. 4171–4186 (2019)
Eberts, M., Ulges, A.: Span-based joint entity and relation extraction with transformer pre-training. arXiv preprint arXiv:1909.07755 (2019)
Fisher, J., Vlachos, A.: Merge and label: a novel neural network architecture for nested ner. In: Proceedings of ACL, pp. 5840–5850 (2019)
Ju, M., Miwa, M., Ananiadou, S.: A neural layered model for nested named entity recognition. In: Proceedings of NAACL, pp. 1446–1459 (2018)
Jue, W., Shou, L., Chen, K., Chen, G.: Pyramid: a layered model for nested named entity recognition. In: Proceedings of ACL, pp. 5918–5928 (2020)
Kim, J.D., Ohta, T., Tateisi, Y., Tsujii, J.: Genia corpus–a semantically annotated corpus for bio-text mining. Bioinformatics 19(suppl\(\_\)1), i180–i182 (2003)
Lafferty, J., McCallum, A., Pereira, F.C.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data (2001)
dos Santos, C., Xiang, B., Zhou, B.: Classifying relations by ranking with convolutional neural networks. In: Proceedings of ACL, pp. 626–634 (2015)
Sennrich, R., Haddow, B., Birch, A.: Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909 (2015)
Shibuya, T., Hovy, E.: Nested named entity recognition via second-best sequence learning and decoding. arXiv preprint arXiv:1909.02250 (2019)
Sohrab, M.G., Miwa, M.: Deep exhaustive model for nested named entity recognition. In: Proceedings of EMNLP, pp. 2843–2849 (2018)
Straková, J., Straka, M., Hajic, J.: Neural architectures for nested NER through linearization. In: Proceedings of ACL pp. 5326–5331 (2019)
Tan, C., Qiu, W., Chen, M., Wang, R., Huang, F.: Boundary enhanced neural span classification for nested named entity recognition, pp. 9016–9023. AAAI (2020)
Wang, B., Lu, W., Wang, Y., Jin, H.: A neural transition-based model for nested mention recognition. In: Proceedings of EMNLP, pp. 1011–1017 (2018)
Xu, M., Jiang, H., Watcharawittayakul, S.: A local detection approach for named entity recognition and mention detection. In: Proceedings of ACL, pp. 1237–1247 (2017)
Ye, W., Li, B., Xie, R., Sheng, Z., Chen, L., Zhang, S.: Exploiting entity bio tag embeddings and multi-task learning for relation extraction with imbalanced data. In: Proceedings of ACL, pp. 1351–1360 (2019)
Yu, J., Bohnet, B., Poesio, M.: Named entity recognition as dependency parsing. arXiv preprint arXiv:2005.07150 (2020)
Zheng, C., Cai, Y., Xu, J., Leung, H.f., Xu, G.: A boundary-aware neural model for nested named entity recognition. In: Proceedings of EMNLP, pp. 357–366 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, C., Fan, H., Liu, J. (2021). Span-Based Nested Named Entity Recognition with Pretrained Language Model. In: Jensen, C.S., et al. Database Systems for Advanced Applications. DASFAA 2021. Lecture Notes in Computer Science(), vol 12682. Springer, Cham. https://doi.org/10.1007/978-3-030-73197-7_42
Download citation
DOI: https://doi.org/10.1007/978-3-030-73197-7_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-73196-0
Online ISBN: 978-3-030-73197-7
eBook Packages: Computer ScienceComputer Science (R0)