Skip to main content
SpringerLink
Log in

Enhancing Ontology Knowledge for Domain-Specific Joint Entity and Relation Extraction

  • Conference paper
  • First Online:
Chinese Computational Linguistics (CCL 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14232))

Included in the following conference series:

  • 319 Accesses

Abstract

Pre-trained language models (PLMs) have been widely used in entity and relation extraction methods in recent years. However, due to the semantic gap between general-domain text used for pre-training and domain-specific text, these methods encounter semantic redundancy and domain semantics insufficiency when it comes to domain-specific tasks. To mitigate this issue, we propose a low-cost and effective knowledge-enhanced method to facilitate domain-specific semantics modeling in joint entity and relation extraction. Precisely, we use ontology and entity type descriptions as domain knowledge sources, which are encoded and incorporated into the downstream entity and relation extraction model to improve its understanding of domain-specific information. We construct a dataset called SSUIE-RE for Chinese entity and relation extraction in space science and utilization domain of China Manned Space Engineering, which contains a wealth of domain-specific knowledge. The experimental results on SSUIE-RE demonstrate the effectiveness of our method, achieving a 1.4% absolute improvement in relation F1 score over previous best approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.wikidata.org.

  2. 2.

    http://www.cmse.gov.cn.

  3. 3.

    https://brat.nlplab.org/.

References

  1. Alsentzer, E., et al.: Publicly available clinical BERT embeddings. In: Proceedings of the 2nd Clinical Natural Language Processing Workshop, Minneapolis, Minnesota, USA, June 2019, pp. 72–78. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/W19-1909. https://aclanthology.org/W19-1909

  2. Araci, D.: FinBERT: financial sentiment analysis with pre-trained language models. arXiv preprint arXiv:1908.10063 (2019)

  3. Aronson, A.R., Lang, F.M.: An overview of MetaMap: historical perspective and recent advances. J. Am. Med. Inform. Assoc. 17(3), 229–236 (2010)

    Article  Google Scholar 

  4. Bodenreider, O.: The unified medical language system (UMLS): integrating biomedical terminology. Nucleic Acids Res. 32(suppl_1), D267–D270 (2004)

    Google Scholar 

  5. Chalkidis, I., Fergadiotis, M., Malakasiotis, P., Aletras, N., Androutsopoulos, I.: LEGAL-BERT: the Muppets straight out of law school. In: Findings of the Association for Computational Linguistics: EMNLP 2020, Online, November 2020, pp. 2898–2904. Association for Computational Linguistics (2020). https://doi.org/10.18653/v1/2020.findings-emnlp.261. https://aclanthology.org/2020.findings-emnlp.261

  6. Chan, Y.S., Roth, D.: Exploiting syntactico-semantic structures for relation extraction. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, Oregon, USA, June 2011, pp. 551–560. Association for Computational Linguistics (2011). https://aclanthology.org/P11-1056

  7. Cui, Y., Che, W., Liu, T., Qin, B., Yang, Z.: Pre-training with whole word masking for Chinese BERT. IEEE Trans. Audio Speech Lang. Process. (2021). https://doi.org/10.1109/TASLP.2021.3124365. https://ieeexplore.ieee.org/document/9599397

  8. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, June 2019, pp. 4171–4186. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/N19-1423. https://aclanthology.org/N19-1423

  9. Gormley, M.R., Yu, M., Dredze, M.: Improved relation extraction with feature-rich compositional embedding models. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal, September 2015, pp. 1774–1784 (2015). Association for Computational Linguistics (2015). https://doi.org/10.18653/v1/D15-1205. https://aclanthology.org/D15-1205

  10. Gu, Y., et al.: Domain-specific language model pretraining for biomedical natural language processing. ACM Trans. Comput. Healthc. (HEALTH) 3(1), 1–23 (2021)

    Google Scholar 

  11. Lai, T., Ji, H., Zhai, C., Tran, Q.H.: Joint biomedical entity and relation extraction with knowledge-enhanced collective inference. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Online, August 2021, pp. 6248–6260. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.acl-long.488. https://aclanthology.org/2021.acl-long.488

  12. Lee, J., et al.: BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36(4), 1234–1240 (2020)

    Article  MathSciNet  Google Scholar 

  13. Li, Q., Ji, H.: Incremental joint extraction of entity mentions and relations. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Baltimore, Maryland, June 2014, pp. 402–412. Association for Computational Linguistics (2014). https://doi.org/10.3115/v1/P14-1038. https://aclanthology.org/P14-1038

  14. Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: Twenty-Ninth AAAI Conference on Artificial Intelligence (2015)

    Google Scholar 

  15. Miwa, M., Sasaki, Y.: Modeling joint entity and relation extraction with table representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, October 2014, pp. 1858–1869. Association for Computational Linguistics (2014). https://doi.org/10.3115/v1/D14-1200. https://aclanthology.org/D14-1200

  16. Nayak, T., Ng, H.T.: Effective modeling of encoder-decoder architecture for joint entity and relation extraction. In: Proceedings of AAAI (2020)

    Google Scholar 

  17. Peng, Y., Yan, S., Lu, Z.: Transfer learning in biomedical natural language processing: an evaluation of BERT and ELMo on ten benchmarking datasets. In: Proceedings of the 18th BioNLP Workshop and Shared Task, Florence, Italy, August 2019, pp. 58–65. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/W19-5006. https://aclanthology.org/W19-5006

  18. Radford, A., Narasimhan, K., Salimans, T., Sutskever, I., et al.: Improving language understanding by generative pre-training (2018)

    Google Scholar 

  19. Roy, A., Pan, S.: Incorporating medical knowledge in BERT for clinical relation extraction. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican Republic, November 2021, pp. 5357–5366. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.emnlp-main.435. https://aclanthology.org/2021.emnlp-main.435

  20. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  21. Wang, J., Lu, W.: Two are better than one: joint entity and relation extraction with table-sequence encoders. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online, November 2020, pp. 1706–1721. Association for Computational Linguistics (2020). https://doi.org/10.18653/v1/2020.emnlp-main.133. https://aclanthology.org/2020.emnlp-main.133

  22. Wang, S., Zhang, Y., Che, W., Liu, T.: Joint extraction of entities and relations based on a novel graph scheme. In: IJCAI, Yokohama, pp. 4461–4467 (2018)

    Google Scholar 

  23. Wang, Y., Yu, B., Zhang, Y., Liu, T., Zhu, H., Sun, L.: TPLinker: single-stage joint extraction of entities and relations through token pair linking. In: Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, Spain (Online), December 2020, pp. 1572–1582. International Committee on Computational Linguistics (2020). https://doi.org/10.18653/v1/2020.coling-main.138. https://aclanthology.org/2020.coling-main.138

  24. Wei, Z., Su, J., Wang, Y., Tian, Y., Chang, Y.: A novel cascade binary tagging framework for relational triple extraction. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, July 2020, pp. 1476–1488. Association for Computational Linguistics (2020). https://doi.org/10.18653/v1/2020.acl-main.136. https://aclanthology.org/2020.acl-main.136

  25. Xiong, X., Yunfei, L., Anqi, L., Shuai, G., Shengyang, L.: A multi-gate encoder for joint entity and relation extraction. In: Proceedings of the 21st Chinese National Conference on Computational Linguistics, Nanchang, China, October 2022, pp. 848–860. Chinese Information Processing Society of China (2022). https://aclanthology.org/2022.ccl-1.75

  26. Yan, Z., Zhang, C., Fu, J., Zhang, Q., Wei, Z.: A partition filter network for joint entity and relation extraction. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican Republic, November 2021, pp. 185–197. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.emnlp-main.17. https://aclanthology.org/2021.emnlp-main.17

  27. Yang, S., Zhang, Y., Niu, G., Zhao, Q., Pu, S.: Entity concept-enhanced few-shot relation extraction. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), Online, August 2021, pp. 987–991. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.acl-short.124. https://aclanthology.org/2021.acl-short.124

  28. Ye, D., Lin, Y., Li, P., Sun, M.: Packed levitated marker for entity and relation extraction. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Dublin, Ireland, May 2022, pp. 4904–4917. Association for Computational Linguistics (2022). https://doi.org/10.18653/v1/2022.acl-long.337. https://aclanthology.org/2022.acl-long.337

  29. Yu, B., et al.: Joint extraction of entities and relations based on a novel decomposition strategy. In: Proceedings of ECAI (2020)

    Google Scholar 

  30. Yu, X., Lam, W.: Jointly identifying entities and extracting relations in encyclopedia text via a graphical model approach. In: Coling 2010: Posters, Beijing, China, August 2010, pp. 1399–1407. Coling 2010 Organizing Committee (2010). https://aclanthology.org/C10-2160

  31. Zeng, X., Zeng, D., He, S., Liu, K., Zhao, J.: Extracting relational facts by an end-to-end neural model with copy mechanism. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Melbourne, Australia, July 2018, pp. 506–514. Association for Computational Linguistics (2018). https://doi.org/10.18653/v1/P18-1047. https://aclanthology.org/P18-1047

  32. Zhang, S., Ng, P., Wang, Z., Xiang, B.: REKnow: enhanced knowledge for joint entity and relation extraction. In: NAACL 2022 Workshop on SUKI (2022). https://www.amazon.science/publications/reknow-enhanced-knowledge-for-joint-entity-and-relation-extraction

  33. Zheng, S., Wang, F., Bao, H., Hao, Y., Zhou, P., Xu, B.: Joint extraction of entities and relations based on a novel tagging scheme. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, Canada, July 2017, pp. 1227–1236. Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/P17-1113. https://aclanthology.org/P17-1113

  34. Zhong, Z., Chen, D.: A frustratingly easy approach for entity and relation extraction. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online, June 2021, pp. 50–61. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.naacl-main.5. https://aclanthology.org/2021.naacl-main.5

  35. Zhou, G., Su, J., Zhang, J., Zhang, M.: Exploring various knowledge in relation extraction. In: Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, Ann Arbor, Michigan, June 2005, ACL 2005, pp. 427–434. Association for Computational Linguistics (2005). https://doi.org/10.3115/1219840.1219893. https://aclanthology.org/P05-1053

Download references

Acknowledgements

This work was supported by the National Defense Science and Technology Key Laboratory Fund Project of the Chinese Academy of Sciences: Space Science and Application of Big Data Knowledge Graph Construction and Intelligent Application Research and Manned Space Engineering Project: Research on Technology and Method of Engineering Big Data Knowledge Mining.

Author information

Authors and Affiliations

  1. Key Laboratory of Space Utilization, Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences, Beijing, China

    Xiong Xiong, Chen Wang, Yunfei Liu & Shengyang Li

  2. University of Chinese Academy of Sciences, Beijing, China

    Xiong Xiong, Chen Wang, Yunfei Liu & Shengyang Li

Authors
  1. Xiong Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yunfei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shengyang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shengyang Li .

Editor information

Editors and Affiliations

  1. Department of Computer Science and Technology, Tsinghua University, Beijing, China

    Maosong Sun

  2. Harbin Institute of Technology, Harbin, China

    Bing Qin

  3. Fudan University, Shanghai, China

    Xipeng Qiu

  4. School of Computing and Information, Singapore Management University, Singapore, Singapore

    Jiang Jing

  5. Institute of Software, Chinese Academy of Sciences, Beijing, China

    Xianpei Han

  6. Beijing Language and Culture University, Beijing, China

    Gaoqi Rao

  7. Chinese Academy of Sciences, Institute of Automation, Beijing, China

    Yubo Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xiong, X., Wang, C., Liu, Y., Li, S. (2023). Enhancing Ontology Knowledge for Domain-Specific Joint Entity and Relation Extraction. In: Sun, M., et al. Chinese Computational Linguistics. CCL 2023. Lecture Notes in Computer Science(), vol 14232. Springer, Singapore. https://doi.org/10.1007/978-981-99-6207-5_15

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-6207-5_15

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-6206-8

  • Online ISBN: 978-981-99-6207-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation