Skip to main content
SpringerLink
Log in

ML2FNet: A Simple but Effective Multi-level Feature Fusion Network for Document-Level Relation Extraction

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1969))

Included in the following conference series:

  • 413 Accesses

Abstract

Document-level relation extraction presents new challenges compared to its sentence-level counterpart, which aims to extract relations from multiple sentences. Current graph-based and transformer-based models have achieved certain success. However, most approaches focus only on local information, independently on a certain entity, without considering the global interdependency among the relational triples. To solve this problem, this paper proposes a novel relation extraction model with a Multi-Level Feature Fusion Network (ML2FNet). Specifically, we first establish the interaction between entities by constructing an entity-level relation matrix. Then, we employ an enhanced U-shaped network to fuse the multi-level feature of entity pairs from local to global. Finally, the relation classification of entity pairs is performed by a bilinear classifier. We conduct experiments on three public document-level relation extraction datasets, and the results show that ML2FNet outperforms the other baselines. Our code is available at https://github.com/zzhinlp/ML2FNet.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/NVIDIA/apex.

References

  1. Wang, L., Cao, Z., de Melo, G., Liu, Z.: Relation classification via multi-level attention CNNs. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pp. 1298–1307 (2016)

    Google Scholar 

  2. Wei, Z., Su, J., Wang, Y., Tian, Y., Chang, Y.: A novel cascade binary tagging framework for relational triple extraction. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1476–1488 (2020)

    Google Scholar 

  3. Yao, Y., et al.: DocRED: a large-scale document-level relation extraction dataset. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 764–777 (2019)

    Google Scholar 

  4. Gardent, C., Shimorina, A., Narayan, S., Perez-Beltrachini, L.: Creating training corpora for NLG micro-planning. In: 55th Annual Meeting of the Association for Computational Linguistics, pp. 179–188 (2017)

    Google Scholar 

  5. Li, J., et al.: Biocreative v CDR task corpus: a resource for chemical disease relation extraction. Database: J. Biol. Datab. Curat. (2016)

    Google Scholar 

  6. Wu, Y., Luo, R., Leung, H.C.M., Ting, H.F., Lam, T.W.: Renet: a deep learning approach for extracting gene-disease associations from literature. In: Research in Computational Molecular Biology, pp. 272–284 (2019)

    Google Scholar 

  7. Verga, P., Strubell, E., McCallum, A.: Simultaneously self-attending to all mentions for full-abstract biological relation extraction. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 872–884 (2018)

    Google Scholar 

  8. Pantel, P., Pennacchiotti, M.: Espresso: leveraging generic patterns for automatically harvesting semantic relations. In: Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pp. 113–120 (2006)

    Google Scholar 

  9. Mintz, M.D., Bills, S., Snow, R., Jurafsky, D.: Distant supervision for relation extraction without labeled data. In: Annual Meeting of the Association for Computational Linguistics, pp. 1003–1011 (2009)

    Google Scholar 

  10. Zeng, D., Liu, K., Lai, S., Zhou, G., Zhao, J.: Relation classification via convolutional deep neural network. In: International Conference on Computational Linguistics, pp. 2335–2344 (2014)

    Google Scholar 

  11. Zhou, P., et al.: Attention-based bidirectional long short-term memory networks for relation classification. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pp. 207–212 (2016)

    Google Scholar 

  12. Vaswani, A., et al.: Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS 2017), pp. 6000–6010 (2017)

    Google Scholar 

  13. Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45, 2673–2681 (1997)

    Article  Google Scholar 

  14. Quirk, C., Poon, H.: Distant supervision for relation extraction beyond the sentence boundary. In: Conference of the European Chapter of the Association for Computational Linguistics, pp. 1003–1011 (2016)

    Google Scholar 

  15. Christopoulou, F., Miwa, M., Ananiadou, S.: Connecting the dots: document-level neural relation extraction with edge-oriented graphs. In: Conference on Empirical Methods in Natural Language Processing, pp. 4925–4936 (2019)

    Google Scholar 

  16. Nan, G., Guo, Z., Sekulic, I., Lu, W.: Reasoning with latent structure refinement for document-level relation extraction. In: Annual Meeting of the Association for Computational Linguistics, pp. 1546–1557 (2020)

    Google Scholar 

  17. Zeng, S., Xu, R., Chang, B., Li, L.: Double graph based reasoning for document-level relation extraction. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pp. 1630–1640 (2020)

    Google Scholar 

  18. Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. IEEE Trans. Neural Netw. 20, 61–80 (2009)

    Article  Google Scholar 

  19. Xu, B., Wang, Q., Lyu, Y., Zhu, Y., Mao, Z.: Entity structure within and throughout: modeling mention dependencies for document-level relation extraction. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 14149–14157 (2021)

    Google Scholar 

  20. Wang, H., Focke, C., Sylvester, R., Mishra, N., Wang, W.: Fine-tune Bert for DocRED with Two-step Process. arXiv preprint arXiv:1909.11898 (2019)

  21. Tang, H., et al.: HIN: hierarchical inference network for document-level relation extraction. Adv. Knowl. Discov. Data Mining 12084, 197–209 (2020)

    Google Scholar 

  22. Zhou, W., Huang, K., Ma, T., Huang, J.: Document-level relation extraction with adaptive thresholding and localized context pooling. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 14612–14620 (2021)

    Google Scholar 

  23. Peng, H., et al.: Learning from context or names? An empirical study on neural relation extraction. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pp. 3661–3672 (2020)

    Google Scholar 

  24. Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., Liang, J.: Unet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans. Med. Imaging 39(6), 1856–1867 (2020)

    Article  Google Scholar 

  25. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention, pp. 234–241 (2015)

    Google Scholar 

  26. Zheng, H., Fu, J., Zha, Z.J., Luo, J.: Learning deep bilinear transformation for fine-grained image representation. In: Advances in Neural Information Processing Systems, vol. 32, pp. 4279–4288 (2019)

    Google Scholar 

  27. Zhang, N., et al.: Document-level relation extraction as semantic segmentation. In: IJCAI, pp. 3999–4006 (2021)

    Google Scholar 

  28. Wolf, T., et al.: Transformers: state-of-the-art natural language processing. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38–45 (2020)

    Google Scholar 

  29. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 4171–4186 (2019)

    Google Scholar 

  30. Beltagy, I., Lo, K., Cohan, A.: Scibert: pretrained language model for scientific text. In: EMNLP, pp. 3615–3620 (2019)

    Google Scholar 

  31. Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: ICLR (2017)

    Google Scholar 

  32. Goyal, P., et al.: Accurate, large minibatch SGD: training imagenet in 1 hour. arXiv preprint arXiv:1706.02677 (2017)

  33. Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014)

    MathSciNet  Google Scholar 

  34. Guo, Z., Zhang, Y., Lu, W.: Attention guided graph convolutional networks for relation extraction. In: ACL, pp. 241–251 (2019)

    Google Scholar 

  35. Wang, D., Hu, W., Cao, E., Sun, W.: Global-to-local neural networks for document-level relation extraction. In: EMNLP, pp. 3711–3721 (2020)

    Google Scholar 

  36. Xu, W., Chen, K., Zhao, T.: Document-level relation extraction with reconstruction. In: The Thirty-Fifth AAAI Conference on Artificial Intelligence (2021)

    Google Scholar 

  37. Ye, D., Lin, Y., Du, J., Liu, Z., Sun, M., Liu, Z.: Coreferential reasoning learning for language representation. In: EMNLP, pp. 7170–7186 (2020)

    Google Scholar 

  38. Nguyen, D.Q., Verspoor, K.M.: Convolutional neural networks for chemical-disease relation extraction are improved with character-based word embeddings. In: Proceedings of the BioNLP 2018 Workshop, pp. 129–136 (2018)

    Google Scholar 

  39. Zhang, Z., et al.: Document-level relation extraction with dual-tier heterogeneous graph. In: Proceedings of the 28th International Conference on Computational Linguistics, pp. 1630–1641 (2020)

    Google Scholar 

Download references

Acknowledgements

This work was supported by the Anhui Provincial Science Foundation (NO.1908085MF202) and the Independent Scientific Research Program of National University of Defense Science and Technology (NO. ZK18-03-14).

Author information

Authors and Affiliations

  1. College of Electronic Engineering, National University of Defense Technology, Hefei, 230037, China

    Zhi Zhang, Junan Yang & Hui Liu

  2. Anhui Province Key Laboratory of Electronic Restriction, Hefei, 230037, China

    Zhi Zhang, Junan Yang & Hui Liu

Authors
  1. Zhi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Junan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hui Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Junan Yang .

Editor information

Editors and Affiliations

  1. School of Automation, Central South University, Changsha, China

    Biao Luo

  2. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Long Cheng

  3. Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou, China

    Zheng-Guang Wu

  4. School of Automation, Guangdong University of Technology, Guangzhou, China

    Hongyi Li

  5. School of Electrical Engineering and Telecommunications, UNSW Sydney, Sydney, NSW, Australia

    Chaojie Li

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, Z., Yang, J., Liu, H. (2024). ML2FNet: A Simple but Effective Multi-level Feature Fusion Network for Document-Level Relation Extraction. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Communications in Computer and Information Science, vol 1969. Springer, Singapore. https://doi.org/10.1007/978-981-99-8184-7_23

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-8184-7_23

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-8183-0

  • Online ISBN: 978-981-99-8184-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation