Skip to main content
SpringerLink
Log in

A Novel, Cognitively Inspired, Unified Graph-based Multi-Task Framework for Information Extraction

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Information extraction (IE) transforms unstructured textual sources into structured knowledge, closely resembling human reasoning. IE involves several subtasks, such as named entity recognition (NER), relation extraction (RE), and coreference resolution (CR). The early IE models tend to treat each subtask as a separate task or to apply a sequential pipeline approach, which can lead to cascading errors and obfuscation of the inherent relationship between tasks. Recent research has shown that it is advantageous to incorporate the interdependence of subtasks and optimize performance through joint learning. However, they do not properly model the interaction between tasks, either by modeling the subtasks sequentially or by using shared input data. Inspired by human reasoning, a graph-based multitask IE framework is presented that facilitates the interaction between several IE tasks capable of capturing both local and global information. Graphs were constructed by selecting the most confident entity spans and coupling them with a confidence-weighted relation type and a confidence-weighted coreference. Additionally, in the study, a dynamic span graph approach was employed, where span updates were propagated across both the coreference and the relation graph. This allowed useful information to be learned from a broader context by enhancing interaction across different IE tasks. The input data were globally shared, and the interaction between subtasks was fully exploited, avoiding cascading errors. Experiments demonstrate that the proposed multitask IE framework outperforms the state-of-the-art in multiple information extraction tasks spanning a variety of datasets. The relative error reductions range from 0.19 to 3.74%. This paper presents the feasibility of a cognitively-inspired unified graph-based information extraction framework, which is shown to achieve state-of-the-art results on multiple IE tasks across various domains. The framework’s ability to enhance interaction across tasks allows it to learn valuable information from a broader context.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Data Availability

Data is available through the corresponding authors on reasonable request.

References

  1. Sarawagi Sunita. Information extraction. Foundations and trends in databases. 2008;1(3):261–377.

    Article  MATH  Google Scholar 

  2. Finkel RF. Trond Grenager, and Christopher Manning. Incorporating non-local information into information extraction systems by Gibbs sampling. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 363–370, Ann Arbor, Michigan, June 2005. Association for Computational Linguistics 2005.

  3. Yang B, Mitchell, TM. Joint extraction of events and entities within a document context. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 289–299, San Diego, California, June 2016. Association for Computational Linguistics 2016.

  4. Miwa M, Sasaki Y. Modeling joint entity and relation extraction with table representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1858–1869, Doha, Qatar, October 2014. Association for Computational Linguistics 2014.

  5. Kirschnick J, Hemsen H, Markl V. JEDI: Joint entity and relation detection using type inference. In Proceedings of ACL-2016 System Demonstrations, pages 61–66, Berlin, Germany, August 2016. Association for Computational Linguistics. 2016.

  6. Zhang M, Zhang Y, Fu G. End-to-end neural relation extraction with global optimization. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1730–1740, Copenhagen, Denmark, September 2017. Association for Computational Linguistics 2017.

  7. Zeng X, Zeng D, He S, Liu K, Zhao J. Extracting relational facts by an end-to-end neural model with copy mechanism. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 506–514, Melbourne, Australia, July 2018. Association for Computational Linguistics 2018.

  8. Wei Z, Su J,  Wang Y, Tian Y, Chang Y. A novel cascade binary tagging framework for relational triple extraction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1476–1488, 2020.

  9. Fu TJ, Li PH, Ma WY. GraphRel: Modeling text as relational graphs for joint entity and relation extraction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1409–1418, Florence, Italy, July 2019. Association for Computational Linguistics 2019.

  10. Collobert R, Weston J. A unified architecture for natural language processing: deep neural networks with multitask learning. In ICML ’08, 2008.

  11. Shinyama Y  Sekine S. Named entity discovery using comparable news articles. In COLING, 2004.

  12. Grishman R. Information extraction: techniques and challenges. In Maria Teresa Pazienza, editor, Information Extraction A Multidisciplinary Approach to an Emerging Information Technology, pages 10–27, Berlin, Heidelberg, 1997. Springer Berlin Heidelberg. 1997.

  13. Adnan K, Akbar R. An analytical study of information extraction from unstructured and multidimensional big data. J Big Data 2019;6(1).

  14. Vincent Ng. Supervised noun phrase coreference research: the first fifteen years. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1396–1411, Uppsala, Sweden, July 2010. Association for Computational Linguistics.

  15. Anders Søgaard and Yoav Goldberg. Deep multi-task learning with low level tasks supervised at lower layers. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 231–235, Berlin, Germany, August 2016. Association for Computational Linguistics.

  16. Qi Li, Heng Ji. Incremental joint extraction of entity mentions and relations. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 402–412, Baltimore, Maryland, June 2014. Association for Computational Linguistics 2014.

  17. Singh S, Riedel S, Martin B, Zheng J, McCallum A. Joint inference of entities, relations, and coreference. In Proceedings of the 2013 Workshop on Automated Knowledge Base Construction, AKBC ’13, page 1-6, New York, NY, USA, 2013. Association for Computing Machinery 2013.

  18. Zhong Z, Chen D. A frustratingly easy approach for entity and relation extraction. In North American Association for Computational Linguistics (NAACL), 2021.

  19. Wang Y, Sun C, Wu Y, Yan J, Gao P, Xie G. Pre-training entity relation encoder with intra-span and inter-span information. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1692–1705, Online, November 2020. Association for Computational Linguistics 2020.

  20. Shen Y, Ma X, Tang Y, Lu W. A trigger-sense memory flow framework for joint entity and relation extraction. In Proceedings of the Web Conference 2021, WWW ’21, page 1704-1715, New York, NY, USA, 2021. Assoc Comput Machine 2021.

  21. Yang B, Tau Yih W, He X, Gao J, Deng L. Embedding entities and relations for learning and inference in knowledge bases. CoRR abs/1412.6575, 2015.

  22. Huang L, Wu L, Wang L. Knowledge graph-augmented abstractive summarization with semantic-driven cloze reward. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5094–5107, Online, July 2020. Association for Computational Linguistics 2020.

  23. Fernandes P, Allamanis M, Brockschmidt M. Structured neural summarization. CoRR, abs/1811.01824, 2018.

  24. Hogan A, Blomqvist E, Cochez M, D’amato C, De Melo G, Gutierrez C, Kirrane S, Gayo JEL, Roberto Navigli, Sebastian Neumaier, Axel-Cyrille Ngonga Ngomo, Axel Polleres, Sabbir M. Rashid, Anisa Rula, Lukas Schmelzeisen, Juan Sequeda, Steffen Staab, and Antoine Zimmermann. Knowledge graphs. ACM Comput. Surv., 54(4), jul 2021.

  25. Christopoulou F, Miwa M, Ananiadou S. A walk-based model on entity graphs for relation extraction. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 81–88, 2018.

  26. Song L, Zhang Y, Wang Z, Gildea D. N-ary relation extraction using graph state lstm. In Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2018.

  27. Wu F, Souza A, Zhang T, Fifty C, Yu T, Weinberger K. Simplifying graph convolutional networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 6861–6871. PMLR, 09–15 Jun 2019.

  28. Zhu Y, Xu Y, Yu F, Liu Q, Wu S, Wang L. Graph contrastive learning with adaptive augmentation. In Proceedings of the Web Conference 2021, WWW ’21, page 2069-2080, New York, NY, USA, Assoc Comput Machine 2021.

  29. Devlin J, Chang MW, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Assoc Comput Linguist 2019.

  30. Lee K, He L, Lewis M, Zettlemoyer L. End-to-end neural coreference resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188–197, Copenhagen, Denmark, September. Association for Computational Linguistics 2017

  31. Doddington G, Mitchell A, Przybocki M, Ramshaw L, Strassel S, Weischedel R. The automatic content extraction (ACE) program – tasks, data, and evaluation. In Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04), Lisbon, Portugal. European Language Resources Association (ELRA) 2004.

  32. Luan Y, He L, Ostendorf M, Hajishirzi H. Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3219–3232, Brussels, Belgium, October-November 2018 Assoc Comput Linguist 2018.

  33. Kim J.-D, Ohta T, Tateisi Y, Tsujii J. Genia corpus - a semantically annotated corpus for bio-textmining. Bioinformatics, 19(suppl_1):i180–i182, 07 2003.

  34. Kulkarni C, Xu W, Ritter A, Machiraju R. An annotated corpus for machine reading of instructions in wet lab protocols. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 97–106, New Orleans, Louisiana, June 2018. Assoc Comput Linguist 2018.

  35. Yuan Y, Zhou X, Pan S, Zhu Q, Song Z, Guo L. A relation-specific attention network for joint entity and relation extraction. In IJCAI, 2020.

  36. Jiang Z, Xu W, Araki J, Neubig G. Generalizing natural language analysis through span-relation representations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2120–2133, Online, July 2020. Association for Computational Linguistics 2020.

  37. Luo Y Zhao H. Bipartite flat-graph network for nested named entity recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6408–6418, Online, July 2020. Association for Computational Linguistics 2020.

  38. Beltagy I, Lo K, Cohan A. Scibert: A pretrained language model for scientific text. In EMNLP. Scibert: A pretrained language model for scientific text. In EMNLP; 2019.

    Google Scholar 

  39. Miwa M, Bansal M. End-to-end relation extraction using LSTMs on sequences and tree structures. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1105–1116, Berlin, Germany, August 2016. Association for Computational Linguistics 2016.

  40. Li X, Yin F, Sun Z, Li X, Yuan A, Chai D, Zhou M, Li J. Entity-relation extraction as multi-turn question answering. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1340–1350, Florence, Italy, July 2019. Association for Computational Linguistics 2019.

  41. Zhao T, Yan Z, Cao Y, Li Z. Asking effective and diverse questions: a machine reading comprehension based framework for joint entity-relation extraction. In IJCAI, 2020.

  42. Sanh V, Wolf T, Ruder S. A hierarchical multi-task approach for learning embeddings from semantic tasks. ArXiv abs/1811.06031, 2019.

  43. Shen Y, Ma X, Tang Y,  Lu W. A trigger-sense memory flow framework for joint entity and relation extraction. Proceedings of the Web Conference 2021.

  44. Zhong Z, Chen D. A frustratingly easy approach for entity and relation extraction. In NAACL, 2021.

  45. Wang Y, Sun C, Wu Y, Yan J, Gao P, Xie GT. Pre-training entity relation encoder with intra-span and inter-span information. In EMNLP, 2020.

  46. Yan Z, Zhang C, Fu J, Zhang Q, Wei Z. A partition filter network for joint entity and relation extraction. In EMNLP, 2021.

  47. Jiang Z, Xu W, Araki J, Neubig G. Generalizing natural language analysis through span-relation representations. In ACL, 2020.

  48. Wang B, Lu W. Neural segmental hypergraphs for overlapping mention recognition. In EMNLP, 2018.

  49. Shibuya T, Hovy EH. Nested named entity recognition via second-best sequence learning and decoding. Transactions of the Association for Computational Linguistics. 2020;8:605–20.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Nanyang Technological University, Singapore, Singapore

    Yandan Zheng & Luu Anh Tuan

  2. IGP-HealthTech, Nanyang Technological University, Singapore, Singapore

    Yandan Zheng

Authors
  1. Yandan Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luu Anh Tuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yandan Zheng.

Ethics declarations

Ethics Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zheng, Y., Tuan, L.A. A Novel, Cognitively Inspired, Unified Graph-based Multi-Task Framework for Information Extraction. Cogn Comput 15, 2004–2013 (2023). https://doi.org/10.1007/s12559-023-10163-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-023-10163-2

Keywords

Search

Navigation