Skip to main content
SpringerLink
Log in

Performance of representation fusion model for entity and relationship extraction within unstructured text

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Automated identification of entities and their relationships from unstructured text is a critical aspect of information extraction. For this, joint entity and relation extraction is a promising research area aimed at achieving this goal. As a priori knowledge, relation label plays a key role in relation extraction. However, there is a lack of emphasis on incorporating relation labels into the analysis of the unstructured text. This limits the accuracy and effectiveness of entity and relation extraction. Inspired by this motivation, we propose the representation fusion model based on the attention mechanism (RFARE) to help extract relations. We take the vector encoded by the prior relation label as the relation information, which is used to replace the relation embedding matrix with less relation information. To obtain vectors with rich semantic information for relation extraction, we use the attention mechanism to fuse important semantic features between sentences and relation labels selectively. By conducting experiments on two public datasets, NYT and WebNLG, we demonstrate that RFARE effectively leverages relation labels to enrich relation information and improve the semantic interaction between sentences and relationships. Our proposed fusion layer plays an essential role in facilitating this process and improving the overall performance of relation extraction with a simple structure. Specifically, our model achieves F1 score of 92.4% and 92.9% on NYT and WebNLG, respectively, outperforming other baseline models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data Availability

Authors do not have permission to share the datasets used.

References

  1. Chan YS, Roth D (2011) Exploiting syntactico-semantic structures for relation extraction. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, 551–560

  2. Diao C, Zhang D, Liang W et al (2022) A novel spatial-temporal multi-scale alignment graph neural network security model for vehicles prediction. IEEE Trans Intell Transp Syst

  3. Fu T-J, Li P-H, Ma W-Y (2019) Graphrel: Modeling text as relational graphs for joint entity and relation extraction. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 1409–1418

  4. Gardent C, Shimorina A, Narayan S, Perez-Beltrachini L (2017) Creating training corpora for nlg micro-planning. In: 55th Annual Meeting of the Association for Computational Linguistics (acl), 179–188

  5. Geng Z, Zhang Y, Han Y (2021) Joint entity and relation extraction model based on rich semantics. Neurocomputing 429:132–140

    Article  Google Scholar 

  6. Gormley MR, Yu M, Dredze M (2015) Improved relation extraction with feature-rich compositional embedding models. 1774–1784

  7. Gupta P, Schütze H, Andrassy B (2016) Table filling multi-task recurrent neural network for joint entity and relation extraction. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, 2537–2547

  8. Huang W, Mao Y, Yang L, Yang Z, Long J (2021) Local-to-global gcn with knowledge-aware representation for distantly supervised relation extraction. Knowl-Based Syst 234:107565

    Article  Google Scholar 

  9. JIANG Lei L, Qi Z, et al (2022) Review on information extraction techniques for knowledge graph. Comput Syst Appl 007:031

  10. Kenton JDM-WC, Toutanova LK (2019) Bert: Pre-training of deep bidirectional transformers for language understanding. 4171–4186

  11. Konur O, Kingma D, Ba J (2015) Adam: A method for stochastic optimization. 1–15

  12. Lei X, Fan Y, Castiglione A et al (2021) High-precision linearized interpretation for fully connected neural network. Appl Soft Comput 109:107572

    Article  Google Scholar 

  13. Li Q, Ji H (2014) Incremental joint extraction of entity mentions and relations. In: 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014, 402–412

  14. Li X, Li Y, Yang J, Liu H, Hu P (2022) A relation aware embedding mechanism for relation extraction. Appl Intell, 1–10

  15. Li X, Luo X, Dong C, Yang, D, Luan B, He Z (2021) Tdeer: an efficient translating decoding schema for joint extraction of entities and relations. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 8055–8064

  16. Li Y, Liang W, Li P et al (2023) Predicting drug-target interactions via dual-stream graph neural network. IEEE/ACM Trans Comput Biol Bioinf. https://doi.org/10.1109/TCBB.2022.3204188

    Article  Google Scholar 

  17. Liang W, Li Y, Xie K et al (2023) Spatial-temporal aware inductive graph neural network for c-its data recovery. IEEE Trans Intell Transp Syst. https://doi.org/10.1109/TITS.2022.3156266

    Article  Google Scholar 

  18. Long J, Liang W, Marino MD et al (2022) A regularized cross-layer ladder network for intrusion detection in industrial internet of things. IEEE Trans Industr Inf 19(2):1747–1755

    Article  Google Scholar 

  19. Miwa M, Bansal M (2016) End-to-end relation extraction using lstms on sequences and tree structures. 1105–1116

  20. Miwa M, Sasaki Y (2014) Modeling joint entity and relation extraction with table representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (emnlp), 1858–1869

  21. Ren X, Wu Z, He W, Qu M, Voss CR, Ji H, Abdelzaher TF, Han J (2017) Cotype: Joint extraction of typed entities and relations with knowledge bases. 1015–1024

  22. Riedel S, Yao L, McCallum A (2010) Modeling relations and their mentions without labeled text. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 148–163

  23. Shunxiang Z, Aoqiang Z, Guangli Z et al (2023) Building fake review detection model based on sentiment intensity and PU learning. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2023.3234427 (forthcoming)

  24. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 30:6000–6010

    Google Scholar 

  25. Wang J, Xu W, Fu X, Xu G, Wu Y (2020) Astral: adversarial trained lstm-cnn for named entity recognition. Knowl-Based Syst 197:105842

    Article  Google Scholar 

  26. Wang Y, Yu B, Zhang Y, Liu T, Zhu H, Sun L (2020b) Tplinker: Single-stage joint extraction of entities and relations through token pair linking. 1572–1582

  27. Wei Z, Su J, Wang Y, Tian Y, Chang Y (2020) A novel cascade binary tagging framework for relational triple extraction. 1476–1488

  28. Yu B, Zhang Z, Shu X, Liu T, Wang Y, Wang B, Li S (2020) Joint extraction of entities and relations based on a novel decomposition strategy. 2282–2289

  29. Yu, X. and Lam, W. (2010). Jointly identifying entities and extracting relations in encyclopedia text via a graphical model approach. In: Coling 2010: Posters, 1399–1407

  30. Zeng D, Zhang H, Liu Q (2020) Copymtl: Copy mechanism for joint extraction of entities and relations with multi-task learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 34, pp 9507–9514

  31. Zeng X, He S, Zeng D, Liu K, Liu S, Zhao J (2019) Learning the extraction order of multiple relational facts in a sentence with reinforcement learning. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 367–377

  32. Zeng X, Zeng D, He S, Liu K, Zhao J (2018) Extracting relational facts by an end-to-end neural model with copy mechanism. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Volume 1: Long Papers, pp 506–514

  33. Zhang S, Hu Z, Zhu G, Jin M, Li K-C (2021) Sentiment classification model for chinese micro-blog comments based on key sentences extraction. Soft Comput 25:463–476

    Article  Google Scholar 

  34. Zhang S, Xu H, Zhu G et al (2023) A data processing method based on sequence labeling and syntactic analysis for extracting new sentiment words from product reviews. Soft Comput 26:853–866

    Article  Google Scholar 

  35. Zhang S, Zhao T, Wu H et al (2023) Ts-gcn: Aspect-level sentiment classification model for consumer reviews. Comput Sci Inf Syst 20:117–136

    Article  Google Scholar 

  36. Zhang S, Zhu H, Xu H et al (2022) A named entity recognition method towards product reviews based on bilstm-attention-crf. Int J Comput Sci Eng 25(5):479–489

    Google Scholar 

  37. Zhao K, Xu H, Cheng Y, Li X, Gao K (2021) Representation iterative fusion based on heterogeneous graph neural network for joint entity and relation extraction. Knowl-Based Syst 219:106888

    Article  Google Scholar 

  38. Zheng S, Wang F, Bao H, Hao Y, Zhou P, Xu B (2017) Joint extraction of entities and relations based on a novel tagging scheme. 1227–1236

  39. Zheng S, Xu J, Zhou P, Bao H, Qi Z, Xu B (2016) A neural network framework for relation extraction: Learning entity semantic and relation pattern. Knowl-Based Syst 114:12–23

    Article  Google Scholar 

Download references

Funding

Not applicable.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Hunan University of Science and Technology, Hunan, China

    Jing Liao, Xiande Su & Lei Jiang

  2. Department of Computer Science and Information Engineering, Providence University, Taichung, Taiwan

    Kuan-Ching Li & Tien-Hsiung Weng

  3. Department of Computer Science and Engineering, University of Aizu, Fukushima, Japan

    Subhash Bhalla

Authors
  1. Jing Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiande Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lei Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kuan-Ching Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tien-Hsiung Weng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Subhash Bhalla
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

XS: Methodology, writing—original draft preparation, software; JL: conceptualization, methodology, writing—original draft preparation, resources, supervision; LJ: methodology, data curation, resources, software, validation; KCL: methodology, data curation, validation, visualization, supervision, writing—reviewing and editing; THW: methodology, visualization, writing—reviewing and editing; SB: writing—reviewing and editing.

Corresponding author

Correspondence to Tien-Hsiung Weng.

Ethics declarations

Ethical Approval

Not applicable.

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liao, J., Su, X., Jiang, L. et al. Performance of representation fusion model for entity and relationship extraction within unstructured text. J Supercomput 79, 20826–20844 (2023). https://doi.org/10.1007/s11227-023-05442-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-023-05442-6

Keywords

Search

Navigation