Skip to main content
SpringerLink
Log in

Corpus-Based Relation Extraction by Identifying and Refining Relation Patterns

  • Conference paper
  • First Online:
Machine Learning and Knowledge Discovery in Databases: Research Track (ECML PKDD 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14172))

Abstract

Automated relation extraction without extensive human-annotated data is a crucial yet challenging task in text mining. Existing studies typically use lexical patterns to label a small set of high-precision relation triples and then employ distributional methods to enhance detection recall. This precision-first approach works well for common relation types but struggles with unconventional and infrequent ones. In this work, we propose a recall-first approach that first leverages high-recall patterns (e.g., a per:siblings relation normally requires both the head and tail entities in the person type) to provide initial candidate relation triples with weak labels and then clusters these candidate relation triples in a latent spherical space to extract high-quality weak supervisions. Specifically, we present a novel framework, RClus, where each relation triple is represented by its head/tail entity type and the shortest dependency path between the entity mentions. RClus first applies high-recall patterns to narrow down each relation type’s candidate space. Then, it embeds candidate relation triples in a latent space and conducts spherical clustering to further filter out noisy candidates and identify high-quality weakly-labeled triples. Finally, RClus leverages the above-obtained triples to prompt-tune a pre-trained language model and utilizes it for improved extraction coverage. We conduct extensive experiments on three public datasets and demonstrate that RClus outperforms the weakly-supervised baselines by a large margin and achieves generally better performance than fully-supervised methods in low-resource settings.

S. Zhou and S. Ge—Equal contribution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    For simplicity in feature acquisitions, we adopts BERT-Large [10] as the pre-trained language model for all the encoding.

  2. 2.

    For convenience, we use the Stanford CoreNLP toolkit [19].

  3. 3.

    \(S^{d-1}:= \{z\in \mathbb {R}^{d} | \Vert z \Vert = 1 \}\). We assume that \(d \ll \min (\dim (\mathbf {H_h}), \dim (\mathbf {H_r}), \dim (\mathbf {H_t}))\).

  4. 4.

    For this work, we use RoBERTa_Large [48] as the backbone model and maintain the consistency between baselines in experiments.

  5. 5.

    The code for this work is available at https://github.com/KevinSRR/RClus.

  6. 6.

    no_relation for TACREV and ReTACRED.

  7. 7.

    https://github.com/KevinSRR/RClus.

References

  1. Agarap, A.F.: Deep learning using rectified linear units (relu). CoRR abs/1803.08375 (2018). http://arxiv.org/abs/1803.08375

  2. Aina, L., Gulordava, K., Boleda, G.: Putting words in context: LSTM language models and lexical ambiguity. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 3342–3348. Association for Computational Linguistics, Florence, Italy (2019). https://doi.org/10.18653/v1/P19-1324, https://aclanthology.org/P19-1324

  3. Alt, C., Gabryszak, A., Hennig, L.: TACRED revisited: a thorough evaluation of the TACRED relation extraction task. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1558–1569. Association for Computational Linguistics, Online (2020). https://doi.org/10.18653/v1/2020.acl-main.142, https://aclanthology.org/2020.acl-main.142

  4. Banerjee, A., Dhillon, I.S., Ghosh, J., Sra, S.: Clustering on the unit hypersphere using von Mises-Fisher distributions. J. Mach. Learn. Res. 6, 1345–1382 (2005)

    MathSciNet  MATH  Google Scholar 

  5. Batista, D.S., Martins, B., Silva, M.J.: Semi-supervised bootstrapping of relationship extractors with distributional semantics. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 499–504. Association for Computational Linguistics, Lisbon, Portugal (2015). https://doi.org/10.18653/v1/D15-1056, https://aclanthology.org/D15-1056

  6. Brown, T., et al.: Language models are few-shot learners. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, pp. 1877–1901. Curran Associates, Inc. (2020). https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf

  7. Chen, Y.N., Hakkani-Tür, D., Tur, G.: Deriving local relational surface forms from dependency-based entity embeddings for unsupervised spoken language understanding. In: 2014 IEEE Spoken Language Technology Workshop (SLT), pp. 242–247 (2014). https://doi.org/10.1109/SLT.2014.7078581

  8. Curran, J., Murphy, T., Scholz, B.: Minimising semantic drift with mutual exclusion bootstrapping. In: Proceedings of the 10th Conference of the Pacific Association for Computational Linguistics, pp. 172–180 (2008)

    Google Scholar 

  9. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. Royal Stat. Soc. Ser. B (Methodological) 39(1), 1–38 (1977). http://www.jstor.org/stable/2984875

  10. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics, Minneapolis, Minnesota (2019). https://doi.org/10.18653/v1/N19-1423, https://aclanthology.org/N19-1423

  11. Ding, N., et al.: Prompt-learning for fine-grained entity typing. ArXiv abs/2108.10604 (2021)

    Google Scholar 

  12. Küffner, K., Zimmer, R., Fundel, R.: Relex - relation extraction using dependency parse trees. Bioinformatics 23(3), 365–71 (2007)

    Google Scholar 

  13. Han, X., Zhao, W., Ding, N., Liu, Z., Sun, M.: PTR: prompt tuning with rules for text classification. AI Open 3, 182–192 (2022). https://doi.org/10.1016/j.aiopen.2022.11.003, https://www.sciencedirect.com/science/article/pii/S2666651022000183

  14. Hancock, B., Varma, P., Wang, S., Bringmann, M., Liang, P., Ré, C.: Training classifiers with natural language explanations. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1884–1895. Association for Computational Linguistics, Melbourne, Australia (2018). https://doi.org/10.18653/v1/P18-1175, https://aclanthology.org/P18-1175

  15. Hinton, G.E., Zemel, R.S.: Autoencoders, minimum description length and Helmholtz free energy. In: Proceedings of the 6th International Conference on Neural Information Processing Systems, pp. 3–10. NIPS’93, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (1993)

    Google Scholar 

  16. Hu, S., Ding, N., Wang, H., Liu, Z., Li, J.Z., Sun, M.: Knowledgeable prompt-tuning: Incorporating knowledge into prompt verbalizer for text classification. In: Annual Meeting of the Association for Computational Linguistics (2021)

    Google Scholar 

  17. Lu, Y., Bartolo, M., Moore, A., Riedel, S., Stenetorp, P.: Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. CoRR abs/2104.08786 (2021). https://arxiv.org/abs/2104.08786

  18. van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(86), 2579–2605 (2008). http://jmlr.org/papers/v9/vandermaaten08a.html

  19. Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., McClosky, D.: The Stanford CoreNLP natural language processing toolkit. In: Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 55–60. Association for Computational Linguistics, Baltimore, Maryland (2014). https://doi.org/10.3115/v1/P14-5010, https://aclanthology.org/P14-5010

  20. Mausam, Schmitz, M., Soderland, S., Bart, R., Etzioni, O.: Open language learning for information extraction. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 523–534. Association for Computational Linguistics, Jeju Island, Korea (2012). https://aclanthology.org/D12-1048

  21. Meng, Y., et al.: Spherical text embedding. In: Advances in Neural Information Processing Systems (2019)

    Google Scholar 

  22. Meng, Y., Shen, J., Zhang, C., Han, J.: Weakly-supervised neural text classification. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pp. 983–992. CIKM ’18, Association for Computing Machinery, New York, NY, USA (2018). https://doi.org/10.1145/3269206.3271737, https://doi.org/10.1145/3269206.3271737

  23. Meng, Y., Zhang, Y., Huang, J., Zhang, Y., Zhang, C., Han, J.: Hierarchical topic mining via joint spherical tree and text embedding. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1908–1917. KDD ’20, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3394486.3403242, https://doi.org/10.1145/3394486.3403242

  24. Nakashole, N., Weikum, G., Suchanek, F.: PATTY: A taxonomy of relational patterns with semantic types. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 1135–1145. Association for Computational Linguistics, Jeju Island, Korea (2012). https://aclanthology.org/D12-1104

  25. Nayak, T., Majumder, N., Goyal, P., Poria, S.: Deep neural approaches to relation triplets extraction: a comprehensive survey. Cognitive Comput. 13, 1215–1232 (2021)

    Google Scholar 

  26. Qu, M., Ren, X., Zhang, Y., Han, J.: Weakly-supervised relation extraction by pattern-enhanced embedding learning. In: Proceedings of the 2018 World Wide Web Conference, pp. 1257–1266. WWW ’18, International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE (2018). https://doi.org/10.1145/3178876.3186024, https://doi.org/10.1145/3178876.3186024

  27. Ratner, A., Bach, S.H., Ehrenberg, H., Fries, J., Wu, S., Ré, C.: Snorkel: rapid training data creation with weak supervision. Proc. VLDB Endowment 11(3), 269–282 (2017). https://doi.org/10.14778/3157794.3157797

  28. Ratner, A., Sa, C.D., Wu, S., Selsam, D., Ré, C.: Data programming: creating large training sets, quickly. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, pp. 3574–3582. NIPS’16, Curran Associates Inc., Red Hook, NY, USA (2016)

    Google Scholar 

  29. Ren, W., Li, Y., Su, H., Kartchner, D., Mitchell, C., Zhang, C.: Denoising multi-source weak supervision for neural text classification. In: Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 3739–3754. Association for Computational Linguistics, Online (2020). https://doi.org/10.18653/v1/2020.findings-emnlp.334, https://aclanthology.org/2020.findings-emnlp.334

  30. Schick, T., Schmid, H., Schütze, H.: Automatically identifying words that can serve as labels for few-shot text classification. In: Proceedings of the 28th International Conference on Computational Linguistics, pp. 5569–5578. International Committee on Computational Linguistics, Barcelona, Spain (Online) (2020). https://doi.org/10.18653/v1/2020.coling-main.488, https://aclanthology.org/2020.coling-main.488

  31. Shen, J., Zhang, Y., Ji, H., Han, J.: Corpus-based open-domain event type induction. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 5427–5440. Association for Computational Linguistics, Online and Punta Cana, Dominican Republic (2021). https://doi.org/10.18653/v1/2021.emnlp-main.441, https://aclanthology.org/2021.emnlp-main.441

  32. Shin, T., Razeghi, Y., IV, R.L.L., Wallace, E., Singh, S.: Eliciting knowledge from language models using automatically generated prompts. ArXiv abs/2010.15980 (2020)

    Google Scholar 

  33. Shwartz, V., Goldberg, Y., Dagan, I.: Improving hypernymy detection with an integrated path-based and distributional method. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2389–2398. Association for Computational Linguistics, Berlin, Germany (2016). https://doi.org/10.18653/v1/P16-1226, https://aclanthology.org/P16-1226

  34. Simmons, R.F.: Answering English questions by computer: a survey. Commun. ACM 8(1), 53–70 (1965). https://doi.org/10.1145/363707.363732

  35. Socher, R., Karpathy, A., Le, Q.V., Manning, C.D., Ng, A.Y.: Grounded compositional semantics for finding and describing images with sentences. Trans. Assoc. Comput. Linguist. 2, 207–218 (2014). https://doi.org/10.1162/tacl_a_00177, https://aclanthology.org/Q14-1017

  36. Stoica, G., Platanios, E.A., P’oczos, B.: Re-TACRED: addressing shortcomings of the TACRED dataset. In: AAAI Conference on Artificial Intelligence (2021)

    Google Scholar 

  37. Varma, P., Ré, C.: Snuba: Automating weak supervision to label training data. Proc. VLDB Endow. 12(3), 223–236 (2018). https://doi.org/10.14778/3291264.3291268

  38. Wang, C., Kalyanpur, A., Fan, J., Boguraev, B.K., Gondek, D.C.: Relation extraction and scoring in deepqa. IBM J. Res. Dev. 56(3.4), 9:1–9:12 (2012). https://doi.org/10.1147/JRD.2012.2187239

  39. Wang, H., Liu, B., Li, C., Yang, Y., Li, T.: Learning with noisy labels for sentence-level sentiment classification. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 6286–6292. Association for Computational Linguistics, Hong Kong, China (2019). https://doi.org/10.18653/v1/D19-1655, https://aclanthology.org/D19-1655

  40. Wang, H., Tian, F., Gao, B., Zhu, C., Bian, J., Liu, T.Y.: Solving verbal questions in IQ test by knowledge-powered word embedding. In: Conference on Empirical Methods in Natural Language Processing (2015)

    Google Scholar 

  41. Xie, J., Girshick, R., Farhadi, A.: Unsupervised deep embedding for clustering analysis. In: Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48, pp. 478–487. ICML’16, JMLR.org (2016)

    Google Scholar 

  42. Xu, Y., Mou, L., Li, G., Chen, Y., Peng, H., Jin, Z.: Classifying relations via long short term memory networks along shortest dependency paths. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1785–1794. Association for Computational Linguistics, Lisbon, Portugal (2015). https://doi.org/10.18653/v1/D15-1206, https://aclanthology.org/D15-1206

  43. Xue, F., Sun, A., Zhang, H., Chng, E.S.: GDPNet: refining latent multi-view graph for relation extraction. In: AAAI Conference on Artificial Intelligence (2020)

    Google Scholar 

  44. Yu, Y., Zuo, S., Jiang, H., Ren, W., Zhao, T., Zhang, C.: Fine-tuning pre-trained language model with weak supervision: a contrastive-regularized self-training approach. ArXiv abs/2010.07835 (2020)

    Google Scholar 

  45. Zhang, J., et al.: WRENCH: a comprehensive benchmark for weak supervision. In: Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2) (2021). https://openreview.net/forum?id=Q9SKS5k8io

  46. Zhang, Y., Zhong, V., Chen, D., Angeli, G., Manning, C.D.: Position-aware attention and supervised data improve slot filling. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 35–45. Association for Computational Linguistics, Copenhagen, Denmark (2017). https://doi.org/10.18653/v1/D17-1004, https://aclanthology.org/D17-1004

  47. Zhou, W., et al.: NERO: a neural rule grounding framework for label-efficient relation extraction. In: Proceedings of The Web Conference 2020, pp. 2166–2176. WWW ’20, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3366423.3380282

  48. Zhuang, L., Wayne, L., Ya, S., Jun, Z.: A robustly optimized BERT pre-training approach with post-training. In: Proceedings of the 20th Chinese National Conference on Computational Linguistics, pp. 1218–1227. Chinese Information Processing Society of China, Huhhot, China (2021). https://aclanthology.org/2021.ccl-1.108

Download references

Acknowledgements

Research was supported in part by US DARPA KAIROS Program No. FA8750-19-2-1004 and INCAS Program No. HR001121C0165, National Science Foundation IIS-19-56151, IIS-17-41317, and IIS 17-04532, and the Molecule Maker Lab Institute: An AI Research Institutes program supported by NSF under Award No. 2019897, and the Institute for Geospatial Understanding through an Integrative Discovery Environment (I-GUIDE) by NSF under Award No. 2118329. Any opinions, findings, and conclusions or recommendations expressed herein are those of the authors and do not necessarily represent the views, either expressed or implied, of DARPA or the U.S. Government.

Author information

Authors and Affiliations

  1. Shanghai Jiao Tong University, Shanghai, China

    Sizhe Zhou

  2. University of Illinois Urbana-Champaign, Urbana, USA

    Suyu Ge & Jiawei Han

  3. Google Research, New York, USA

    Jiaming Shen

Authors
  1. Sizhe Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Suyu Ge
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiaming Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiawei Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiawei Han .

Editor information

Editors and Affiliations

  1. University of Michigan, Ann Arbor, MI, USA

    Danai Koutra

  2. University of Vienna, Vienna, Austria

    Claudia Plant

  3. Max Planck Institute for Software Systems, Kaiserslautern, Germany

    Manuel Gomez Rodriguez

  4. Politecnico di Torino, Turin, Italy

    Elena Baralis

  5. CENTAI, Turin, Italy

    Francesco Bonchi

Ethics declarations

Ethical Statement

To the best of our knowledge, there is no specific ethical concern for the methodology of RClus. However, since RClus is dependent on external entity typing tools, pre-trained language models and also the given corpus, potential errors or bias should be given appropriate awareness and be taken good care of.

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhou, S., Ge, S., Shen, J., Han, J. (2023). Corpus-Based Relation Extraction by Identifying and Refining Relation Patterns. In: Koutra, D., Plant, C., Gomez Rodriguez, M., Baralis, E., Bonchi, F. (eds) Machine Learning and Knowledge Discovery in Databases: Research Track. ECML PKDD 2023. Lecture Notes in Computer Science(), vol 14172. Springer, Cham. https://doi.org/10.1007/978-3-031-43421-1_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43421-1_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43420-4

  • Online ISBN: 978-3-031-43421-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation