Abstract
Open information extraction (Open IE) is a core task of natural language processing (NLP). Even many efforts have been made in this area, and there are still many problems that need to be tackled. Conventional Open IE approaches use a set of handcrafted patterns to extract relational tuples from the corpus. Secondly, many NLP tools are employed in their procedure; therefore, they face error propagation. To address these problems and inspired by the recent success of Generative Adversarial Networks (GANs), we employ an adversarial training architecture and name it Adversarial-OIE. In Adversarial-OIE, the training of the Open IE model is assisted by a discriminator, which is a (Convolutional Neural Network) CNN model. The goal of the discriminator is to differentiate the extraction result generated by the Open IE model from the training data. The goal of the Open IE model is to produce high-quality triples to cheat the discriminator. A policy gradient method is leveraged to co-train the Open IE model and the discriminator. In particular, due to insufficient training, the discriminator usually leads to the instability of GAN training. We use the distant supervision method to generate training data for the Adversarial-OIE model to solve this problem. To demonstrate our approach, an empirical study on two large benchmark dataset shows that our approach significantly outperforms many existing baselines.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Agichtein E, Gravano L (2000) Snowball: extracting relations from large plain-text collections. In: Proceedings of the fifth ACM Conference on Digital Libraries, June 2-7, 2000, San Antonio, TX, USA, ACM, pp 85–94. https://doi.org/10.1145/336597.336644
Angeli Gabor, Premkumar (2015) Leveraging linguistic structure for open domain information extraction. In: Proceedings of the 53rd annual meeting of the association for computational linguistics, the 7th international joint conference on natural language processing (Volume 1: Long Papers), pp 344–354
Bahdanau D (2014) Neural machine translation by jointly learning to align, translate. arXiv:1409.0473
Brin S (1998) Extracting patterns, relations from the world wide web. In: Paolo Atzeni GM Alberto O Mendelzon (ed) The World Wide Web, Databases, International Workshop WebDB’98, Valencia, Spain, March 27–28, 1998, Selected Papers, Springer, Lecture Notes in Computer Science, vol 1590, pp 172–183. https://doi.org/10.1007/10704656_11
Cetto, Matthias, Niklaus (2018) Graphene: semantically-linked propositions in open information extraction. arXiv:1807.11276
Cho K, Merriënboer V (2014) Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv:1406.1078
Christensen, Janara, Mausam (2010) Semantic role labeling for open information extraction. In: Proceedings of the NAACL HLT 2010 first international workshop on formalisms, methodology for learning by reading, association for computational linguistics, USA, FAM-LbR ’10, pp 52–60
Corro D, Luciano, Gemulla, Rainer (2013) Clausie: clause-based open information extraction. In: Proceedings of the 22nd international conference on World Wide Web, pp 355–366
Cui L, Wei F, Zhou M (2018) Neural open information extraction. In: Gurevych I, Miyao Y (eds) Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 2: Short Papers, Association for Computational Linguistics, pp 407–413. https://doi.org/10.18653/v1/P18-2065
Etzioni O, Banko M (2008) Open information extraction from the web. Commun ACM 51(12):68–74
Etzioni, Oren, Fader (2011) Open information extraction: the second generation. In: Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, Volume 1, AAAI Press, IJCAI’11, p 3–10
Fader, Anthony, Soderland (2011) Identifying relations for open information extraction. In: Proceedings of the 2011 conference on empirical methods in natural language processing, pp 1535–1545
Gashteovski, Kiril (2017) Minie: minimizing facts in open information extraction. In: Advances in neural information processing systems, Association for Computational Linguistics
Gong X, Chang S, Jiang Y, Wang Z (2019) Autogan: neural architecture search for generative adversarial networks. In: 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27–November 2, 2019, IEEE, pp 3223–3233. https://doi.org/10.1109/ICCV.2019.00332
Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B (2014) Generative adversarial nets. In: Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger KQ (eds) Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8–13 2014, Montreal, Quebec, Canada, pp 2672–2680. http://papers.nips.cc/paper/5423-generative-adversarial-nets
Goyal A, Lamb A, Zhang Y, Zhang S (2016) Professor forcing: a new algorithm for training recurrent networks. In: Lee DD, Sugiyama M, von Luxburg U (eds) Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5–10, 2016, Barcelona, Spain, pp 4601–4609. http://papers.nips.cc/paper/6099-professor-forcing-a-new-algorithm-for-training-recurrent-networks
Guarasci R, Damiano E, Minutoloo A (2020) Lexicon-grammar based open information extraction from natural language sentences in Italian. Expert Syst Appl 143
Hermann, Moritz K, Kocisky, Tomas (2015) Teaching machines to read, comprehend. In: Advances in neural information processing systems, pp 1693–1701
Hill, Felix, Bordes, Antoine (2015) The goldilocks principle: reading children’s books with explicit memory representations. arXiv:1511.02301
Karras T, Laine S, Aila T (2019) A style-based generator architecture for generative adversarial networks. In: IEEE Conference on Computer Vision, Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, Computer Vision Foundation / IEEE, pp 4401–4410, https://doi.org/10.1109/CVPR.2019.00453. http://openaccess.thecvf.com/content_CVPR_2019/html/Karras_A_Style-Based_Generator_Architecture_for_Generative_Adversarial_Networks_CVPR_2019_paper.html
Li, Chen, Zhang, Zhen, Lee S, Wee, Lee H, Gim (2018) Convolutional sequence to sequence model for human dynamics. In: Proceedings of the IEEE conference on computer vision, pattern recognition, pp 5226–5234
Liu H, Wang W, Wang Y, Liu H, Liu Z, Tang J (2020) Mitigating gender bias for neural dialogue generation with adversarial learning. https://arxiv.org/abs/2009.13028
Arjovsky Martín, B L, Chintala Soumith (2017) Wasserstein GAN. http://arxiv.org/abs/1701.07875
Mausam (2016) Open information extraction systems, downstream applications. In: Proceedings of the twenty-fifth international joint conference on artificial intelligence, pp 4074–4077
Nallapati, Ramesh, Zhou, Bowen (2016) Abstractive text summarization using sequence-to-sequence rnns, beyond. arXiv:1602.06023
Pal, Harinder (2016) Demonyms compound relational nouns in nominal open IE. In: Proceedings of the 5th Workshop on Automated Knowledge Base Construction, Association for Computational Linguistics, San Diego, CA, pp 35–39. https://doi.org/10.18653/v1/W16-1307. https://www.aclweb.org/anthology/W16-1307
Radford A, Metz L, Chintala S (2016) Unsupervised representation learning with deep convolutional generative adversarial networks. In: Yoshua Bengio YL (ed) 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2–4, 2016, Conference Track Proceedings. http://arxiv.org/abs/1511.06434
Rajpurkar, Pranav, Zhang, Jian (2016) Squad: 100,000+ questions for machine comprehension of text. arXiv:1606.05250
Richard S, Sutton RGB (1998) Reinforcement learning: an introduction. IEEE Trans Neural Netw 9(5):1054–1054
Riloff E, Jones R (1999) Learning dictionaries for information extraction by multi-level bootstrapping. In: Jim Hendler DS (ed) Proceedings of the Sixteenth National Conference on Artificial Intelligence, Eleventh Conference on Innovative Applications of Artificial Intelligence, July 18–22, 1999, Orlando, Florida, USA, AAAI Press / The MIT Press, pp 474–479. http://www.aaai.org/Library/AAAI/1999/aaai99-068.php
Rush MA, Chopra, Sumit, Weston, Jason (2015) A neural attention model for abstractive sentence summarization. arXiv:1509.00685
Saha, Swarnadeep, Pal, Harinder, Mausam (2017) Bootstrapping for numerical open IE. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Association for Computational Linguistics, Vancouver, Canada, pp 317–323. https://doi.org/10.18653/v1/P17-2050
Sahnoun S, Elloumi S, Yahia SB (2020) Event detection based on open information extraction and ontology. J Inf Telecommun 4(3):383–403
Santoro, Adam, Raposo, David (2017) A simple neural network module for relational reasoning. In: Advances in neural information processing systems, pp 4967–4976
Schmitz M, Soderland M, Bart S, Etzioni R, Oren (2012) Open language learning for information extraction. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing, Computational Natural Language Learning, Association for Computational Linguistics, Jeju Island, pp 523–534. https://www.aclweb.org/anthology/D12-1048
Schneider, Oberhauser R, Klatt T (2017) Analysing errors of open information extraction systems. arXiv:1707.07499
See, Abigail, Liu, J P, Manning, D C (2017) Get to the point: Summarization with pointer-generator networks.arXiv:1704.04368
Solawetz J, Larson S (2021) LSOIE: a large-scale dataset for supervised open information extraction. In: Merlo P, Tiedemann J, Tsarfaty R (eds) Proceedings of the 16th Conference of the European chapter of the association for computational linguistics: main volume, EACL 2021, Online, April 19–23, 2021, association for computational linguistics, pp 2595–2600. https://www.aclweb.org/anthology/2021.eacl-main.222/
Stanovsky, Gabriel, Ficler (2016) Getting more out of syntax with props. arXiv:1603.01648
Stanovsky G, Dagan I (2016) Creating a large benchmark for open information extraction. In: Su J, Carreras X, Duh K (eds) Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1–4, 2016, The Association for Computational Linguistics, pp 2300–2305. https://doi.org/10.18653/v1/d16-1252
Sutskever, Ilya, Vinyals, Oriol, Le, VQ (2014) Sequence to sequence learning with neural networks. In: Advances in neural information processing systems, pp 3104–3112
Tian Y, Wang Q, Huang Z, Li W (2020) Off-policy reinforcement learning for efficient, effective GAN architecture search. https://arxiv.org/abs/2007.09180
Vaswani, Ashish, Shazeer (2017a) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008
Vaswani A, Shazeer N, Parmar N (2017b) Attention is all you need. In: Guyon I, von Luxburg U, Bengio S (eds) Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, 4–9 December 2017, Long Beach, CA, USA, pp 5998–6008. http://papers.nips.cc/paper/7181-attention-is-all-you-need
Weston, Jason, Bordes (2015) Towards ai-complete question answering: a set of prerequisite toy tasks. arXiv:1502.05698
Williams RJ (1992) Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach Learn 8:229–256
Wingfield, Arthur, Stine-Morrow, AL E (2000) Language, speech. Commun ACM
Wiseman S, Rush AM (2016) Sequence-to-sequence learning as beam-search optimization. In: Jian Su KD Xavier Carreras (ed) Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1–4, 2016, The Association for Computational Linguistics, pp 1296–1306. https://doi.org/10.18653/v1/d16-1137
Wu, Fei, Weld, S D (2010) Open information extraction using wikipedia. In: Proceedings of the 48th annual meeting of the association for computational linguistics, pp 118–127
Wu, Yonghui, Schuster (2016) Google’s neural machine translation system: bridging the gap between human, machine translation. arXiv:1609.08144
Xiong, Caiminm, Merity (2016) Dynamic memory networks for visual, textual question answering. In: International conference on machine learning, pp 2397–2406
Yang Z, Chen W, Wang F, Xu B (2018) Generative adversarial training for neural machine translation. Neurocomputing 321:146–155
Yates, Banko A, Michele (2007) Textrunner: open information extraction on the web. In: Proceedings of human language technologies: the annual conference of the North American chapter of the association for computational linguistics (NAACL-HLT), pp 25–26
Yu B, Zhang Z, Sheng J, Liu T, Wang Y, Wang Y, Wang B (2021) Semi-open information extraction. In: Leskovec J, Grobelnik M, Najork M, Tang J, Zia L (eds) WWW ’21: the Web Conference 2021, Virtual Event / Ljubljana, Slovenia, April 19–23, 2021, ACM / IW3C2, pp 1661–1672. https://doi.org/10.1145/3442381.3450029
Yu L, Zhang W, Wang J, Yu Y (2017) Seqgan: Sequence generative adversarial nets with policy gradient. In: Satinder P Singh SM (ed) Proceedings of the thirty-first AAAI conference on artificial intelligence, February 4–9, 2017, San Francisco, California, USA, AAAI Press, pp 2852–2858. http://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14344
Zhan J, Zhao H (2020) Span model for open information extraction on accurate corpus. In: The thirty-fourth AAAI conference on artificial intelligence, AAAI 2020, the thirty-second innovative applications of artificial intelligence conference, IAAI 2020, the tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7–12, 2020, AAAI Press, pp 9523–9530. https://aaai.org/ojs/index.php/AAAI/article/view/6497
Zhang Yong, Fang Yang, Weidong Xiao (2017) Deep keyphrase generation with a convolutional sequence to sequence model. In: 2017 4th International Conference on Systems. Informatics (ICSAI), IEEE, pp 1477–1485
Acknowledgements
This paper was partially supported by NSFC grant U1866602, 61772157.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that we have no conflict of interest.
Rights and permissions
About this article
Cite this article
Han, J., Wang, H. Generative adversarial networks for open information extraction. Adv. in Comp. Int. 1, 6 (2021). https://doi.org/10.1007/s43674-021-00006-8
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s43674-021-00006-8