Skip to main content
SpringerLink
Log in

Language Adaptation for Entity Relation Classification via Adversarial Neural Networks

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

Entity relation classification aims to classify the semantic relationship between two marked entities in a given sentence, and plays a vital role in various natural language processing applications. However, existing studies focus on exploiting mono-lingual data in English, due to the lack of labeled data in other languages. How to effectively benefit from a richly-labeled language to help a poorly-labeled language is still an open problem. In this paper, we come up with a language adaptation framework for cross-lingual entity relation classification. The basic idea is to employ adversarial neural networks (AdvNN) to transfer feature representations from one language to another. Especially, such a language adaptation framework enables feature imitation via the competition between a sentence encoder and a rival language discriminator to generate effective representations. To verify the effectiveness of AdvNN, we introduce two kinds of adversarial structures, dual-channel AdvNN and single-channel AdvNN. Experimental results on the ACE 2005 multilingual training corpus show that our single-channel AdvNN achieves the best performance on both unsupervised and semi-supervised scenarios, yield- ing an improvement of 6.61% and 2.98% over the state-of-the-art, respectively. Compared with baselines which directly adopt a machine translation module, we find that both dual-channel and single-channel AdvNN significantly improve the performances (F1) of cross-lingual entity relation classification. Moreover, extensive analysis and discussion demonstrate the appropriateness and effectiveness of different parameter settings in our language adaptation framework.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Yu M, Yin W, Hasan K S, dos Santos C, Xiang B, Zhou B. Improved neural relation detection for knowledge base question answering. In Proc. the 55th Annual Meeting of the Association for Computational Linguistics, July 2017, pp.571-581. https://doi.org/10.18653/v1/P17-1053.

  2. Min B, Grishman R. Challenges in the knowledge base population slot filling task. In Proc. the 8th International Conference on Language Resources and Evaluation, May 2012, pp.1137-1142.

  3. Liu Y, Li S, Wei F, Ji H. Relation classification via modeling augmented dependency paths. IEEE/ACM Transactions on Audio, Speech and Language Processing, 2016, 24(9): 1589-1598. https://doi.org/10.1109/TASLP.2016.2573050.

    Article  Google Scholar 

  4. Qian L, Hui H, Hu Y, Zhou G, Zhu Q. Bilingual active learning for relation classification via pseudo parallel corpora. In Proc. the 52nd Annual Meeting of the Association for Computational Linguistics, June 2014, pp.582-592. https://doi.org/10.3115/v1/P14-1055.

  5. Min B, Jiang Z, Freedman M, Weischedel R. Learning transferable representation for bilingual relation extraction via convolutional neural networks. In Proc. the 8th International Joint Conference on Natural Language Processing, November 2017, pp.674-684.

  6. Bousmalis K, Trigeorgis G, Silberman N, Krishnan D, Erhan D. Domain separation networks. In Proc. the 29th Conference on Neural Information Processing Systems, December 2016, pp.343-351.

  7. Park G, Im W. Image-text multi-modal representation learning by adversarial backpropagation. arXiv:1612.08354, 2016. https://arxiv.org/abs/1612.08354, Dec. 2019.

  8. Ganin Y, Ustinova E, Ajakan H, Germain P, Larochelle H, Laviolette F, Marchand M, Lempitsky V. Domain-adversarial training of neural networks. Journal of Machine Learning Research, 2016, 17: Article No. 59. https://doi.org/10.555/2946645.2946704.

  9. Qin L, Zhang Z, Zhao H, Hu Z, Xing E P. Adversarial connective-exploiting networks for implicit discourse relation classification. In Proc. the 55th Annual Meeting of the Association for Computational Linguistics, July 2017, pp.1006-1017. https://doi.org/10.18653/v1/P17-1093.

  10. Zou B, Xu Z, Hong Y, Zhou G. Adversarial feature adaptation for cross-lingual relation classification. In Proc. the 27th International Conference on Computational Linguistics, August 2018, pp.437-448.

  11. Hendrickx I, Kim S N, Kozareva Z et al. SemEval-2010 task 8: Multiway classification of semantic relations between pairs of nominals. In Proc. the 5th International Workshop on Semantic Evaluation, July 2010, pp.33-38.

  12. Gábor K, Buscaldi D, Schumann A K, QasemiZadeh B, Zargayouna H, Charnois T. SemEval-2018 Task 7: Semantic relation extraction and classification in scientific papers. In Proc. the 12th International Workshop on Semantic Evaluation, June 2018, pp.679-688. https://doi.org/10.18653/v1/S18-1111.

  13. Kambhatla N. Combining lexical, syntactic, and semantic features with maximum entropy models for extracting relations. In Proc. the 42nd Annual Meeting on Association for Computational Linguistics on Interactive Poster and Demonstration Sessions, July 2004, Article No. 22. https://doi.org/10.3115/1219044.1219066.

  14. Suchanek F M, Ifrim G, Weikum G. Combining linguistic and statistical analysis to extract relations from web documents. In Proc. the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, August 2006, pp.712-717. https://doi.org/10.1145/1150402.1150492.

  15. Bunescu R C, Mooney R J. Subsequence kernels for relation extraction. In Proc. International Conference on Neural Information Processing Systems, December 2005, pp.171-178. https://doi.org/10.5555/2976248.2976270.

  16. Qian L, Zhou G, Kong F, Zhu Q, Qian P. Exploiting constituent dependencies for tree kernel-based semantic relation extraction. In Proc. the 22nd International Conference on Computational Linguistics, August 2008, pp.697-704. https://doi.org/10.3115/1599081.1599169.

  17. Zeng D, Liu K, Chen Y, Zhao J. Distant supervision for relation extraction via piecewise convolutional neural networks. In Proc. the 2015 Conference on Empirical Methods in Natural Language Processing, September 2015, pp.1753-1762. https://doi.org/10.18653/v1/D15-1203.

  18. dos Santos C N, Xiang B, Zhou B. Classifying relations by ranking with convolutional neural networks. In Proc. the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, July 2015, pp.626-634. https://doi.org/10.3115/v1/P15-1061.

  19. Wang L, Cao Z, de Melo G, Liu Z. Relation classification via multi-level attention CNNs. In Proc. the 54th Annual Meeting of the Association for Computational Linguistics, August 2016, pp.1298-1307. https://doi.org/10.18653/v1/P16-1123.

  20. Kim S, Jeong M, Lee J, Lee G G. Cross-lingual annotation projection for weakly-supervised relation extraction. ACM Transactions on Asian Language Information Processing, 2014, 13(1): Article No. 3. https://doi.org/10.1145/2529994.

  21. Faruqui M, Kumar S. Multilingual open relation extraction using cross-lingual projection. In Proc. the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, May 2015, pp.1351-1356. https://doi.org/10.3115/v1/N15-1151.

  22. Verga P, Belanger D, Strubell E, Roth B, McCallum A. Multilingual relation extraction using compositional universal schema. In Proc. the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 2016, pp.886-896. https://doi.org/10.18653/v1/N16-1103.

  23. Lin Y, Liu Z, Sun M. Neural relation extraction with multilingual attention. In Proc. the 55th Annual Meeting of the Association for Computational Linguistics, July 2017, pp.34-43. https://doi.org/10.18653/v1/P17-1004.

  24. Goodfellow I J, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial nets. In Proc. Annual Conference on Neural Information Processing Systems Conference, December 2014, pp.2672-2680.

  25. Makhzani A, Shlens J, Jaitly N, Goodfellow I, Frey B. Adversarial autoencoders. In Proc. the International Conference on Learning Representations, May 2016.

  26. Fu L, Nguyen T H, Min B, Grishman R. Domain adaptation for relation extraction with domain adversarial neural network. In Proc. the 8th International Joint Conference on Natural Language Processing, November 2017, pp.425-429.

  27. Chen X, Sun Y, Athiwaratkun B, Cardie C, Weinberger K. Adversarial deep averaging networks for cross-lingual sentiment classification. arXiv:1606.01614, 2016. https://arxiv.org/abs/1606.01614, Aug. 2018.

  28. Sato M, Manabe H, Noji H, Matsumoto Y. Adversarial training for cross-domain universal dependency parsing. In Proc. the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, August 2017, pp.71-79. https://doi.org/10.18653/v1/K17-3007.

  29. Huang L, Ji H, May J. Cross-lingual multi-level adversarial transfer to enhance low-resource name tagging. In Proc. the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 2019, pp.3823-3833. https://doi.org/10.18653/v1/N19-1383.

  30. Zeng D, Liu K, Lai S, Zhou G, Zhao J. Relation classification via convolutional deep neural network. In Proc. the 25th International Conference on Computational Linguistics: Technical Papers, August 2014, pp.2335-2344.

  31. Shi T, Liu Z, Liu Y, Sun M. Learning cross-lingual word embeddings via matrix co-factorization. In Proc. the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, July 2015, pp.567-572. https://doi.org/10.3115/v1/P15-2093.

  32. Zeiler M D. ADADELTA: An adaptive learning rate method. arXiv:1212.5701, 2012. https://arxiv.org/abs/12-12.5701, Dec. 2019.

  33. Yin W, Kann K, Yu M, Schütze H. Comparative study of CNN and RNN for natural language processing. arXiv:1702.01923, 2017. https://arxiv.org/abs/1702.01923, Feb. 2020.

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Soochow University, Suzhou, 215006, China

    Bo-Wei Zou, Rong-Tao Huang, Zeng-Zhuang Xu, Yu Hong & Guo-Dong Zhou

  2. Institute for Infocomm Research, Agency for Science, Technology and Research, Singapore, 138635, Singapore

    Bo-Wei Zou

Authors
  1. Bo-Wei Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rong-Tao Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zeng-Zhuang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yu Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guo-Dong Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yu Hong.

Supplementary Information

ESM 1

(PDF 115 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zou, BW., Huang, RT., Xu, ZZ. et al. Language Adaptation for Entity Relation Classification via Adversarial Neural Networks. J. Comput. Sci. Technol. 36, 207–220 (2021). https://doi.org/10.1007/s11390-020-9713-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-020-9713-0

Keywords

Search

Navigation