Advertisement

Comparative Investigation of Deep Learning Components for End-to-end Implicit Discourse Relationship Parser

  • Dejian Li
  • Man LanEmail author
  • Yuanbin WuEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11856)

Abstract

The neural components in deep learning framework are crucial for the performance of many natural language processing tasks. So far there is no systematic work to investigate the influence of neural components on the performance of implicit discourse relation recognition. To address it, in this work we compare many different components and build two implicit discourse parsers base on the sequence and structure of sentence respectively. Experimental results show due to different linguistic features, the neural components have different effects in English and Chinese. Besides, our models achieve state-of-the-art performance on CoNLL-2016 English and Chinese datasets.

Keywords

Deep learning Implicit discourse relation classification Word embedding Neural network 

References

  1. 1.
    Biran, O., McKeown, K.: Aggregated word pair features for implicit discourse relation disambiguation (2013)Google Scholar
  2. 2.
    Braud, C., Denis, P.: Combining natural and artificial examples to improve implicit discourse relation identification. In: Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pp. 1694–1705 (2014)Google Scholar
  3. 3.
    Chen, J., Zhang, Q., Liu, P., Qiu, X., Huang, X.: Implicit discourse relation detection via a deep architecture with gated relevance network. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 1726–1735 (2016)Google Scholar
  4. 4.
    Dai, Z., Huang, R.: Improving implicit discourse relation classification by modeling inter-dependencies of discourse units in a paragraph. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 141–151 (2018)Google Scholar
  5. 5.
    Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186 (2019)Google Scholar
  6. 6.
    Ji, Y., Zhang, G., Eisenstein, J.: Closing the gap: domain adaptation from explicit to implicit discourse relations. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 2219–2224 (2015)Google Scholar
  7. 7.
    Li, J.J., Carpuat, M., Nenkova, A.: Assessing the discourse factors that influence the quality of machine translation. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), vol. 2, pp. 283–288 (2014)Google Scholar
  8. 8.
    Liu, Y., Li, S., Zhang, X., Sui, Z.: Implicit discourse relation classification via multi-task neural networks. In: Thirtieth AAAI Conference on Artificial Intelligence (2016)Google Scholar
  9. 9.
    Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., McClosky, D.: The Stanford CoreNLP natural language processing toolkit. In: Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 55–60 (2014)Google Scholar
  10. 10.
    McClosky, D., Charniak, E., Johnson, M.: Bllip North American news text, complete. In: Linguistic Data Consortium, p. 4:3 (2008)Google Scholar
  11. 11.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, S.G.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems (2013)Google Scholar
  12. 12.
    Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (2014)Google Scholar
  13. 13.
    Peters, M.E., et al.: Deep contextualized word representations. In: Proceedings of NAACL (2018)Google Scholar
  14. 14.
    Pitler, E., Nenkova, A.: Using syntax to disambiguate explicit discourse connectives in text. In: Proceedings of the ACL-IJCNLP 2009 Conference Short Papers, pp. 13–16. Association for Computational Linguistics (2009)Google Scholar
  15. 15.
    Qin, L., Zhang, Z., Zhao, H.: Shallow discourse parsing using convolutional neural network. In: Proceedings of the CoNLL-16 Shared Task, pp. 70–77 (2016)Google Scholar
  16. 16.
    Qin, L., Zhang, Z., Zhao, H., Hu, Z., Xing, E.: Adversarial connective-exploiting networks for implicit discourse relation classification. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1006–1017 (2017)Google Scholar
  17. 17.
    Rönnqvist, S., Schenk, N., Chiarcos, C.: A recurrent neural model with attention for the recognition of Chinese implicit discourse relations. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp. 256–262. Association for Computational Linguistics, Vancouver (2017)Google Scholar
  18. 18.
    Rutherford, A., Xue, N.: Discovering implicit discourse relations through brown cluster pair representation and coreference patterns. In: Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pp. 645–654 (2014)Google Scholar
  19. 19.
    Rutherford, A., Xue, N.: Improving the inference of implicit discourse relations via classifying explicit discourse connectives. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 799–808 (2015)Google Scholar
  20. 20.
    Rutherford, A., Xue, N.: Robust non-explicit neural discourse parser in English and Chinese. In: Proceedings of the CoNLL-16 Shared Task, pp. 55–59 (2016)Google Scholar
  21. 21.
    Schenk, N., Chiarcos, C., Donandt, K., Rönnqvist, S., Stepanov, E., Riccardi, G.: Do we really need all those rich linguistic features? A neural network-based approach to implicit sense labeling. In: Proceedings of the CoNLL-16 Shared Task, pp. 41–49 (2016)Google Scholar
  22. 22.
    Tai, K.S., Socher, R., Manning, C.D.: Improved semantic representations from tree-structured long short-term memory networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1556–1566 (2015)Google Scholar
  23. 23.
    Wang, J., Lan, M.: Two end-to-end shallow discourse parsers for English and Chinese in CoNLL-2016 shared task. In: Proceedings of the CoNLL-16 Shared Task, pp. 33–40 (2016)Google Scholar
  24. 24.
    Wang, X., Yoshida, Y., Hirao, T., Sudoh, K., Nagata, M.: Summarization based on task-oriented discourse parsing. IEEE Trans. Audio Speech Lang. Process. 23(8), 1358–1367 (2015)CrossRefGoogle Scholar
  25. 25.
    Wang, Y., Li, S., Yang, J., Sun, X., Wang, H.: Tag-enhanced tree-structured neural networks for implicit discourse relation classification. In: Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 496–505 (2017)Google Scholar
  26. 26.
    Wu, C., Shi, X., Chen, Y., Su, J., Wang, B.: Improving implicit discourse relation recognition with discourse-specific word embeddings. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 269–274 (2017)Google Scholar
  27. 27.
    Xu, Y., Hong, Y., Ruan, H., Yao, J., Zhang, M., Zhou, G.: Using active learning to expand training data for implicit discourse relation recognition. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 725–731 (2018)Google Scholar
  28. 28.
    Yang, B., Cardie, C.: Context-aware learning for sentence-level sentiment analysis with posterior regularization. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 325–335 (2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of Computer ScienceEast China Normal UniversityShanghaiPeople’s Republic of China

Personalised recommendations