Advertisement

RST Discourse Parser for Russian: An Experimental Study of Deep Learning Models

Conference paper
  • 175 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12602)

Abstract

This work presents the first fully-fledged discourse parser for Russian based on the Rhetorical Structure Theory of Mann and Thompson (1988). For the segmentation, discourse tree construction, and discourse relation classification we employ deep learning models. With the help of multiple word embedding techniques, the new state of the art for discourse segmentation of Russian texts is achieved. We found that the neural classifiers using contextual word representations outperform previously proposed feature-based models for discourse relation classification. By ensembling both methods, we are able to further improve the performance of the discourse relation classification achieving the new state of the art for Russian.

Keywords

Rhetorical structure theory Discourse parsing Deep learning Pre-trained language models 

Notes

Acknowledgements

This work was partially supported by the Ministry of Science and Higher Education of the Russian Federation, project No. 075-15-2020-799.

References

  1. 1.
    Atutxa, A., Bengoetxea, K., de Ilarraza, A.D., Iruskieta, M.: Towards a top-down approach for an automatic discourse analysis for Basque: segmentation and central unit detection tool. PLOS ONE 14(9), 1–25 (2019)CrossRefGoogle Scholar
  2. 2.
    Bhatia, P., Ji, Y., Eisenstein, J.: Better document-level sentiment analysis from RST discourse parsing. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 2212–2218. Association for Computational Linguistics (2015) Google Scholar
  3. 3.
    Braud, C., Coavoux, M., Søgaard, A.: Cross-lingual RST discourse parsing. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pp. 292–304 (2017)Google Scholar
  4. 4.
    Carlson, L., Marcu, D., Okurowski, M.E.: Building a discourse-tagged corpus in the framework of rhetorical structure theory. In: van Kuppevelt, J., Smith, R.W. (eds.) Current and New Directions in Discourse and Dialogue. Text, Speech and Language Technology, vol. 22, pp. 85–112. Springer, Dordrecht (2003).  https://doi.org/10.1007/978-94-010-0019-2_5CrossRefGoogle Scholar
  5. 5.
    Chistova, E., Kobozeva, M., Pisarevskaya, D., Shelmanov, A., Smirnov, I., Toldova, S.: Towards the data-driven system for rhetorical parsing of Russian texts. In: Proceedings of the Workshop on Discourse Relation Parsing and Treebanking 2019, pp. 82–87. Association for Computational Linguistics (2019)Google Scholar
  6. 6.
    Chistova, E., Shelmanov, A., Kobozeva, M., Pisarevskaya, D., Smirnov, I., Toldova, S.: Classification models for RST discourse parsing of texts in Russian. In: Computational Linguistics and Intellectual Technologies. Papers from the 2019 Annual International Conference “Dialogue”, vol. 18, pp. 163–176 (2019)Google Scholar
  7. 7.
    Desai, T., Dakle, P.P., Moldovan, D.: Joint learning of syntactic features helps discourse segmentation. In: Proceedings of The 12th Language Resources and Evaluation Conference, pp. 1073–1080 (2020)Google Scholar
  8. 8.
    Feng, V.W., Hirst, G.: A linear-time bottom-up discourse parser with constraints and post-editing. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 511–521 (2014)Google Scholar
  9. 9.
    Guzmán, F., Joty, S., Màrquez, L., Nakov, P.: Using discourse structure improves machine translation evaluation. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pp. 687–698 (2014)Google Scholar
  10. 10.
    Hernault, H., Prendinger, H., Ishizuka, M., et al.: HILDA: a discourse parser using support vector machine classification. Dialogue Discourse 1(3), 1–33 (2010)CrossRefGoogle Scholar
  11. 11.
    Hirao, T., Yoshida, Y., Nishino, M., Yasuda, N., Nagata, M.: Single-document summarization as a tree knapsack problem. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1515–1520 (2013)Google Scholar
  12. 12.
    Hung, S.S., Huang, H.H., Chen, H.H.: A complete shift-reduce Chinese discourse parser with robust dynamic oracle. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 133–138 (2020)Google Scholar
  13. 13.
    Joty, S., Carenini, G., Ng, R.: A novel discriminative framework for sentence-level discourse analysis. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 904–915 (2012)Google Scholar
  14. 14.
    Kibrik, A.: The problem of non-discreteness and spoken discourse structure. Comput. Linguist. intellect. Technol. 1(14(21)), 225–233 (2015)Google Scholar
  15. 15.
    Kobayashi, N., Hirao, T., Kamigaito, H., Okumura, M., Nagata, M.: Top-down RST parsing utilizing granularity levels in documents. In: AAAI, pp. 8099–8106 (2020)Google Scholar
  16. 16.
    Liu, L., Lin, X., Joty, S., Han, S., Bing, L.: Hierarchical pointer net parsing. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 1006–1016 (2019)Google Scholar
  17. 17.
    Liu, X., Ou, J., Song, Y., Jiang, X.: On the importance of word and sentence representation learning in implicit discourse relation classification. In: Proceedings of the 29th International Joint Conference on Artificial Intelligence, IJCAI-20, pp. 3830–3836. International Joint Conferences on Artificial Intelligence Organization (2020)Google Scholar
  18. 18.
    Mann, W.C., Thompson, S.A.: Rhetorical structure theory: toward a functional theory of text organization. Text Interdisc. J. Study Discourse 8(3), 243–281 (1988)CrossRefGoogle Scholar
  19. 19.
    Marcu, D.: The Theory and Practice of Discourse Parsing and Summarization. MIT Press, Cambridge (2000)CrossRefGoogle Scholar
  20. 20.
    Morey, M., Muller, P., Asher, N.: How much progress have we made on RST discourse parsing? A replication study of recent results on the RST-DT. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 1319–1324 (2017)Google Scholar
  21. 21.
    Muller, P., Braud, C., Morey, M.: ToNy: Contextual embeddings for accurate multilingual discourse segmentation of full documents. In: NAACL HLT 2019 (2019)Google Scholar
  22. 22.
    Peters, M.E., et al.: Deep contextualized word representations. In: Proceedings of NAACL-HLT, pp. 2227–2237 (2018)Google Scholar
  23. 23.
    Pisarevskaya, D., et al.: Towards building a discourse-annotated corpus of Russian. In: Computational Linguistics and Intellectual Technologies: Proceedings of the International Conference Dialogue, p. 23 (2017)Google Scholar
  24. 24.
    Pisarevskaya, D., Galitsky, B.: An anatomy of a lie: discourse patterns in ultimate deception dataset. In: Proceedings of the International Conference on Computational Linguistics and Intellectual Technologies “Dialogue 2019” (2019)Google Scholar
  25. 25.
    Rogers, A., Romanov, A., Rumshisky, A., Volkova, S., Gronas, M., Gribov, A.: RuSentiment: an enriched sentiment analysis dataset for social media in Russian. In: Proceedings of the 27th International Conference on Computational Linguistics, pp. 755–763. Association for Computational Linguistics (2018)Google Scholar
  26. 26.
    Soricut, R., Marcu, D.: Sentence level discourse parsing using syntactic and lexical information. In: Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, vol. 1, pp. 149–156 (2003)Google Scholar
  27. 27.
    Straka, M., Straková, J.: Tokenizing, POS tagging, lemmatizing and parsing UD 2.0 with UDPipe. In: Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pp. 88–99 (2017)Google Scholar
  28. 28.
    Subba, R., Di Eugenio, B.: An effective discourse parser that uses rich linguistic information. In: Proceedings of Human Language Technologies: the 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp. 566–574 (2009)Google Scholar
  29. 29.
    Wang, Y., Li, S., Wang, H.: A two-stage parsing method for text-level discourse analysis. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 184–188 (2017)Google Scholar
  30. 30.
    Wang, Y., Li, S., Yang, J.: Toward fast and accurate neural discourse segmentation. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 962–967 (2018)Google Scholar
  31. 31.
    Wang, Z., Hamza, W., Florian, R.: Bilateral multi-perspective matching for natural language sentences. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence, pp. 4144–4150 (2017)Google Scholar
  32. 32.
    Yu, N., Zhang, M., Fu, G.: Transition-based neural RST parsing with implicit syntax features. In: Proceedings of the 27th International Conference on Computational Linguistics, pp. 559–570 (2018)Google Scholar
  33. 33.
    Zeldes, A., Das, D., Maziero, E.G., Antonio, J., Iruskieta, M.: The DISRPT 2019 shared task on elementary discourse unit segmentation and connective detection. In: Proceedings of the Workshop on Discourse Relation Parsing and Treebanking 2019, pp. 97–104. Association for Computational Linguistics (2019)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2021

Authors and Affiliations

  1. 1.FRC CSC RASMoscowRussia
  2. 2.Skolkovo Institute of Science and TechnologyMoscowRussia
  3. 3.NRU Higher School of EconomicsMoscowRussia

Personalised recommendations