Discourse-Level Dialogue Management

  • Boris Galitsky


In this Chapter we learn how to manage a dialogue relying on discourse of its utterances. We first explain how to build an invariant discourse tree for a corpus of texts to arrange a chatbot-facilitated navigation through this corpus. We define extended discourse trees, introduce means to manipulate with them, and outline scenarios of multi-document navigation. We then show how a dialogue structure can be built from an initial utterance. After that, we introduce imaginary discourse tree to address a problem of involving background knowledge on demand, answering questions. Finally, an approach to dialogue management based on lattice walk is described.


  1. Agostaro F, Augello A, Pilato G, Vassallo G, Gaglio S (2005) A conversational agent based on a conceptual interpretation of a data driven semantic space, proceedings of AI*IA. LNAI 3673:381–392Google Scholar
  2. Allan J (1996) Automatic hypertext link typing. In: Hypertext’96, The seventh ACM conference on Hypertext, pp 42–52Google Scholar
  3. Amiridze N, Kutsia T (2018) Anti-unification and natural language processing. In: Fifth workshop on natural language and computer science, NLCS’18, EasyChair Preprint no. 203Google Scholar
  4. Augello A, Gentile M, Dignum F (2017) An overview of open-source chatbots social skills. In: Diplaris S, Satsiou A, Følstad A, Vafopoulos M, Vilarinho T (eds) Internet science, Lecture notes in computer science, vol 10750, pp 236–248CrossRefGoogle Scholar
  5. Barzilay R, Elhadad M (1997) Using lexical chains for text summarization. In: Proceedings of the ACL/EACL’97 workshop on intelligent scalable text summarization. Madrid, Spain, July 1997, pp 10–17.Google Scholar
  6. Barzilay R, Lapata M (2008) Modeling local coherence: An entity-based approach. Comput Linguist 34(1):1–34CrossRefGoogle Scholar
  7. Bordes A, Weston, J (2016) Learning end-to-end goal-oriented dialog. ICRL 2017Google Scholar
  8. Burtsev M, Seliverstov A, Airapetyan R, Arkhipov M, Baymurzina D, Bushkov N, Gureenkova O, Khakhulin T, Kuratov Y, Kuznetsov D, Litinsky A, Logacheva V, Lymar A, Malykh V, Petrov M, Polulyakh V, Pugachev L, Sorokin A, Vikhreva M, Zaynutdinov M (2018) DeepPavlov: open-source library for dialogue systems. In: ACL-system demonstrations, pp 122–127Google Scholar
  9. CarPros (2017)
  10. Chali Y, Joty SR, Hasan SA (2009) Complex question answering: unsupervised learning approaches and experiments. J Artif Int Res 35(1):1–47MathSciNetzbMATHGoogle Scholar
  11. Clarke J, Lapata M (2010) Discourse constraints for document compression. Comput Linguist 36(3):411–441CrossRefGoogle Scholar
  12. Codocedo V, Napoli A (2014) A proposition for combining pattern structures and relational concept analysis. In: Glodeanu CV, Kaytoue M, Sacarea C (eds) ICFCA 2014. LNCS (LNAI), vol 8478. Springer, Heidelberg, pp 96–111Google Scholar
  13. Cohen W (2018) Enron email dataset. Last downloaded 10 July 2018
  14. Elsner M, Charniak E (2008) You talking to me? a corpus and algorithm for conversation disentanglement. In: Proceedings of the 46th annual Meeting of the ACL: HLT (ACL 2008), Columbus, USA, pp 834–842Google Scholar
  15. Feng WV, Hirst G (2014) A linear-time bottom-up discourse parser with constraints and post-editing. In: Proceedings of the 52nd annual meeting of the Association for Computational Lin-guistics (ACL 2014), Baltimore, USA, June.Google Scholar
  16. Galitsky B (2014) Learning parse structure of paragraphs and its applications in search. Eng Appl Artif Intell 32:160–184CrossRefGoogle Scholar
  17. Galitsky B (2016) Providing personalized recommendation for attending events based on individual interest profiles. AI Research 5(1), Sciedu PressGoogle Scholar
  18. Galitsky B (2017) Discovering rhetorical agreement between a request and response. Dialogue Discourse 8(2):167–205Google Scholar
  19. Galitsky B, Ilvovsky D (2017a) Chatbot with a discourse structure-driven dialogue management, EACL demo programGoogle Scholar
  20. Galitsky B, Ilvovsky D (2017b) On a chat bot finding answers with optimal rhetoric representation. In: Proceedings of recent advances in natural language processing, Varna, Bulgaria, 4–6 September, pp 253–259Google Scholar
  21. Galitsky B, Jones R (2017) A chatbot demo about a student being broke. Video link Google Scholar
  22. Galitsky B, Makowski G (2017) Document classifier for a data loss prevention system based on learning rhetoric relations. CICLing 2017, Budapest, Hungary, 17–23 April.Google Scholar
  23. Galitsky B, McKenna EW (2017) Sentiment extraction from consumer reviews for providing product recommendations. US Patent 9646078B2Google Scholar
  24. Galitsky B, Chen H, Du S (2009a) Inverting semantic structure of customer opinions expressed in forums and blogs. In: 17th international conference on conceptual structures, Suppl. Proc.Google Scholar
  25. Galitsky B, González MP, Chesñevar CI (2009b) A novel approach for classifying customer complaints through graphs similarities in argumentative dialogue. Decis Support Syst 46(3):717–729CrossRefGoogle Scholar
  26. Galitsky B, Dobrocsi G, de la Rosa JL (2012) Inferring the semantic properties of sentences by mining syntactic parse trees. Data Knowl Eng v81:21–45Google Scholar
  27. Galitsky B, Kuznetsov SO, Usikov D (2013) Parse thicket representation for multi-sentence search. In: International conference on conceptual structures, pp 153–172Google Scholar
  28. Galitsky B, Ilvovsky D, Kuznetsov SO, Strok F (2014) Finding maximal common sub-parse thickets for multi-sentence search. In: Graph structures for knowledge representation and reasoning, pp 39–57CrossRefGoogle Scholar
  29. Galitsky B, Ilvovsky D, Kuznetsov SO (2015) Text classification into abstract classes based on discourse structure. In: Proceedings of recent advances in natural language processing, Hissar, Bulgaria, 7–9 September 2015, pp 200–207.Google Scholar
  30. Galitsky B, Parnis A, Usikov D (2017) Exploring discourse structure of user-generated content. CICLing 2017, Budapest, Hungary, 17–23 April.Google Scholar
  31. Ganter B, Kuznetsov SO (2001) Pattern structures and their projections. In: International conference on conceptual structures, pp 129–142Google Scholar
  32. Grasso F (1999) Playing with RST: two algorithms for the automated manipulation of discourse trees. In: Matousek V, Mautner P, Ocelíková J, Sojka P (eds) Text, speech and dialogue. TSD 1999. Lecture notes in computer science, vol 1692. Springer, Berlin/HeidelbergGoogle Scholar
  33. Grosz BJ, Sidner CL (1986) Attention, intention and the structure of discourse. Comput Linguist 12(3):175–204Google Scholar
  34. Grosz B, Joshi AK, Weinstein S (1995) Centering: a framework for modeling the local coherence of discourse. Comput Linguist 21(2):203–225Google Scholar
  35. Gundel JK, Hedberg N, Zacharski R (1993) Cognitive status and the form of referring expressions in discourse. Language 69(2):274–307CrossRefGoogle Scholar
  36. Heerschop B, Goossen F, Hogenboom A, Frasincar F, Kaymak U, de Jong F (2011) Polarity analysis of texts using discourse structure. In: Proceedings of the 20th ACM international conference on information and knowledge management, CIKM ‘11, pp 1061–1070, New York, USA, ACMGoogle Scholar
  37. Indri IR (2018) Last downloaded Sept 11, 2018
  38. Jansen P, Surdeanu M, Clark P (2014) Discourse comple-ments lexical semantics for nonfactoid answer reranking. ACLGoogle Scholar
  39. Ji Y, Eisenstein J (2014) Representation learning for text-level discourse parsing. ACL 2014Google Scholar
  40. Joty SR, Moschitti A (2014) Discriminative reranking of discourse parses using tree kernels. In: Proceedings of the 2014 conference on Empirical Methods in Natural Language Processing (EMNLP).?Google Scholar
  41. Joty SR, Carenini G, Ng RT, Mehdad Y (2013) Combining intra-and multi- sentential rhetorical parsing for document-level discourse analysis. In: ACL, vol. 1, pp 486–496Google Scholar
  42. Kaytoue M, Codocedo V, Buzmakov A, Baixeries J, Kuznetsov SO, Napoli A (2015) Pattern structures and concept lattices for data mining and knowledge processing. Joint european conference on machine learning and knowledge discovery in databases. Springer, Cham, pp 227–231Google Scholar
  43. Kelley JF (1984) An iterative design methodology for user-friendly natural language office information applications. ACM Trans Inf Syst 2(1):26–41CrossRefGoogle Scholar
  44. Kerly A, Hall P, Bull S (2007) Bringing chatbots into education: towards natural language negotiation of open learner models. Knowl-Based Syst 20(2):177–185CrossRefGoogle Scholar
  45. Kim SN, Wang LI, Baldwin T (2010) Tagging and linking web forum posts. In: Proceedings of the 14th conference on Computational Natural Language Learning (CoNLL-2010), Uppsala, Sweden, pp 192–202Google Scholar
  46. Koiti H (2010) SemAF: discourse structures. Last downloaded 28 February 2018
  47. Kovalerchuk B, Kovalerchuk M (2017) Toward virtual data scientist with visual means. In: IJCNN.Google Scholar
  48. Kuyten P, Bollegala D, Hollerit B, Prendinger H, Aizawa K (2015) A discourse search engine based on rhetorical structure theory. In: Hanbury A, Kazai G, Rauber A, Fuhr N (eds) Advances in information retrieval. ECIR 2015, Lecture notes in computer science, vol 9022. Springer, ChamGoogle Scholar
  49. Kuznetsov SO, Makhalova T (2018) On interestingness measures of formal concepts. Inf Sci 442:202–219MathSciNetCrossRefGoogle Scholar
  50. LeThanh H, Abeysinghe G, Huyck C (2004) Generating discourse structures for written texts. In: Proceedings of the 20th international conference on computational linguistics, COLING ‘04, Geneva, Switzerland. Association for Computational LinguisticsGoogle Scholar
  51. Lioma C, Larsen B, Lu W (2012). Rhetorical relations for information retrieval. SIGIR. Portland, Oregon, USA, 12–16 August 2012Google Scholar
  52. Louis A, Joshi AK, Nenkova A (2010) Discourse indicators for content selection in summarization. In Fernandez R, Katagiri Y, Komatani K, Lemon O, Nakano M (eds) SIGDIAL conference, The Association for Computer Linguistics, pp 147–156Google Scholar
  53. Lowe RIV, Noseworthy M, Charlin L, Pineau J (2016) On the evaluation of dialogue systems with next utterance classification. In: Special interest group on discourse and dialogueGoogle Scholar
  54. Marcu D (2000) The rhetorical parsing of unrestricted texts: a surface-based approach. Comput Linguist 26:395–448CrossRefGoogle Scholar
  55. Marcu D, Echihabi A (2002) An unsupervised approach to recognizing discourse relations. In: Proceedings of the 40th annual meeting on Association for Computational Linguistics, ACL’02, pp 368–375Google Scholar
  56. Marir F, Haouam K (2004) Rhetorical structure theory for content-based indexing and retrieval of Web documents, ITRE 2004. In: 2nd international conference information technology: research and education, pp 160–164Google Scholar
  57. Morato J, Llorens J, Genova G, Moreiro JA (2003) Experiments in discourse analysis impact on information classification and retrieval algorithms. Info Process Manag 39:825–851CrossRefGoogle Scholar
  58. Nagarajan V, Chandrasekar P (2014) Pivotal sentiment tree classifier. IJSTR V.3, I, 11 November.Google Scholar
  59. Nguyen DT, Joty S (2017) A neural local coherence model. ACL 1:1320–1330Google Scholar
  60. Plotkin GD (1970) A note on inductive generalization. Mach Intell 5(1):153–163MathSciNetzbMATHGoogle Scholar
  61. Poesio M, Stevenson R, Di Eugenio B, Hitzeman J (2004) Centering: A parametric theory and its instantiations. Comput Linguist 30(3):309–363CrossRefGoogle Scholar
  62. Radev DR (2000) A common theory of information fusion from multiple text sources step one: cross-document structure. In: Proceedings of the 1st SIGDIAL workshop on discourse and dialogue (SIGDIAL) ‘00, pp 74–83Google Scholar
  63. Rajpurkar P, Zhang J, Lopyrev K, Liang P (2016) Squad: 100,000+ questions for machine comprehension of text.
  64. Rose CP, Di Eugenio B, Levin LS, Van Ess-Dykema C (1995) Discourse processing of dialogues with multiple threads. In: Proceedings of the 33rd annual meeting of the association for computational linguistics, Cambridge, USA, pp 31–38Google Scholar
  65. Sakai T (2007) Alternatives to Bpref. In: Proceedings of the 30th annual international ACM SIGIR conference on research and development in information retrieval. Amsterdam, The Netherlands, ACM, pp 71–78Google Scholar
  66. Seo JW, Croft B, Smith DA (2009) Online community search using thread structure. In: Proceedings of the 18th ACM Conference on Information and Knowledge Management (CIKM 2009), Hong Kong, China, pp 1907–1910.Google Scholar
  67. Serban IV, Lowe R., Henderson P, Charlin L, Pineau J (2017) A survey of available corpora for building data-driven dialogue systems.
  68. Sidorov G, Velasquez F, Stamatatos E, Gelbukh A, Chanona-Hernández L (2012) Syntactic Dependency-based N-grams as Classification Features. LNAI 7630:1–11Google Scholar
  69. Singh Ospina N, Phillips KA, Rodriguez-Gutierrez R, Castaneda-Guarderas A, Gionfriddo MR, Branda ME, Montori VM (2019) Eliciting the patient’s agenda- secondary analysis of recorded clinical encounters. J Gen Intern Med 34(1):36–40CrossRefGoogle Scholar
  70. Somasundaran S, Namata G, Wiebe J, Getoor L (2009) Supervised and unsupervised methods in employing discourse relations for improving opinion polarity classification. In: EMNLP, ACL, pp 170–179.Google Scholar
  71. Soricut R, Marcu D (2003) Sentence level discourse parsing using syntactic and lexical information. In: HLT-NAACL.Google Scholar
  72. Sporleder C, Lascarides A (2004) Combining hierarchical clustering and machine learning to predict high-level discourse structure. In: Proceedings of the 20th international conference on Computational Linguistics, COLING’04, Geneva, SwitzerlandGoogle Scholar
  73. Sun M, Chai JY (2007) Discourse processing for context question answering based on linguistic knowledge. Know Based Syst 20:511–526CrossRefGoogle Scholar
  74. Surdeanu M, Hicks T, Valenzuela-Escarcega MA (2015) Two practical rhetorical structure theory parsers. In: Proceedings of the conference of the North American Chapter of the Association for Computational Linguistics – Human Language Technologies: Software Demonstrations (NAACL HLT).Google Scholar
  75. Suwandaratna N, Perera U (2010). Discourse marker based topic identification and search results refining. In: Information and automation for sustainability (ICIAFs), 2010 5th International conference on, pp 119–125Google Scholar
  76. Teufel S, Moens M (2002) Summarizing scientific articles: experiments with relevance and rhetorical status. Comput Linguist 28(4):409–445, 2002CrossRefGoogle Scholar
  77. Trigg R, Weiser M (1987) TEXTNET: A network-based approach to text handling. ACM Trans Off Inf Sys 4(1):1–23Google Scholar
  78. Vorontsov K, Potapenko A (2015) Additive regularization of topic models. Mach Learn 101(1–3):303–323MathSciNetCrossRefGoogle Scholar
  79. Wanas N, El-Saban M, Ashour H, Ammar W (2008) Automatic scoring of online discussion posts. In: Proceeding of the 2nd ACM workshop on Information credibility on the web (WICOW’08), Napa Valley, USA, pp 19–26.Google Scholar
  80. Wang Z, Lemon O (2013) A simple and generic belief tracking mechanism for the dialog state tracking challenge: on the believability of observed information. In: Proceedings of the SIGDIALGoogle Scholar
  81. Wang DY, Luk RWP, Wong KF, Kwok KL. (2006) An information retrieval approach based on discourse type. In: Kop C, Fliedl G, Mayr HC, M’etais E (eds), NLDB, volume 3999 of Lecture notes in computer science, Springer, pp 197–202.Google Scholar
  82. Wang W, Su J, Tan CL (2010) Kernel based discourse relation recognition with temporal ordering information. ACLGoogle Scholar
  83. Wang L, Lui M, Kim SN, Nivre J, Baldwin T (2011) Predicting thread discourse structure over technical web forums. In: Proceedings of the 2011 conference on empirical methods in natural language processing, Edinburgh, UK, pp 13–25Google Scholar
  84. Webscope (2017) Yahoo! answers dataset.
  85. Wilks YA (ed) (1999) Machine conversations. Kluwer, BostonGoogle Scholar
  86. Wolf F, Gibson E (2005) Representing discourse coherence: A corpus-based study. Comput Linguist 31(2):249–287CrossRefGoogle Scholar
  87. Young S, Gasic M, Thomson B, Williams J (2013) POMDP-based statistical spoken dialogue systems: a review. In: Proceedings of IEEE, vol 99, pp 1–20Google Scholar
  88. Zeldes A (2016) rstWeb – a browser-based annotation Interface for rhetorical structure theory and discourse relations. In: Proceedings of NAACL-HLT 2016 (demonstrations). San Diego, California, June 12–17, 2016, pp 1–5Google Scholar
  89. Zhao K, Huang L (2017) Joint syntacto-discourse parsing and the syntacto-discourse treebank.
  90. Zhao J, Chevalier F, Collins C, Balakrishnan R (2012) Facilitating discourse analysis with interactive visualization. IEEE Trans Vis Comput Graph 18(12):2639–2648CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Boris Galitsky
    • 1
  1. 1.Oracle (United States)San JoseUSA

Personalised recommendations