Skip to main content
SpringerLink
Log in

Discourse-Level Dialogue Management

  • Chapter
  • First Online:
Developing Enterprise Chatbots

Abstract

In this Chapter we learn how to manage a dialogue relying on discourse of its utterances. We first explain how to build an invariant discourse tree for a corpus of texts to arrange a chatbot-facilitated navigation through this corpus. We define extended discourse trees, introduce means to manipulate with them, and outline scenarios of multi-document navigation. We then show how a dialogue structure can be built from an initial utterance. After that, we introduce imaginary discourse tree to address a problem of involving background knowledge on demand, answering questions. Finally, an approach to dialogue management based on lattice walk is described.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 89.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Agostaro F, Augello A, Pilato G, Vassallo G, Gaglio S (2005) A conversational agent based on a conceptual interpretation of a data driven semantic space, proceedings of AI*IA. LNAI 3673:381–392

    Google Scholar 

  • Alice 3 (2018) Last downloaded July 21, 2018 https://www.oracle.com/webfolder/technetwork/tutorials/OracleAcademy/Alice3SelfStudyV2/index.html#section1s3

  • Allan J (1996) Automatic hypertext link typing. In: Hypertext’96, The seventh ACM conference on Hypertext, pp 42–52

    Google Scholar 

  • Amiridze N, Kutsia T (2018) Anti-unification and natural language processing. In: Fifth workshop on natural language and computer science, NLCS’18, EasyChair Preprint no. 203

    Google Scholar 

  • Augello A, Gentile M, Dignum F (2017) An overview of open-source chatbots social skills. In: Diplaris S, Satsiou A, Følstad A, Vafopoulos M, Vilarinho T (eds) Internet science, Lecture notes in computer science, vol 10750, pp 236–248

    Chapter  Google Scholar 

  • Barzilay R, Elhadad M (1997) Using lexical chains for text summarization. In: Proceedings of the ACL/EACL’97 workshop on intelligent scalable text summarization. Madrid, Spain, July 1997, pp 10–17.

    Google Scholar 

  • Barzilay R, Lapata M (2008) Modeling local coherence: An entity-based approach. Comput Linguist 34(1):1–34

    Article  Google Scholar 

  • Bordes A, Weston, J (2016) Learning end-to-end goal-oriented dialog. ICRL 2017

    Google Scholar 

  • Burtsev M, Seliverstov A, Airapetyan R, Arkhipov M, Baymurzina D, Bushkov N, Gureenkova O, Khakhulin T, Kuratov Y, Kuznetsov D, Litinsky A, Logacheva V, Lymar A, Malykh V, Petrov M, Polulyakh V, Pugachev L, Sorokin A, Vikhreva M, Zaynutdinov M (2018) DeepPavlov: open-source library for dialogue systems. In: ACL-system demonstrations, pp 122–127

    Google Scholar 

  • CarPros (2017) http://www.2carpros.com

  • CarPros Car Repair Dataset (2017) https://github.com/bgalitsky/relevance-based-on-parse-trees/blob/master/examples/CarRepairData_AnswerAnatomyDataset2.csv.zip

  • Chali Y, Joty SR, Hasan SA (2009) Complex question answering: unsupervised learning approaches and experiments. J Artif Int Res 35(1):1–47

    MathSciNet  MATH  Google Scholar 

  • Clarke J, Lapata M (2010) Discourse constraints for document compression. Comput Linguist 36(3):411–441

    Article  Google Scholar 

  • Codocedo V, Napoli A (2014) A proposition for combining pattern structures and relational concept analysis. In: Glodeanu CV, Kaytoue M, Sacarea C (eds) ICFCA 2014. LNCS (LNAI), vol 8478. Springer, Heidelberg, pp 96–111

    Google Scholar 

  • Cohen W (2018) Enron email dataset. https://www.cs.cmu.edu/~./enron/. Last downloaded 10 July 2018

  • Elsner M, Charniak E (2008) You talking to me? a corpus and algorithm for conversation disentanglement. In: Proceedings of the 46th annual Meeting of the ACL: HLT (ACL 2008), Columbus, USA, pp 834–842

    Google Scholar 

  • Feng WV, Hirst G (2014) A linear-time bottom-up discourse parser with constraints and post-editing. In: Proceedings of the 52nd annual meeting of the Association for Computational Lin-guistics (ACL 2014), Baltimore, USA, June.

    Google Scholar 

  • Fidelity (2018) https://github.com/bgalitsky/relevance-based-on-parse-trees/blob/master/examples/Fidelity_FAQs_AnswerAnatomyDataset1.csv.zip

  • Galitsky B (2014) Learning parse structure of paragraphs and its applications in search. Eng Appl Artif Intell 32:160–184

    Article  Google Scholar 

  • Galitsky B (2016) Providing personalized recommendation for attending events based on individual interest profiles. AI Research 5(1), Sciedu Press

    Google Scholar 

  • Galitsky B (2017) Discovering rhetorical agreement between a request and response. Dialogue Discourse 8(2):167–205

    Google Scholar 

  • Galitsky B, Ilvovsky D (2017a) Chatbot with a discourse structure-driven dialogue management, EACL demo program

    Google Scholar 

  • Galitsky B, Ilvovsky D (2017b) On a chat bot finding answers with optimal rhetoric representation. In: Proceedings of recent advances in natural language processing, Varna, Bulgaria, 4–6 September, pp 253–259

    Google Scholar 

  • Galitsky B, Jones R (2017) A chatbot demo about a student being broke. Video link https://drive.google.com/open?id=0B-TymkYCBPsfV3JQSGU3TE9mRVk

    Google Scholar 

  • Galitsky B, Makowski G (2017) Document classifier for a data loss prevention system based on learning rhetoric relations. CICLing 2017, Budapest, Hungary, 17–23 April.

    Google Scholar 

  • Galitsky B, McKenna EW (2017) Sentiment extraction from consumer reviews for providing product recommendations. US Patent 9646078B2

    Google Scholar 

  • Galitsky B, Chen H, Du S (2009a) Inverting semantic structure of customer opinions expressed in forums and blogs. In: 17th international conference on conceptual structures, Suppl. Proc.

    Google Scholar 

  • Galitsky B, González MP, Chesñevar CI (2009b) A novel approach for classifying customer complaints through graphs similarities in argumentative dialogue. Decis Support Syst 46(3):717–729

    Article  Google Scholar 

  • Galitsky B, Dobrocsi G, de la Rosa JL (2012) Inferring the semantic properties of sentences by mining syntactic parse trees. Data Knowl Eng v81:21–45

    Google Scholar 

  • Galitsky B, Kuznetsov SO, Usikov D (2013) Parse thicket representation for multi-sentence search. In: International conference on conceptual structures, pp 153–172

    Google Scholar 

  • Galitsky B, Ilvovsky D, Kuznetsov SO, Strok F (2014) Finding maximal common sub-parse thickets for multi-sentence search. In: Graph structures for knowledge representation and reasoning, pp 39–57

    Chapter  Google Scholar 

  • Galitsky B, Ilvovsky D, Kuznetsov SO (2015) Text classification into abstract classes based on discourse structure. In: Proceedings of recent advances in natural language processing, Hissar, Bulgaria, 7–9 September 2015, pp 200–207.

    Google Scholar 

  • Galitsky B, Parnis A, Usikov D (2017) Exploring discourse structure of user-generated content. CICLing 2017, Budapest, Hungary, 17–23 April.

    Google Scholar 

  • Ganter B, Kuznetsov SO (2001) Pattern structures and their projections. In: International conference on conceptual structures, pp 129–142

    Google Scholar 

  • Grasso F (1999) Playing with RST: two algorithms for the automated manipulation of discourse trees. In: Matousek V, Mautner P, Ocelíková J, Sojka P (eds) Text, speech and dialogue. TSD 1999. Lecture notes in computer science, vol 1692. Springer, Berlin/Heidelberg

    Google Scholar 

  • Grosz BJ, Sidner CL (1986) Attention, intention and the structure of discourse. Comput Linguist 12(3):175–204

    Google Scholar 

  • Grosz B, Joshi AK, Weinstein S (1995) Centering: a framework for modeling the local coherence of discourse. Comput Linguist 21(2):203–225

    Google Scholar 

  • Gundel JK, Hedberg N, Zacharski R (1993) Cognitive status and the form of referring expressions in discourse. Language 69(2):274–307

    Article  Google Scholar 

  • Heerschop B, Goossen F, Hogenboom A, Frasincar F, Kaymak U, de Jong F (2011) Polarity analysis of texts using discourse structure. In: Proceedings of the 20th ACM international conference on information and knowledge management, CIKM ‘11, pp 1061–1070, New York, USA, ACM

    Google Scholar 

  • Indri IR (2018) Last downloaded Sept 11, 2018 https://www.lemurproject.org/indri/

  • Jansen P, Surdeanu M, Clark P (2014) Discourse comple-ments lexical semantics for nonfactoid answer reranking. ACL

    Google Scholar 

  • Ji Y, Eisenstein J (2014) Representation learning for text-level discourse parsing. ACL 2014

    Google Scholar 

  • Joty SR, Moschitti A (2014) Discriminative reranking of discourse parses using tree kernels. In: Proceedings of the 2014 conference on Empirical Methods in Natural Language Processing (EMNLP).?

    Google Scholar 

  • Joty SR, Carenini G, Ng RT, Mehdad Y (2013) Combining intra-and multi- sentential rhetorical parsing for document-level discourse analysis. In: ACL, vol. 1, pp 486–496

    Google Scholar 

  • Kaytoue M, Codocedo V, Buzmakov A, Baixeries J, Kuznetsov SO, Napoli A (2015) Pattern structures and concept lattices for data mining and knowledge processing. Joint european conference on machine learning and knowledge discovery in databases. Springer, Cham, pp 227–231

    Google Scholar 

  • Kelley JF (1984) An iterative design methodology for user-friendly natural language office information applications. ACM Trans Inf Syst 2(1):26–41

    Article  Google Scholar 

  • Kerly A, Hall P, Bull S (2007) Bringing chatbots into education: towards natural language negotiation of open learner models. Knowl-Based Syst 20(2):177–185

    Article  Google Scholar 

  • Kim SN, Wang LI, Baldwin T (2010) Tagging and linking web forum posts. In: Proceedings of the 14th conference on Computational Natural Language Learning (CoNLL-2010), Uppsala, Sweden, pp 192–202

    Google Scholar 

  • Koiti H (2010) SemAF: discourse structures. http://slideplayer.com/slide/6408486/. Last downloaded 28 February 2018

  • Kovalerchuk B, Kovalerchuk M (2017) Toward virtual data scientist with visual means. In: IJCNN.

    Google Scholar 

  • Kuyten P, Bollegala D, Hollerit B, Prendinger H, Aizawa K (2015) A discourse search engine based on rhetorical structure theory. In: Hanbury A, Kazai G, Rauber A, Fuhr N (eds) Advances in information retrieval. ECIR 2015, Lecture notes in computer science, vol 9022. Springer, Cham

    Google Scholar 

  • Kuznetsov SO, Makhalova T (2018) On interestingness measures of formal concepts. Inf Sci 442:202–219

    Article  MathSciNet  Google Scholar 

  • LeThanh H, Abeysinghe G, Huyck C (2004) Generating discourse structures for written texts. In: Proceedings of the 20th international conference on computational linguistics, COLING ‘04, Geneva, Switzerland. Association for Computational Linguistics

    Google Scholar 

  • Lioma C, Larsen B, Lu W (2012). Rhetorical relations for information retrieval. SIGIR. Portland, Oregon, USA, 12–16 August 2012

    Google Scholar 

  • Louis A, Joshi AK, Nenkova A (2010) Discourse indicators for content selection in summarization. In Fernandez R, Katagiri Y, Komatani K, Lemon O, Nakano M (eds) SIGDIAL conference, The Association for Computer Linguistics, pp 147–156

    Google Scholar 

  • Lowe RIV, Noseworthy M, Charlin L, Pineau J (2016) On the evaluation of dialogue systems with next utterance classification. In: Special interest group on discourse and dialogue

    Google Scholar 

  • Marcu D (2000) The rhetorical parsing of unrestricted texts: a surface-based approach. Comput Linguist 26:395–448

    Article  Google Scholar 

  • Marcu D, Echihabi A (2002) An unsupervised approach to recognizing discourse relations. In: Proceedings of the 40th annual meeting on Association for Computational Linguistics, ACL’02, pp 368–375

    Google Scholar 

  • Marir F, Haouam K (2004) Rhetorical structure theory for content-based indexing and retrieval of Web documents, ITRE 2004. In: 2nd international conference information technology: research and education, pp 160–164

    Google Scholar 

  • Morato J, Llorens J, Genova G, Moreiro JA (2003) Experiments in discourse analysis impact on information classification and retrieval algorithms. Info Process Manag 39:825–851

    Article  Google Scholar 

  • Nagarajan V, Chandrasekar P (2014) Pivotal sentiment tree classifier. IJSTR V.3, I, 11 November.

    Google Scholar 

  • Nguyen DT, Joty S (2017) A neural local coherence model. ACL 1:1320–1330

    Google Scholar 

  • Plotkin GD (1970) A note on inductive generalization. Mach Intell 5(1):153–163

    MathSciNet  MATH  Google Scholar 

  • Poesio M, Stevenson R, Di Eugenio B, Hitzeman J (2004) Centering: A parametric theory and its instantiations. Comput Linguist 30(3):309–363

    Article  Google Scholar 

  • Radev DR (2000) A common theory of information fusion from multiple text sources step one: cross-document structure. In: Proceedings of the 1st SIGDIAL workshop on discourse and dialogue (SIGDIAL) ‘00, pp 74–83

    Google Scholar 

  • Rajpurkar P, Zhang J, Lopyrev K, Liang P (2016) Squad: 100,000+ questions for machine comprehension of text. https://arxiv.org/abs/1606.05250

  • Rose CP, Di Eugenio B, Levin LS, Van Ess-Dykema C (1995) Discourse processing of dialogues with multiple threads. In: Proceedings of the 33rd annual meeting of the association for computational linguistics, Cambridge, USA, pp 31–38

    Google Scholar 

  • Sakai T (2007) Alternatives to Bpref. In: Proceedings of the 30th annual international ACM SIGIR conference on research and development in information retrieval. Amsterdam, The Netherlands, ACM, pp 71–78

    Google Scholar 

  • Seo JW, Croft B, Smith DA (2009) Online community search using thread structure. In: Proceedings of the 18th ACM Conference on Information and Knowledge Management (CIKM 2009), Hong Kong, China, pp 1907–1910.

    Google Scholar 

  • Serban IV, Lowe R., Henderson P, Charlin L, Pineau J (2017) A survey of available corpora for building data-driven dialogue systems. https://arxiv.org/abs/1512.05742

  • Sidorov G, Velasquez F, Stamatatos E, Gelbukh A, Chanona-Hernández L (2012) Syntactic Dependency-based N-grams as Classification Features. LNAI 7630:1–11

    Google Scholar 

  • Singh Ospina N, Phillips KA, Rodriguez-Gutierrez R, Castaneda-Guarderas A, Gionfriddo MR, Branda ME, Montori VM (2019) Eliciting the patient’s agenda- secondary analysis of recorded clinical encounters. J Gen Intern Med 34(1):36–40

    Article  Google Scholar 

  • Somasundaran S, Namata G, Wiebe J, Getoor L (2009) Supervised and unsupervised methods in employing discourse relations for improving opinion polarity classification. In: EMNLP, ACL, pp 170–179.

    Google Scholar 

  • Soricut R, Marcu D (2003) Sentence level discourse parsing using syntactic and lexical information. In: HLT-NAACL.

    Google Scholar 

  • Sporleder C, Lascarides A (2004) Combining hierarchical clustering and machine learning to predict high-level discourse structure. In: Proceedings of the 20th international conference on Computational Linguistics, COLING’04, Geneva, Switzerland

    Google Scholar 

  • Sun M, Chai JY (2007) Discourse processing for context question answering based on linguistic knowledge. Know Based Syst 20:511–526

    Article  Google Scholar 

  • Surdeanu M, Hicks T, Valenzuela-Escarcega MA (2015) Two practical rhetorical structure theory parsers. In: Proceedings of the conference of the North American Chapter of the Association for Computational Linguistics – Human Language Technologies: Software Demonstrations (NAACL HLT).

    Google Scholar 

  • Suwandaratna N, Perera U (2010). Discourse marker based topic identification and search results refining. In: Information and automation for sustainability (ICIAFs), 2010 5th International conference on, pp 119–125

    Google Scholar 

  • Teufel S, Moens M (2002) Summarizing scientific articles: experiments with relevance and rhetorical status. Comput Linguist 28(4):409–445, 2002

    Article  Google Scholar 

  • Trigg R, Weiser M (1987) TEXTNET: A network-based approach to text handling. ACM Trans Off Inf Sys 4(1):1–23

    Google Scholar 

  • Vorontsov K, Potapenko A (2015) Additive regularization of topic models. Mach Learn 101(1–3):303–323

    Article  MathSciNet  Google Scholar 

  • Wanas N, El-Saban M, Ashour H, Ammar W (2008) Automatic scoring of online discussion posts. In: Proceeding of the 2nd ACM workshop on Information credibility on the web (WICOW’08), Napa Valley, USA, pp 19–26.

    Google Scholar 

  • Wang Z, Lemon O (2013) A simple and generic belief tracking mechanism for the dialog state tracking challenge: on the believability of observed information. In: Proceedings of the SIGDIAL

    Google Scholar 

  • Wang DY, Luk RWP, Wong KF, Kwok KL. (2006) An information retrieval approach based on discourse type. In: Kop C, Fliedl G, Mayr HC, M’etais E (eds), NLDB, volume 3999 of Lecture notes in computer science, Springer, pp 197–202.

    Google Scholar 

  • Wang W, Su J, Tan CL (2010) Kernel based discourse relation recognition with temporal ordering information. ACL

    Google Scholar 

  • Wang L, Lui M, Kim SN, Nivre J, Baldwin T (2011) Predicting thread discourse structure over technical web forums. In: Proceedings of the 2011 conference on empirical methods in natural language processing, Edinburgh, UK, pp 13–25

    Google Scholar 

  • Webscope (2017) Yahoo! answers dataset. https://webscope.sandbox.yahoo.com/catalog.php?datatype=l

  • Wilks YA (ed) (1999) Machine conversations. Kluwer, Boston

    Google Scholar 

  • Wolf F, Gibson E (2005) Representing discourse coherence: A corpus-based study. Comput Linguist 31(2):249–287

    Article  Google Scholar 

  • Young S, Gasic M, Thomson B, Williams J (2013) POMDP-based statistical spoken dialogue systems: a review. In: Proceedings of IEEE, vol 99, pp 1–20

    Google Scholar 

  • Zeldes A (2016) rstWeb – a browser-based annotation Interface for rhetorical structure theory and discourse relations. In: Proceedings of NAACL-HLT 2016 (demonstrations). San Diego, California, June 12–17, 2016, pp 1–5

    Google Scholar 

  • Zhao K, Huang L (2017) Joint syntacto-discourse parsing and the syntacto-discourse treebank. https://arxiv.org/pdf/1708.08484.pdf

  • Zhao J, Chevalier F, Collins C, Balakrishnan R (2012) Facilitating discourse analysis with interactive visualization. IEEE Trans Vis Comput Graph 18(12):2639–2648

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Oracle (United States), San Jose, CA, USA

    Boris Galitsky

Authors
  1. Boris Galitsky
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Galitsky, B. (2019). Discourse-Level Dialogue Management. In: Developing Enterprise Chatbots. Springer, Cham. https://doi.org/10.1007/978-3-030-04299-8_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-04299-8_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-04298-1

  • Online ISBN: 978-3-030-04299-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation