T2D: Generating Dialogues Between Virtual Agents Automatically from Text

  • Paul Piwek
  • Hugo Hernault
  • Helmut Prendinger
  • Mitsuru Ishizuka
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4722)


The Text2Dialogue (T2D) system that we are developing allows digital content creators to generate attractive multi-modal dialogues presented by two virtual agents—by simply providing textual information as input. We use Rhetorical Structure Theory (RST) to decompose text into segments and to identify rhetorical discourse relations between them. These are then “acted out” by two 3D agents using synthetic speech and appropriate conversational gestures. In this paper, we present version 1.0 of the T2D system and focus on the novel technique that it uses for mapping rhetorical relations to question–answer pairs, thus transforming (monological) text into a form that supports dialogues between virtual agents.


Mapping Rule Input Text Virtual Agent Declarative Sentence Rhetorical Relation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    André, E.: The generation of multimedia presentations. In: Dale, R., Moisl, H., Somers, H. (eds.) Handbook of Natural Language Processing, pp. 305–327. Marcel Dekker, Inc. (2000)Google Scholar
  2. 2.
    André, E., Rist, T., van Mulken, S., Klesen, M., Baldes, S.: The automated design of believable dialogue for animated presentation teams. In: Cassell, J., Sullivan, J., Prevost, S., Churchill, E. (eds.) Embodied Conversational Agents, pp. 220–255. The MIT Press, Cambridge (2000)Google Scholar
  3. 3.
    Bäuerle, R., Zimmermann, T.: Fragesätze. In: von Stechow, A., Wunderlich, D. (eds.) Semantics. An International Handbook of Contemporary Research, Mouton de Gruyter, Berlin/New York, pp. 333–348 (1991)Google Scholar
  4. 4.
    Carlson, L., Marcu, D.: Discourse tagging reference manual. Technical Report ISI-TR-545, ISI (September 2001)Google Scholar
  5. 5.
    Cox, R., McKendree, J., Tobin, R., Lee, J., Mayes, T.: Vicarious learning from dialogue and discourse: A controlled comparison. Instructional Science 27, 431–458 (1999)Google Scholar
  6. 6.
    Craig, S., Gholson, B., Ventura, M., Graesser, A.: Tutoring Research Group: Overhearing dialogues and monologues in virtual tutoring sessions: Effects on questioning and vicarious learning. International Journal of Artificial Intelligence in Education 11, 242–253 (2000)Google Scholar
  7. 7.
    Davis, R.: Writing for Dialogue Scripts. A & C Black Ltd, London (1998)Google Scholar
  8. 8.
    Hofstadter, D.: Gödel, Escher, Bach: an Eternal Golden Braid. Basic Books, USA (1979)Google Scholar
  9. 9.
    Le, H.T., Abeysinghe, G.: A study to improve the efficiency of a discourse parsing system. In: Gelbukh, A. (ed.) CICLing 2003. LNCS, vol. 2588, pp. 101–114. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  10. 10.
    Mann, W.C., Thompson, S.A.: Rethorical structure theory: Toward a functional theory of text organization. Text 8(3), 243–281 (1988)Google Scholar
  11. 11.
    Mitkov, R., Ha, L.A., Karamanis, N.: A computer-aided environment for generating multiple-choice test items. Natural Language Engineering: Special Issue on using NLP for Educational Applications 12(2), 177–194 (2006)Google Scholar
  12. 12.
    Nadamoto, A., Tanaka, K.: Complementing your TV-viewing by web content automatically-transformed into TV-program-type content. In: Proceedings 13th Annual ACM International Conference on Multimedia, pp. 41–50. ACM Press, New York (2005)CrossRefGoogle Scholar
  13. 13.
    Nischt, M., Prendinger, H., André, E., Ishizuka, M.: MPML3D: a reactive framework for the Multimodal Presentation Markup Language. In: Gratch, J., Young, M., Aylett, R., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 218–229. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  14. 14.
    Piwek, P., Power, R., Scott, D., van Deemter, K.: Generating multimedia presentations. From plain text to screenplay. In: Stock, O., Zancanaro, M. (eds.) Multimodal Intelligent Information Presentation, Text, Speech, and Language Technology, pp. 203–225. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  15. 15.
    Piwek, P., Power, R., Williams, S.: Generating scripts for personalized medical dialogues for patients. Technical Report 2006/06, Department of Computing, Faculty of Mathematics and Computing, The Open University, UK (2006)Google Scholar
  16. 16.
    Piwek, P., van Deemter, K.: Towards automated generation of scripted dialogue: some time-honoured strategies. In: EIDLOG 2002. Proceedings 6th Workshop on the Semantics and Pragmatics of Dialogue, pp. 141–148 (2002)Google Scholar
  17. 17.
    Reitter, D.: Rhetorical theory in LaTeX with the rst package,
  18. 18.
    Sumi, K., Tanaka, K.: Transforming E-contents into a storybook world with animations and dialogues using semantic tags. In: SeC 2005. Online Proceedings of WWW-05 Workshop on the Semantic Computing Initiative (2005),
  19. 19.
    Tenny, C.L., Speas, P.: The interaction of clausal syntax, discourse roles, and information structure in questions. In: ESSLLI 2004 Workshop on Syntax, Semantics and Pragmatics of Questions, Université Henri Poincaré, France (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Paul Piwek
    • 1
  • Hugo Hernault
    • 2
  • Helmut Prendinger
    • 2
  • Mitsuru Ishizuka
    • 3
  1. 1.NLG Group, Centre for Research in Computing, The Open University, Walton Hall, Milton Keynes MK7 6AAUK
  2. 2.National Institute of Informatics, 2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo 101-8430Japan
  3. 3.Graduate School of Information Science and Technology, University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656Japan

Personalised recommendations