WIP: The Coordinated Generation of Multimodal Presentations from a Common Representation

  • Wolfgang Wahlster
  • Elisabeth André
  • Som Bandyopadhyay
  • Winfried Graf
  • Thomas Rist
Conference paper
Part of the NATO ASI Series book series (volume 100)

Abstract

The task of the knowledge-based presentation system WIP is the generation of a variety of multimodal documents from an input consisting of a formal description of the communicative intent of a planned presentation. WIP generates illustrated texts that are customized for the intended audience and situation. We present the architecture of WIP and introduce as its major components the presentation planner, the layout manager, the text generator and the graphics generator. An extended notion of coherence for multimodal documents is introduced that can be used to constrain the presentation planning process. The paper focuses on the coordination of contents planning and layout that is necessary to produce a coherent illustrated text. In particular, we discuss layout revisions after contents planning and the influence of layout constraints on text generation. We show that in WIP the design of a multimodal document is viewed as a non-monotonic planning process that includes various revisions of preliminary results in order to achieve a coherent output with an optimal media mix.

Keywords

Content Planning Presentation Planner Coherence Relation Constraint Hierarchy Tree Adjoin Grammar 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Allgayer, J., Harbusch, K., Kobsa, A., Reddig, C, Reithinger, N. & Schmauks, D.: ‘XTRA: A Natural Language Access System to Expert Systems’, International Journal of Man-Machine Studies, 31, 161–195 (1989)CrossRefGoogle Scholar
  2. André, E. & Rist, T.: ‘Towards a Plan-Based Synthesis of Illustrated Documents’, Proc. of the 9th European Conference on Artificial Intelligence, 25–30 (1990)Google Scholar
  3. André, E. & Rist, T.: ‘Synthesizing Illustrated Documents: A Plan-Based Approach’, Proc. of Info Japan 90, vol. 2, 163–170 (1990)Google Scholar
  4. Bandyopadhyay, S.: ‘Towards an Understanding of Coherence in Multimodal Discourse’. Technical Memo DFKI-TM-90-01, German Research Center for Artificial Intelligence, Saarbrücken Site (1990)Google Scholar
  5. Beach, R. J.: ‘Setting Tables and Illustrations with Style’, Xerox PARC, Technical Report CSL-85-3 (1985)Google Scholar
  6. Borning, A. & Duisberg, A.: ‘Constraint-Based Tools for Building User Interfaces’, ACM Trans. on Graphics, 5, 6, 345–374 (1986)CrossRefGoogle Scholar
  7. Borning, A., Freeman-Benson, B. & Wilson, M.: ‘Constraint Hierarchies’, Internal Report, Department of Computer Science and Engineering, FR-35, University of Washington, Seattle (1989)Google Scholar
  8. Feiner, S.: ‘A Grid-Based Approach to Automating Display Layout’, Proc. Graphics Interface +88, Palo Alto, Morgan Kaufmann, 192–197 (1988)Google Scholar
  9. Feiner, S. & McKeown, K.: ‘Coordinating Text and Graphics in Explanation Generation’, DARPA Speech and Natural Language Workshop (1989)Google Scholar
  10. Finkler, W. & Neumann, G.: ‘POPEL-HOW: A Distributed Parallel Model for Incremental Natural Language Production with Feedback’, Proc. of the 11th IJCAI, 1518–1523 (1989)Google Scholar
  11. Freeman-Benson, B., Maloney, J. & Borning, A.: ‘An Incremental Constraint Solver’, Communications of the ACM, 33, 1, 54–63 (1990)CrossRefGoogle Scholar
  12. Graf, W.: ‘Spezielle Aspekte des automatischen Layout-Designs bei der koordinierten Generierung von multimodalen Dokumenten’, GI-Workshop Multimediale elektronische Dokumente (1990)Google Scholar
  13. Grice, H.: ‘Logic and Conversation’. In Cole and Morgan (eds.), Syntax and Semantics, 3, New York, Academic Press (1975)Google Scholar
  14. Grimes, J.E.: The Thread of Discourse. The Hague, Mouton (1975)Google Scholar
  15. Harbusch, K.: ‘Constraining Tree Adjoining Grammars by Unification’, Proc. of the 13th COLING, 167–172 (1990)Google Scholar
  16. Hobbs, J.: ‘Coherence and Coreference’, Cognitive Science, 3, 1 (1979)MathSciNetCrossRefGoogle Scholar
  17. Hobbs, J.: ‘Why is Discourse Coherent?’. In Neubauer (ed.), Coherence in Natural Language Texts, Hamburg, Buske (1983)Google Scholar
  18. Jameson, A. & Wahlster, W.: ‘User Modelling in Anaphora Generation: Ellipsis and Definite Description’, Proc. of the 5th ECAI, 222–221 (1982)Google Scholar
  19. Kjorup, S.: ‘Pictorial Speech Acts’, Erkenntnis, 12, 55–71 (1978)CrossRefGoogle Scholar
  20. Mann, W. & Thompson, S.: ‘Rhetorical Structure Theory: Towards a Functional Theory of Text Organization’, TEXT, 8, 3 (1988)Google Scholar
  21. Marks, J. & Reiter, E.: ‘Avoiding Unwanted Conversational Implicatures in Text and Graphics’, Proc. of the 8th AAAI, 450–455 (1990)Google Scholar
  22. Maaß, W.: ‘Constraint-basierte Repräsentation von graphischem Wissen am Beispiel des Layout-Managers in WIP’. MS thesis, Computer Science Departement, University of Saarbrücken (1991)Google Scholar
  23. McKeown, K. & Feiner, S.: ‘Interactive Multimedia Explanation for Equipment Maintenance and Repair’, DARPA Speech and Natural Language Workshop, 42–47 (1990)Google Scholar
  24. Moore, J. & Paris, C: ‘Planning Text for Advisory Dialogues’, Proc. of the 27th ACL, 203–211 (1989)Google Scholar
  25. Moore, J.D. & Swartout, W.R.: ‘A Reactive Approach to Explanation’, Proc. of the 11th IJCAI, 1504–1510 (1989)Google Scholar
  26. Müller-Brockmann, J.: Grid Systems in Graphic Design. Stuttgart, Hatje (1981)Google Scholar
  27. Neal, J. & Shapiro, S.: ‘Intelligent Multi-Media Interface Technology’, Proc. of the Workshop on Architectures of Intelligent Interfaces: Elements & Prototypes, 69–91 (1988)Google Scholar
  28. Nebel, B.: ‘Reasoning and Revision in Hybrid Representation Systems’, Lecture Notes in AI, 422, Berlin, Springer-Verlag (1990)Google Scholar
  29. Reichmann, R.: Getting Computers to Talk like You and Me. Cambridge, MA, MIT Press (1985)Google Scholar
  30. Rist, T. & André, E.: ‘Wissensbasierte Perspektivenwahl für die automatische Erzeugung von 3D-Objektdarstellungen’ In Kansy, K. and Wißkirchen P. (eds.), Graphik und KL IFB 239, Berlin, Springer-Verlag, 48–57 (1990)CrossRefGoogle Scholar
  31. Roth, S., Mattis, J. & Mesnard, X.: ‘Graphics and Natural Language as Components of Automatic Explanation’, Proc. of the Workshop on Architectures of Intelligent Interfaces: Elements & Prototypes, 109–128 (1988)Google Scholar
  32. Searle, J.: Speech Acts: An Essay in the Philosophy of Language. Cambridge, MA., Cambridge University Press (1969)CrossRefGoogle Scholar
  33. Schauder, A.: ‘Inkrementelle syntaktische Generierung natürlicher Sprache mit Tree Adjoining Grammars’. MS thesis, Computer Science Departement, University of Saarbrücken (1990)Google Scholar
  34. Stock, O.: ‘Natural Language and Exploration of an Information Space: the ALFresco Interactive System’. Proceedings of the 12th IJCAI, 972–978 (1991)Google Scholar
  35. van Dijk, T.: Textwissenschaft. München, DTV (1980)CrossRefGoogle Scholar
  36. Wahlster, W.: ‘User and Discourse Models for Multimodal Communication’. In Sullivan, J. and Tyler, S. (eds.), Architectures for Intelligent User Interfaces: Elements & Prototypes, Reading, MA., Addison-Wesley (1991)Google Scholar
  37. Wahlster, W. & Kobsa, A.: ‘User Models in Dialog Systems’. In Kobsa, A. and Wahlster, W. (eds.), User Models in Dialog Systems, Symbolic Computation Series, Berlin, Springer-Verlag, 4–34 (1989)CrossRefGoogle Scholar
  38. Wahlster, W., André, E., Hecking, M. & Rist, T.: ‘WIP: Knowledge-based Presentation of Information’. Report WIP-1, German Research Center for Artificial Intelligence, Saarbrücken (1989)Google Scholar
  39. Wahlster, W., André, E., Graf, W. & Rist, T.: ‘Designing Illustrated Texts: How Language Production Is Influenced by Graphics Generation’, Proc. of the 5th Conference of the European Chapter of the ACL, Berlin, Springer-Verlag, 8–14 (1991)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1992

Authors and Affiliations

  • Wolfgang Wahlster
    • 1
  • Elisabeth André
    • 1
  • Som Bandyopadhyay
    • 1
  • Winfried Graf
    • 1
  • Thomas Rist
    • 1
  1. 1.German Research Center for Artificial Intelligence (DFKI)Germany

Personalised recommendations