© 2017

Multimodal Interaction with W3C Standards

Toward Natural User Interfaces to Everything

  • Deborah A. Dahl

Table of contents

  1. Front Matter
    Pages i-xxix
  2. Standards

    1. Front Matter
      Pages 1-1
    2. Paolo Baggia, Daniel C. Burnett, Rob Marchand, Val Matula
      Pages 19-36
    3. Felix Burkhardt, Catherine Pelachaud, Björn W. Schuller, Enrico Zovato
      Pages 65-80
    4. Jim Barnett
      Pages 81-107
    5. Harry Bunt, Volha Petukhova, David Traum, Jan Alexandersson
      Pages 109-135
    6. Kouichi Katsurada, Tsuneo Nitta, Masahiro Araki, Kazuyuki Ashimura
      Pages 137-153
    7. Daniel C. Burnett
      Pages 155-169
  3. Implementations

    1. Front Matter
      Pages 171-171
    2. Stefan Radomski, Jens Heuschkel, Dirk Schnelle-Walka, Max Mühlhäuser
      Pages 213-255
    3. Deborah A. Dahl
      Pages 257-269
    4. António Teixeira, Nuno Almeida, Carlos Pereira, Miguel Oliveira e Silva, Diogo Vieira, Samuel Silva
      Pages 271-291
  4. Applications

    1. Front Matter
      Pages 293-293
    2. Vikram Ramanarayanan, David Suendermann-Oeft, Patrick Lange, Robert Mundkowsky, Alexei V. Ivanov, Zhou Yu et al.
      Pages 295-310
  5. Future Directions

    1. Front Matter
      Pages 323-323
    2. B. Helena Rodríguez, Jean-Claude Moissinac
      Pages 325-349

About this book


This book presents new standards for multimodal interaction published by the W3C and other standards bodies in straightforward and accessible language, while also illustrating the standards in operation through case studies and chapters on innovative implementations. The book illustrates how, as smart technology becomes ubiquitous, and appears in more and more different shapes and sizes, vendor-specific approaches to multimodal interaction become impractical, motivating the need for standards. This book covers standards for voice, emotion, natural language understanding, dialog, and multimodal architectures. The book describes the standards in a practical manner, making them accessible to developers, students, and researchers.

  • Comprehensive resource that explains the W3C standards for multimodal interaction clear and straightforward way;
  • Includes case studies of the use of the standards on a wide variety of devices, including mobile devices, tablets, wearables and robots, in applications such as assisted living, language learning, and health care;
  • Features illustrative examples of implementations that use the standards, to help spark innovative ideas for future applications.


Internet of Things (IoT) Multimodal interaction Multimodal technologies Natural user interface Smart technology W3C multimodal standards World Wide Web Consortium standards

Editors and affiliations

  • Deborah A. Dahl
    • 1
  1. 1.Conversational TechnologiesPlymouth MeetingUSA

About the editors

Dr. Deborah Dahl has over 20 years of experience in speech and natural language technologies, including working on research, defense, and commercial systems. She is also active in speech and multimodal standards activities in the World Wide Web Consortium, serving as Chair of the Multimodal Interaction Working Group and Co-Chair of the Hypertext Coordination Group. She is an editor of the EMMA (Extensible MultiModal Annotation specification). Dr. Dahl received the prestigious "Speech Luminary" award from Speech Technology Magazine, in 2012 and 2014.

Bibliographic information


“This 341-page book provides relatively comprehensive coverage of programming language concepts, a core topic in the ACM model computer science curriculum and a required topic for ABET accreditation of a computer science program. As such the book is appropriately aimed at senior students in an undergraduate curriculum.” (Michael Oudshoorn, Computing Reviews, February, 2018)​