Skip to main content
SpringerLink
Log in

Development of a Data-driven Framework for Multimodal Interactive Systems

  • Conference paper
  • First Online:
Proceedings of the Paralinguistic Information and its Integration in Spoken Dialogue Systems Workshop

Abstract

The objective of the present study was to develop a data-driven framework for MMI (multimodal interaction) system development. In the area of software engineering, data-driven development methods have proven to be reusable and extensible in the object-oriented paradigm. We have designed an object-oriented data modeling language for MMI systems and an extended elements of HTML for interaction description. The proposed framework uses the extended Rails framework as a generator of the MVC (Model-View-Controller) elements from the proposed objectoriented data modeling language. As the generated view code is written in extended HTML, we also implemented a speech browser for the extended HTML. We evaluated the framework by rapidly implementing a messaging system and examined its extensibility by a functionality expansion test.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Masahiro Araki, Kazunori Komatani, Taishi Hirata and Shuji Doshit: A Dialogue Library for Task-oriented Spoken Dialogue Systems, In Proc. IJCAI Workshop on Knowledge and Reasoning in Practical Dialogue Systems, 1–7, (1999)

    Google Scholar 

  2. Masahiro Araki, Akiko Kouzawa and Kenji Tachibana: Proposal of a multimodal interaction description language for various interac-tive agents, IEICE transactions on information and systems, E88-D(11), 2469–2476 (2005)

    Google Scholar 

  3. Masahiro Araki, Tsuneo Nitta, Kouichi Katsurada, Takuya Nishimoto, Tetsuo Amakasu and Shinnichi Kawamoto: Proposal of a Hierarchical Architecture for Multimodal Interactive Systems. W3C MMI workshop. (2007) Available via http://www.w3.org/2007/08/mmi-arch/slides/ araki.pdfCited5May2011

  4. Masahiro Araki and Takashi Hattori: Proposal of a Practical Spoken Dialogue System Development Method: Data-management Centered Approach, In W. Minker et al. (eds) Spoken Dialogue Systems Technology and Design, Springer, 187–211 (2010)

    Google Scholar 

  5. Masahiro Araki, Kana Shibahara and Yuko Mizukami: Spoken dialogue system for learning Braille, In Proc. COMPSAC 2011 (to appear)

    Google Scholar 

  6. Jim Barnett (eds.): Multimodal Architecture and Interfaces. W3C Working Draft 25 January 2011. Available via http://www.w3.org/TR/ mmi-arch/Cited5May2011

  7. Philip Hanna, Ian O’Neill, Xingkun Liu and Michael McTear: Developing Extensible and Reusable Spoken Dialogue Components: An Examination of the Queen’s Communicator, In Proc. Interspeech, 1865–1868 (2005)

    Google Scholar 

  8. Tobias Heinroth, Dan Denich and Gregor Bertrand: Ontology-based Spoken Dialogue Modeling, In Proc. the 1st IWSDS, (2009)

    Google Scholar 

  9. Koichi Katsurada, Hiroaki Adachi, Kunitoshi Sato, Hirobumi Yamada and Tsuneo Nitta: Rapid Prototyping Tool for Constructing Web-based MMI Applications, In Proc. Interspeech, 1861–1864 (2005)

    Google Scholar 

  10. Michael McTear: Spoken Dialogue Technology, Springer. (2004)

    Google Scholar 

  11. Bertrand Meyer: Object-Oriented Software Construction (2nd Edition). Prentice-Hall International Series in Computer Science. (2000)

    Google Scholar 

  12. Stephanie Seneff, Ed Hurley, Raymond Lau, Christine Pao, Philipp Schmid, and Victor Zue: GALAXY-II: A Reference Architecture for Conversational System Development. In Proc. ICSLP, 931–934 (1998)

    Google Scholar 

  13. Daniel Sonntag, Robert Nesselrath, Gerhard Sonnenberg, and Gerd Herzog: Supporting a Rapid Dialogue System Engineering Process, In Proc. the 1st IWSDS, (2009)

    Google Scholar 

Download references

Acknowledgements

The present research was supported in part by the Ministry of Education, Science, Sports, and Culture through a Grant-in-Aid for Scientific Research (C), 22500153, 2010.

Author information

Authors and Affiliations

  1. Kyoto Institute of Technology, Matsugasaki Sakyo-ku, Kyoto, 6068585, Japan

    Masahiro Araki & Yuko Mizukami

Authors
  1. Masahiro Araki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuko Mizukami
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Masahiro Araki .

Editor information

Editors and Affiliations

  1. , Dept. of Languages and Computer Systems, University of Granada, Granada, 18071, Spain

    Ramón López-Cózar Delgado

  2. , Dept. of Computer Science & Engineering, Waseda University, Okubo 3-4-1, Tokyo, 169-8555, Japan

    Tetsunori Kobayashi

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer Science+Business Media, LLC

About this paper

Cite this paper

Araki, M., Mizukami, Y. (2011). Development of a Data-driven Framework for Multimodal Interactive Systems. In: Delgado, RC., Kobayashi, T. (eds) Proceedings of the Paralinguistic Information and its Integration in Spoken Dialogue Systems Workshop. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-1335-6_11

Download citation

  • DOI: https://doi.org/10.1007/978-1-4614-1335-6_11

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4614-1334-9

  • Online ISBN: 978-1-4614-1335-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation