Skip to main content
SpringerLink
Log in

Multimodal Interaction Description Language Based on Data Modeling

  • Chapter
  • First Online:
Multimodal Interaction with W3C Standards

  • 709 Accesses

Abstract

A methodology for developing a multimodal interactive system is required to be implemented as the number of input and output devices is increasing. Previous MMI (MultiModal Interaction) description languages used a state transition model for defining interaction, which has low maintainability and extensibility. In this paper, we design and implement a new MMI description language named MrailsScript from the software engineering point of view. MrailsScript consists of the data model definition that can be inherited from existing semantic web class. By expanding the existing web application development framework, our Mrails framework can automatically generate prototype code of MMI applications that are based on the MVC (Model-View-Controller) model referring to the annotations of task type and dialogue initiative. We also develop a helper application for the MMI system, MrailsBuilder, which assists in the coding of MrailsScript and contents management using semantic web standards, such as OWL, RDFS, and RDF.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://grails.org/

  2. 2.

    https://schema.org/

  3. 3.

    http://wiki.dbpedia.org/

References

  1. Hori, C., Ohtake, K., Misu, T., Kashioka, H., & Nakamura, S. (2009). Statistical dialog management applied to WFST-based dialog systems. In Proceedings of ICASSP 2009, Taipei, Taiwan, pp. 4793–4796.

    Google Scholar 

  2. Williams, J. D., & Young, S. (2007). Partially observable Markov decision processes for spoken dialog systems. Computer Speech and Language, 21(2), 393–422.

    Article  Google Scholar 

  3. Heinroth, T., Denich, D., & Bertrand, G. (2009). Ontology-based spoken dialogue modeling. In Proceedings of the IWSDS 2009, Irsee, Germany

    Google Scholar 

  4. Kogure, S., & Nakagawa, S. (2001). A development tool for spoken dialogue systems and its evaluation. In Proceedings of TSD2001 (LNAI 2166), Zelezna Ruda, Czech Republic, pp. 373–380.

    Google Scholar 

  5. Dean, M., & Schreiber, G. (Eds.). (2004). OWL Web Ontology Language Reference, W3C Recommendation. 10 February 2004. https://www.w3.org/TR/2004/REC-owl-ref-20040210/.

  6. Araki, M., & Mizukami, Y. (2011). Development of a data-driven framework for multimodal interactive systems. In Proceedings of IWSDS 2011, Granada, Spain, pp. 91–101.

    Google Scholar 

  7. Araki, M. (2012a). Rapid development process of spoken dialogue systems using collaboratively constructed semantic resources. In Proceedings of SIGDial 2012, Seoul, South Korea, pp. 70–73.

    Google Scholar 

  8. Takegoshi, D., & Araki, M. (2014). Development environment for multimodal interactive system based on ontological knowledge. In Proceedings of IIAI AAI 2014, Kitakyushu, Japan, pp. 785–788. doi:10.1109/IIAI-AAI.2014.158.

  9. Prud’hommeaux, E., & Seaborne, A. (Eds.). (2008). SPARQL Query Language for RDF. W3C Recommendation 15 January 2008. https://www.w3.org/TR/rdf-sparql-query/.

  10. Shires, G., & Wennborg, H. (Eds.). (2012). Web Speech API Specification. https://dvcs.w3.org/hg/speech-api/raw-file/tip/speechapi.html.

  11. Araki, M. (2012b). Multimodal motion learning system for traditional arts. In Proceedings of AHFE2012, San Francisco, USA, pp. 5274–5281.

    Google Scholar 

  12. Araki, M., & Takegoshi, D. (2012). A rapid development framework for multilingual spoken dialogue systems. In Proceedings of COMPSAC 2012, Izmir, Turkey, pp. 195–202.

    Google Scholar 

Download references

Acknowledgment

The collaborator of this research is Mr. Daisuke Takegoshi.

Author information

Authors and Affiliations

  1. Department of Information Science, Kyoto Institute of Technology, Kyoto, Japan

    Masahiro Araki

Authors
  1. Masahiro Araki
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Masahiro Araki .

Editor information

Editors and Affiliations

  1. Conversational Technologies, Plymouth Meeting, Pennsylvania, USA

    Deborah A. Dahl

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Araki, M. (2017). Multimodal Interaction Description Language Based on Data Modeling. In: Dahl, D. (eds) Multimodal Interaction with W3C Standards. Springer, Cham. https://doi.org/10.1007/978-3-319-42816-1_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-42816-1_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-42814-7

  • Online ISBN: 978-3-319-42816-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation