Integrating Multimodal Cues Using Grammar Based Models

  • Manuel Giuliani
  • Alois Knoll
Conference paper

DOI: 10.1007/978-3-540-73281-5_94

Part of the Lecture Notes in Computer Science book series (LNCS, volume 4555)
Cite this paper as:
Giuliani M., Knoll A. (2007) Integrating Multimodal Cues Using Grammar Based Models. In: Stephanidis C. (eds) Universal Access in Human-Computer Interaction. Ambient Interaction. UAHCI 2007. Lecture Notes in Computer Science, vol 4555. Springer, Berlin, Heidelberg

Abstract

Multimodal systems must process several input streams efficiently and represent the input in a way that allows the establishment of connections between modalities. This paper describes a multimodal system that uses Combinatory Categorial Grammars to parse several input streams and translate them into logical formulas. These logical formulas are expressed in Hybrid Logic, which is very suitable for multimodal integration because it can represent temporal relationships between modes in an abstract way. This level of abstraction makes it possible to define rules for multimodal processing in a straightforward way.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Manuel Giuliani
    • 1
  • Alois Knoll
    • 1
  1. 1.Robotics and Embedded Systems Group, Department of Informatics, Technische Universität München, Boltzmannstraße 3, D-85748 Garching bei MünchenGermany

Personalised recommendations