Mobile Networks and Applications

, Volume 13, Issue 6, pp 569–582 | Cite as

Mobile Multimodality: A Theoretical Approach to Facilitate Virtual Device Environments



The classical strategy of multimodal interactions is to simultaneously utilise several modalities offered on a ‘single’ device. However, practical restrictions appear, especially in mobile communications, due to the limitations of device’s interfaces and the continuously changing communication conditions. Taking multimodality out of this single-device scenario, this paper introduces a theoretical framework for dynamic Multi User Interfaces-Device Binding (MID-B). MID-B facilitates dynamic adaptation and customisation of physically distributed devices (and their user interfaces) into one ‘virtual’ multimodal environment, thus introducing ambient awareness into user interfaces. In MID-B, one device assumes the role of a “controller” which detects and binds the interfaces and modalities available on devices in the vicinity. After discovery, it binds these modalities to deliver the actual mobile service.


multimodal interaction virtual device ambient aware user interfaces interface and modality binding mobile ubiquitous Modality Service Tree 


  1. 1.
    Cellario M (2001) Human-centered intelligent vehicles: toward multimodal interface integration. IEEE Intell Syst 26:78–81CrossRefGoogle Scholar
  2. 2.
    Schraefel MC, Eilson M, Russel A, Smith DA (2006) mSpace: improving information access to multimedia domains with multimodal exploratory search. Commun ACM 49:47–49CrossRefGoogle Scholar
  3. 3.
    Ahn LV, Ginosar S, Kedia M, Liu R, Blum M (2006) Improving accessibility of the web with a computer game. In: ACM Conference on Human Factor in Computer System: ACM, New York, pp 78–82Google Scholar
  4. 4.
    Oviatt S (2002) Multimodal interfaces. In: Sears JJAA (ed) Handbook of human–computer interaction. Lawrence Erlbaum, New Jersey, pp 286–304Google Scholar
  5. 5.
    Niklfeld G, Anegg H, Gassner A (2005) Device independent mobile multimodal user interfaces with the MONA multimodal presentation server. In: Eurescom Summit 2005:ubiquitous services and applications exploiting the potential, Heidelberg, GermanyGoogle Scholar
  6. 6.
    Riekki J, Davidyuk O, Forstadius J, Sun J, Sauvola J. Enabling context-aware services for mobile users. In: IFIP, INTELLCOMM ’04Google Scholar
  7. 7.
    Tarasewich P (2003) Wireless devices for mobile commerce: user interface design and usability. In: Mobile commerce: technology, theory, and applications, pp 26–50Google Scholar
  8. 8.
    Kobayashi N, Tokunaga E, Kimura H (2005) An input widget framework for multi-modal and multi-device environments. Presented at 3rd IEEE Workshop on Software Technologies for Future Embedded and Ubiquitous System (SEUS’05)Google Scholar
  9. 9.
    “WAG UAProf,” Aireless Application Protocol Form, Ltd WAP-248-UAPROF-20011020-a, 2001Google Scholar
  10. 10.
    Belotti R, Decurtins C, Norrie MC, Phthon E (2005) A context-aware component registry for ubiquitous and mobile applications. In: CAiSE Workshop on Ubiquitous Mobile Information and Collaboration Systems (UMICS 2005), Porto, PortugalGoogle Scholar
  11. 11.
    Padovitz A, Loke SW, Zaslavsky A (2004) Towards a theory of context spaces. In: 2nd IEEE Annual Conference on Pervasive Computing and Communications Workshop. Florida, USA: IEEE Computer SocietyGoogle Scholar
  12. 12.
    Curzon P, Blandford A (2000) Using a verification system to reason about post-completion errors. In: Paterno PPAF (ed) 7th International Workshop on Design, Specification and Verification of Interactive System, pp 290–308Google Scholar
  13. 13.
    Thimbleby H, Gow J (2004) Computer algebra in interface design research. In: 9th International Conference on Intelligent User Interface. Madeira, PortugalGoogle Scholar
  14. 14.
    Jeschke S, Richiter T (2005) Mathematics in virtual knowledge spaces: user adaptation by intelligent assistants. Presented at PISA-Workshop, Palmerston North, New ZealandGoogle Scholar
  15. 15.
    Hrbacek K, Jech T (1999) Introduction to set theory, 3rd edn. Macel Dekker, New YorkMATHGoogle Scholar
  16. 16.
    Razak AHF (2004) Understanding interaction experience in mobile learning. In: Asia Pacific Computer Human Interaction (APCHI) (Doctoral Consortium)Google Scholar
  17. 17.
    Axelsson J, Potter S, Raman TV, Raggett D (2004) Modality component to host environment dom requirements and capabilities assessment. (Last accessed: 04.05.08).
  18. 18.
    Floreen P, Przybilski M, Nurmi P, Koolwaaij J, Tarlano A, Wagner M (2005) Towards a context management framework for mobilife. In: IST Mobile & Communication Summit Dresden, GermanyGoogle Scholar
  19. 19.
    Deitel HM, Deitel PJ, Nieto TR, Lin TM, Sadhu P (2001) Creating markup with XML. In: Horton M (ed) XML how to program. Prentice Hall, Upper Saddle River, pp 112–132Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.National e-Science CentreThe University of EdinburghEdinburghUK
  2. 2.Mobile Communications Research, Centre for Communication Systems ResearchUniversity of SurreySurreyUK

Personalised recommendations