Journal on Multimodal User Interfaces

, Volume 3, Issue 3, pp 215–225 | Cite as

A wizard of oz component-based approach for rapidly prototyping and testing input multimodal interfaces

Original Paper


In this paper we present a novel approach for prototyping, testing and evaluating multimodal interfaces, OpenWizard. OpenWizard allows the designer and the developer to rapidly evaluate a non-fully functional multimodal prototype by replacing one modality or a composition of modalities that are not yet available by wizard of oz techniques. OpenWizard is based on a conceptual component-based approach for rapidly developing multimodal interfaces, an approach first implemented in the ICARE software tool and more recently in the OpenInterface tool. We present a set of wizard of oz components that are implemented in OpenInterface. While some wizard of oz (WoZ) components are generic to be reused for different multimodal applications, our approach allows the integration of tailored WoZ components. We illustrate OpenWizard using a multimodal map navigator.


Multimodality Wizard of oz Component-based approach 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Allen J (1983) Maintaining knowledge about temporal intervals. J Commun ACM 26(11):832–843 CrossRefMATHGoogle Scholar
  2. 2.
    Buxton W (1983) Lexical and pragmatic considerations of input structures. In: ACM SIGGRAPH, pp 31–37 Google Scholar
  3. 3.
    Cheng H, Bratt H, Mishra R, Shriberg E, Upson S, Chen J, Weng F, Peters S, Cavedon L, Niekrasz J (2004) A wizard of oz framework for collecting spoken human-computer dialogs. In: Intl. conf. on spoken language processing Google Scholar
  4. 4.
    Coiana M, Conconi A, Nigay L, Ortega M (2008) Test-bed for multimodal games on mobile devices. In: Proceedings of fun and games 2008. Springer, Berlin, pp 75–87 CrossRefGoogle Scholar
  5. 5.
    Coutaz J, Salber D, Carraux E, Portolan N (1996) Neimo, a multiworkstation usability lab for observing and analyzing multimodal interaction. In: Proceedings of CHI’96. ACM, New York, pp 402–403 Google Scholar
  6. 6.
    Davis RC, Saponas TS, Shilman M, Landay JA (2007) Sketchwizard: Wizard of oz prototyping of pen-based user interfaces. In: Proceedings of UIST’07. ACM, New York, pp 119–128 Google Scholar
  7. 7.
    Dietz P, Leigh D (2001) Diamondtouch: a multi-user touch technology. In: Proceedings of UIST’01. ACM, New York, pp 219–226 Google Scholar
  8. 8.
  9. 9.
    Eyetoy. By Sony,
  10. 10.
    Gray P, Ramsay A, Serrano M (2007) A demonstration of the openinterface interaction development environment. In: Adjunct proceedings of UIST’07. ACM, New York, pp 39–40 Google Scholar
  11. 11.
    Höysniemi J, Hämäläinen P, Turkki L (2004) Wizard of oz prototyping of computer vision based action games for children. In: Proceedings of IDC’04. ACM, New York, pp 27–34 Google Scholar
  12. 12.
  13. 13.
    Klein A, Schwank I, Généreux M, Trost H (2001) Evaluating multi-modal input modes in a wizard-of-oz study for the domain of web search. In: People and computers XV. Springer, Berlin, pp 475–483 Google Scholar
  14. 14.
    Klemmer SR, Sinha AK, Chen J, Landay JA, Aboobaker N, Wang A (2000) Suede, a wizard of oz prototyping tool for speech user interfaces. In: Proc of UIST’00. ACM, New York, p 110 Google Scholar
  15. 15.
    Lee M, Billinghurst M (2008) A wizard of oz study for an ar multimodal interface. In: Proceedings of ICMI’08. ACM, New York, pp 249–256 Google Scholar
  16. 16.
    Li Y, Hong JI, Landay JA (2007) Design challenges and principles for wizard of oz testing of location-enhanced applications In: Proceedings of IEEE pervasive computing, vol 6, pp 70–75 Google Scholar
  17. 17.
    Lim Y-K, Stolterman E, Tenenberg J (2008) The anatomy of prototypes: Prototypes as filters, prototypes as manifestations of design ideas. In: Transactions on computer-human interaction (TOCHI), vol 15. ACM, New York Google Scholar
  18. 18.
    Nigay L, Coutaz J (1997) Multifeature systems: the CARE properties and their impact on software design. intelligence and multimodality in multimedia interfaces. AAAI Press, Menlo Park Google Scholar
  19. 19.
  20. 20.
    OpenInterface. European project. IST Framework 6 STREP funded by the European Commission (FP6-35182).
  21. 21.
    Oviatt S (1999) Ten myths of multimodal interaction. Commun ACM 42(11):74–81 CrossRefGoogle Scholar
  22. 22.
    Oviatt S, Coulston R, Tomko S, Xiao B, Lunsford R, Wesson M, Carmichael L (2003) Toward a theory of organized multimodal integration patterns during human-computer interaction. In: Proceedings of ICMI’03. ACM, New York, pp 44–51 Google Scholar
  23. 23.
    Oviatt S, Lunsford R, Coulston R (2005) Individual differences in multimodal integration patterns: What are they and why do they exist? In: Proceedings of CHI’05. ACM, New York, pp 241–249 Google Scholar
  24. 24.
    Salber D, Coutaz J (1993) Applying the wizard-of-oz technique to the study of multimodal systems. In: Proc of the European workshop on HCI’93 Google Scholar
  25. 25.
    Serrano M, Nigay L, Demumieux R, Descos J, Losquin P (2006) Multimodal interaction on mobile phones: Development and evaluation using acicare. In: Proceedings of MobileHCI’06. ACM, New York, pp 129–136 Google Scholar
  26. 26.
    Serrano M, Juras D, Nigay L (2008) A three-dimensional characterization space of software components for rapidly developing multimodal interfaces. In: Proceedings of ICMI’08. ACM, New York, pp 149–156 Google Scholar
  27. 27.
    Wang S, Xiong X, Xu Y, Wang C, Zhang W, Dai X, Zhang D (2006) Face tracking as an augmented input in video games: Enhancing presence, role-playing and control. In: Proceedings of CHI’06. ACM, New York, pp 1097–1106 Google Scholar

Copyright information

© OpenInterface Association 2010

Authors and Affiliations

  1. 1.University of GrenobleGrenobleFrance

Personalised recommendations