A Formal Description of Multimodal Interaction Techniques for Immersive Virtual Reality Applications

  • David Navarre
  • Philippe Palanque
  • Rémi Bastide
  • Amélie Schyn
  • Marco Winckler
  • Luciana P. Nedel
  • Carla M. D. S. Freitas
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3585)


Nowadays, designers of Virtual Reality (VR) applications are faced with the choice of a large number of different input and output devices leading to a growing number of interaction techniques. Usually VR interaction techniques are described informally, based on the actions users can perform within the VR environment. At implementation time, such informal descriptions (made at design time) yield to ambiguous interpretations by the developers. In addition, informal descriptions make it difficult to foresee the impact throughout the application of a modification of the interaction techniques. This paper discusses the advantages of using a formal description technique (called ICO) to model interaction techniques and dialogues for VR applications. This notation is presented via a case study featuring an immersive VR application. The case study is then used to show, through analysis of models, how the formal notation can help to ensure the usability, reliability and efficiency of virtual reality systems.


Virtual Reality Motion Captor Interaction Technique Virtual Reality Environment Informal Description 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Bach, C., Scapin, D.: Adaptation of Ergonomic Criteria to Human-Virtual Environments Interactions. In: INTERACT 2003, pp. 880–883. IOS Press, Amsterdam (2003)Google Scholar
  2. 2.
    Bass, L., Pellegrino, R., Reed, S., Seacord, R., Sheppard, R., Szezur, M.R.: The Arch model: Seeheim revisited. In: User Interface Developer’s workshop version 1.0 (1991)Google Scholar
  3. 3.
    Bastide, R., Navarre, D., Palanque, P., Schyn, A., Dragicevic, P.: A Model-Based Approach for Real-Time Embedded Multimodal Systems in Military Aircrafts. In: Sixth International Conference on Multimodal Interfaces (ICMI 2004), Pennsylvania State University, USA, October 14-15 (2004)Google Scholar
  4. 4.
    Bastide, R., Navarre, D., Palanque, P.: A Model-Based Tool for Interactive Prototyping of Highly Interactive Applications. In: ACM SIGCHI 2002 (Extended Abstracts), pp. 516–517 (2002)Google Scholar
  5. 5.
    Bastide, R., Palanque, P., Le Duc, H., Muñoz, J.: Integrating Rendering Specification into a Formalism for the Design of Interactive Systems. In: 5th Eurographics Workshop on Design, Specification and Verification of Interactive Systems (DSV-IS 1998). Springer, Heidelberg (1998)Google Scholar
  6. 6.
    Bowman, D., Johnson, D.B., Hodges, L.F.: Testbed evaluation of virtual environments interaction techniques. In: ACM Symposium on Virtual Reality Software and Technology, pp. 26–33 (1999)Google Scholar
  7. 7.
    Bowman, D., Kruijff, E., Laviola Jr., J.J., Poupyrev, I.: An introduction to 3-D User Interface Design. Presence: Teleoperators and Virtual Environments 10(1), 96–108 (2001)CrossRefGoogle Scholar
  8. 8.
    Buxton, W.: A three-state model of graphical input. In: 3rd IFIP International Conference on Human-Computer Interaction, INTERACT 1990, Cambridge, UK, August 27-31, pp. 449–456 (1990)Google Scholar
  9. 9.
    Campos, J.C., Harrison, M.D.: Formally verifying interactive systems: A review. In: Design, Specification and Verification of Interactive Systems 1997, pp. 109–124. Springer Computer Science, Heidelberg (1997)Google Scholar
  10. 10.
    Coutaz, J., Paterno, P., Faconti, G., Nigay, L.: A comparison of Approaches for Specifying Multimodal Interactive Systems. In: Proceedings of ERCIM, Nancy, France (1993)Google Scholar
  11. 11.
    Dubois, E., Gray, P.D., Nigay, L.: ASUR++: a Design Notation for Mobile Mixed Systems. IWC Journal, Special Issue on Mobile HCI 15(4), 497–520 (2003)zbMATHGoogle Scholar
  12. 12.
    Genrich, H.J.: Predicte/Transiion Nets. In: Jensen, K., Rozenberg, G. (eds.) High-Level Petri: Theory and Applications, pp. 3–43. Springer, Heidelberg (1991)CrossRefGoogle Scholar
  13. 13.
    Hinckley, K., Czerwinski, M., Sinclair, M.: Interaction and Modeling Techniques for Desktop Two-Handed Input (1998),
  14. 14.
    Latoschik, M.E.: Designing Transition Networks for Multimodal VR-Interactions Using a Markup Language. In: IEEE International Conference on Multimodal Interfaces (ICMI 2002) Proceedings (2002)Google Scholar
  15. 15.
    Märtin, C.: A method engineering framework for modeling and generating interactive applications. In: 3rd International Conference on Computer-Aided Design of User Interfaces, Belgium (1999)Google Scholar
  16. 16.
    Nedel, L.P., Freitas, C.M.D.S., Jacob, L.J., Pimenta, M.S.: Testing the Use of Egocentric Interactive Techniques in Immersive Virtual Environments. In: IFIP TC 13 Conference INTERACT 2003, on Human Computer Interaction, pp. 471–478. IOS Press, Zurich (2003)Google Scholar
  17. 17.
    Palanque, P., Schyn, A.: A Model-Based Approach for Engineering Multimodal Interactive Systems. In: IFIP TC 13 INTERACT 2003 conference. IOS Press, Amsterdam (2003)Google Scholar
  18. 18.
    Poupyrev, I., Weghorst, S., Billinghurst, M., Ichikawa, T.: Egocentric Object Manipulation in Virtual Environments: Empirical Evaluation of Interaction Techniques. In: Computer Graphics Forum, Eurographics 1998 issue, vol. 17(3), pp. 41–52 (1998)Google Scholar
  19. 19.
    Jacob, R., Deligiannidis, L., Morrison, S.: A software model and specification language for non-WIMP user interfaces. ACM ToCHI 6(1), 1–46 (1999)CrossRefGoogle Scholar
  20. 20.
    Smith, S., Duke, D.: Virtual environments as hybrid systems. In: Eurographics UK 17th Annual Conference Proceedings, pp. 113–128 (1999)Google Scholar
  21. 21.
    Smith, S., Duke, D.: The Hybrid World of Virtual Environments. In: Computer Graphics Forum, vol. 18(3), The Eurographics Association and Blackwell Publishers (1999)Google Scholar
  22. 22.
    Smith, S., Duke, D.: Using CSP to specify interaction in virtual environment. Technical report YCS 321, University ok York – Department of Computer Science (1999)Google Scholar
  23. 23.
    Sutcliffe, A., Gault, B., de Bruijn, O.: Comparing Interaction in the Real World and CAVE virtual environments. In: 18th HCI 2004, Leeds Metropolitan University, UK, September 6-10 (2004)Google Scholar
  24. 24.
    Van Schooten, B.W., Donk, O.A., Zwiers, J.: Modelling Interaction in Virtual Environments using Process Algebra. In: Proceedings TWLT 15: Interactions in Virtual Worlds, May 19-21 (1999)Google Scholar
  25. 25.
    Willans, J., Harrison, M.: A toolset supported approach for designing and testing virtual environment interaction techniques. International Journal of Human-Computer Studies 55(2), 145–165 (2001)CrossRefzbMATHGoogle Scholar

Copyright information

© IFIP International Federation for Information Processing 2005

Authors and Affiliations

  • David Navarre
    • 1
  • Philippe Palanque
    • 1
  • Rémi Bastide
    • 1
  • Amélie Schyn
    • 1
  • Marco Winckler
    • 1
  • Luciana P. Nedel
    • 2
  • Carla M. D. S. Freitas
    • 2
  1. 1.LIIHS-IRIT (Université Paul Sabatier)ToulouseFrance
  2. 2.Informatics InstituteFederal University of Rio Grande do SulPorto AlegreBrazil

Personalised recommendations