Extending a Dialog Model with Contextual Knowledge

  • Lode Vanacken
  • Erwin Cuppens
  • Tim Clerckx
  • Karin Coninx
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4849)


Designing and exploring multimodal interaction techniques, such as those used in virtual environments, can be facilitated by using high-level notations. Besides task modelling, notations have been introduced at the dialog level such as our notation NiMMiT. For advanced interaction techniques, there is not yet an established approach to decide when to stop detailing the task model and continue modelling at the dialog level. Also, context-awareness is usually introduced at the task level and not at the dialog level. We show that this might cause an explosion in the amount of dialog states in situations where context-aware multimodal interaction is used in one and the same task. Therefore, we propose an approach which attempts to introduce contextual knowledge at the dialog level where transitions are chosen upon context information. We validate our approach in a case study from which we conclude that the augmented notation is easy to use and successfully introduces context at the dialog level.


model-based user interface development multimodal user interfaces contextual knowledge 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cuppens, E., Raymaekers, C., Coninx, K.: A model-based design process for interactive virtual environments. In: DSVIS 2005. Proceedings of 12th International Workshop on Design, Specification and Verification of Interactive Systems, Newcastle upon Tyne, UK, pp. 225–236 (2005)Google Scholar
  2. 2.
    Willans, J., Harrison, M.: A toolset supported approach for designing and testing virtual environment interaction techniques. International Journal of Human-Computer Studies 55, 145–165 (2001)zbMATHCrossRefGoogle Scholar
  3. 3.
    Kulas, C., Sandor, C., Klinker, G.: Towards a development methodology for augmented reality user interfaces. In: Proc. of the International Workshop exploring the Design and Engineering of Mixed Reality Systems - MIXER 2004. CEUR Workshop Proceedings, Funchal, Madeira (2004)Google Scholar
  4. 4.
    Paternò, F.: Model-Based Design and Evaluation of Interactive Applications. Springer, Heidelberg (1999)Google Scholar
  5. 5.
    Vanacken, D., De Boeck, J., Raymaekers, C., Coninx, K.: NiMMiT: A notation for modeling multimodal interaction techniques. In: GRAPP 2006. Proceedings of the International Conference on CG Theory and Applications, Setubal, Portugal (2006)Google Scholar
  6. 6.
    Navarre, D., Palanque, P., Bastide, R., Schyn, A., Winckler, M., Nedel, L., Freitas, C.: A formal description of multimodal interaction techniques for immersive virtual reality applications. In: Proceedings of Tenth IFIP TC13 International Conference on Human-Computer Interaction, Rome, IT (2005)Google Scholar
  7. 7.
    Carr, D.: Interaction object graphs: An executable graphical notation for specifying user interfaces. In: Formal Methods for Computer-Human Interaction, pp. 141–156. Springer, Heidelberg (1997)Google Scholar
  8. 8.
    Figueroa, P., Green, M., Hoover, H.J.: InTml: A description language for VR applications. In: Proceedings of Web3D 2002, Arizona, USA, pp. 53–58 (2002)Google Scholar
  9. 9.
    Dragicevic, P., Fekete, J.D.: Support for input adaptability in the ICON toolkit. In (ICMI 2004). Proceedings of the 6th international conference on multimodal interfaces, State College, PA, USA, pp. 212–219 (2004)Google Scholar
  10. 10.
    Wingrave, C., Bowman, D.: CHASM: Bridging description and implementation of 3D interfaces. In: New Directions in 3D User Interfaces Workshop in IEEE Virtual Reality, Bonn, Germany, March 12, 2005 (2005)Google Scholar
  11. 11.
    Dey, A.K.: Providing Architectural Support for Building Context-Aware Applications. PhD thesis, College of Computing, Georgia Institute of Technology (2000)Google Scholar
  12. 12.
    Preuveneers, D., Van den Bergh, J., Wagelaar, D., Georges, A., Rigole, P., Clerckx, T., Berbers, Y., Coninx, K., Jonckers, V., Bosschere, K.D.: Towards an Extensible Context Ontology for Ambient Intelligence. In: Markopoulos, P., Eggen, B., Aarts, E., Crowley, J.L. (eds.) EUSAI 2004. LNCS, vol. 3295, pp. 148–159. Springer, Heidelberg (2004)Google Scholar
  13. 13.
    Clerckx, T., Van den Bergh, J., Coninx, K.: Modeling Multi-Level Context Influence on the User Interface, pp. 57–61. IEEE Computer Society, Los Alamitos (2006)Google Scholar
  14. 14.
    Coutaz, J., Nigay, L., Salber, D., Blandford, A., May, J., Young, R.M.: Four easy pieces for assessing the usability of multimodal interaction: the care properties. In: IFIP Conference Proceedings, pp. 115–120. Chapman & Hall, Sydney, Australia (1995)Google Scholar
  15. 15.
    Pribeanu, C., Limbourg, Q., Vanderdonckt, J.: Task Modelling for Context-Sensitive User Interfaces. In: Johnson, C. (ed.) Interactive Systems: Design, Specification, and Verification, pp. 60–76 (2001)Google Scholar
  16. 16.
    Van den Bergh, J., Coninx, K.: Contextual concurtasktrees: Integrating dynamic contexts in task based design. In: PerCom Workshops, pp. 13–17. IEEE Computer Society, Los Alamitos (2004)Google Scholar
  17. 17.
    Paternó, F., Santoro, C.: One model, many interfaces. In: Kolski, C., Vanderdonckt, J. (eds.) Computer-Aided Design of User Interfaces III, vol.  3, pp. 143–154. Kluwer Academic, Dordrecht (2002)Google Scholar
  18. 18.
    Clerckx, T., Luyten, K., Coninx, K.: DynaMo-AID: A Design Process and a Runtime Architecture for Dynamic Model-Based User Interface Development. In: Bastide, R., Palanque, P., Roth, J. (eds.) Engineering Human Computer Interaction and Interactive Systems. LNCS, vol. 3425, pp. 77–95. Springer, Heidelberg (2005)Google Scholar
  19. 19.
    Clerckx, T., Vandervelpen, C., Coninx, K.: Task-Based Design and Runtime Support for Multimodal User Interface Distribution. In: Engineering Interactive Systems 2007: EHCI/HCSE/DSV-IS (2007)Google Scholar
  20. 20.
    De Boeck, J., Gonzalez Calleros, J.M., Coninx, K., Vanderdonckt, J.: Open issues for the development of 3d multimodal applications from an MDE perspective. In: MDDAUI workshop 2006, Genova, Italy (2006)Google Scholar
  21. 21.
    Clerckx, T., Luyten, K., Coninx, K.: Generating Context-Sensitive Multiple Device Interfaces from Design, pp. 281–294. Kluwer, Dordrecht (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Lode Vanacken
    • 1
  • Erwin Cuppens
    • 1
  • Tim Clerckx
    • 1
  • Karin Coninx
    • 1
  1. 1.Hasselt University, Expertise Centre for Digital Media (EDM), and transnationale Universiteit Limburg, Wetenschapspark 2, B-3590 DiepenbeekBelgium

Personalised recommendations