Skip to main content
SpringerLink
Log in

User Interaction Adaptation Within Ambient Environments

  • Chapter
Next Generation Intelligent Environments

Abstract

Ambient environments introduce new user interaction issues. The interaction environment which was static and closed becomes open, heterogeneous and dynamic. The variety of users, devices and physical environments leads to a more complex interaction context. As a consequence, the user interface has to adapt itself to preserve its utility and usability. It is no longer reasonable to continue to propose static and rigid interfaces while users, systems and environments are more and more diversified. To the dynamic nature of the interaction context introduced by ambient environments, the user interface must also respond by a dynamic adaptation. Thanks to the interaction richness it can offer, multimodality represents an interesting solution to this adaptation problem. The objective is to exploit all the interaction capabilities available to the system at a given moment, to instantiate and evolve user interfaces. In this chapter, we start by presenting a survey of the state of the art on user interaction adaptation . After, discussing the limitations of the existing approaches, we present our proposals to achieve user interaction adaptation within ambient environments. Then we describe the derived software architecture and the user evaluation it led to. We conclude by some directions for future work.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The model we propose in following sections was designed to take into account the global vision and will be expanded to include such plasticity in future versions.

  2. 2.

    Wiimote is a sensor for video games that was designed by Nintendo in 2005. It is a remote control that includes vibrators, orientation sensors and screen pointing capabilities.

  3. 3.

    http://iroom.limsi.fr/atraco/InteractionOntology.owl.

  4. 4.

    More details can also be found on the official jess website http://www.jessrules.com.

  5. 5.

    That is the bundle of code to be executed within a specific mediator.

  6. 6.

    In our implementation, available OIOs and styles are statically specified in the Interaction Ontology, but they could also be specified and retrieved from a remote library if necessary.

  7. 7.

    We assume here that such techniques exist. We showed in Sect. 3.4 that even though it is the case, the reliability of those techniques is variable. Our model is thus flexible to the extent that third-party ontologies that describe ecology devices are of a sufficient quality to allow alignment.

References

  1. André, E.: The generation of multimedia presentations. In: Handbook of Natural Language Processing, pp. 305–327. Marcel Dekker Inc, Basel (2000)

    Google Scholar 

  2. Appert, C., Huot, S., Dragicevic, P., Beaudouin-Lafon, M.: Flowstates: prototypage d’application interactives avec des flots de donnees et des machines a etats. In: Proceedings of 21eme Conference Francophone sur l’Interaction Homme-Machine, IHM 2009, pp. 119–128. ACM Press, New York (2009)

    Google Scholar 

  3. Athanasopoulos, D., Zarras, A., Issarny, V., Pitoura, E., Vassiliadis, P.: CoWSAMI: interface-aware context gathering in ambient intelligence environments. Pervasive Mob. Comput. 4(3), 360–389 (2008)

    Article  Google Scholar 

  4. Balme, L., Demeure, A., Barralon, N., Coutaz, J., Calvary, G., et al.: Cameleon-rt: a software architecture reference model for distributed, migratable, and plastic user interfaces. In: Ambient Intelligence. Lecture Notes in Computer Science, vol. 3295, pp. 291–302, Springer, Berlin (2004)

    Google Scholar 

  5. Bass, L., Faneuf, R., Little, R., Mayer, N., Pellegrino, B., Reed, S., Seacord, R., Sheppard, S., Szczur, M.: A metamodel for the runtime architecture of an interactive system. ACM SIGCHI Bull. 24(1), 32–37 (1992)

    Article  Google Scholar 

  6. Bellik, Y.: Interfaces multimodales: concepts, modèles et architectures. Ph.D. Thesis, Université d’Orsay Paris-Sud (1995)

    Google Scholar 

  7. Bellik, Y., Rebai, I., Machrouh, E., Barzaj, Y., Jacquet, C., Pruvost, G., Sansonnet, J.: Multimodal interaction within ambient environments: an exploratory study. In: Human-Computer Interaction, INTERACT’09: IFIP International Conference on Human-Computer Interaction; Uppsala, Sweden (2009)

    Google Scholar 

  8. Bernsen, N.: Modality Theory in support of multimodal interface design. In: Proceedings of Intelligent Multi-Media Multi-Modal Systems, pp. 37–44 (1994)

    Google Scholar 

  9. Berti, S., Paternó, F.: Migratory multimodal interfaces in multidevice environments. In: Proceedings of the International Conference on Multimodal Interfaces, pp. 92–99. ACM Press, New York (2005)

    Google Scholar 

  10. Bohus, D., Rudnicky, A.I.: The ravenclaw dialog management framework: architecture and systems. Comput. Speech Lang. 23, 332–361 (2009)

    Article  Google Scholar 

  11. Bordegoni, M., Faconti, G., Maybury, M.T., Rist, T., Ruggieri, S., Trahanias, P., Wilson, M.: A standard reference model for intelligent multimedia presentation systems. Comput. Stand. Interfaces 18(6), 477–496 (1997)

    Article  Google Scholar 

  12. Braffort, A., Choisier, A., Collet, C., Dalle, P., Gianni, F., Lenseigne, B., Segouat, J.: Toward an annotation software for video of Sign Language, including image processing tools and signing space modelling. In: 4th International Conference on Language Resources and Evaluation, held in Lisbon, Portugal (2004)

    Google Scholar 

  13. Browne, D., Totterdell, P., Norman, M.: Adaptive User Interfaces. Academic Press, London (1990)

    Google Scholar 

  14. Brusilovsky, P.: Adaptive hypermedia. User Model. User-Adap. Inter. 11, 87–110 (2001)

    Article  MATH  Google Scholar 

  15. Calvary, G., Coutaz, J., Thevenin, D., Limbourg, Q., Bouillon, L., Vanderdonckt, J.: A unifying reference framework for multi-target user interfaces. J. Interacting with computers 15/3, 289–308 (2003). Elsevier Science B.V

    Google Scholar 

  16. Chikofsky, E., Cross II, J.: Reverse engineering and design recovery: a taxonomy. IEEE Softw. 7(1), 13–17 (1990)

    Article  Google Scholar 

  17. Coutaz, J.: Meta-user interfaces for ambient spaces. In: Task Models and Diagrams for Users Interface Design. Lecture Notes in Computer Science, vol. 4385, pp. 1–15. Springer, Berlin (2007)

    Google Scholar 

  18. Coutaz, J., Nigay, L., Salber, D., Blandford, A., May, J., Young, R.: Four easy pieces for assessing the usability of multimodal interaction: the CARE properties. In: Proceedings of INTERACT, vol. 95, pp. 115–120 (1995)

    Google Scholar 

  19. Dey, A.K.: Providing architectural support for building context-aware applications. Ph.D. Thesis, Georgia Institute of Technology (2000)

    Google Scholar 

  20. Dieterich, H., Malinowski, U., Kühme, T., Schneider-Hufschmidt, M.: State of the art in adaptive user interfaces. Hum. Factors Inform. Technol. 10, 13–13 (1993)

    Google Scholar 

  21. Dragicevic, P., Fekete, J.: Input device selection and interaction configuration with ICON. In: People and Computers, pp. 543–558. Springer, London (2001)

    Google Scholar 

  22. Duarte, C., Carriço, L.: A conceptual framework for developing adaptive multimodal applications. In: Proceedings of the 11th International Conference on Intelligent User Interfaces, pp. 132–139. ACM Press, New York (2006)

    Google Scholar 

  23. Eisenstein, J., Vanderdonckt, J., Puerta, A.: Applying model-based techniques to the development of UIs for mobile computers. In: Proceedings of the 6th International Conference on Intelligent User Interfaces, pp. 69–76. ACM Press Publication, Santa Fe, New Mexico, USA (2001)

    Google Scholar 

  24. Florins, M., Trevisan, D., Vanderdonckt, J.: The continuity property in mixed reality and multiplatform systems: a comparative study. In: Proceedings of CADUI’04, pp. 13–16, Madeira Island (2004)

    Google Scholar 

  25. Forgy, C.: Rete: a fast algorithm for the many pattern/many object pattern match problem* 1. Artif. Intell. 19(1), 17–37 (1982)

    Article  Google Scholar 

  26. Frasincar, F., Houben, G.J.: Hypermedia presentation adaptation on the semantic web. In: Proceedings of the 2nd International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems, LNCS 2347, pp. 133–142. Malaga, Spain (2002)

    Google Scholar 

  27. Friedman, E.: Jess in action: rule-based systems in java. Manning Publications Co., Greenwich, CT, USA (2003)

    Google Scholar 

  28. Frohlich, D.M.: The design space of interfaces, multimedia systems, interaction and applications. In: Proceedings of 1st Eurographics Workshop, held in Stockholm, Sweden (1991)

    Google Scholar 

  29. Gram, C., Cockton, G.: Design Principles for Interactive Software (IFIP International Federation for Information Processing). Chapman & Hall Ltd, London (1996)

    Google Scholar 

  30. Hagras, H., Goumopoulos, C., Bellik, Y., Minker, W., Meliones, A.: Research Results on Adaptation & Evolution. Technical Report, 7th Framework program, ATRACO Project Report - Deliverable D13 (2011)

    Google Scholar 

  31. Heinroth, T., Denich, D., Schmitt, A.: Owlspeak - adaptive spoken dialogue within intelligent environments. In: 8th IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops), pp. 666–671. Mannheim (2010). http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5470518

  32. Henricksen, K., Indulska, J., Rakotonirainy, A.: Modeling context information in pervasive computing systems. Pervasive Comput. 2414, 79–117 (2002)

    Google Scholar 

  33. Horrocks, I., Patel-Schneider, P., Boley, H., Tabet, S., Grosof, B., Dean, M.: SWRL: a semantic web rule language combining OWL and RuleML. W3C Member submission 21 (2004). http://www.w3.org/Submission/2004/03/

  34. Jaimes, A., Sebe, N.: Multimodal human-computer interaction: a survey. Comput. Vis. Image Underst. 108(1–2), 116–134 (2007)

    Article  Google Scholar 

  35. Johnston, M., Bangalore, S.: Finite-state multimodal integration and understanding. Nat. Lang. Eng. 11(02), 159–187 (2005)

    Article  Google Scholar 

  36. Kobsa, A., Koenemann, J., Pohl, W.: Personalised hypermedia presentation techniques for improving online customer relationships. Knowl. Eng. Rev. 16(2), 111–155 (2001)

    Article  MATH  Google Scholar 

  37. Kolski, C., Le Strugeon, E.: A review of intelligent human-machine interfaces in the lights of the arch model. Int. J. Hum. Comput. Interact. 10(3), 193–231 (1998)

    Article  Google Scholar 

  38. Kolski, C., Forbrig, P., David, B., Girard, P., Tran, C., Ezzedine, H.: Agent-based architecture for interactive system design: current approaches, perspectives and evaluation. In: Jacko, J.A. (ed.) Human-Computer Interaction. New Trends, Lecture Notes in Computer Science, vol. 5610, pp. 624–633. Springer, Berlin (2009). doi:10.1007/978-3-642-02574-7_70. http://dx.doi.org/10.1007/978-3-642-02574-7_70

    Google Scholar 

  39. Kurkovsky, S.: Multimodality in Mobile Computing and Mobile Devices: Methods for Adaptable Usability. IGI Global, 701 E. Chocolate Avenue Hershey, PA 17033, USA (2010)

    Google Scholar 

  40. Landragin, F.: Physical, semantic and pragmatic levels for multimodal fusion and fission (2007). Accessible on HAL server: http://hal.archives-ouvertes.fr

  41. Larsson, S., Traum, D.: Information state and dialogue management in the TRINDI Dialogue Move Engine Toolkit. Nat. Lang. Eng. (Special Issue) 6(3), 323–340 (2000)

    Google Scholar 

  42. Lee, G.G., Mariani, J., Minker, W. (eds.): Spoken Dialogue Systems for Ambient Environments. Lecture Notes in Computer Science, vol. 6392. Springer, Heidelberg (2010)

    Google Scholar 

  43. Limbourg, Q., Vanderdonckt, J., Michotte, B., Bouillon, L., López-Jaquero, V.: Usixml: a language supporting multi-path development of user interfaces. In: Engineering Human Computer Interaction and Interactive Systems, pp. 200–220. Springer, Berlin (2005)

    Google Scholar 

  44. Mori, G., Paternò, F., Santoro, C.: Tool support for designing nomadic applications. In: Proceedings of the 8th International Conference on Intelligent User Interfaces, pp. 141–148. ACM Press, New York (2003)

    Google Scholar 

  45. Mori, G., Paternó, F., Santoro, C.: Design and development of multidevice user interfaces through multiple logical descriptions. IEEE Trans. Softw. Eng. 30(8), 507–520 (2004)

    Article  Google Scholar 

  46. Navarre, D., Palanque, P., Bastide, R., Schyn, A., Winckler, M., Nedel, L., Freitas, C.: A formal description of multimodal interaction techniques for immersive virtual reality applications. In: Human-Computer Interaction-INTERACT 2005, pp. 170–183, Springer, Berlin (2005)

    Google Scholar 

  47. Nigay, L., Coutaz, J.: A generic platform for addressing the multimodal challenge. In: Proceedings of the Conference on Human Factors in Computing Systems, CHI’95 held in Denver, Colorado, USA (1995)

    Google Scholar 

  48. Oshry, M., Auburn, R., Baggia, P., Bodell, M., Burke, D., Burnett, D., Candell, E., Carter, J., Mcglashan, S., Lee, A., Porter, B., Rehor, K.: Voice extensible markup language (voicexml) version 2.1. Technical Report, W3C – Voice Browser Working Group (2007)

    Google Scholar 

  49. Padovitz, A., Loke, S., Zaslavsky, A.: The ECORA framework: a hybrid architecture for context-oriented pervasive computing. Pervasive Mob. Comput. 4(2), 182–215 (2008)

    Article  Google Scholar 

  50. Paganelli, L., Paternó, F.: Automatic reconstruction of the underlying interaction design of web applications. In: Proceedings of SEKE 2002. Ischia, Italy (2002)

    Google Scholar 

  51. Paternó, F., Mancini, C., Meniconi, S.: Concurtasktrees: A diagrammatic notation for specifying task models. In: INTERACT ’97: Proceedings of the IFIP TC13 International Conference on Human-Computer Interaction, pp. 362–369. Chapman & Hall, Ltd (1997)

    Google Scholar 

  52. Potel, M.: MVP: Model-View-Presenter The Taligent Programming Model for C++ and Java. Technical Report, Taligent Inc (1996). Available on http://www.wildcrest.com/Potel/Portfolio/mvp.pdf

  53. Reenskaug, T.: Models - views - controllers. Technical Report, Xerox PARC (1979). http://heim.ifi.uio.no/~trygver/1979/mvc-2/1979-12-MVC.pdf

  54. Riley, G.: Clips: an expert system building tool. In: NASA, Washington, Technology 2001: The Second National Technology Transfer Conference and Exposition, vol. 2 (1991)

    Google Scholar 

  55. Rist, T.: Supporting mobile users through adaptive information presentation. Multimodal Intell. Inf. Presentation 27, 113–139 (2005)

    Article  Google Scholar 

  56. Rist, T., Andre, E.: Building smart embodied virtual characters. In: Smart Graphics. Lecture Notes in Computer Science, vol. 2733, pp. 123–130. Springer, Berlin (2003)

    Google Scholar 

  57. Román, M., Hess, C., Cerqueira, R., Ranganathan, A., Campbell, R., Nahrstedt, K.: A middleware infrastructure for active spaces. IEEE Pervasive Comput. 1(4), 74–83 (2002)

    Article  Google Scholar 

  58. Rousseau, C.: Présentation multimodale et contextuelle de l’information. Ph.D. Thesis, Université d’Orsay Paris-Sud (2006)

    Google Scholar 

  59. Rousseau, C., Bellik, Y., Vernier, F., Bazalgette, D.: A framework for the intelligent multimodal presentation of information. Signal Process. 86(12), 3696–3713 (2006)

    Article  MATH  Google Scholar 

  60. Samaan, K., Tarpin-Bernard, F.: Task models and interaction models in a multiple user interfaces generation process. In: In Proceedings of TAMODIA 2004, vol. 86, pp. 137–144. ACM Press, Czech Republic (2004)

    Google Scholar 

  61. Scapin, D., Bastien, J.: Ergonomic criteria for evaluating the ergonomic quality of interactive systems. Behav. Inform. Technol. 16(4), 220–231 (1997)

    Article  Google Scholar 

  62. Serrano, M., Nigay, L., Lawson, J.Y.L., Ramsay, A., Murray-Smith, R., Denef, S.: The openinterface framework: a tool for multimodal interaction. In: CHI ’08 Extended Abstracts on Human Factors in Computing Systems, pp. 3501–3506. ACM, New York, NY (2008). doi:http://doi.acm.org/10.1145/1358628.1358881

  63. Stanciulescu, A., Limbourg, Q., Vanderdonckt, J., Michotte, B., Montero, F.: A transformational approach for multimodal web user interfaces based on UsiXML. In: Proceedings of the 7th International Conference on Multimodal Interfaces, pp. 259–266. ACM, New York, NY (2005)

    Google Scholar 

  64. Stephanidis, C., Karagiannidis, C., Koumpis, A.: Decision making in intelligent user interfaces. In: Intelligent User Interfaces, IUI’97 held in Orlando, Florida, USA, pp. 195–202 (1997)

    Google Scholar 

  65. Stephanidis, C., Paramythis, A., Sfyrakis, M., Stergiou, A., Maou, N., Leventis, A., Paparoulis, G., Karagiandidis, C.: Adaptable and adaptive user interfaces for disabled users in avanti project. In: Intelligence in Services and Networks: Technology for Ubiquitous Telecom Services. Lecture Notes In Computer Science. vol. 1430, pp. 153–16. Springer, Berlin (1998)

    Google Scholar 

  66. Stephanidis, C., Savidis, A.: Universal access in the information society: methods, tools, and interaction technologies. UAIS J. 1(1), 40–55 (2001)

    Google Scholar 

  67. Tandler, P.: The BEACH Application Model and Software Framework for Synchronous Collaboration in Ubiquitous Computing Environments. J. Syst. Softw. 69(3), 267–296 (2004)

    Article  Google Scholar 

  68. Tarpin-Bernard, F.: Interaction Homme-Machine Adaptative. Habilitation a diriger des recherches (hdr), Université Claude Bernard de Lyon (2006)

    Google Scholar 

  69. Teil, D., Bellik, Y.: The Structure of Multimodal Dialog II. Multimodal Interaction Interface Using Voice and Gesture, chap. 19, pp. 349–366. John Benjamins Publishing Co., P.O.Box 75577, 1070 AN Amsterdam, The Netherlands, John Benjamins Noth America, P.O.Box 27519, Philadelphia PA 19118-0519, USA (2000)

    Google Scholar 

  70. Thevenin, D., Coutaz, J.: Plasticity of user interfaces: framework and research agenda. In: Human-computer Interaction, INTERACT’99: IFIP TC. 13 International Conference on Human-Computer Interaction, 30th August-3rd September 1999, Edinburgh, UK, p. 110. IOS Press (1999)

    Google Scholar 

  71. Thevenin, D., Coutaz, J.: Adaptation des IHM: taxonomies et archi. logicielle. In: Proceedings of the 14th French-Speaking Conference on Human-Computer Interaction, pp. 207–210. ACM, New York, NY (2002)

    Google Scholar 

  72. van Helvert, J., Hagras, H., Kameas, A.: D27 - prototype testing and validation. Restricted deliverable, The ATRACO Project (FP7/2007-2013 grant agreement 216837) (2009)

    Google Scholar 

  73. Vanderdonckt, J., Grolaux, D., Van Roy, P., Limbourg, Q., Macq, B., Michel, B.: A design space for context-sensitive user interfaces. In: Proceedings of IASSE 2005, held in Toronto, Canada (2005)

    Google Scholar 

  74. Young, S., Williams, J., Schatzmann, J., Stuttle, M., Weilhammer, K.: D4.3: Bayes net prototype - the hidden information state dialogue manager. Technical Report, TALK - Talk and Look: Tools for Ambient Linguistic Knowledge, IST-507802, 6th FP (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Center for Scientific Research (LIMSI-CNRS), BP 133, 91403, Orsay cedex, France

    Gaëtan Pruvost & Yacine Bellik

  2. Institute of Information Technology, Ulm University, Ulm, 89081, Germany

    Tobias Heinroth & Wolfgang Minker

Authors
  1. Gaëtan Pruvost
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tobias Heinroth
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yacine Bellik
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wolfgang Minker
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yacine Bellik .

Editor information

Editors and Affiliations

  1. University of Ulm, Ulm, Germany

    Stefan Ultes

  2. University of Ulm, Ulm, Germany

    Florian Nothdurft

  3. University of Ulm, Ulm, Germany

    Tobias Heinroth

  4. University of Ulm, Ulm, Germany

    Wolfgang Minker

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Pruvost, G., Heinroth, T., Bellik, Y., Minker, W. (2016). User Interaction Adaptation Within Ambient Environments. In: Ultes, S., Nothdurft, F., Heinroth, T., Minker, W. (eds) Next Generation Intelligent Environments. Springer, Cham. https://doi.org/10.1007/978-3-319-23452-6_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-23452-6_6

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-23451-9

  • Online ISBN: 978-3-319-23452-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation