Beyond Responsive Design: Context-Dependent Multimodal Augmentation of Web Applications

  • Giuseppe Ghiani
  • Marco Manca
  • Fabio Paternò
  • Claudio Porta
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8640)


Context-dependent adaptation is becoming a continuous necessity since we access our applications in more and more variegated contexts. Multimodality can be a significant support in such changing settings. We present a solution for obtaining automatic augmentation of Web applications in such a way as to enable them to exploit various combinations of graphical and vocal modalities. We report on the software architecture supporting such augmentations and its underlying context manager, as well as some example applications and first user tests.


Adaptation Context-Awareness Multimodal Interfaces Vocal Interaction Web applications 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Avouac, P.-A., Lalanda, P., Nigay, L.: Autonomic management of multimodal interaction: DynaMo in action. In: Proc. EICS 2012, pp. 35–44. ACM (2012)Google Scholar
  2. 2.
    Bellucci, F., Ghiani, G., Paternò, F., Porta, C.: Automatic Reverse Engineering of Interactive Dynamic Web Applications to Support Adaptation across Platforms. In: Proc. IUI 2012, pp. 217–226. ACM (2012)Google Scholar
  3. 3.
    Cantera, J.M., González, J., Meixner, G., Paternò, F., Pullmann, J., Raggett, D., Schwabe, D., Vanderdonckt, J.: Model-Based UI XG Final Report,
  4. 4.
    Coelho, J., Duarte, C.: The Contribution of Multimodal Adaptation Techniques to the GUIDE Interface. In: Stephanidis, C. (ed.) Universal Access in HCI, Part I, HCII 2011. LNCS, vol. 6765, pp. 337–346. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  5. 5.
    Coutaz, J., Nigay, L., Salber, D., Blandford, A., May, J., Young, R.: Four Easy Pieces for Assessing the Usability of Multimodal Interaction: the CARE Properties. In: Proc. INTERACT 1995, pp. 115–120. Chapman and Hall (1995)Google Scholar
  6. 6.
    Díaz, O., Arellano, C., Azanza, M.: A language for end-user web augmentation: Caring for producers and consumers alike. ACM Transactions on Web (2), 9 (2013)Google Scholar
  7. 7.
    Duarte, C., Carriço, L.: A conceptual framework for developing adaptive multimodal applications. In: Proc. IUI 2006, pp. 132–139. ACM (2006)Google Scholar
  8. 8.
    Dumas, B., Solórzano, M., Signer, B.: Design Guidelines for Adaptive Multimodal Mobile Input Solutions. In: Proc. MobileHCI 2013, pp. 285–294. ACM (2013)Google Scholar
  9. 9.
    Gajos, K.Z., Hurst, A., Findlater, L.: Personalized dynamic accessibility. Interactions 19(2), 69–73 (2012)CrossRefGoogle Scholar
  10. 10.
    Jumisko-Pyykkö, S., Vainio, T.: Framing the Context of Use for Mobile HCI. International Journal of Mobile-Human-Computer-Interaction (IJMHCI) 3(4), 1–28 (2010)CrossRefGoogle Scholar
  11. 11.
    Kane, S.K., Wobbrock, J.O., Smith, I.E.: Getting off the treadmill: Evaluating walking user interfaces for mobile devices in public spaces. In: Proc. MobileHCI 2008, pp. 109–118. ACM Press (2008)Google Scholar
  12. 12.
    Kumar, R., Talton, J.O., Ahmad, S., Klemmer, S.R.: Bricolage: Example-Based Retargeting for Web Design. In: Proc. CHI 2011, pp. 2197–2206. ACM (2011)Google Scholar
  13. 13.
    Manca, M., Paternò, F., Santoro, C., Spano, L.D.: Generation of Multi-Device Adaptive MultiModal Web Applications. In: Daniel, F., Papadopoulos, G.A., Thiran, P. (eds.) MobiWIS 2013. LNCS, vol. 8093, pp. 218–232. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  14. 14.
    Marcotte, E.: Responsive Web Design, A Book Apart (2011),
  15. 15.
    Nebeling, M., Speicher, M., Norrie, M.C.: W3Touch: Metrics-based Web Page Adaptation for Touch. In: Proc. CHI 2013, pp. 2311–2320. ACM (2013)Google Scholar
  16. 16.
    Octavia, J.R., Vanacken, L., Raymaekers, C., Coninx, K., Flerackers, E.: Facilitating Adaptation in Virtual Environments Using a Context-Aware Model-Based Design Process. In: England, D., Palanque, P., Vanderdonckt, J., Wild, P.J. (eds.) TAMODIA 2009. LNCS, vol. 5963, pp. 58–71. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  17. 17.
    Paternò, F., Santoro, C., Spano, L.D.: MARIA: A Universal Language for Service-Oriented Applications in Ubiquitous Environment. ACM Transactions on Computer-Human Interaction 16(4), 19:1–19:30 (2009)Google Scholar
  18. 18.
    Peissner, M., Häbe, D., Janssen, D., Sellner, T.: MyUI: generating accessible user interfaces from multimodal design patterns. In: Proc. EICS 2012, pp. 81–90. ACM Press (2012)Google Scholar
  19. 19.
    Salber, D., Anind, D., Abowd, G.: The context toolkit: Aiding the development of context-enabled applications. In: Proc. CHI 1999 Conference, pp. 434–441. ACM (1999)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Giuseppe Ghiani
    • 1
  • Marco Manca
    • 1
  • Fabio Paternò
    • 1
  • Claudio Porta
    • 1
  1. 1.HIIS LaboratoryCNR-ISTIPisaItaly

Personalised recommendations