Multimodality for Plastic User Interfaces: Models, Methods, and Principles

Abstract

In this chapter, we bring together the main results from the developments in plastic user interfaces (UI) and in multimodal UIs using a theoretic and conceptual perspective as a unifying approach. In particular, we provide an operational model for the notion of modality as well as for the combination of multiple modalities in terms of the CARE properties. Then, we clarify the problem space of UI plasticity and show how multimodality can be used as a means for UI adaptation. To address the challenges raised by the development of plastic multimodal UIs, we propose three principles that have been applied to several proof-of-concept exemplars: (1) Blur the distinction between design-time and run-time, (2) Mix close and open adaptiveness and (3) Keep humans in the loop.