Applications of the Multimodal Interaction Architecture in Ambient Assisted Living
Developing applications for ambient assisted living (AAL) scenarios requires dealing with diverse user groups, heterogeneous environments, and a large plethora of devices. These requirements pose several challenges on how to design and develop user interaction with the proposed applications and services. In this context, the versatility provided by multimodal interaction (MMI) is paramount and the adopted architecture should be instrumental in harnessing its full potential. This chapter offers an insight on how AAL challenges can be tackled by multimodal-based solutions. It presents the authors’ views and research outcomes in multimodal application development for AAL grounded on an architecture for MMI aligned with the W3C recommendations.
KeywordsAmbient assisted living (AAL) Multimodal interaction (MMI) Ubiquitous AAL for All Speech for interaction
The research was partially funded by IEETA Research Unit funding (Incentivo/EEI/UI0127/2014), Marie Curie IAPP project IRIS (ref. 610986, FP7-PEOPLE-2013-IAPP), project Smart Phones for Seniors (S4S), a QREN project (QREN 21541), co-funded by COMPETE and FEDER, project PaeLIFE (AAL-08-1-2001-0001), and project AAL4ALL (AAL/0015/2009).
The authors thank all W3C MMI recommendations contributors for their insightful and inspiring approaches to MMI.
- 1.Steg, H., Strese, H., Loroff, C., Hull, J., & Schmidt, S. (2006). Ambient assisted living–European overview report, March 2006. EU Specific Support Action.Google Scholar
- 2.Kung, A., & Jean-Bart, B. (2010). Making AAL platforms a reality. Ambient Intelligence, 6439, Lecture Notes in Computer Science, 187–196.Google Scholar
- 3.Mikulecký, P., Lišková, T., Čech, P., & Bureš, V. (2009). Ambient intelligence perspectives: Selected papers from the First International Ambient Intelligence Forum 2008. Amsterdam: IOS Press.Google Scholar
- 4.Hämäläinen, A., Teixeira, A., Almeida, N., Meinedo, H., Fegyó, T., & Dias, M. S. (2015). Multilingual speech recognition for the elderly: The AALFred personal life assistant. Proceedings of the 6th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Infoexclusion (DSAI 2015) June 10–12, 2015 Fraunhofer FIT, Sankt Augustin, GermanyGoogle Scholar
- 5.Teixeira, A., Ferreira, F., Almeida, N., Silva, S., Rosa, A. F., Pereira, J. C., et al. (2016). Design and development of Medication Assistant: older adults centred design to go beyond simple medication reminders. Universal Access in the Information Society.Google Scholar
- 12.Feki, M. A., Kawsar, F., Boussard, M., & Trappeniers, L. (2013). The internet of things: The next technological revolution. Computer (Long Beach California), 46(2), 24–25.Google Scholar
- 13.Gomes, B., Muniz, L., da Silva e Silva, F. J., Ríos, L. E. T., & Endler, M. (2015). A comprehensive cloud-based IoT software infrastructure for ambient assisted living. In 2015 International Conference on Cloud Technologies and Applications (CloudTech), pp. 1–8.Google Scholar
- 14.Moschetti, A., Fiorini, L., Aquilano, M., Cavallo, F., & Dario, P. (2014). Preliminary findings of the AALIANCE2 ambient assisted living roadmap. In S. Longhi, P. Siciliano, M. Germani, & A. Monteriù (Eds.), Ambient assisted living: Italian forum 2013 (pp. 335–342). Cham: Springer.CrossRefGoogle Scholar
- 15.Teixeira, A. (2014). A critical analysis of speech-based interaction in healthcare robots: Making a case for the increased use of speech in medical and assistive robots. In A. Neustein (Ed.), Speech and automata in health care (pp. 1–29). Boston: De Gruyter.Google Scholar
- 16.Teixeira, A., Braga, D., Coelho, L., & Fonseca, A. (2009). Speech as the basic interface for assistive technology. In International Conference on Software Development for Enhancing Accessibility and Fighting Info-Exclusion. Lisbon: UTAD.Google Scholar
- 19.Gövercin, M., Meyer, S., Schellenbach, M., Steinhagen-Thiessen, E., Weiss, B., & Haesner, M. (2016). SmartSenior@home: Acceptance of an integrated ambient assisted living system. Results of a clinical field trial in 35 households. Informatics for Health and Social Care, 41(4), 430–447.CrossRefGoogle Scholar
- 20.Teixeira, A., Almeida, N., Pereira, C., Silva, M. O., & Pereira, J. C. (2013). Serviços de Suporte à Interação Multimodal. In A. Teixeira, A. Queirós, & N. Rocha (Eds.), Laboratório Vivo de Usabilidade (pp. 151–165). Portugal: ARC Publishing.Google Scholar
- 21.Teixeira, A., Pereira, C., e Silva, M. O., Pacheco, O., Neves, A., & Casimiro, J. (2011). AdaptO—Adaptive multimodal output. In Proceedings of PECCS. Vila Moura, Algarve: INSTICC.Google Scholar
- 22.Teixeira, A., Pereira, C., e Silva, M. O., Alvarelhão, J., Silva, A., Cerqueira, M., et al. (2013). New telerehabilitation services for the elderly. In M. M. Cruz-Cunha (Ed.), Handbook of research on ICTs for healthcare and social services: Developments and applications (pp. 109–132). IGI Global, Hershey, Pennsylvania.Google Scholar
- 23.Johnston, M. (2009). Building multimodal applications with EMMA. In Proceedings of the 2009 International Conference on Multimodal Interfaces—ICMI-MLMI’09, Cambridge, MA, USA (p. 47).Google Scholar
- 24.Baggia, P., Burnett, D. C., Carter, J., Dahl, D. A., McCobb, G., & Raggett, D. EMMA: Extensible MultiModal Annotation markup language. W3C Recommendation. http://www.w3.org/TR/emma/. Accessed 10 Feb 2009.
- 26.Teixeira, A., Francisco, P., Almeida, N., Pereira, C., & Silva, S. (2014). Services to support use and development of speech input for multilingual multimodal applications for mobile scenarios. In The Ninth International Conference on Internet and Web Applications and Services (ICIW 2014), Track WSSA—Web Services-based Systems and Applications. IARIA, Paris.Google Scholar
- 27.Almeida, N., Teixeira, A., Rosa, A. F., Braga, D., Freitas, J., Dias, M. S., et al. (2015). Giving voices to multimodal applications. In M. Kurosu (Ed.), Human–computer interaction: Interaction technologies is a LNCS volume of the Proceedings of the 17th International Conference HCI International Los Angeles, CA, USA, August 2–7, 2015 (pp. 273–283). Cham: Springer.Google Scholar
- 28.World Health Organization (2002). Active ageing: A policy framework. A contribution of the World Health Organization to the Second United Nations World Assembly on Ageing, Madrid, Spain, April 2002, WHO/NMH/NPH/02.8. http://www.who.int/ageing/publications/active_ageing/en/
- 30.Burdea, G. (2002). Keynote address: Virtual rehabilitation-benefits and challenges. In 1st International Workshop on Virtual Reality Rehabilitation (Mental Health, Neurological, Physical, Vocational) VRMHR, Lausanne, Switzerland (pp. 1–11). Vila Moura, Algarve, INSTICC Portugal.Google Scholar
- 32.Teixeira, A., Pereira, C., Oliveira e Silva, M., Almeida, N., Pinto, J., Teixeira, C., et al. (2012). Health@home scenario: Creating a new support system for home telerehabilitation. In Proceedings of the 2nd International Living Usability Lab Workshop on AAL Latest Solutions, Trends and Applications, AAL 2012, in Conjunction with BIOSTEC 2012 (pp. 37–47). INSTICC: Vila Moura.Google Scholar
- 33.Pereira, C., Almeida, N., Martins, A. I., Silva, S., Rosa, A. F., Silva, M. O., et al. (2015). Evaluation of complex distributed multimodal applications evaluating a telerehabilitation system when it really matters. In J. Zhou & G. Salvendy (Eds.), Human aspects of IT for the aged population: Design for everyday life (part II) (Proceedings of the 1st International Conference on Human Aspects of IT for the Aged Population, part of HCI International 2015). Los Angeles, CA: Springer. published as LNCS volume 9194.Google Scholar
- 34.Vieira, D. (2015). Enhanced multimodal interaction framework and applications. Master thesis, Universidade de Aveiro, Aveiro.Google Scholar
- 35.Vieira, D., Freitas, J. D., Acartürk, C., Teixeira, A., Sousa, L., Silva, S., et al. (2015). Read that article: Exploring synergies between gaze and speech interaction. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers and Accessibility (pp. 341–342). New York, NY.Google Scholar