The CaMeLi Framework—A Multimodal Virtual Companion for Older Adults

  • Christiana Tsiourti
  • João Quintas
  • Maher Ben-Moussa
  • Sten Hanke
  • Niels Alexander Nijdam
  • Dimitri Konstantas
Conference paper
First Online:
Part of the Studies in Computational Intelligence book series (SCI, volume 751)

Abstract

Artificial Social Companions are a promising solution for the increasing challenges in elderly care. This chapter describes the CaMeLi autonomous conversational agent system which simulates human-like affective behaviour and acts as a companion for older adults living alone at home. The agent employs synthetic speech, gaze, facial expressions, and gestures to support multimodal natural interaction with its users and assists them in a number of daily life scenarios.We present the agent’s overall architecture, with a focus on the perception, decision making and synthesis components which give rise to the agent’s intelligent affective behavior. The agent was evaluated in an exploratory study where it was introduced in 20 homes of older adults (aged 65+) in three European countries (Switzerland, the Netherlands, Portugal) for a total duration of 12 weeks. We present the results of the evaluation study with regards to acceptance, perceived usability, and usefulness of the agent, and discuss future opportunities for fellow researchers who are striving to bring virtual agents out of the laboratories into successful real world applications.

This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

This work was supported by the European research projects CaMeLi (Grant No. 010000-2012-16), Miraculous Life (Grant No. 616421), Vizier (Grant No. AAL-2015-2-145), GrowMeUp (Grant No. 643647) and GEO-SAFE (Grant No. 691161). We express our gratitude to all the study participants and to all the project partners, including SIEMENS AG and Noldus IT who led the design of CaMeLi’s GUI and the end-users VIVA and Zuyderland who performed tremendous work in supporting the user studies and trials.

References

  1. 1.
    Aldoma, A., Tombari, F., Di Stefano, L., Vincze, M.: A Global Hypotheses Verification Method for 3D Object Recognition, In: ECCV, pp. 511–524 (2012)Google Scholar
  2. 2.
    Austin, J.T., Vancouver, J.B.: Goal constructs in psychology: structure, process, and content. Psychol. Bull. 120(3), 338–375 (1996)Google Scholar
  3. 3.
    Ben Moussa, M., Magnenat-Thalmann, N.: Toward socially responsible agents: integrating attachment and learning in emotional decision-making. Comput. Animat. Virtual Worlds. vol. 24, no. 34, pp. 327–334 (2013)Google Scholar
  4. 4.
    Bickmore, T.W., Caruso, L., Clough-Gorr, K., Heeren, T.: Its just like you talk to a friend relational agents for older adults. Interact. Comput. 17(6), 711–735 (2005)Google Scholar
  5. 5.
    Bickmore, T.W., Gruber, A., Picard, R.W.: Establishing the computer-patient working alliance in automated health behavior change interventions. Patient Educ. Couns. 59(1), 2130 (2005)Google Scholar
  6. 6.
    Bickmore, T.W., Silliman, R.A., Nelson, K., Cheng, M., Winter, M., Henault, L., Paasche-Orlow, M.K.: A randomized controlled trial of an automated exercise coach for older adults. J. Am. Geriatr. Soc. 61(10), 1676–1683 (2013)Google Scholar
  7. 7.
    Bickmore, T.W.: Relational agents: effecting change through human-computer relationships. Massachusetts Institute of Technology (2003)Google Scholar
  8. 8.
    Bickmore, T.W., Schulman, D., Sidner, C.L.: A reusable framework for health counseling dialogue systems based on a behavioral medicine ontology. J. Biomed. Inform. 44(2), 183197 (2011)CrossRefGoogle Scholar
  9. 9.
    Brooke, J.: SUS —a quick and dirty usability scale. Usability Eval. Ind. 189(194), 47 (1996)Google Scholar
  10. 10.
    Cassell, J., Bickmore, T., Campbell, L., Vilhjlmsson, H., Yan, H.: Human conversation as a system framework: designing embodied conversational agents. 29–63 (2001)Google Scholar
  11. 11.
    Cassell, J.: More than just another pretty face: embodied conversational interface agents. Commun. 43, 70–78. ACM (2000)Google Scholar
  12. 12.
    Colby, B.N., Ortony, A., Clore, G.L., Collins, A.: The Cognitive Structure of Emotions. Contemp. Sociol. 18(6), 957 (1989)Google Scholar
  13. 13.
    Dantas, C., Jegundo, A.L., Quintas, J., Martins, A.I., Queirós, A., Rocha, N.P.: European portuguese validation of usefulness, satisfaction and ease of use questionnaire (USE). In: World Conference on Information Systems and Technologies, pp. 561–570 (2017)Google Scholar
  14. 14.
    de Rosis, F., Novielli, N., Carofiglio, V., Cavalluzzi, A., De Carolis, B.: User modeling and adaptation in health promotion dialogs with an animated character. J. Biomed. Inform. 39(5), 51431 (2006)CrossRefGoogle Scholar
  15. 15.
    Ekman, P., Keltner, D.: Universal facial expressions of emotion. Calif. Ment. Heal. Res. Dig. 8(4), 151–158 (1970)Google Scholar
  16. 16.
    Fagel, S., Ben-Moussa, M., Cereghetti, D.: How avatars in care context should show affect. In: Pervasive Health 2016 Workshop on Affective Interaction with Virtual Assistants within the Healthcare Context (2016)Google Scholar
  17. 17.
    Group, T.W.: Development of the world health organization WHOQOL-BREF quality of life assessment. The WHOQOL Group. Psychol. Med. 28(3), 551–558 (1998)Google Scholar
  18. 18.
    Hanke, S., Tsiourti, C., Sili, M., Christodoulou, E.: Embodied ambient intelligent systems. in ambient intelligence and smart environments: recent advances in ambient assisted living, bridging assistive technologies, e-health and personalized health care, vol. 20, pp. 65–85. IOS Press (2015)Google Scholar
  19. 19.
    Kasap, Z., Magnenat-Thalmann, N.: Building long-term relationships with virtual and robotic characters: the role of remembering. Vis. Comput. 28(1), 87–97 (2012)Google Scholar
  20. 20.
    Kelley, J.F.: An iterative design methodology for user-friendly natural language office information applications. ACM Trans. Inf. Syst. 2(1), 26–41 (1984)Google Scholar
  21. 21.
    Khoshhal, K., Aliakbarpour, H., Quintas, J., Drews, P., Dias, J.: Probabilistic LMA-based classification of human behaviour understanding using power spectrum technique. In: 2010 13th International Conference of Information Fusion, pp. 17 (2010)Google Scholar
  22. 22.
    Klein, J., Moon, Y., Picard, R.W.: This computer responds to user frustration—theory, design, results and implications. In: Proceedings of CHI 99 Extended Abstracts on Human Factors in Computing Systems, pp. 242 (1999)Google Scholar
  23. 23.
    Kramer, M., Yaghoubzadeh, R., Kopp, S., Pitsch, K.: A conversational virtual human as autonomous assistant for elderly and cognitively impaired users?. Social acceptability and design considerations. Lect. Notes Inf. (2013)Google Scholar
  24. 24.
    Lim, M.Y.: Memory models for intelligent social companions. Human-Computer Interaction: The Agency Perspective, pp. 241–262. Springer (2012)Google Scholar
  25. 25.
    Lisetti, C., Yasavur, U., de Leon, C., Amini, R., Rishe, N., Visserv, U.: Building an on-demand avatar-based health intervention for behavior change. In: Proceedings of Twenty-Fifth International Florida Intelligence Research Soceity Conference, no. Mi, pp. 175–180 (2012)Google Scholar
  26. 26.
    Nijholt, A.: Disappearing computers, social actors and embodied agents. In: Proceedings of the International Conference on Cyberworlds, pp. 128–134 (2003)Google Scholar
  27. 27.
    Ortiz, A., Carretero, P., Oyarzun, D., Yanguas, J.J., Buiza, C., Gonzalez, M.F., Etxeberria, I.: Elderly users in ambient intelligence : does an avatar improve the interaction? Intelligence 4397, 99–114 (2002)Google Scholar
  28. 28.
    Quintas, J., Paulo, M., Jorge, D.: Information model and architecture specification for context awareness interaction decision support in cyber-physical human-machine systems. IEEE Trans. Hum. Mach. Syst. 47(3), 323–331 (2016)Google Scholar
  29. 29.
    Rusu, R.B., Cousins, S.: 3D is here: point cloud library. In: IEEE International Conference on Robotics and Automation, pp. 1–4 (2011)Google Scholar
  30. 30.
    Sakai, Y., Nonaka, Y., Yasuda, K., Nakano, Y.I.: Listener agent for elderly people with dementia. In: 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 199–200 (2012)Google Scholar
  31. 31.
    Thiebaux, M., Marsella, S., Marshall, A.N., Kallmann, M.: SmartBody: behavior realization for embodied conversational agents. In: Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS ’08), pp. 151–158 (2008)Google Scholar
  32. 32.
    Traum, D., Larsson, S.: The information state approach to dialogue management. Current and New Directions in Discourse and Dialogue, pp. 325–353 (2003)Google Scholar
  33. 33.
    Tsiourti, C., Ben Moussa, M., Joly, E., Wings, C., Wac, K.: Virtual assistive companions for older adults: qualitative field study and design implications. In: 8th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) (2014)Google Scholar
  34. 34.
    Tsiourti, C., Ben-Moussa, M., Quintas, J., Loke, B., Jochem, I., Lopes, J.A., Konstantas, D.: A virtual assistive companion for older adults: design and evaluation of a real-world application. In: Proceedings of SAI Intelligent Systems Conference 2016, London, UK (2016)Google Scholar
  35. 35.
    Van Kuilenburg, H., Wiering, M., Den Uyl, M.: A model based method for automatic facial expression recognition. In: Proceeding of Machine Learning: ECML 2005, pp. 194–205. Springer (2005)Google Scholar
  36. 36.
    Vardoulakis, L.P., Ring, L., Barry, B., Sidner, C.L., Bickmore, T.: Designing relational agents as long term social companions for older adults. In: Proceedings of the 12th International Conference on Intelligent Virtual Agents, vol. 7502, pp. 289–302 (2012)Google Scholar
  37. 37.
    Verberne, F.M.F., Ham, J., Ponnada, A., Midden, C.J.H.: Trusting digital chameleons: the effect of mimicry by a virtual social agent on user trust. In: Berkovsky, S., Freyne, J. (eds.) Persuasive Technology, vol. 7822, pp. 234–245, Springer, Berlin, Heidelberg (2013)Google Scholar
  38. 38.
    Woelfel, M., McDonough, J.: Distant Speech Recognition. Wiley (2009)Google Scholar
  39. 39.
    Yaghoubzadeh, R., Kramer, M., Pitsch, K., Kopp, S.: Virtual Agents as daily assistants for elderly or cognitively impaired people. In: Proceedings 13th International Conference of Intelligence Virtual Agents, vol. 8108, pp. 91 (2013)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Christiana Tsiourti
    • 1
  • João Quintas
    • 2
  • Maher Ben-Moussa
    • 1
  • Sten Hanke
    • 3
  • Niels Alexander Nijdam
    • 1
  • Dimitri Konstantas
    • 1
  1. 1.University of GenevaGenevaSwitzerland
  2. 2.Instituto Pedro NunesCoimbraPortugal
  3. 3.Austrian Institue of TechnologyViennaAustria

Personalised recommendations