Toward a Companion Agent for the Elderly – The Methods to Estimate At-Home and Outside-Home Daily Life Activities of the Elderly Who Live Alone

  • Yuma Takeda
  • Hung-Hsuan Huang
  • Kyoji Kawagoe
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8515)


With advances in medical technology, people’s life have been extended, and there are more and more older adults isolated. If they do not maintain social life with others, they may feel loneliness and anxiety. For their mental health, it is reported effective to keep their social relationship with others, for example, the conversation with their caregivers or other elderly people. Active listening is a communication technique that the volunteer listener listens to the speaker (the elderly) carefully and attentively by confirming or asking for more details about what they heard. This helps to make the elderly feel cared and to relieve their anxiety and loneliness. This paper presents our in-progress project aiming to develop a framework of a virtual companion agent who is always with the user and can engage active listening to maintain a long-term relationship with elderly users. In order to achieve the agent’s companionship with the user for a longer period, we believe that it is essential to make the agent to understand the user as best as it can. This kind of user-fitted conversation is not addressed in previous companion agent work. The proposed approach is the acquisition of the “memory” of the user’s daily life in two situations, at-home and outside-home. In the former one, multiple Microsoft Kinect depth sensors were adopted. The depth information is integrated to detect the user’s position and posture and then to estimate the user’s daily activity. In the outside-home configuration, the prototype application is an Android smartphone application that recognizes the user’s moving status with the information from the on-board three-axis accelerometer as well as the location of the user from GPS information. These data are then used to estimate the user’s outside-home activity. All estimated daily activities are recorded in an activity history database. Both the at-home and outside-home activity estimation methods have been developed and have been evaluated in a laboratory environment with student subjects at a moderate accuracy. The interface of the companion agent is being designed with the results from human-human and human-agent (driven by the data from the human listener condition) subject experiments. After the technologies are more matured, we would like to conduct real-world experiment with elderly subjects in near future.


Resource Description Framework Dementia Patient Elderly User Active Listening Companion Agent 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Butler, R.N.: Successful aging and the role of the life review. Journal of the American Geriatrics Society 22(12), 529–535 (1974)Google Scholar
  2. 2.
    Lazarov, O., Robinson, J., Tang, Y.P., Hairston, I.S., Korade-Mirnics, Z., Lee, V.M.Y., Hersh, L.B., Sapolsky, R.M., Mirnics, K., Sisodia, S.S.: Environmental enrichment reduces Aβ levels and amyloid deposition in transgenic mice. Cell 120(5), 701–713 (2005)CrossRefGoogle Scholar
  3. 3.
    Kempermann, G., Gast, D., Gage, F.H.: Neuroplasticity in old age: Sustained fivefold induction of hippocampal neurogenesis by long-term environmental enrichment. Annals of Neurology 52(2), 135–143 (2002)Google Scholar
  4. 4.
    Bickmore, T.W., Picard, R.W.: Towards caring machines. In: Proceeding CHI EA 2004 CHI 2004 Extended Abstracts on Human Factors in Computing Systems, pp. 1489–1492 (April 2004)Google Scholar
  5. 5.
    Heerink, M., Kröse, B., Evers, V., Wielinga, B.: Studying the acceptance of a robotic agent by elderly users. International Journal of ARM 7(3), 33–43 (2006)Google Scholar
  6. 6.
    Cassell, J.: Nudge Nudge Wink Wink: Elements of Face-to-Face Conversation for Embodied Conversational Agents. In: Embodied Conversational Agents, pp. 1–27. The MIT Press (2000)Google Scholar
  7. 7.
    Kanoh, M., Oida, Y., Nomura, Y., Araki, A., Konagaya, Y., Ihara, K., Shimizu, T., Kimura, K.: Examination of practicability of communication robot-assisted activity program for elderly people. Journal of Robotics and Mechatronics 23(1), 3–12 (2011)Google Scholar
  8. 8.
    Leite, I., Mascarenhas, S., Pereira, A., Martinho, C., Prada, R., Paiva, A.: ”Why can’t we be friends?” an empathic game companion for long-term interaction. In: Safonova, A. (ed.) IVA 2010. LNCS, vol. 6356, pp. 315–321. Springer, Heidelberg (2010)Google Scholar
  9. 9.
    Smith, C., Crook, N., Boye, J., Charlton, D., Dobnik, S., Pizzi, D., Cavazza, M., Pulman, S., de la Camara, R.S., Turunen, M.: Interaction strategies for an affective conversational agent. In: Safonova, A. (ed.) IVA 2010. LNCS, vol. 6356, pp. 301–314. Springer, Heidelberg (2010)Google Scholar
  10. 10.
    Lim, M.Y., Aylett, R., Ho, W.C., Enz, S., Vargas, P.: A socially-aware memory for companion agents. In: Ruttkay, Z., Kipp, M., Nijholt, A., Vilhjálmsson, H.H. (eds.) IVA 2009. LNCS, vol. 5773, pp. 20–26. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  11. 11.
    Campos, J., Paiva, A.: May: My memories are yours. In: Safonova, A. (ed.) IVA 2010. LNCS, vol. 6356, pp. 406–412. Springer, Heidelberg (2010)Google Scholar
  12. 12.
    Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The weka data mining software: An update. ACM SIGKDD Explorations 11(1), 11–18 (2009)CrossRefGoogle Scholar
  13. 13.
    Ogawa, K., Ono, T.: ITACO: Effects to interactions by relationships between humans and artifacts. In: Prendinger, H., Lester, J.C., Ishizuka, M. (eds.) IVA 2008. LNCS (LNAI), vol. 5208, pp. 296–307. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  14. 14.
    Sieber, G., Krenn, B.: Towards an episodic memory for companion dialogue. In: Safonova, A. (ed.) IVA 2010. LNCS, vol. 6356, pp. 322–328. Springer, Heidelberg (2010)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Yuma Takeda
    • 1
  • Hung-Hsuan Huang
    • 1
  • Kyoji Kawagoe
    • 1
  1. 1.Graduate School of Information Science & EngineeringRitsumeikan UniversityJapan

Personalised recommendations