Multimedia Tools and Applications

, Volume 76, Issue 4, pp 5073–5094 | Cite as

Simulating empathic behavior in a social assistive robot

  • Berardina De Carolis
  • Stefano Ferilli
  • Giuseppe Palestra


When used as an interface in the context of Ambient Assisted Living (AAL), a social robot should not just provide a task-oriented support. It should also try to establish a social empathic relation with the user. To this aim, it is crucial to endow the robot with the capability of recognizing the user’s affective state and reason on it for triggering the most appropriate communicative behavior. In this paper we describe how such an affective reasoning has been implemented in the NAO robot for simulating empathic behaviors in the context of AAL. In particular, the robot is able to recognize the emotion of the user by analyzing communicative signals extracted from speech and facial expressions. The recognized emotion allows triggering the robot’s affective state and, consequently, the most appropriate empathic behavior. The robot’s empathic behaviors have been evaluated both by experts in communication and through a user study aimed at assessing the perception and interpretation of empathy by elderly users. Results are quite satisfactory and encourage us to further extend the social and affective capabilities of the robot.


Social assistive robots Affective computing Empathy 


  1. 1.
    Anderson C, Keltner D (2002) The role of empathy in the formation and maintenance of social bonds. Behav Brain Sci 25:21–22Google Scholar
  2. 2.
    Baron-Cohen S (2011) The science of evil: on empathy and the origins of cruelty. Basic Books, New York, NYGoogle Scholar
  3. 3.
    Beadle JN, Sheehan AH, Dahlben B, Gutchess AH (2013) Aging, empathy, and prosociality. J Gerontol Ser B Psychol Sci Soc Sci:1–10Google Scholar
  4. 4.
    Boersma P, Weenink D (2007) Praat: doing phonetics by computer (version 4.5.15) [computer program]. Retrieved 24.02.2007
  5. 5.
    Broekens J, Heerink M, Rosendal H (2009) Assistive social robots in elderly care: a review. Gerontechnology 8(2):94–103. doi:10.4017/gt.2009. CrossRefGoogle Scholar
  6. 6.
    Cesta A, Cortellessa G, Pecora F, Rasconi R (2007) Supporting interaction in the robocare intelligent assistive environment, AAAI Spring Symposium: Interaction Challenges for Intelligent Assistants, pp. 18–25Google Scholar
  7. 7.
    Coradeschi S, Cesta A, Cortellessa G, Coraci L, Galindo C, Gonzalez J, Karlsson L, Forsberg A, Frennert S, Furfari F, Loutfi A, Orlandini A, Palumbo F, Pecora F, von Rump S, Stimec A, Ullberg J, Ötslund B (2014) GiraffPlus: a system for monitoring activities and physiological parameters and promoting social interaction for elderly in human-computer systems interaction: backgrounds and applications 3. Adv Intell Syst Comput 300:261–271. doi:10.1007/978-3-319-08491-6_22 CrossRefGoogle Scholar
  8. 8.
    Cramer H, Goddijn J, Wielinga B, Evers V (2010) Effects of (In) accurate empathy and situational valence on attitudes towards robots. ACM/IEEE International Conference on Human Robot Interaction (HRI). 141-2. New York. March 2005Google Scholar
  9. 9.
    Davis M (1980) A multidimensional approach to individual differences in empathy. Cat Sel Doc Psychol 10:85Google Scholar
  10. 10.
    De Carolis B, Ferilli S, Greco D (2013) Towards a caring home for assisted living. In: Proceedings of the workshop “the challenge of ageing society: technological roles and opportunities for artificial intelligence” co-located with the 13th Conference of the Italian Association for Artificial Intelligence (AI*IA 2013), Turin, 6 Dec 2013.
  11. 11.
    De Carolis B, Mazzotta I, Novielli N, Pizzutilo S (2013) User modeling in social interaction with a Caring Agent. In: UMADR - User modeling and adaptation for daily routines, Springer HCI Series. SpringerGoogle Scholar
  12. 12.
    de Rosis F, De Carolis B, Carofiglio V, Pizzutilo S (2003) Shallow and inner forms of emotional intelligence in advisory dialog simulation. In: Prendinger H, Ishizuka M (eds) “Life-like characters. Tools, affective functions and applications. pp 271-294. SpringerGoogle Scholar
  13. 13.
    Eisenberg N, Miller PA (1987) The relation of empathy to prosocial and related behaviors. Psychol Bull 101:91–119CrossRefGoogle Scholar
  14. 14.
    Ekman P, Oster H (1979) Facial expressions of emotion. Annu Rev Psychol 30:527–554CrossRefGoogle Scholar
  15. 15.
    Elliott C, Siegle G (1993) Variables influencing the intensity of simulated affective states. In Proceedings of the AAAI Spring Symposium on Mental States.’93:58-67Google Scholar
  16. 16.
    Graf B, Hans M, Shaft RD (2004) Care-O-bot II – development of a next generation robotics home assistant. Auton Robots 16(2):193–205Google Scholar
  17. 17.
    Hoffman ML (1981) The development of empathy. In: Rushton J, Sorrentino R (eds) Altruism and helping behavior: Social personality and developmental perspectives. Erlbaum, Hillsdale, pp. 41–63Google Scholar
  18. 18.
    Jensen FV (2001) Bayesian networks and decision graphs, statistics for engineering and information science. Springer New York, New York, doi:10.1007/978-1-4757-3502-4
  19. 19.
    Jiang H, Vidal JM, Huhns MN (2007) EBDI: an architecture for emotional agents, Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems, May 14-18, Honolulu, Hawaii, ACM, p. 11Google Scholar
  20. 20.
    Johnson DO, Cuijpers RH, van der Pol D (2013) Imitating human emotions with artificial facial expressions. Int J Soc Robot 5(4):503–513CrossRefGoogle Scholar
  21. 21.
    Klein J, Moon Y, Picard R (1999) This computer responds to user frustration. In Proceedings of the Conference on Human Factors in Computing Systems. Pittsburgh, USA, ACM Press, pp 242–243Google Scholar
  22. 22.
    Leite I, Martinho C, Paiva A (2013) Social robots for long-term interaction: a survey. Int J Soc Robot. doi:10.1007/s12369-013-0178-y Google Scholar
  23. 23.
    Mataric M, Tapus A, Feil-Seifer D (2007) Personalized socially assistive robotics, workshop on intelligent systems for assisted cognition, rochester, New York, USAGoogle Scholar
  24. 24.
    Mehrabian A (1968) Communication without words. Psychol Today 2:53–56Google Scholar
  25. 25.
    Milborrow S, Nicolls F (2008) Locating facial features with an extended active shape model. In Computer Vision–ECCV 2008 (pp. 504-513). Springer Berlin HeidelbergGoogle Scholar
  26. 26.
    Milborrow S, Nicolls F (2014) Active shape models with SIFT descriptors and MARS, In Proceedings of the 9th International Conference on Computer Vision Theory and Applications, Lisbon, pp. 5–8Google Scholar
  27. 27.
    Nehmer J, Becker M, Karshmer A, Lamm R (2006) Living assistance systems: an ambient intelligence approach, in International Conference on Software Engineering, ACM, pp. 43–50Google Scholar
  28. 28.
    Niewiadomski R, Ochs M, Pelachaud C (2008) Expressions of empathy in ECAs, in Proceedings of the 8t Int.Conf. on IVA, LNAI, vol. 5208. Springer-Verlag, Berlin, Heidelberg, pp 37-44Google Scholar
  29. 29.
    Nijholt A, de Ruyter B, Heylen D, Privender S (2006) Social interfaces for ambient intelligence environments. Chapter 14. In: Aarts E, Encarnaçao J (eds) True visions: the emergence of ambient intelligence. Springer, New York, pp. 275–289CrossRefGoogle Scholar
  30. 30.
    Oatley K, Johnson-Laird PN (1987) Towards a cognitive theory of emotions. Cognit Emot 29–50Google Scholar
  31. 31.
    Ortony A, Clore GL, Collins A (1988) The cognitive structure of emotions. Cambridge University PressGoogle Scholar
  32. 32.
    Paiva A, Dias J, Sobral D, Aylett R, Sobreperez P, Woods S, Hall L (2004) Caring for agents and agents that care: building empathic relations with synthetic agents. In Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems-Volume 1 (pp. 194-201). IEEE Computer Society., ACM PressGoogle Scholar
  33. 33.
    Paiva A et al. (2011) Empathy in social agents. Int J sVirtual Real 10(1):65–68Google Scholar
  34. 34.
    Palestra G, Pettinicchio A, Del Coco M, Carcagnì P, Leo M, Distante C (2015) Improved performance in facial expression recognition using 32 geometric features. ICIAP, lecture notes in computer science, Springer, pp. 518–528Google Scholar
  35. 35.
    Picard R (1997) Affective computing. MIT Press, Cambridge, MACrossRefGoogle Scholar
  36. 36.
    Pineau J, Montemerlo M, Pollack M, Roy N, Thrun S (2003) Towards robotic assistants in nursing homes: challenges and results, robotics and autonomous systems 42(3–4):271–281Google Scholar
  37. 37.
    Reeves B, Nass C (1996) The media equation: how people treat computers, television, and new media like real people and places. Cambridge University Press, New York, NYGoogle Scholar
  38. 38.
    Reiter E, Dale R (2000) Building natural language generation systems. Studies in natural language processing. Cambridge University Press, Cambridge, United Kingdom. ISBN 0-521-62036-8Google Scholar
  39. 39.
    Russell J (1980) A circumplex model of affect. J Pers Soc Psychol 39:1161–1178CrossRefGoogle Scholar
  40. 40.
    Speigel C. Greczek J, Matarić M (2016) Creating a baseline for empathy in socially assistive robotics.
  41. 41.
    van Ruiten AM, Haitas D, Bingley P, Hoonhout HCM, Meerbeek BW, Terken JMB (2007) Attitude of elderly towards a robotic game-and-train- buddy: evaluation of empathy and objective control. In: Cowie R, de Rosis F (eds) Proceedings of the Doctoral consortium, in the scope of ACII2007 ConferenceGoogle Scholar
  42. 42.
    Viola PA, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154CrossRefGoogle Scholar
  43. 43.
    Vogt T, Andrié E, Bee N (2008) EmoVoice - a framework for online recognition of emotions from voice. In Proceedings of the 4th IEEE tutorial and research workshop on Perception and Interactive Technologies for Speech-Based Systems: Perception in Multimodal Dialogue Systems (PIT '08), Springer-Verlag, Berlin, 188-199Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Berardina De Carolis
    • 1
  • Stefano Ferilli
    • 1
  • Giuseppe Palestra
    • 1
  1. 1.Dipartimento di InformaticaUniverisita’ di Bari, Aldo MoroBariItaly

Personalised recommendations