Skip to main content

Seniors’ Sensing of Agents’ Personality from Facial Expressions

Part of the Lecture Notes in Computer Science book series (LNISA,volume 10897)


The presented study investigated the preferences of seniors towards artificial avatars showing personality both from a pragmatic and a hedonic point of view. Also, preferences for technological devices were considered. The involved participants were 45 adults (20 female) aged 65+ years in good health. They were asked to watch video clips of 4 agents (two males and two females) showing different personality traits (i.e. angry, depressed, joyful, and practical), and subsequently had to complete a questionnaire. Subjects were not informed about an avatar’s personality and not openly interviewed regarding this subject. Rather, the administered questionnaire was devoted to test their perception of agents and whether such complies with the intended characteristics. Results show that subjects prefer female agents with a positive personality (joyful and practical) on both pragmatic and hedonic dimensions of the interactive system.


  • Emotional artificial agents
  • Facial expressions
  • Assisted living

1 Introduction

Several research studies have shown that emotions and facial emotional expressions play an important role in everyday life [7, 11, 12]. Indeed, there is a wide field of research dealing with questions concerning emotional facial expressions as a signal that mediates social interaction. Interactional exchanges constitute the main source of emotional appraisals and emotions are indispensable for starting, maintaining, modifying, strengthening or breaking relationships with others [2]. In this context, facial emotional expressions exemplify the visible component of emotions, providing information on the emotional states of the interacting person. To this aim, facial emotional expressions have been subjected to numerous studies and over time they have become the pivotal topic of many researches, some of which focused on the visual appearance of virtual agents in order to investigate the types of reactions such may trigger in individuals. The Semaine projectFootnote 1, for example, aimed at the implementation of artificial conversational agents allowing for “emotionally colored interactions between a person and a machine”. To achieve this, the agents had to exploit mostly non-verbal facial expression behaviors in order to elicit similar behavior in human interlocutors [1]. The Empathic projectFootnote 2 attempts to expand this work by “developing causal models of [agent] coach-user interactional exchanges that engage elders in emotionally believable interactions […]”. Doing so, Empathic is age contextualized and will account for social and cultural variations in Western Europe, collecting data in Spain, France, Norway, and Italy. It will follow a user-centered design approach so as to establish trusting human-machine relationships, take appropriate actions to provide long-term care, and detect emotional states and negative moods, such as depression. To achieve this, we will build upon previous work [e.g. 4,5,6], yet aim at developing solutions which are more efficient and more effective (e.g. [3]).

A first pilot study focused on eliciting initial guidelines for designing such user-centered and adaptable interfaces. The study explored user’s requirements and expectations with respect to their age and familiarity with technology. The goal was to (1) assess seniors’ preferences in initiating conversations with an agent visually expressing emotional behavior; (2) summon elder’s emotional responses arising from such non-verbal agent behavior; and (3) measure their interest in these interactions.

2 Material and Methodology

The research sample consisted of 45 healthy seniors (20 females), aged 65+ years (AVG = 70.28 years; SD = ±5.52). They were recruited in Campania, a region in the south of Italy. Participation was voluntary and approved by the ethical committee of the Dept. of Psych. at Univ. della Campania Luigi Vanvitelli (Code No. 25/2017).

Our stimuli were build based upon four conversational agents possessing different personality features able to arise user specific emotional states [10]. For each agent, a video-clip was extracted. Agents’ names and videos were assessed by one expert and three naïve, before the following 4 video clips, each 10 s long, showing an agent’s half torso while speaking (note: audio was muted), were selected:

  • Serena, female, based on Semaine’s Poppy, committed to expressing optimism

  • Gerardo, male, based on Semaine’s Obadiah, deputed to express pessimism

  • Pasquale, male, based on Semaine’s Spike, deputed to express aggression

  • Francesca, female, based on Semaine’s Prudence, aimed at expressing pragmatism

We used a tripartite questionnaire to collect user feedback. Section 1 focused on participants’ socio-demographic information, their degree of technology experience, and their used technological devices with respective ratings on perceived usage difficulty. Section 2 established users’ preferences regarding the interaction with each of the proposed agents. This section was clustered in four subsections, each consisting of 7 items, investigating the practicality (PQ), pleasure feelings (HQI, and HQS), and attractiveness (ATT) experienced while watching the video-clips [8]. Finally, Sect. 3 collected feedback on 10 agent characteristics rated on 5-point Likert scales ranging from 1 = strongly agree to 5 = strongly disagree (note: 3 = I don’t know). Section 1 was completed first. Then participants watched the 4 videos, each time followed by a separate Sect. 2, before they eventually completed Sect. 3.

3 Results

Our results show that, 57.8% of the participants regularly use at least one of the proposed technological devices (i.e. smartphone, tablet, laptop), 26.7% use one from time to time, 2.2% had attempted to use one, and 13.3% never used one. The most popular device seems to be the smartphone (68.9% regular usage), followed by the tablet (22.2% regular usage) and laptop (17.8% regular usage). The smartphone appears to be also the device with the highest usability, rated as easy to use by 75.6% of the participants. Asked about their acceptance for the shown avatars, 71.1% liked Francesca, Serena was accepted by 66.7% and Gerardo by 46.7%. Pasquale was the least favored, accepted by only 22.2% of the participants.

Analyzing the Pragmatic Qualities (PQ), Hedonic Qualities of Identification (HQI) and Stimulation (HQS), and the attractiveness (ATT) of the agents, a 2(gender) × 4(PQ, HQS, HQI, ATT) ANOVA was applied (α = .01). No interactions were found between gender and avatars. Significant differences were, however, found among avatars for PQ (F(1, 43) = 40.04, p < .001), HQI (F(1, 43) = 42.32, p < .001), HQS (F(1, 43) = 47.84, p < .001), and ATT (F(1, 43) = 47.51, p < .001). Bonferroni post hoc tests revealed that these significant differences were always between Pasquale and Francesca (with p < .001 for PQ, HQI, HQS, and ATT), between Pasquale and Serena (with p < .001, for PQ, HQI, HQS, and ATT), between Gerardo and Francesca (with p < .001, for PQ, HQI, HQS, and ATT), and between Gerardo and Serena (with p < .001 for PQ, and ATT, and p = .002 for HQI). No significant HQS differences were found between Gerardo and Pasquale, Francesca and Serena, and Gerardo and Serena.

To summarize the preference scores obtained in each cluster of Sect. 2 (PQ, HQI, HQS, ATT), we classified them in Strong/High (7–13 points), Moderate (14-20 points), Mild (21–27 points), and None (28–35 points). According to this arrangement, 64.4% of the participants considered Francesca moderately pragmatic (PQ) (Serena = 53.3%, Gerardo = 42.2%, Pasquale = 24.5%). They also found her to be moderately mind-pleasing (HQI) (66.5%; Serena = 51.1%, Gerardo = 35.6%, Pasquale = 22.2%), moderately stimulating (HQS) (75.5%; Serena = 66.6%, Gerardo = 40.0%, Pasquale = 20.0%), and moderately attractive (ATT) (64.5%; Serena = 55.6%, Gerardo = 44.4%, Pasquale = 31.1%).

4 Conclusions

Our study showed that a significant number of seniors use at least one technological device, where the smartphone seems to be preferred and perceived as most usable. Furthermore, it was shown that generally seniors show a positive attitude towards interactions with an artificial avatar, and that they favor avatars which express a positive personality. That is, regarding pragmatic, hedonic, and attractiveness features our avatars Francesca and Serena did consistently score higher than Pasquale and Gerardo. Although participants were not informed about an avatar’s personality, they were able to perceive relevant facial expressions, suggesting that they have preferences for positive facial dynamics. Future work requires a deeper investigation of this capability as those avatars showing positive facial dynamics were females, indicating a potential gender influence on the processing of emotional facial expressions [9]. Also, the effect of voice has to be explored, as for now the utilized avatars were moving their lips yet audio output was deliberately muted.


  1. 1.

  2. 2.


  1. Bevacqua, E., Pammi, S., Hyniewska, S.J., Schröder, M., Pelachaud, C.: Multimodal backchannels for embodied conversational agents. In: Allbeck, J., Badler, N., Bickmore, T., Pelachaud, C., Safonova, A. (eds.) IVA 2010. LNCS (LNAI), vol. 6356, pp. 194–200. Springer, Heidelberg (2010).

    CrossRef  Google Scholar 

  2. Campos, J.J., Barrett, K.C.: Toward a new understanding of emotions and their development. In: Izard, C., Kagan, J., Zajonc, R. (eds.) Emotions, cognition, and behavior, pp. 229–263. Cambridge University Press, New York (1984)

    Google Scholar 

  3. Cordasco, G., Esposito, M., Masucci, F., Riviello, M.T., Esposito, A., Chollet, G., Schlögl, S., Milhorat, P., Pelosi, G.: Assessing voice user interfaces: the vAssist system prototype. In Proceedings of the 5th IEEE International Conference on Cognitive InfoCommunications, Vietri sul Mare, Italy (2014)

    Google Scholar 

  4. Esposito, A., Esposito, A. M., Vogel. C.: Needs and challenges in human computer interaction for processing social emotional information. Patt. Recog. Lett. 66, 41–51 (2015)

    CrossRef  Google Scholar 

  5. Esposito, A., Fortunati, L., Lugano, G.: Modeling emotion, behaviour and context in socially believable robots and ICT interfaces. Cognit. Comput. 6(4), 623–627 (2014)

    CrossRef  Google Scholar 

  6. Esposito, A.: The situated multimodal facets of human communication. In: Rojc, M., Campbell, N. (eds.) Coverbal Synchrony in Human-Machine Interaction, pp. 173–202. CRC Press, Taylor & Francis Group, Boca Raton (2013)

    CrossRef  Google Scholar 

  7. Esposito, A., Esposito, A.M.: On the recognition of emotional vocal expressions: Motivations for an holistic approach. Cognit. Process. J. 13(2), 541–550 (2012)

    CrossRef  Google Scholar 

  8. Hassenzahl, M.: The interplay of beauty, goodness, and usability in interactive products. Hum. Comput. Interact. 19, 319–349 (2004)

    CrossRef  Google Scholar 

  9. Marsh, A.A., Ambady, N., Kleck, R.E.: The effects of fear and anger facial expressions on approach- and avoidance-related behaviors. Emotion 5(1), 119–124 (2005)

    CrossRef  Google Scholar 

  10. Ochs, M., Niewiadomski, R., Pelachaud, C.: How a virtual agent should smile? In: Allbeck, J., Badler, N., Bickmore, T., Pelachaud, C., Safonova, A. (eds.) IVA 2010. LNCS (LNAI), vol. 6356, pp. 427–440. Springer, Heidelberg (2010).

    CrossRef  Google Scholar 

  11. Seibt, B., Mühlberger, A., Likowski, K.U., Weyers, P.: Facial mimicry in its social setting. Front. Psychol. 6, 1122 (2015)

    CrossRef  Google Scholar 

  12. Seidela, E.V., Habela, U., Kirschnerc, M., Gurd, R.C., Derntla, B.: The impact of facial emotional expressions on behavioral tendencies in females and males. J. Exp. Psychol. Hum. Percept. Perform 36(2), 500–507 (2010)

    CrossRef  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Stephan Schlögl .

Rights and permissions

<SimplePara><Emphasis Type="Bold">Open Access</Emphasis> This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.</SimplePara> <SimplePara>The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.</SimplePara>

Reprints and Permissions

Copyright information

© 2018 The Author(s)

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Esposito, A. et al. (2018). Seniors’ Sensing of Agents’ Personality from Facial Expressions. In: Miesenberger, K., Kouroupetroglou, G. (eds) Computers Helping People with Special Needs. ICCHP 2018. Lecture Notes in Computer Science(), vol 10897. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-94273-5

  • Online ISBN: 978-3-319-94274-2

  • eBook Packages: Computer ScienceComputer Science (R0)