Skip to main content
SpringerLink
Log in

A listener model: introducing personality traits

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

We present a computational model that generates listening behaviour for a virtual agent. It triggers backchannel signals according to the user’s visual and acoustic behaviour. The appropriateness of the backchannel algorithm in a user-agent situation of storytelling, has been evaluated by naïve participants, who judged the algorithm-ruled timing of backchannels more positively than a random timing. The system can generate different types of backchannels. The choice of the type and the frequency of the backchannels to be displayed is performed considering the agent’s personality traits. The personality of the agent is defined in terms of two dimensions, extroversion and neuroticism. We link agents with a higher level of extroversion to a higher tendency to perform more backchannels than introverted ones, and we link neuroticism to less mimicry production and more response and reactive signals sent. We run a perception study to test these relations in agent-user interactions, as evaluated by third parties. We find that the selection of the frequency of backchannels performed by our algorithm contributes to the correct interpretation of the agent’s behaviour in terms of personality traits.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Allwood J, Cerrato L (2003) A study of gestural feedback expressions. In: Paggio P, Jokinen K, Jonsson A (eds) First Nordic symposium on multimodal communication, Copenhagen, pp 7–22

    Google Scholar 

  2. Allwood J, Nivre J, Ahlsén E (1992) On the semantics and pragmatics of linguistic feedback. J Semant 1–26

  3. van Baaren RB, Holland RW, Steenaert B, van Knippenberg A (2003) Mimicry for money: behavioral consequences of imitation. J Exp Soc Psychol 39:393–398

    Article  Google Scholar 

  4. Bertrand R, Blache P, Espesser R, Ferré G, Meunier C, Priego-Valverde B, Rauzy S (2006) Le cid—corpus of interactional data: protocoles, conventions, annotations. In: Travaux Interdisciplinaires du Laboratoire Parole et Langage d’Aix-en-Provence (TIPA), vol 25, pp 25–55

    Google Scholar 

  5. Bevacqua E, Heylen D, Tellier M, Pelachaud C (2007) Facial feedback signals for ECAs. In: AISB’07 annual convention, workshop “Mindful Environments”, Newcastle upon Tyne, UK, pp 147–153

    Google Scholar 

  6. Bevacqua E, Mancini M, Pelachaud C (2008) A listening agent exhibiting variable behaviour. In: Prendinger H, Lester JC, Ishizuka M (eds) Proceedings of 8th international conference on intelligent virtual agents. Springer, Tokyo, pp 262–269

    Google Scholar 

  7. Borkenau P, Liebler A (1992) Trait inferences: sources of validity at zero acquaintance. J Pers Soc Psychol 62:645–657

    Article  Google Scholar 

  8. Cassell J, Bickmore T (1999) Embodiment in conversational interfaces: Rea. In: Human factors in computing systems, Pittsburgh, PA

    Google Scholar 

  9. Cassell J, Nakano Y, Bickmore T, Sidner C, Rich C (2001) Non-verbal cues for discourse structure. In: Proceedings of the 39th annual meeting on association for computational linguistics. Association for Computational Linguistics, Morristown, Morristown, pp 114–123

    Google Scholar 

  10. Cerrato L, Skhiri M (2003) Analysis and measurment of head movements signalling feedback in face-to-face human dialogues. In: Paggio P, Jokinen K, Jónsson A (eds) First Nordic symposium on multimodal communication, Copenhagen, pp 43–52

    Google Scholar 

  11. Chartrand T, Bargh J (1999) The chameleon effect: the perception-behavior link and social interaction. Pers Soc Psychol 76:893–910

    Article  Google Scholar 

  12. Chartrand T, Maddux W, Lakin J (2005) Beyond the perception-behavior link: the ubiquitous utility and motivational moderators of nonconscious mimicry. In: The new unconscious. Oxford University Press, New York, pp 334–361

    Google Scholar 

  13. Costa PT, McCrae RR (1992) Four ways five factors are basic. Pers Individ Differ 13:653–665

    Article  Google Scholar 

  14. Decety J, Jackson P (2004) The functional architecture of human empathy. Behav Cogn Neurosci Rev 3:71–100

    Article  Google Scholar 

  15. Eisenberg N, Fabes RA, Murphy B, Karbon M, Smith M, Maszk P (1996) The relations of children’s dispositional empathy-related responding to their emotionality, regulation, and social functioning. Dev Psychol 32:195–209

    Article  Google Scholar 

  16. Eisenberg N, Fabes RA, Shepard SA, Murphy BC, Jones S, Guthrie IK (1998) Contemporaneous and longitudinal prediction of children’s sympathy from dispositional regulation and emotionality. Dev Psychol 34:910–924

    Article  Google Scholar 

  17. Eyben F, Wöllmer M, Schuller B (2010) OpenSMILE—the Munich versatile and fast open-source audio feature extractor. In: ACM multimedia (MM), Florence, Italy, pp 1459–1462

    Google Scholar 

  18. Eysenck HJ (1991) Dimensions of personality: criteria for a taxonomic paradigm. Pers Individ Differ 12:773–779

    Article  Google Scholar 

  19. Eysenck S, Eysenck H (1978) Impulsiveness and venturesomeness—their position in a dimensional system of personality description. Psychol Rep 43:1247–1255

    Article  Google Scholar 

  20. Gunes H, Pantic M (2010) Dimensional emotion prediction from spontaneous head gestures for interaction with sensitive artificial listeners. In: International conference on intelligent virtual agents, Philadelphia, USA, pp 371–377

    Chapter  Google Scholar 

  21. Huang L, Morency LP, Gratch J (2010) Parasocial consensus sampling: combining multiple perspectives to learn virtual human behavior. In: The 9th international conference on autonomous agents and multiagent systems (AAMAS 2010), Toronto, Canada

    Google Scholar 

  22. Isbister K, Nass C (2000) Consistency of personality in interactive characters: verbal cues, non-verbal cues, and user characteristics. Int J Hum-Comput Stud 53(2):251–267

    Article  Google Scholar 

  23. Kopp S, Allwood J, Grammer K, Ahlsen E, Stocksmeier T (2008) Modeling embodied feedback with virtual humans. Lect Notes Comput Sci 4930:18

    Article  Google Scholar 

  24. Lakin JL, Jefferis VA, Cheng CM, Chartrand TL (2003) Chameleon effect as social glue: evidence for the evolutionary significance of nonconsious mimicry. J Nonverbal Behav 27(3):145–162

    Article  Google Scholar 

  25. Maatman RM, Gratch J, Marsella S (2005) Natural behavior of a listening agent. In: 5th international conference on interactive virtual agents, Kos, Greece

    Google Scholar 

  26. McKeown G, Valstar M, Cowie R, Pantic M (2010) The semaine corpus of emotionally coloured character interactions. In: IEEE int’l conf multimedia & expo, Singapore, Singapore, pp 1079–1084

    Google Scholar 

  27. McRorie M, Sneddon I, de Sevin E, Bevacqua E, Pelachaud C (2009) A model of personality and emotional traits. In: Intelligent virtual agents 2009, IVA’09, Amsterdam, Holland

    Google Scholar 

  28. Morency LP, de Kok I, Gratch J (2008) Predicting listener backchannels: a probabilistic multimodal approach. In: Prendinger H, Lester JC, Ishizuka M (eds) Proceedings of 8th international conference on intelligent virtual agents. Lecture notes in computer science, vol 5208. Springer, Tokyo

    Google Scholar 

  29. Nass C, Steuer J, Tauber ER (1994) Computers are social actors. In: CHI, pp 72–78

    Google Scholar 

  30. Noor F, Evans D (2003) The effect of facial symmetry on perceptions of personality and attractiveness. J Res Pers 37:339–347

    Article  Google Scholar 

  31. Poggi I (2003) Mind markers. In: Trigo N, Rector M, Poggi I (eds) Gestures. Meaning and use. University Fernando Pessoa Press, Oporto

    Google Scholar 

  32. Poggi I (2007) Mind, hands, face and body. A goal and belief view of multimodal communication. Weidler, Berlin

    Google Scholar 

  33. Von der Pütten A, Krämer NC, Gratch J, Kang S (2010, in press) “It doesn’t matter what you are!” Explaining social effects of agents and avatars. Comput Hum Behav

  34. Reeves B, Nass C (1996) The media equation: how people treat computers, television and new media like real people and places. CSLI Publications, Stanford

    Google Scholar 

  35. Schröder M (2010) The semaine api: towards a standards-based framework for building emotion-oriented systems. Adv Hum-Comput Interact 2010

  36. Schröder M, Bevacqua E, Cowie R, Eyben F, Gunes H, Heylen D, ter Maat M, McKeown G, Pammi S, Pantic M, Pelachaud C, Schuller B, de Sevin E, Valstar M, Wollmer M (to appear) Building autonomous sensitive artificial listeners. J Trans Affect Comput

  37. Sonnby-Borgstrom M (2002) Automatic mimicry reactions as related to differences in emotional empathy. Scand J Psychol 43:433–443

    Article  Google Scholar 

  38. Sonnby-Borgström M, Jönsson P, Svensson O (2003) Emotional empathy as related to mimicry reactions at different levels of information processing. J Nonverbal Behav 27:3–23. http://dx.doi.org/10.1023/A:1023608506243

    Article  Google Scholar 

  39. Thórisson KR (1996) Communicative humanoids: a computational model of psychosocial dialogue skills. PhD thesis, MIT Media Laboratory

  40. Valstar M, Martinez B, Binefa X, Pantic M (2010) Facial point detection using boosted regression and graph models. In: IEEE int’l conf computer vision and pattern recognition, San Francisco, USA, pp 2729–2736

    Google Scholar 

  41. Ward N, Tsukahara W (2000) Prosodic features which cue back-channel responses in English and Japanese. J Pragmat 23:1177–1207

    Article  Google Scholar 

  42. Watson D (1988) Strangers? ratings of the five robust personality factors evidence of a surprising convergence with self-report. J Pers Soc Psychol 57:120–128

    Google Scholar 

  43. Wiggins J, Trapnell P, Phillips N (1988) Psychometric and geometric characteristics of the revised interpersonal adjectives scale (IAS-R). J Multivar Behav Res 23:217–305

    Google Scholar 

  44. Yngve V (1970) On getting a word in edgewise. In: Papers from the sixth regional meeting of the Chicago linguistic society, pp 567–577

    Google Scholar 

Download references

Acknowledgements

This work has been funded by the STREP SEMAINE project IST-211486 (http://www.semaine-project.eu) and Web 2.0 MyPresentingAvatar project.

Author information

Authors and Affiliations

  1. Lab-STICC, CERV—École Nationale d’Ingénieurs de Brest, Parvis Blaise Pascal, Technopôle Brest-Iroise, 29280, Plouzané, France

    Elisabetta Bevacqua

  2. Université Pierre et Marie Curie—LIP6, 4 place Jussieu, 75005, Paris, France

    Etienne de Sevin

  3. CNRS—Telecom ParisTech, 37/39 rue Dareau, 75014, Paris, France

    Sylwia Julia Hyniewska & Catherine Pelachaud

Authors
  1. Elisabetta Bevacqua
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Etienne de Sevin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sylwia Julia Hyniewska
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Catherine Pelachaud
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Elisabetta Bevacqua.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bevacqua, E., de Sevin, E., Hyniewska, S.J. et al. A listener model: introducing personality traits. J Multimodal User Interfaces 6, 27–38 (2012). https://doi.org/10.1007/s12193-012-0094-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-012-0094-8

Keywords

Search

Navigation