Evaluating the Effect of Gesture and Language on Personality Perception in Conversational Agents

  • Michael Neff
  • Yingying Wang
  • Rob Abbott
  • Marilyn Walker
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6356)


A significant goal in multi-modal virtual agent research is to determine how to vary expressive qualities of a character so that it is perceived in a desired way. The “Big Five” model of personality offers a potential framework for organizing these expressive variations. In this work, we focus on one parameter in this model – extraversion – and demonstrate how both verbal and non-verbal factors impact its perception. Relevant findings from the psychology literature are summarized. Based on these, an experiment was conducted with a virtual agent that demonstrates how language generation, gesture rate and a set of movement performance parameters can be varied to increase or decrease the perceived extraversion. Each of these factors was shown to be significant. These results offer guidance to agent designers on how best to create specific characters.


personality gesture conversational and non-verbal behavior evaluation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    André, E., Rist, T., van Mulken, S., Klesen, M., Baldes, S.: The automated design of believable dialogues for animated presentation teams. In: Embodied conversational agents, pp. 220–255 (2000)Google Scholar
  2. 2.
    Argyle, M.: Bodily communication. Taylor & Francis, Abington (1988)Google Scholar
  3. 3.
    Brebner, J.: Personality theory and movement. Individual differences in movement, 27–41 (1985)Google Scholar
  4. 4.
    Cassell, J., Vilhjálmsson, H., Bickmore, T.: BEAT: the Behavior Expression Animation Toolkit. In: Proceedings of SIGGRAPH 2001, pp. 477–486 (2001)Google Scholar
  5. 5.
    Chi, D.M., Costa, M., Zhao, L., Badler, N.I.: The EMOTE model for effort and shape. In: Proc. SIGGRAPH 2000, pp. 173–182 (2000)Google Scholar
  6. 6.
    Frank, K.: Posture & Perception in the Context of the Tonic Function Model of Structural Integration: an Introduction. In: IASI Yearbook 2007, pp. 27–35 (2007)Google Scholar
  7. 7.
    Funder, D.C.: The Personality Puzzle, 2nd edn. W. W. Norton & Company, New York (1997)Google Scholar
  8. 8.
    Furnham, A.: Language and personality. In: Giles, H., Robinson, W. (eds.) Handbook of Language and Social Psychology, Winley (1990)Google Scholar
  9. 9.
    Giles, H., Street, R.: Communicator characteristics and behavior. Handbook of interpersonal communication 2, 103–161 (1994)Google Scholar
  10. 10.
    Gosling, S.D., Rentfrow, P.J., Swann, W.B.: A very brief measure of the big five personality domains. Journal of Research in Personality 37, 504–528 (2003)CrossRefGoogle Scholar
  11. 11.
    Hartmann, B., Mancini, M., Pelachaud, C.: Formational parameters and adaptive prototype installation for MPEG-4 compliant gesture synthesis. In: Proc. Computer Animation 2002. pp. 111–119 (2002)Google Scholar
  12. 12.
    Hartmann, B., Mancini, M., Pelachaud, C.: Implementing expressive gesture synthesis for embodied conversational agents. In: Gibet, S., Courty, N., Kamp, J.-F. (eds.) GW 2005. LNCS (LNAI), vol. 3881, pp. 45–55. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  13. 13.
    Heloir, A., Kipp, M.: EMBR – A Realtime Animation Engine for Interactive Embodied Agents. In: Ruttkay, Z., Kipp, M., Nijholt, A., Vilhjálmsson, H.H. (eds.) IVA 2009. LNCS, vol. 5773, pp. 393–404. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  14. 14.
    Heylighen, F., Dewaele, J.M.: Variation in the contextuality of language: an empirical measure. Context in Context, Special issue of Foundations of Science 7(3), 293–340 (2002)Google Scholar
  15. 15.
    Isbister, K., Nass, C.: Consistency of personality in interactive characters: Verbal cues, non-verbal cues, and user characteristics. International Journal of Human Computer Studies 53(2), 251–268 (2000)CrossRefGoogle Scholar
  16. 16.
    Kipp, M., Neff, M., Kipp, K., Albrecht, I.: Towards natural gesture synthesis: Evaluating gesture units in a data-driven approach to gesture synthesis. In: Pelachaud, C., Martin, J.-C., André, E., Chollet, G., Karpouzis, K., Pelé, D. (eds.) IVA 2007. LNCS (LNAI), vol. 4722, pp. 15–28. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  17. 17.
    Knapp, M., Hall, J.: Nonverbal communication in human interaction. Holt, New York (1978)Google Scholar
  18. 18.
    Kopp, S., Wachsmuth, I.: Synthesizing multimodal utterances for conversational agents. Computer Animation and Virtual Worlds 15, 39–52 (2004)CrossRefGoogle Scholar
  19. 19.
    Levine, S., Theobalt, C., Koltun, V.: Real-time prosody-driven synthesis of body language. ACM Transactions on Graphics (TOG) 28(5), 1–10 (2009)CrossRefGoogle Scholar
  20. 20.
    Lippa, R.: The nonverbal display and judgment of extraversion, masculinity, femininity, and gender diagnosticity: A lens model analysis. Journal of Research in Personality 32(1), 80–107 (1998)CrossRefGoogle Scholar
  21. 21.
    Mairesse, F., Walker, M.A.: PERSONAGE: Personality generation for dialogue. In: Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 496–503 (2007)Google Scholar
  22. 22.
    Mairesse, F., Walker, M.A.: Trainable generation of Big-Five personality styles through data-driven parameter estimation. In: Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics, ACL (2008)Google Scholar
  23. 23.
    McQuiggan, S., Mott, B., Lester, J.: Modeling self-efficacy in intelligent tutoring systems: An inductive approach. User Modeling and User-Adapted Interaction 18(1), 81–123 (2008)CrossRefGoogle Scholar
  24. 24.
    Mehl, M.R., Gosling, S.D., Pennebaker, J.W.: Personality in its natural habitat: Manifestations and implicit folk theories of personality in daily life. Journal of Personality and Social Psychology 90, 862–877 (2006)CrossRefGoogle Scholar
  25. 25.
    Mehrabian, A.: Significance of posture and position in the communication of attitude and status relationships. Psychological Bulletin 71(5), 359–372 (1969)CrossRefGoogle Scholar
  26. 26.
    Neff, M., Kim, Y.: Interactive editing of motion style using drives and correlations. In: Proceedings of the 2009 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 103–112. ACM Press, New York (2009)CrossRefGoogle Scholar
  27. 27.
    Neff, M., Fiume, E.: AER: Aesthetic Exploration and Refinement for expressive character animation. In: Proc. ACM SIGGRAPH/Eurographics Symposium on Computer Animation 2005, pp. 161–170 (2005)Google Scholar
  28. 28.
    Neff, M., Kipp, M., Albrecht, I., Seidel, H.P.: Gesture modeling and animation based on a probabilistic re-creation of speaker style. ACM Transactions on Graphics 27(1), 5:1–5:24 (2008)Google Scholar
  29. 29.
    Norman, W.T.: Toward an adequate taxonomy of personality attributes: Replicated factor structure in peer nomination personality rating. Journal of Abnormal and Social Psychology 66, 574–583 (1963)CrossRefGoogle Scholar
  30. 30.
    North, M.: Personality assessment through movement. Macdonald and Evans (1972)Google Scholar
  31. 31.
    Pennebaker, J.W., King, L.A.: Linguistic styles: Language use as an individual difference. Journal of Personality and Social Psychology 77, 1296–1312 (1999)CrossRefGoogle Scholar
  32. 32.
    Piwek, P.: A flexible pragmatics-driven language generator for animated agents. In: Proceedings of Annual Meeting of the European Chapter of the Association for Computational Linguistics, EACL (2003)Google Scholar
  33. 33.
    Riggio, R., Friedman, H.: Impression formation: The role of expressive behavior. Journal of Personality and Social Psychology 50(2), 421–427 (1986)CrossRefGoogle Scholar
  34. 34.
    Scherer, K.R.: Personality markers in speech. In: Scherer, K.R., Giles, H. (eds.) Social markers in speech, pp. 147–209. Cambridge University Press, Cambridge (1979)Google Scholar
  35. 35.
    Takala, M.: Studies of psychomotor personality tests, 1-. Suomalainen Tiedeakatemia (1953)Google Scholar
  36. 36.
    Thiebaux, M., Marshall, A., Marsella, S., Kallman, M.: Smartbody: Behavior realization for embodied conversational agents. In: Proc. of 7th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2008), pp. 151–158 (2008)Google Scholar
  37. 37.
    Wang, N., Johnson, W.L., Mayer, R.E., Rizzo, P., Shaw, E., Collins, H.: The politeness effect: Pedagogical agents and learning gains. Frontiers in Artificial Intelligence and Applications 125, 686–693 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Michael Neff
    • 1
  • Yingying Wang
    • 1
  • Rob Abbott
    • 2
  • Marilyn Walker
    • 2
  1. 1.University of CaliforniaDavis
  2. 2.University of CaliforniaSanta Cruz

Personalised recommendations