Virtual Rapport 2.0

  • Lixing Huang
  • Louis-Philippe Morency
  • Jonathan Gratch
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6895)


Rapport, the feeling of being “in sync” with your conversational partners, is argued to underlie many desirable social effects. By generating proper verbal and nonverbal behaviors, virtual humans have been seen to create rapport during interactions with human users. In this paper, we introduce our approach to creating rapport following Tickle-Degnen and Rosenberg’s threefactor (positivity, mutual attention and coordination) theory of rapport. By comparing with a previously published virtual agent, the Rapport Agent, we show that our virtual human predicts the timing of backchannel feedback and end-of-turn more precisely, performs more natural behaviors and, thereby creates much stronger feelings of rapport between users and virtual agents.


Rapport Virtual human Positivity Mutual attention Coordination 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Drolet, A.L., Morris, M.W.: Rapport in conflict resolution: accounting for how face-to-face contact fosters mutual cooperation in mixed-motive conflicts. Experimental Social Psychology 36, 26–50 (2000)CrossRefGoogle Scholar
  2. 2.
    Burns, M.: Rapport and relationships: the basis of child care. J. Child Care 2, 47–57 (1984)Google Scholar
  3. 3.
    Tatar, D.: Social and personal consequences of a preoccupied listener. Stanford University, Stanford (1997)Google Scholar
  4. 4.
    Bernieri, F.J., Rosenthal, R.: Interpersonal coordination: Behavior matching and interactional synchrony. In: Feldman, R.S., Rime, B. (eds.) Fundamentals of Nonverbal Behaviors. Studies in emotion and social interaction, pp. 401–432. Cambridge University Press, Cambridge (1991)Google Scholar
  5. 5.
    Tickle-Degnen, L., Rosenthal, R.: The Nature of Rapport and its Nonverbal Correlates. Psychological Inquiry 1(4), 285–293 (1990)CrossRefGoogle Scholar
  6. 6.
    Cassell, J., Gill, A.J., Tepper, P.A.: Coordination in conversation and rapport. In: Proceedings of the Workshop on Embodied Language Processing, pp. 41–50 (2007)Google Scholar
  7. 7.
    Bailenson, J.N., Yee, N.: Digital Chameleons: Automatic assimilation of nonverbal gestures in immersive virtual environments. Psychological Science 16, 814–819 (2005)CrossRefGoogle Scholar
  8. 8.
    Bickmore, T., Puskar, K., Schlenk, E., Pfeifer, L., Sereika, S.: Maintaining Reality: Relational Agents for Antipsychotic Medication Adherence. J. Interacting with Computers special issue on Mental Health (2010)Google Scholar
  9. 9.
    Bevacqua, E., Mancini, M., Pelachaud, C.: A listening agent exhibiting variable behaviour. In: Prendinger, H., Lester, J.C., Ishizuka, M. (eds.) IVA 2008. LNCS (LNAI), vol. 5208, pp. 262–269. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  10. 10.
    Kang, S.-H., Gratch, J., Watts, J.: The Effect of Affective Iconic Realism on Anonymous Interactants’ Self-Disclosure. In: Proceedings of Interaction Conference for Human-Computer Interaction (2009)Google Scholar
  11. 11.
    Gratch, J., Wang, N., Gerten, J., Fast, E., Duffy, R.: Creating rapport with virtual agents. In: Pelachaud, C., Martin, J.-C., André, E., Chollet, G., Karpouzis, K., Pelé, D. (eds.) IVA 2007. LNCS (LNAI), vol. 4722, pp. 125–138. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  12. 12.
    Gratch, J., Okhmatovskaia, A., Lamothe, F., Marsella, S.C., Morales, M., van der Werf, R.J., Morency, L.-P.: Virtual rapport. In: Gratch, J., Young, M., Aylett, R.S., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 14–27. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  13. 13.
    Morency, L.-P., et al.: Contextual Recognition of Head Gestures. In: Proceedings of 7th International Conference on Multimodal Interactions (2005)Google Scholar
  14. 14.
    Thiebaux, M., Marshall, A., Marsella, S., Kallmann, M.: SmartBody: Behavior Realization for Embodied Conversational Agents. In: Proceedings of 7th International Conference on Autonomous Agents and Multi-Agent Systems (2008)Google Scholar
  15. 15.
    Wang, N., Gratch, J.: Don’t just stare at me. In: Proceedings of 28th Annual CHI Conference on Human Factors in Computing Systems (2010)Google Scholar
  16. 16.
    von der Pütten, A., Krämer, N., Gratch, J.: Who’s there? Can a Virtual Agent Really Elicit Social Presence. In: Proceedings of 12th Annual International Workshop on Presence (2009)Google Scholar
  17. 17.
    Wang, N., Gratch, J.: Rapport and Facial Expression. In: Proceedings of the International Conference on Affective Computing and Intelligent Interaction (2009)Google Scholar
  18. 18.
  19. 19.
    Jonsdottir, G.R., Thorisson, K.R., Nivel, E.: Learning smooth, human-like turntaking in realtime dialogue. In: Prendinger, H., Lester, J.C., Ishizuka, M. (eds.) IVA 2008. LNCS (LNAI), vol. 5208, pp. 162–175. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  20. 20.
    Huang, L., Morency, L.-P., Gratch, J.: Parasocial Consensus Sampling: Combining Multiple Perspectives to Learn Virtual Human Behavior. In: Proceedings of 9th International Conference on Autonomous Agents and Multiagent Systems (2010)Google Scholar
  21. 21.
    Morency, L.-P., de Kok, I., Gratch, J.: Predicting listener backchannels: A probabilistic multimodal approach. In: Prendinger, H., Lester, J.C., Ishizuka, M. (eds.) IVA 2008. LNCS (LNAI), vol. 5208, pp. 176–190. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  22. 22.
    Lee, J., Marsella, S.: Learning a Model of Speaker Head Nods using Gesture Corpora. In: Proceedings of 8th International Conference on Autonomous Agent and Multiagent System, pp. 189–296 (2009)Google Scholar
  23. 23.
    ter Maat, M., Truong, K., Heylen, D.: How turn-taking strategies influence users’ impressions of an agent. In: Proceedings of 10th International Conference on Intelligent Virtual Agents (2010)Google Scholar
  24. 24.
    Huang, L., Morency, L.-P., Gratch, J.: A Multimodal End-of-Turn Prediction Model: Learning from Parasocial Consensus Sampling. In: Proceedings of 10th International Conference on Autonomous Agents and Multiagent Systems (2011)Google Scholar
  25. 25.
    Moon, Y.: Intimate exchanges: Using computers to elicit self-disclosure from consumers. J. Consumer Research 26(4), 323–339 (2000)CrossRefGoogle Scholar
  26. 26.
    Bailenson, J.N., Blascovich, J., Beall, A.C., Loomis, J.M.: Equilibrium revisited: Mutual gaze and personal space in virtual environment. PRESENCE: Teleoperators and Virtual Environments 10, 583–598 (2001)CrossRefGoogle Scholar
  27. 27.
    Lafferty, J., McCallum, A., Pereira, F.: Conditional Random Field: Probabilistic Models for Segmenting and Labeling Sequence Data. In: Proceedings of 18th International Conference on Machine Learning (2001)Google Scholar
  28. 28.
    Kang, S.-H., Gratch, J.: People Like Virtual Counselors that Highly-Disclose About Themselves. The Annual Review of Cybertherapy and Telemedicine (2011)Google Scholar
  29. 29.
    Kang, S.-H., Watts, J., Gratch, J.: Associations between interactants’ personality traits and their feelings of rapport in interactions with virtual humans. In: 59th Annual Conference of the International Communication Association (2009)Google Scholar
  30. 30.
    Murphy, K.: Dynamic Bayesian Networks: Representation, Inference and Learning. Ph.D. Thesis, UC Berkeley, Computer Science Division (July 2002)Google Scholar
  31. 31.
    Huang, L., Morency, L.-P., Gratch, J.: Learning Backchannel Prediction Model from Parasocial Consensus Sampling: A Subjective Evaluation. In: Proceedings of 10th International Conference on Intelligent Virtual Agent (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Lixing Huang
    • 1
  • Louis-Philippe Morency
    • 1
  • Jonathan Gratch
    • 1
  1. 1.Institute for Creative TechnologiesUniversity of Southern CaliforniaPlaya VistaUSA

Personalised recommendations