Culturally Aware Agent Communication

  • Matthias Rehm
  • Yukiko Nakano
  • Tomoko Koda
  • Heike Winschiers-Theophilus
Part of the Studies in Computational Intelligence book series (SCI, volume 396)


Agent based interaction in the form of Embodied Conversational Agents (ECAs) has matured over the last decade and agents have become more and more sophisticated in terms of their verbal and nonverbal behavior like facial expressions or gestures. Having such “natural” communication channels available for expressing not only task-relevant but also socially and psychologically relevant information makes it necessary to take influences into account that are not readily implemented like emotions or cultural heuristics. These influences have a huge impact on the success of an interaction but are often neither consciously generated nor consciously interpreted. In this chapter, we focus on culture as a parameter for successful agent interactions and start with an attempt of grasping what is meant by this multiply defined notion. For the remainder of the chapter, we analyze in detail how culture influences the expression of multimodal behavior in agents.


Facial Expression Multiagent System Nonverbal Behavior Virtual Agent Conversational Agent 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aylett, R., Paiva, A., Vannini, N., Enz, S., André, E., Hall, L.: But that was in another country: agents and intercultural empathy. In: Proceedings of 8th International Conferecne on Autonomous Agents and Multiagent Systems (AAMAS), pp. 329–336 (2009)Google Scholar
  2. 2.
    Bull, P.E.: Posture and Gesture. Pergamon Press (1987)Google Scholar
  3. 3.
    Carney, D.R., Hall, J.A., LeBeau, L.S.: Beliefs about the non-verbal expression of social power. Journal of Nonverbal Behavior 29(2), 105–123 (2005)CrossRefGoogle Scholar
  4. 4.
    Cassell, J., Nakano, Y.I., Bickmore, T.W., Sidner, C.L., Rich, C.: Non-Verbal Cues for Discourse Structure. In: Meeting of the Association for Computational Linguistics, pp. 106–115 (2001)Google Scholar
  5. 5.
    de Meijer, M.: The contribution of general features of body movement to the attribution of emotions. Journal of Nonverbal Behavior 13(4), 247–268 (1989)CrossRefGoogle Scholar
  6. 6.
    Efron, D.: Gesture, Race and Culture. Mouton and Co. (1972)Google Scholar
  7. 7.
    Ekman, P., Friesen, W.V., O’Sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., Krause, R., LeCompte, W.A., Pitcairn, T., Ricci-Bitti, P.E., Scherer, K.R., Tomita, M., Tzavaras, A.: Universals and cultural differences in the judgment of facial expressions of emotion. Journal of Personality and Social Psychology 53, 712–717 (1987)CrossRefGoogle Scholar
  8. 8.
    Ekman, P., Friesen, W.: The repertoire of nonverbal behavior: categories, origins, usage and coding. Semiotica 1, 49–98 (1969)Google Scholar
  9. 9.
    Ekman, P., Rosenberg, E. (eds.): What the Face Reveals: Basic & Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS). Oxford University Press (1998)Google Scholar
  10. 10.
    Elfenbein, H.A., Ambady, N.A.: Universals and cultural differences in understanding emotions. Current Directions in Psychological Science 12(5), 159–164 (2003)CrossRefGoogle Scholar
  11. 11.
    Endrass, B., Rehm, M., André, E.: Planning small talk behavior with cultural influences for multiagent systems. Computer, Speech and Language (in press)Google Scholar
  12. 12.
    Exline, R.V.: Visual behavior as an aspect of power role relationships. In: Pliner, P., Krames, L., Alloway, T. (eds.) Nonverbal communication of aggression, pp. 21–52 (1975)Google Scholar
  13. 13.
    Fogg, B.J.: Persuasive Technology: Using Computers to Change What We Think and Do. Morgan Kaufmann, San Francisco (2003)Google Scholar
  14. 14.
    P.E. Gallaher. Individual Differences in Nonverbal Behavior: Dimensions of Style. Journal of Personality and Social Psychology, 63(1):133–145, 1992.CrossRefGoogle Scholar
  15. 15.
    Goodwin, C.: Conversational Organization — Interaction between Speakers and Hearers. Academic Press, New York (1981)Google Scholar
  16. 16.
    Hall, E.T.: The Silent Language. Doubleday (1959)Google Scholar
  17. 17.
    Hall, E.T.: The Hidden Dimension. Doubleday (1966)Google Scholar
  18. 18.
    Hartmann, B., Mancini, M., Buisine, S., Pelachaud, C.: Design and evaluation of expressive gesture synthesis for embodied conversational agents. In: Proceedings of the Fourth International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), pp. 1095–1096 (2005)Google Scholar
  19. 19.
    Hofstede, G.: Cultures Consequences: Comparing Values, Behaviors, Institutions, and Organizations Across Nations. Sage Publications, Thousand Oaks (2001)Google Scholar
  20. 20.
    Hofstede, G.J., Pedersen, P.B., Hofstede, G.: Exploring Culture: Exercises, Stories, and Synthetic Cultures. Intercultural Press, Yarmouth (2002)Google Scholar
  21. 21.
    Isbister, K., Nakanishi, H., Ishida, T., Nass, C.: Helper agent: designing an assistant for human-human interaction in a virtual meeting space. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 57–64 (2000)Google Scholar
  22. 22.
    Jack, R.E., Blais, C., Scheepers, C., Schyns, P.G., Caldara, R.: Cultural confusions show facial expressions are not universal. Current Biology 19(18), 1543–1548 (2009)CrossRefGoogle Scholar
  23. 23.
    Jan, D., Herrera, D., Martinovski, B., Novick, D., Traum, D.R.: A Computational Model of Culture-Specific Conversational Behavior. In: Pelachaud, C., Martin, J.-C., André, E., Chollet, G., Karpouzis, K., Pelé, D. (eds.) IVA 2007. LNCS (LNAI), vol. 4722, pp. 45–56. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  24. 24.
    Lewis Johnson, W., Valente, A.: Tactical language and culture training systems: Using artificial intelligence to teach foreign languages and cultures. In: Proceedings of IAAI (2008)Google Scholar
  25. 25.
    Kendon, A.: Conducting Interaction: Patterns of Behavior in Focused Encounters. Cambridge Univ Press, Cambridge (1991)Google Scholar
  26. 26.
    Khaled, R., Barr, P., Fischer, R., Noble, J., Biddle, R.: Factoring Culture into the Design of a Persuasive Game. In: OZCHI 2006, pp. 213–220 (2006)Google Scholar
  27. 27.
    Khaled, R., Biddle, R., Noble, J., Barr, P., Fischer, R.: Persuasive interaction for collectivist cultures. In: Piekarski, W. (ed.) The Seventh Australasian User Interface Conference (AUIC 2006), pp. 73–80 (2006)Google Scholar
  28. 28.
    Kluckhohn, F., Strodtbeck, F.: Variations in value orientations. Row, Peterson, New York (1961)Google Scholar
  29. 29.
    Koda, T., Ruttkay, Z., Nakagawa, Y., Tabuchi, K.: Cross-Cultural Study on Facial Regions as Cues to Recognize Emotions of Virtual Agents. In: Ishida, T. (ed.) Culture And Computing. LNCS, vol. 6259, pp. 16–27. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  30. 30.
    Koda, T., Ishida, T., Rehm, M., André, E.: Avatar Culture: Cross-Cultural Evaluations of Avatar Facial ExpressionsAvatar Culture: Cross-Cultural Evaluations of Avatar Facial Expressions. AI & Society, Special Issue on Enculturating HCI 24(3), 237–250 (2009)Google Scholar
  31. 31.
    Labarre, W.: Teh cultural basis of emotions and gestures. Journal of Personality 16, 49–68 (1947)CrossRefGoogle Scholar
  32. 32.
    Laga, H., Amaoka, T.: Modeling the spatial behavior of virtual agents in groups for non-verbal communication in virtual worlds. In: Proceedings of the 3rd International Universal Communication Symposium (IUCS), pp. 154–159 (2009)Google Scholar
  33. 33.
    Chad Lane, H., Hays, M.J.: Getting down to business: Teaching cross-cultural social interaction skills in a serious game. In: Workshop on Culturally Aware Tutoring Systems (CATS), pp. 35–46 (2008)Google Scholar
  34. 34.
    Lipi, A.A., Nakano, Y., Rehm, M.: A parameter-based model for generating culturally adaptive nonverbal behaviors in embodied conversational agents. In: Stephanidis, C. (ed.) Universal Access in HCI, HCII 2009, pp. 631–640. Springer, Heidelberg (2009)Google Scholar
  35. 35.
    Lipi, A.A., Nakano, Y., Rehm, M.: Culture and social relationship as factors of affecting communicative non-verbal behaviors. Transactions of the Japanese Society for Artificial Intelligence 25, 6 (inpress)Google Scholar
  36. 36.
    Matsumoto, D., Ekman, P.: American-japanese cultural differences in intensity ratings. Motivation and Emotion 13, 143–157 (1989)CrossRefGoogle Scholar
  37. 37.
    McNeill, D.: Hand and Mind — What Gestures Reveal about Thought. The University of Chicago Press, London (1992)Google Scholar
  38. 38.
    Pedica, C., Vilhjálmsson, H.H.: Spontaneous Avatar Behavior for Human Territoriality. In: Ruttkay, Z., Kipp, M., Nijholt, A., Vilhjálmsson, H.H., et al. (eds.) IVA 2009. LNCS, vol. 5773, pp. 344–357. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  39. 39.
    Reeves, B., Nass, C.: The Media Equation — How People Treat Computers, Television, and New Media Like Real People and Places. Cambridge University Press, Cambridge (1996)Google Scholar
  40. 40.
    Rehm, M.: Non-symbolic gesture usage for ambient intelligence. In: Human-Centric Interfaces for Ambient Intelligence. Elsevier (2010)Google Scholar
  41. 41.
    Rehm, M., André, E., Bee, N., Endrass, B., Wissner, M., Nakano, Y., Akhter Lipi, A., Nishida, T., Huang, H.-H.: Creating Standardized Video Recordings of Multimodal Interactions Across Cultures. In: Kipp, M., Martin, J.-C., Paggio, P., Heylen, D., et al. (eds.) Multimodal Corpora. LNCS, vol. 5509, pp. 138–159. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  42. 42.
    Rehm, M., André, E., Nischt, M.: Let’s Come Together — Social Navigation Behaviors of Virtual and Real Humans. In: Maybury, M., Stock, O., Wahlster, W. (eds.) INTETAIN 2005. LNCS (LNAI), vol. 3814, pp. 124–133. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  43. 43.
    Rehm, M., Nakano, Y., André, E., Nishida, T., Bee, N., Endrass, B., Wissner, M., Lipi, A.A., Huang, H.-H.: From Observation to Simulation — Generating Culture Specific Behavior for Interactive Systems. AI & Society 24, 267–280 (2009)CrossRefGoogle Scholar
  44. 44.
    Reitberger, W., Meschtscherjakov, A., Mirlacher, T., Scherndl, T., Huber, H., Tscheligi, M.: A persuasive interactive mannequin for shop windows. In: Proceedings of Persuasive 2009 (2009)Google Scholar
  45. 45.
    Rodrigues, I.G.: Gesture Space and Gesture Choreography in European Portuguese and African Portuguese Interactions: A Pilot Study of Two Cases. In: Kopp, S., Wachsmuth, I. (eds.) GW 2009. LNCS, vol. 5934, pp. 23–33. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  46. 46.
    Sanchez-Burks, J., Blount, S., Bartel, C.: Fluidity and performance in intercultural workplace interactions: The role of behavioral mirroring and social sensitivity. Journal of Applied Psychology 94(1), 216–223 (2009)CrossRefGoogle Scholar
  47. 47.
    Scheflen, A.E.: Communicational structure: Analysis of a psychotherapy transaction. Indiana University Press (1973)Google Scholar
  48. 48.
    Schwartz, S.H., Sagiv, L.: Identifying culture-specifics in the content and structure of values. Journal of Cross-Cultural Psychology 26(1), 92–116 (1995)CrossRefGoogle Scholar
  49. 49.
    Spencer-Oatey, H.: Reconsidering power and distance. Journal of Pragmatics 26, 1–24 (1996)CrossRefGoogle Scholar
  50. 50.
    Ting-Toomey, S.: Communicating Across Cultures. The Guilford Press, New York (1999)Google Scholar
  51. 51.
    Vossen, S., Ham, J., Midden, C.: Social influence of a persuasive agent: the role of agent embodiment and evaluative feedback. In: Proceedings of Persuasive 2009 (2009)Google Scholar
  52. 52.
    Yuki, M., Maddux, W.W., Masuda, T.: Are the Windows to the Soul the Same in the East and West? Cultural differences in using the eyes and mouth as cues to recognize emotions in Japan and the United States. Journal of Experimental Social Psychology 43, 303–311 (2007)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Matthias Rehm
    • 1
  • Yukiko Nakano
    • 2
  • Tomoko Koda
    • 3
  • Heike Winschiers-Theophilus
    • 4
  1. 1.Dept. of Architecture, Design, and Media TechnologyAalborg UniversityAalborgDenmark
  2. 2.Dept. of Computer and Information Science, Faculty of Science and TechnologySeikei UniversityTokyoJapan
  3. 3.Dept. of Information Science and TechnologyOsaka Institute of TechnologyOsaka CityJapan
  4. 4.School of Information TechnologyPolytechnic of NamibiaWindhoekNamibia

Personalised recommendations