Skip to main content

Creating Standardized Video Recordings of Multimodal Interactions across Cultures

  • Chapter
Multimodal Corpora (MMCorp 2008)

Abstract

Trying to adapt the behavior of an interactive system to the cultural background of the user requires information on how relevant behaviors differ as a function of the user’s cultural background. To gain such insights in the interrelation of culture and behavior patterns, the information from the literature is often too anecdotal to serve as the basis for modeling a system’s behavior, making it necessary to collect multimodal corpora in a standardized fashion in different cultures. In this chapter, the challenges of such an endeavor are introduced and solutions are presented by examples from a German-Japanese project that aims at modeling culture-specific behaviors for Embodied Conversational Agents.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Argyle, M.: Bodily Communication. Methuen & Co. Ltd., London (1975)

    Google Scholar 

  2. Bennett, M.J.: A developmental approach to training for intercultural sensitivity. International Journal for Intercultural Relations 10(2), 179–195 (1986)

    Article  Google Scholar 

  3. Berger, J., Fisek, M.H., Norman, R.Z., Zelditch Jr., M.: Status Characteristics and Social Interaction: An Expectation-States Approach. Elsevier, Amsterdam (1977)

    Google Scholar 

  4. Bull, P.E.: Posture and Gesture. Pergamon Press, Oxford (1987)

    Google Scholar 

  5. Burgoon, J.K., Buller, D.B., Woodall, W.G.: Nonverbal Communication: The Unspoken Dialogue. Harper and Row, New York (1989)

    Google Scholar 

  6. Core, M., Allen, J.: Coding Dialogs with the DAMSL Annotation Scheme. In: Proceedings of the AAAI Fall Symposium on Communicative Action in Humans and Machines, Boston, MA (1997)

    Google Scholar 

  7. Efron, D.: Gesture, Race and Culture. Mouton and Co., Netherlands (1972)

    Google Scholar 

  8. Ellyson, S.L., Dovidio, J.F., Fehr, B.J.: Visual behavior and dominance in women and men. In: Mayo, C., Henely, N.M. (eds.) Gender and nonverbal behavior. Springer, Heidelberg (1981)

    Google Scholar 

  9. Endrass, B., Rehm, M., André, E.: Culture-specific communication management for virtual agents. In: Proceedings of AAMAS (2009)

    Google Scholar 

  10. Gallaher, P.E.: Individual Differences in Nonverbal Behavior: Dimensions of Style. Journal of Personality and Social Psychology 63(1), 133–145 (1992)

    Article  Google Scholar 

  11. Greenbaum, P.E., Rosenfeld, H.M.: Varieties of touching in greeting: Sequential structure and sex-related differences. Journal of Nonverbal Behavior 5, 13–25 (1980)

    Article  Google Scholar 

  12. Hall, E.T.: The Silent Language. Doubleday (1959)

    Google Scholar 

  13. Hall, E.T.: The Hidden Dimension. Doubleday (1966)

    Google Scholar 

  14. Hall, J.A., Friedman, G.B.: Status, gender, and nonverbal behavior: A study of structured interactions between employees of a company. Personality and Social Psychology Bulletin 25(9), 1082–1091 (1999)

    Article  Google Scholar 

  15. Hofstede, G.: Cultures Consequences: Comparing Values, Behaviors, Institutions, and Organizations Across Nations. Sage Publications, Thousand Oaks (2001)

    Google Scholar 

  16. Hofstede, G.J., Pedersen, P.B., Hofstede, G.: Exploring Culture: Exercises, Stories, and Synthetic Cultures. Intercultural Press, Yarmouth (2002)

    Google Scholar 

  17. Johnson, C.: Gender, legitimate authority, and leader-subordinate conversations. American Sociological Review 59, 122–135 (1994)

    Article  Google Scholar 

  18. Kendon, A.: Conducting Interaction: Patterns of Behavior in Focused Encounters. Cambridge Univ. Press, Cambridge (1991)

    Google Scholar 

  19. Knapp, M.L., Vangelisti, A.L.: Interpersonal Communication and Human Relationships. Pearson Education, Inc., Boston (1984)

    Google Scholar 

  20. Knoterus, J.D., Greenstein, T.N.: Status and performance characteristics in social interaction: A theory of status validation. Social Psychology Quarterly 44(4), 338–349 (1981)

    Article  Google Scholar 

  21. Knudsen, M.W., Martin, J.-C., Dybkjær, L., Ayuso, M.J.M., Bernsen, N.O., Carletta, J., Heid, U., Kita, S., Llisterri, J., Pelachaud, C., Poggi, I., Reithinger, N., van Elswijk, G., Wittenburg, P.: ISLE Natural Interactivity and Multimodality Working Group Deliverable D9.1: Survey of Multimodal Coding Schemes and Best Practice (2002), http://isle.nis.sdu.dk/reports/wp9/D9.1-7.3.2002-F.pdf (07.02.07)

  22. Leffler, A., Gillespie, D.L., Conaty, J.C.: The effects of status differentiation on nonverbal behavior. Social Psychology Quarterly 45(3), 153–161 (1982)

    Article  Google Scholar 

  23. McCrae, R.R., Allik, J. (eds.): The Five-Factor Model of Personality Across Cultures. Kluwer Academics, Dordrecht (2002)

    Google Scholar 

  24. McNeill, D.: Hand and Mind — What Gestures Reveal about Thought. The University of Chicago Press, Chicago (1992)

    Google Scholar 

  25. Nazir, A., Lim, M.Y., Kriegel, M., Aylett, R., Cawsey, A., Enz, S., Zoll, C.: Culture-personality based affective model. In: Proceedings of the IUI workshop on Enculturating Conversational Interfaces, Gran Canaria (2008)

    Google Scholar 

  26. Quaddus, M.A., Tung, L.L.: Explaining cultural differences in decision conferencing. Communications of the ACM 45, 93–98 (2002)

    Article  Google Scholar 

  27. Rehm, M., Nakano, Y., André, E., Nishida, T.: Culture-specific first meeting encounters between virtual agents. In: Prendinger, H., Lester, J.C., Ishizuka, M., et al. (eds.) IVA 2008. LNCS (LNAI), vol. 5208, pp. 223–236. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  28. Rehm, M., Nakano, Y., André, E., Nishida, T., Bee, N., Endrass, B., Huang, H.-H., Lipi, A.A., Wissner, M.: From Observation to Simulation — Generating Culture Specific Behavior for Interactive Systems. In: AI & Society (in press)

    Google Scholar 

  29. Schwartz, S.H., Sagiv, L.: Identifying culture-specifics in the content and structure of values. Journal of Cross-Cultural Psychology 26(1), 92–116 (1995)

    Article  Google Scholar 

  30. Teng, J.T.C., Calhoun, K.J., Cheon, M.J., Raeburn, S., Wong, W.: Is the east really different from the west: a cross-cultural study on information technology and decision making. In: Proceedings of the 20th international conference on Information Systems, pp. 40–46 (1999)

    Google Scholar 

  31. Ting-Toomey, S.: Communicating Across Cultures. The Guilford Press, New York (1999)

    Google Scholar 

  32. Traum, D., Swartout, W., Marsella, S., Gratch, J.: Fight, Flight, or Negotiate: Believable Strategies for Conversing Under Crisis. In: Panayiotopoulos, T., Gratch, J., Aylett, R.S., Ballin, D., Olivier, P., Rist, T. (eds.) IVA 2005. LNCS (LNAI), vol. 3661, pp. 52–64. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  33. Triandis, H.C., Suh, E.M.: Cultural influences on personality. Annual Review of Psychology 53, 133–160 (2002)

    Article  Google Scholar 

  34. Trompenaars, F., Hampden-Turner, C.: Riding the Waves of Culture. McGraw-Hill, New York (1998)

    Google Scholar 

  35. Ungar, S.: The effects of status and excuse on interpersonal reactions to deviant behavior. Social Psychology Quarterly 44(3), 260–263 (1981)

    Article  Google Scholar 

  36. http://www.jobware.de/ra/fue/vhs/41.html (last visited: November 25, 2008)

  37. Warwick, T.: Function analysis for team problem solving. In: SAVE Annual Proceedings (1994)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Rehm, M. et al. (2009). Creating Standardized Video Recordings of Multimodal Interactions across Cultures. In: Kipp, M., Martin, JC., Paggio, P., Heylen, D. (eds) Multimodal Corpora. MMCorp 2008. Lecture Notes in Computer Science(), vol 5509. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04793-0_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-04793-0_9

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-04792-3

  • Online ISBN: 978-3-642-04793-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics