Abstract
We posit that observed differences in negotiation performance across cultures can be explained by participants trying to optimize across multiple values, where the relative importance of values differs across cultures. We look at two ways for specifying weights on values for different cultures: one in which the weights of the model are hand-crafted, based on intuition interpreting Hofstede dimensions for the cultures, and one in which the weights of the model are learned from data using inverse reinforcement learning (IRL). We apply this model to the Ultimatum Game and integrate it into a virtual human dialog system. We show that weights learned from IRL surpass both a weak baseline with random weights and a strong baseline considering only one factor for maximizing gain in own wealth in accounting for the behavior of human players from four different cultures. We also show that the weights learned with our model for one culture outperform weights learned for other cultures when playing against opponents of the first culture.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
We used only some of the many reward functions learned in experiment 1 for each culture to learn policies for other culture data. In this table we show results for the reward functions that are closest to the median values reported in Table 7, but in some cases they are not identical.
Abbreviations
- MARV:
-
Multi-attribute relational value
- RL:
-
Reinforcement learning
- IRL:
-
Inverse reinforcement learning
- MDP:
-
Markov decision process
- SU:
-
Simulated user
- KL divergence:
-
Kullback–Leibler divergence
References
Abbeel P, Ng AY (2004) Apprenticeship learning via inverse reinforcement learning. In: Proceedings of the 21st international conference on machine learning (ICML)
Allais M (1953) Le comportement de l’homme rationnel devant le risqué: critique des postulats et axioms de l’ecole americaine. Econometrica 21(4):503–546
Brett JM, Gelfand MJ (2006) A cultural analysis of the underlying assumptions of negotiation theory. In: Thompson L (ed) Frontiers of negotiation research. Psychology Press, Philadelphia, pp 173–201
Buchan NR, Croson RTA, Johnson EJ (1999) Understanding what’s fair: contrasting perceptions of fairness in ultimatum bargaining in Japan and the United States. Discussion paper, University of Wisconsin
Camerer CF (2003) Behavioral game theory—experiments in strategic interaction. Princeton University Press, Princeton
Gal Y, Pfeffer A, Marzo F, Grosz BJ (2004) Learning social preferences in games. In: Proceedings of the 19th national conference on artificial intelligence
Gandhe S, DeVault D, Roque A, Martinovski B, Artstein R, Leuski A, Gerten J, Traum DR (2008) From domain specification to virtual humans: an integrated approach to authoring tactical questioning characters. In: Proceedings of the annual conference of the international speech communication association (INTERSPEECH)
Georgila K, Henderson J, Lemon O (2006) User simulation for spoken dialogue systems: learning and evaluation. In: Proceedings of the annual conference of the international speech communication association (INTERSPEECH)
Güth W, Schmittberger R, Schwarze B (1982) An experimental analysis of ultimatum bargaining. J Econ Behav Organ 3(4):367–388
Hartholt A, Gratch J, Weiss L, Leuski A, Morency L-P, Liewer M, Thiebaux M, Marsella S, Doraiswamy P, Tsiartas A, LeMasters K, Fast E, Sadek R, Marshall A, Lee J, Pickens L (2009) At the virtual frontier: introducing Gunslinger, a multi-character, mixed-reality, story-driven experience. In: Proceedings of the 9th international conference on intelligent virtual agents (IVA)
Henrich J (2000) Does culture matter in economic behavior? Ultimatum game bargaining among the Machiguenga of the Peruvian Amazon. Am Econ Rev 90:973–979
Henrich J, Boyd R, Bowles S, Camerer C, Fehr E, Gintis H, McElreath R, Alvard M, Barr A, Ensminger J, Henrich NS, Hill K, Gil-White F, Gurven M, Marlowe FW, Patton JQ, Tracer D (2005) Cross-cultural perspective: behavioral experiments in 15 small-scale societies. Behav Brain Sci 28(6):795–815
Hofstede GH (2001) Culture’s consequences: comparing values, behaviors, institutions, and organizations across nations. SAGE, Thousand Oaks
Lewis DK (1969) Convention: a philosophical study. Harvard University Press, Cambridge
Mascarenhas S, Dias J, Afonso N, Enz S, Paiva A (2009) Using rituals to express cultural differences in synthetic characters. In: Proceedings of the 8th international conference on autonomous agents and multiagent systems (AAMAS)
Neumann JV, Morgenstern O (1944) Theory of games and economic behavior. Princeton University Press, Princeton
Okada A, Riedl A (1999) When culture does not matter: experimental evidence from coalition ultimatum games in Austria and Japan. Discussion paper, University of Amsterdam
Oosterbeek H, Sloof R, van de Kuilen G (2004) Cultural differences in ultimatum game experiments: evidence from a meta-analysis. Exp Econ 7:171–188
Rawls J (1971) A theory of justice. The Belknap Press of Harvard University Press, Cambridge
Roth AE, Prasnikar V, Okuno-Fujiwara M, Zamir S (1991) Bargaining and market behavior in Jerusalem, Ljubljana, Pittsburgh, and Tokyo: an experimental study. Am Econ Rev 81(5):1068–1095
Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge
Turan N, Dudik M, Gordon G, Weingart LR (2011) Modeling group negotiation: three computational approaches that can inform behavioral sciences. In: Mannix EA, Neale MA, Overbeck JR (eds) Negotiation and groups (research on managing groups and teams), vol 14. Emerald Group Publishing, London, pp 189–205
Acknowledgments
This work was funded by the NSF Grant IIS-1117313 and a MURI award through ARO Grant No. W911NF-08-1-0301.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Nouri, E., Georgila, K. & Traum, D. Culture-specific models of negotiation for virtual characters: multi-attribute decision-making based on culture-specific values. AI & Soc 32, 51–63 (2017). https://doi.org/10.1007/s00146-014-0570-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00146-014-0570-7