A study of computational and human strategies in revelation games

Noam Peled; Ya’akov (Kobi) Gal; Sarit Kraus

doi:10.1007/s10458-014-9253-5

Autonomous Agents and Multi-Agent Systems

January 2015, Volume 29, Issue 1, pp 73–97 | Cite as

A study of computational and human strategies in revelation games

Authors
Authors and affiliations

Noam Peled
Ya’akov (Kobi) GalEmail author
Sarit Kraus

Article

First Online: 20 February 2014

336 Downloads
8 Citations

Abstract

Many negotiations in the real world are characterized by incomplete information, and participants’ success depends on their ability to reveal information in a way that facilitates agreements without compromising their individual gain. This paper presents an agent-design that is able to negotiate proficiently with people in settings in which agents can choose to truthfully reveal their private information before engaging in multiple rounds of negotiation. Such settings are analogous to real-world situations in which people need to decide whether to disclose information such as when negotiating over health plans and business transactions. The agent combined a decision-theoretic approach with traditional machine-learning techniques to reason about the social factors that affect the players’ revelation decisions on people’s negotiation behavior. It was shown to outperform people as well as agents playing the equilibrium strategy of the game in empirical studies spanning hundreds of subjects. It was also more likely to reach agreement than people or agents playing equilibrium strategies. In addition, it had a positive effect on people’s play, allowing them to reach significantly better performance when compared to people’s play with other people. These results are shown to generalize for two different settings that varied how players depend on each other in the negotiation.

Keywords

Human–computer negotiation Opponent modeling Empirical studies

This is a preview of subscription content, to check access.

Notes

Acknowledgments

This research is supported in part by the U.S. Army Research Laboratory and the U.S. Army Research Ofce under Grant number W911NF-08-1-0144, by ERC Grant #267523, EU Grant FP7-ICT-2011-9 #600854 and Marie Curie Grant #268362.

Appendix

Before proving Proposition 1, we will postulate the following lemma:

Lemma 1

Suppose that player $j$ uses response strategy $a_{j}^{2}(\omega ,t_j)$ (Eq. 2) to accept proposals $\omega $ at round 2. Then player $i$ is always indifferent between revealing and not revealing its type, regardless of the revelation decision of player $j$.

Proof

We will show that the the revelation decisions of player $i$ cannot affect its outcome in the game. The strategy $a_{j}^{2}(\omega ,t_j)$ of player $j$ to accept the proposal by $i$ at round 2 does not depend on $i$’s type. Hence the revelation decision of $i$ in round 0 does not affect its benefit from proposals in round 2. Suppose that player $i$ revealed its type. If it can get a more beneficial proposal from player $j$ by not revealing its type, then it will reject $j$’s offer and make the more beneficial proposal in round 2. A similar argument can be used to show that player $i$ can get its more preferred outcome when it does not reveal its type. Hence the revelation decision of $i$ does not affect its outcome.

We provide a proof to Proposition 1 by showing that the strategies specified in Sect. 4.1 correspond to a perfect Bayesian equilibrium given the sufficient conditions.

Proof

We denote the possible types for $j$ as $t_{j_1}$ and $t_{j_2}$ and the possible types for $i$ as $t_{i_1}$ and $t_{i_2}$. We prove that when one type of player $j$ is preferable to the other type (Eq. 6), there always exists a revelation equilibrium in which players reveal their goals for both types.¹⁰ In round 2, player $i$ makes the proposal $\omega _{i,{j}}^{2^{*}}(t_j)$ using Eq. 4. This proposal gives the maximum benefit for player $i$ among the proposals that are accepted by player $j$. Any proposal with higher benefit for $i$ will be rejected by player $j$.¹¹ Player $j$ also cannot benefit from deviation as rejecting the proposal incurs zero benefit to both players.

We proceed to round 1. We first specify why the belief update rules are part of a perfect Bayesian equilibrium. Suppose $j$ is of type $t_{j_2}$ and that this type is preferable for $j$ (without loss of generality). If $j$ revealed its type, then $i$ uses Bayes rule to assign probability 1 to type $t_{j_2}$, as specified in equilibrium. If $j$ does not reveal its type, then the resulting information set is off the equilibrium path. In this case $i$ makes an arbitrary belief update to assign probability 0 to type $t_{j_2}$. Using a similar argument, we can show that the belief update rules are part of a perfect Bayesian equilibrium when type $t_{j_1}$ is preferable to $j$. Suppose that player $i$ is of type $t_{i_1}$. If $i$ reveals its type, then $j$ uses Bayes rule to assign probability 1 to $t_{i_2}$, as specified in equilibrium. If $i$ does not reveal its type, then the resulting information set is off the equilibrium path. In this case $j$ makes an arbitrary belief update to assign probability 1 to $t_{i_1}$. Using a similar argument, we can show that the belief update rules are part of a perfect Bayesian equilibrium when $i$ is of type $t_{i_2}$.

Next, we specify why players’ actions are in equilibrium in round 1. Player $j$ makes the proposal $w^{1*}_{j,i}(t_i)$ using Eq. 5. Any proposal with a higher benefit to $j$ will necessarily be rejected by player $i$. As a result, player $j$’s proposal maximizes its benefit given $i$’s response strategy. Player $i$ also cannot benefit from deviation because player $j$’s proposal gives it the same benefit as its counter-proposal $\omega _{i,j}^{2*}(t_j)$.

Finally, we consider round 0, in which both players $i$ and $j$ reveal their types for both of their types. A deviation for player $j$ of type $t_{j_2}$ in round 0 means that $j$ chooses not to reveal its type. According to the belief-update rules, player $i$ will update its beliefs to assign probability 1 to type $t_{j_1}$. As a result, it will make a proposal in round 2 to $j$ that is less preferable to the one it would receive if it revealed its type. Thus, deviation from its revelation decision is not strictly beneficial for $j$ when it is of type $t_{j_2}$. In a similar fashion, we can show that deviation from a revelation decision is not beneficial for $j$ when it is of type $t_{j_1}$. Thus the revelation strategies of round 0 are in Equilibrium for $j$.

Due to Lemma 1, a decision by player $i$ to reveal its type in round 0 for each of its types is in equilibrium. Thus there always exists a revelation equilibrium in which players reveal their goals for both types.

We provide a proof to Proposition 1 by showing that the strategies specified in Sect. 4.2 specify a perfect Bayesian Equilibrium.

Proof

We use backward induction. In Round 2 there are two cases. On the equilibrium path. In this case the players did not update their beliefs according to the equilibrium specification.

Player $i$ makes the proposal $\omega _{i,{j}}^{2^{*}}(h^1,t_i)$ using Eq. 10. This proposal gives the maximum expected benefit for player $i$, so it cannot on expectation benefit from deviation. Player $j$ also cannot benefit from deviation as rejecting the proposal incurs zero benefit to both players.

Lets examine the cases where the information set is off the equilibrium path. The first case is related to round 0: if $i$ revealed its type, this will not affect the proposal at round 2 (Proposition 2). If $j$ revealed its type, then $i$ knows its true type, and the strategies are in equilibrium as in the revelation equilibrium. The other case is related to round 1: In this case, as defined in the Equilibrium, the players do not change their beliefs, and the strategies in round 2 are as in the equilibrium. We now proceed to Round 1. we show neither player has incentive to deviate from its strategy. Suppose player $j$ of type $t_j$ wishes to deviate and propose a different proposal. If the proposal is worse off for player $i$, given its type $t_i$, then player $i$ will reject the offer and player $j$ will not benefit. If player $i$ accepts the proposal (according to Eq. 12), this means that it is greater or equal in benefit to $i$ than the equilibrium proposal in round 2. By the second condition (Eq. 14), this means that the proposal will be at most equal to the benefit that player $j$ will receive from the equilibrium proposal. So player $j$ will not benefit from making this proposal. For player $i$, it cannot benefit from rejecting $j$’s proposal. According to Eq. 12, player $i$ accepts player $j$’s proposal only if it cannot get a higher benefit from its own counter-proposal.

Finally, Lets examine if the player can benefit from deviation in Round 0. Suppose that player $j$ of type $t_{j_1}$ deviates and reveals its type. By definition, then player $i$ will update its beliefs to its true type. In this case, player $i$ will make the proposal $\omega _{i,j}^{2^*}(t_{j_1})$. However, by the first condition (Eq. 13), this proposal is at most equal to the proposal that player $j$ will get from not revealing its type. Therefore it is not beneficial for player $j$ to deviate. By Proposition 1, player $i$ is indifferent between its two types. Thus the strategy for both players satisfies a perfect Bayesian equilibrium.

We conclude by showing that the sufficient conditions of proposal 2 hold in the symmetric (Fig. 1a) and the asymmetric (Fig. 1b) boards described in the paper:

Symmetric board:: For the first condition (13), $\pi _j(\omega _{i,j_l}^{2*}(t_j)\mid t_l)=20$ for $l\in {1,2}$, is smaller than $\pi _j(\omega _{i,j}^{2*}(h^1,t_i)=25$, thus the condition holds. For the second condition (14), $\omega _{i,j}^{2*}(h^1,t_i)$ let player $i$ reach its goal, and so for both types of player $j$. In such case, the only way for a player to get more points is by asking for more chips, which means that the game turns to be a zero-sum game. In such case, there is no proposal that can be more beneficial for both players, thus the condition holds.
Asymmetric board:: For the first condition (13), $\pi _j(\omega _{i,j_l}^{2*}(t_j)\mid t_1)=30$ for $t_1$ as the weak type, and $\pi _j(\omega _{i,j_l}^{2*}(t_j)\mid t_1)=10$ for $t_2$ as the strong type. $\pi _j(\omega _{i,j}^{2*}(h^1,t_i)=35$ for $t_1$ and $15$ for $t_2$, thus the condition holds. For the second condition (14), like in the symmetric board $\omega _{i,j}^{2*}(h^1,t_i)$ let both types of player $j$ and player $i$ to reach their goal, so the game turns to be a zero-sum game, and the condition holds.

References

1.
Amir, O., Rand, D. G., et al. (2012). Economic games on the internet: The effect of ${\$}$1 stakes. PloS ONE, 7(2), e31461.CrossRefGoogle Scholar
2.
Azaria, A., Rabinovich, Z., Kraus, S., & Goldman, C. V. (2011). Strategic information disclosure to people with multiple alternatives. In Proceedings of AAAI.Google Scholar
3.
Baarslag, T., Fujita, K., Gerding, E. H., Hindriks, K., Ito, T., Jennings, N. R., et al. (2013). Evaluating practical negotiating agents: Results and analysis of the 2011 international competition. Artificial Intelligence, 198, 73–103.CrossRefGoogle Scholar
4.
Banks, J., Camerer, C. F., & Porter, D. (1994). Experimental tests of nash refinements in signaling games. Games and Economic Behavior, 6, 1–31.CrossRefMATHMathSciNetGoogle Scholar
5.
Bench-Capon, T., Atkinson, K., & McBurney, P. (2009). Altruism and agents: an argumentation based approach to designing agent decision mechanisms. In Proceedings of AAMAS.Google Scholar
6.
Coehoorn, R. M., & Jennings, N. R. (2004). Learning on opponent’s preferences to make effective multi-issue negotiation trade-offs. In Proceedings of EC.Google Scholar
7.
de Melo, C., Carnevale, P., Read, S., Antos, D., & Gratch, J. (2012). Bayesian model of the social effects of emotion in decision-making in multiagent systems. In Proceedings of AAMAS.Google Scholar
8.
Erev, I., & Roth, A. E. (1998). Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria. American Economic Review, 88(4), 848–881.Google Scholar
9.
Gal, Y., Grosz, B., Kraus, S., Pfeffer, A., & Shieber, S. (2010). Agent decision-making in open-mixed networks. Artificial Intelligence, 174(18), 1460–1480.CrossRefMathSciNetGoogle Scholar
10.
Gal, Y., Kraus, S., Gelfand, M., Khashan, H., & Salmon, E. (2012). Negotiating with people across cultures using an adaptive agent. ACM Transactions on Intelligent Systems and Technology, 3(1).Google Scholar
11.
Gal, Y., & Pfeffer, A. (2007). Modeling reciprocity in human bilateral negotiation. In Proceedings of AAAI.Google Scholar
12.
Grosz, B., Kraus, S., Talman, S., & Stossel, B. (2004). The influence of social dependencies on Decision-Making. In Proceedings of AAMAS.Google Scholar
13.
Haim, G., Gal, Y., Kraus, S., & Gelfand, M. J. (2012). A cultural sensitive agent for Human–Computer negotiation. In Proceedings of AAMAS.Google Scholar
14.
Hajaj, C., Hazon, N., Sarne, D., & Elmalech, A. (2013). Search more, disclose less. In Proceedings of AAAI.Google Scholar
15.
Hindriks, K., & Tykhonov, D. (2008). Opponent modelling in automated multi-issue negotiation using bayesian learning. In Proceedings of AAMAS, (pp. 331–338).Google Scholar
16.
Jonker, C. M., Robu, V., & Treur, J. (2007). An agent architecture for multi-attribute negotiation using incomplete preference information. Autonomous Agents and Multi-Agent Systems, 15(2), 221–252.CrossRefGoogle Scholar
17.
Lin, R., & Kraus, S. (2010). Can automated agents proficiently negotiate with humans? Communications of the ACM, 53(1), 78–88.CrossRefGoogle Scholar
18.
McKelvey, R. D., & Palfrey, T. R. (1992). An experimental study of the centipede game. Econometrica: Journal of the Econometric Society, 60(4), 803–836.CrossRefMATHGoogle Scholar
19.
Osborne, M. J., & Rubinstein, A. (1999). A course in game theory. Cambridge, MA: MIT Press.Google Scholar
20.
Oshrat, Y., Lin, R., & Kraus, S. (2009). Facing the challenge of human-agent negotiations via effective general opponent modeling. In Proceedings of AAMAS.Google Scholar
21.
Pasquier, P., Hollands, R., Dignum, F., Rahwan, I., & Sonenberg, L. (2007). An empirical study of interest-based negotiation. In Proceedings of ICEC.Google Scholar
22.
Peled, N., Gal, Y., & Kraus, S. (2011). A study of computational and human strategies in revelation games. In Proceedings of AAMAS.Google Scholar
23.
Rosenfeld, A., & Kraus, S. (2012). Modeling agents based on aspiration adaptation theory. Autonomous Agents and Multi-Agent Systems, 24(2), 221–254.CrossRefGoogle Scholar
24.
Sarne, D., Elmalech, A., Grosz, B. J., & Geva, M. (2011). Less is more: Restructuring decisions to improve agent search. In Proceedings of AAMAS.Google Scholar
25.
Spence, A. M. (1974). Market signaling. Cambridge, MA: Harvard University Press.Google Scholar
26.
Sycara, K. P. (1990). Persuasive argumentation in negotiation. Theory and decision, 28(3), 203–242.CrossRefGoogle Scholar
27.
Wedekind, C., & Milinski, M. (1996). Human cooperation in the simultaneous and the alternating prisoner’s dilemma. PNAS, 93(7), 2686.CrossRefGoogle Scholar

Copyright information

Authors and Affiliations

Noam Peled
- 1
Ya’akov (Kobi) Gal
- 2
Email author
Sarit Kraus
- 3
- 4

1.Gonda Brain Research CenterBar-Ilan UniversityRamat GanIsrael
2.Department of Information Systems EngineeringBen-Gurion UniversityBeer ShevaIsrael
3.Department of Computer ScienceBar-Ilan UniversityRamat GanIsrael
4.Institute of Advanced Computer StudiesUniversity of MarylandCollege ParkUSA

A study of computational and human strategies in revelation games

Abstract

Keywords

Notes

Acknowledgments

References

Copyright information

Authors and Affiliations

Personalised recommendations

Cite article