The field of convention emergence studies how agents involved in repeated coordination games can reach consensus through only local interactions. The literature on this topic is vast and is motivated by human societies, mainly addressing coordination problems between human agents, such as who gets to redial after a dropped telephone call. In contrast, real-world engineering problems, such as coordination in wireless sensor networks, involve agents with limited resources and knowledge and thus pose certain restrictions on the complexity of the coordination mechanisms. Due to these restrictions, strategies proposed for human coordination may not be suitable for engineering applications and need to be further explored in the context of real-world application domains. In this article we take the role of designers of large decentralized multi-agent systems. We investigate the factors that speed up the convergence process of agents arranged in different static and dynamic topologies and under different interaction models, typical for engineering applications. We also study coordination problems both under partial observability and in the presence of faults (or noise). The main contributions of this article are that we propose an approach for emergent coordination, motivated by highly constrained devices, such as wireless nodes and swarm bots, in the absence of a central entity and perform extensive theoretical and empirical studies. Our approach is called Win-Stay Lose-probabilistic-Shift, generalizing two well-known strategies in game theory that have been applied in other domains. We demonstrate that our approach performs well in different settings under limited information and imposes minimal system requirements, due to its simplicity. Moreover, our technique outperforms state-of-the-art coordination mechanisms, guarantees full convergence in any topology and has the property that all convention states are absorbing.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Confirmed through personal communication with the first author.
Axelrod, R. (1984). The evolution of cooperation. New York: Basic Books.
Axelrod, R. (1986). An evolutionary approach to norms. The American Political Science Review, 80(4), 1095–1111.
Barabasi, A. L., Albert, R., & Jeong, H. (1999). Mean-field theory for scale-free random networks. Physica A: Statistical Mechanics and its Applications, 272(1–2), 19.
Barrett, J., & Zollman, K. J. S. (2009). The role of forgetting in the evolution and learning of language. Journal of Experimental & Theoretical Artificial Intelligence, 21(4), 293–309.
Bendor, J., Mookherjee, D., & Ray, D. (1994). Aspirations, adaptive learning and cooperation in repeated games. Tech. rep., Tilburg University, Center for Economic Research.
Bowling, M., & Veloso, M. (2002). Multiagent learning using a variable learning rate. Artificial Intelligence, 136(2), 215–250.
Bramoullé, Y., López-Pintado, D., Goyal, S., & Vega-Redondo, F. (2004). Network formation and anti-coordination games. International Journal of Games Theory, 33(1), 1–19.
Brooks, L., Iba, W., & Sen, S. (2011). Modeling the emergence and convergence of norms. In Proceedings of the Twenty-Second international joint conference on Artificial Intelligence (Vol. 1, pp. 97–102). Menlo Park, CA: AAAI Press.
Castellano, C., Fortunato, S., & Loreto, V. (2009). Statistical physics of social dynamics. Reviews of modern physics, 81(2), 591.
De Hauwere, Y.M. (2011). Sparse interactions in multi-agent reinforcement learning (Ph.D. thesis, Vrije Universiteit Brussel, 2011).
De Vylder B. (2007). The evolution of conventions in multi-agent systems (Ph.D. thesis, Vrije Universiteit Brussel, 2007).
Delgado, J., Pujol, J., & Sanguesa, R. (2003). Emergence of coordination in scale-free networks. Web Intelligence and Agent Systems, 1(2), 131–138.
Farinelli, A., Rogers, A., Petcu, A., & Jennings, N. R. (2008). Decentralised coordination of lowpower embedded devices using the max-sum algorithm. In Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent System (pp. 639–646). Richland, SC.
Fischer, M. J., Na, Lynch, & Paterson, M. S. (1985). Impossibility of distributed consensus with one faulty process. Journal of the ACM, 32(2), 374–382.
Franks, H., Griffiths, N., & Jhumka, A. (2013). Manipulating convention emergence using influencer agents. Journal of Autonomous Agents and Multi-Agent Systems, 26(3), 315–353.
Grenager, T., Powers, R., & Shoham, Y. (2002). Dispersion games: general definitions and some specific learning results. In Proceedings of the Eighteenth National Conference on Artificial Intelligence (pp. 398–403). Alpern: AAAI Press.
de Jong, S., Uyttendaele, S., & Tuyls, K. (2008). Learning to reach agreement in a continuous ultimatum game. Journal of Artificial Intelligence Research (JAIR), 33, 551–574.
Karandikar, R., Mookherjee, D., Ray, D., & Vega-Redondo, F. (1998). Evolving aspirations and cooperation. Journal of Economic Theory, 80(2), 292–331.
Kelley, H., Thibaut, J., Radloff, R., & Mundy, D. (1962). The development of cooperation in the minimal social situation. Psychological Monographs: General and Applied, 76(19), 1–19.
Kemeny, J., & Snell, J. (1969). Finite Markov chains. New York: VanNostrand.
Kittock, J. (1993). Emergent conventions and the structure of multi-agent systems. In Proceedings of the 1993 Santa Fe Institute Complex Systems Summer School (Vol. 5, pp. 1–14). Citeseer.
Knoester, D. B., & McKinley, P. K. (2009). Evolving virtual fireflies. In Proceedings of the 10th European Conference on Artificial Life, Budapest, Hungary.
Kojima, F., & Takahashi, S. (2007). Anti-coordination games and dynamic stability. International Game Theory Review, 9(4), 667–688.
Lemmens, B., Steenhaut, K., Ruckebusch, P., Moerman, I., & Nowé, A. (2012). Network-wide synchronization in wireless sensor networks. In Proceedings of the 19th IEEE Symposium on Communications and Vehicular Technology in the Benelux.
Lewis, D. (1969). Convention: A philosophical study. Cambridge: Harvard University Press.
Lu, G., Krishnamachari, B., & Raghavendra, C. (2004). An adaptive energy-efficient and low-latency MAC for data gathering in wireless sensor networks. In Proceedings of the 18th International Symposium on Parallel and Distributed Processing (p. 224).
Mihaylov, M. (2012). Decentralized coordination in multi-agent systems (Ph.D. thesis, Vrije Universiteit Brussel, 2012).
Mihaylov, M., Le Borgne, Y. A., Tuyls, K., & Nowé, A. (2013). Reinforcement learning for self-organizing wake-up scheduling in wireless sensor networks. Agents and artificial intelligence (Vol. 271, pp. 382–396). Berlin, Heidelberg: Springer.
Mobilia, M. (2003). Does a single zealot affect an infinite group of voters? Physical Review Letters, 91(2), 028,701.
Mukherjee, P., Sen, S., & Airiau, S. (2008). Norm emergence under constrained interactions in diverse societies. In Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems (Vol. 2, pp. 779–786). International Foundation for Autonomous Agents and Multiagent Systems.
Nowak, M., & Sigmund, K. (1993). A strategy of Win-Stay, Lose-Shift that outperforms tit-for-tat in the Prisoner’s Dilemma game. Nature, 364, 56–58.
Savarimuthu, B. T. R., Cranefield, S., Purvis, M. K., & Purvis, M. A. (2009). Norm emergence in agent societies formed by dynamically changing networks. Web Intelligence and Agent Systems, 7(3), 223–232.
Schelling, T. C. (1960). The strategy of conflict. Cambridge: Harvard University Press.
Segbroeck, S.V., Santos, F.C., Lenaerts, T., Pacheco, J.M. (2009) Emergence of cooperation in adaptive social networks with behavioral diversity. In Proceedings of the 10th European Conference on Artificial Life (ECAL) (pp. 434–441).
Sen, S., & Airiau, S. (2007). Emergence of norms through social learning. In Proceedings of the Twentieth International Joint Conference on Artificial Intelligence (pp. 1507–1512).
Shoham, Y., & Tennenholtz, M. (1993). Co-learning and the evolution of social activity. Tech. rep. Stanford University.
Shoham, Y., & Tennenholtz, M. (1997). On the emergence of social conventions: Modeling, analysis, and simulations. Artificial Intelligence, 94, 139–166.
Steels, L. (1997). The synthetic modeling of language origins. Evolution of Communication, 1(1), 1–34.
Traag, V. (2011). Indirect reciprocity through gossiping can lead to cooperative clusters. In IEEE ALIFE (pp. 154–161).
Urbano, P., Ja, B., Antunes, L., & Moniz, L. (2009). Force versus majority: A comparison in convention emergence efficiency. In Coordination, Organizations, Institutions and Norms in Agent Systems IV (pp. 48–63).
Villatoro, D., Sabater-Mir, J., & Sen, S. (2011a). Social Instruments for Robust Convention Emergence. In Twenty-Second International Joint Conference On Artificial Intelligence (IJCAI) (p. 6). Barcelona, Spain.
Villatoro, D., Sen, S., & Sabater-Mir, J. (2011b). Exploring the dimensions of convention emergence in multiagent systems. Advances in Complex Systems, 14(02), 201–227.
Ye, W., Heidemann, J., & Estrin, D. (2004). Medium access control with coordinated adaptive sleeping for wireless sensor networks. IEEE/ACM Transactions on Networking, 12(3), 493–506.
Young, H. P. (1993). The evolution of conventions. Econometrica, 61(1), 57–84.
About this article
Cite this article
Mihaylov, M., Tuyls, K. & Nowé, A. A decentralized approach for convention emergence in multi-agent systems. Auton Agent Multi-Agent Syst 28, 749–778 (2014). https://doi.org/10.1007/s10458-013-9240-2
- Coordination games
- Convention emergence
- Decentralized systems