A decentralized approach for convention emergence in multi-agent systems


The field of convention emergence studies how agents involved in repeated coordination games can reach consensus through only local interactions. The literature on this topic is vast and is motivated by human societies, mainly addressing coordination problems between human agents, such as who gets to redial after a dropped telephone call. In contrast, real-world engineering problems, such as coordination in wireless sensor networks, involve agents with limited resources and knowledge and thus pose certain restrictions on the complexity of the coordination mechanisms. Due to these restrictions, strategies proposed for human coordination may not be suitable for engineering applications and need to be further explored in the context of real-world application domains. In this article we take the role of designers of large decentralized multi-agent systems. We investigate the factors that speed up the convergence process of agents arranged in different static and dynamic topologies and under different interaction models, typical for engineering applications. We also study coordination problems both under partial observability and in the presence of faults (or noise). The main contributions of this article are that we propose an approach for emergent coordination, motivated by highly constrained devices, such as wireless nodes and swarm bots, in the absence of a central entity and perform extensive theoretical and empirical studies. Our approach is called Win-Stay Lose-probabilistic-Shift, generalizing two well-known strategies in game theory that have been applied in other domains. We demonstrate that our approach performs well in different settings under limited information and imposes minimal system requirements, due to its simplicity. Moreover, our technique outperforms state-of-the-art coordination mechanisms, guarantees full convergence in any topology and has the property that all convention states are absorbing.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10


  1. 1.

    Confirmed through personal communication with the first author.


  1. 1.

    Axelrod, R. (1984). The evolution of cooperation. New York: Basic Books.

    Google Scholar 

  2. 2.

    Axelrod, R. (1986). An evolutionary approach to norms. The American Political Science Review, 80(4), 1095–1111.

    Article  Google Scholar 

  3. 3.

    Barabasi, A. L., Albert, R., & Jeong, H. (1999). Mean-field theory for scale-free random networks. Physica A: Statistical Mechanics and its Applications, 272(1–2), 19.

    Google Scholar 

  4. 4.

    Barrett, J., & Zollman, K. J. S. (2009). The role of forgetting in the evolution and learning of language. Journal of Experimental & Theoretical Artificial Intelligence, 21(4), 293–309.

    Article  MATH  Google Scholar 

  5. 5.

    Bendor, J., Mookherjee, D., & Ray, D. (1994). Aspirations, adaptive learning and cooperation in repeated games. Tech. rep., Tilburg University, Center for Economic Research.

  6. 6.

    Bowling, M., & Veloso, M. (2002). Multiagent learning using a variable learning rate. Artificial Intelligence, 136(2), 215–250.

    Article  MATH  MathSciNet  Google Scholar 

  7. 7.

    Bramoullé, Y., López-Pintado, D., Goyal, S., & Vega-Redondo, F. (2004). Network formation and anti-coordination games. International Journal of Games Theory, 33(1), 1–19.

    Article  MATH  Google Scholar 

  8. 8.

    Brooks, L., Iba, W., & Sen, S. (2011). Modeling the emergence and convergence of norms. In Proceedings of the Twenty-Second international joint conference on Artificial Intelligence (Vol. 1, pp. 97–102). Menlo Park, CA: AAAI Press.

  9. 9.

    Castellano, C., Fortunato, S., & Loreto, V. (2009). Statistical physics of social dynamics. Reviews of modern physics, 81(2), 591.

    Article  Google Scholar 

  10. 10.

    De Hauwere, Y.M. (2011). Sparse interactions in multi-agent reinforcement learning (Ph.D. thesis, Vrije Universiteit Brussel, 2011).

  11. 11.

    De Vylder B. (2007). The evolution of conventions in multi-agent systems (Ph.D. thesis, Vrije Universiteit Brussel, 2007).

  12. 12.

    Delgado, J., Pujol, J., & Sanguesa, R. (2003). Emergence of coordination in scale-free networks. Web Intelligence and Agent Systems, 1(2), 131–138.

    Google Scholar 

  13. 13.

    Farinelli, A., Rogers, A., Petcu, A., & Jennings, N. R. (2008). Decentralised coordination of lowpower embedded devices using the max-sum algorithm. In Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent System (pp. 639–646). Richland, SC.

  14. 14.

    Fischer, M. J., Na, Lynch, & Paterson, M. S. (1985). Impossibility of distributed consensus with one faulty process. Journal of the ACM, 32(2), 374–382.

    Article  MATH  Google Scholar 

  15. 15.

    Franks, H., Griffiths, N., & Jhumka, A. (2013). Manipulating convention emergence using influencer agents. Journal of Autonomous Agents and Multi-Agent Systems, 26(3), 315–353.

    Article  Google Scholar 

  16. 16.

    Grenager, T., Powers, R., & Shoham, Y. (2002). Dispersion games: general definitions and some specific learning results. In Proceedings of the Eighteenth National Conference on Artificial Intelligence (pp. 398–403). Alpern: AAAI Press.

  17. 17.

    de Jong, S., Uyttendaele, S., & Tuyls, K. (2008). Learning to reach agreement in a continuous ultimatum game. Journal of Artificial Intelligence Research (JAIR), 33, 551–574.

    MATH  Google Scholar 

  18. 18.

    Karandikar, R., Mookherjee, D., Ray, D., & Vega-Redondo, F. (1998). Evolving aspirations and cooperation. Journal of Economic Theory, 80(2), 292–331.

    Article  MATH  MathSciNet  Google Scholar 

  19. 19.

    Kelley, H., Thibaut, J., Radloff, R., & Mundy, D. (1962). The development of cooperation in the minimal social situation. Psychological Monographs: General and Applied, 76(19), 1–19.

    Article  Google Scholar 

  20. 20.

    Kemeny, J., & Snell, J. (1969). Finite Markov chains. New York: VanNostrand.

    Google Scholar 

  21. 21.

    Kittock, J. (1993). Emergent conventions and the structure of multi-agent systems. In Proceedings of the 1993 Santa Fe Institute Complex Systems Summer School (Vol. 5, pp. 1–14). Citeseer.

  22. 22.

    Knoester, D. B., & McKinley, P. K. (2009). Evolving virtual fireflies. In Proceedings of the 10th European Conference on Artificial Life, Budapest, Hungary.

  23. 23.

    Kojima, F., & Takahashi, S. (2007). Anti-coordination games and dynamic stability. International Game Theory Review, 9(4), 667–688.

    Article  MATH  MathSciNet  Google Scholar 

  24. 24.

    Lemmens, B., Steenhaut, K., Ruckebusch, P., Moerman, I., & Nowé, A. (2012). Network-wide synchronization in wireless sensor networks. In Proceedings of the 19th IEEE Symposium on Communications and Vehicular Technology in the Benelux.

  25. 25.

    Lewis, D. (1969). Convention: A philosophical study. Cambridge: Harvard University Press.

    Google Scholar 

  26. 26.

    Lu, G., Krishnamachari, B., & Raghavendra, C. (2004). An adaptive energy-efficient and low-latency MAC for data gathering in wireless sensor networks. In Proceedings of the 18th International Symposium on Parallel and Distributed Processing (p. 224).

  27. 27.

    Mihaylov, M. (2012). Decentralized coordination in multi-agent systems (Ph.D. thesis, Vrije Universiteit Brussel, 2012).

  28. 28.

    Mihaylov, M., Le Borgne, Y. A., Tuyls, K., & Nowé, A. (2013). Reinforcement learning for self-organizing wake-up scheduling in wireless sensor networks. Agents and artificial intelligence (Vol. 271, pp. 382–396). Berlin, Heidelberg: Springer.

  29. 29.

    Mobilia, M. (2003). Does a single zealot affect an infinite group of voters? Physical Review Letters, 91(2), 028,701.

    Article  Google Scholar 

  30. 30.

    Mukherjee, P., Sen, S., & Airiau, S. (2008). Norm emergence under constrained interactions in diverse societies. In Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems (Vol. 2, pp. 779–786). International Foundation for Autonomous Agents and Multiagent Systems.

  31. 31.

    Nowak, M., & Sigmund, K. (1993). A strategy of Win-Stay, Lose-Shift that outperforms tit-for-tat in the Prisoner’s Dilemma game. Nature, 364, 56–58.

    Article  Google Scholar 

  32. 32.

    Savarimuthu, B. T. R., Cranefield, S., Purvis, M. K., & Purvis, M. A. (2009). Norm emergence in agent societies formed by dynamically changing networks. Web Intelligence and Agent Systems, 7(3), 223–232.

    Google Scholar 

  33. 33.

    Schelling, T. C. (1960). The strategy of conflict. Cambridge: Harvard University Press.

    Google Scholar 

  34. 34.

    Segbroeck, S.V., Santos, F.C., Lenaerts, T., Pacheco, J.M. (2009) Emergence of cooperation in adaptive social networks with behavioral diversity. In Proceedings of the 10th European Conference on Artificial Life (ECAL) (pp. 434–441).

  35. 35.

    Sen, S., & Airiau, S. (2007). Emergence of norms through social learning. In Proceedings of the Twentieth International Joint Conference on Artificial Intelligence (pp. 1507–1512).

  36. 36.

    Shoham, Y., & Tennenholtz, M. (1993). Co-learning and the evolution of social activity. Tech. rep. Stanford University.

  37. 37.

    Shoham, Y., & Tennenholtz, M. (1997). On the emergence of social conventions: Modeling, analysis, and simulations. Artificial Intelligence, 94, 139–166.

    Article  MATH  Google Scholar 

  38. 38.

    Steels, L. (1997). The synthetic modeling of language origins. Evolution of Communication, 1(1), 1–34.

    Article  Google Scholar 

  39. 39.

    Traag, V. (2011). Indirect reciprocity through gossiping can lead to cooperative clusters. In IEEE ALIFE (pp. 154–161).

  40. 40.

    Urbano, P., Ja, B., Antunes, L., & Moniz, L. (2009). Force versus majority: A comparison in convention emergence efficiency. In Coordination, Organizations, Institutions and Norms in Agent Systems IV (pp. 48–63).

  41. 41.

    Villatoro, D., Sabater-Mir, J., & Sen, S. (2011a). Social Instruments for Robust Convention Emergence. In Twenty-Second International Joint Conference On Artificial Intelligence (IJCAI) (p. 6). Barcelona, Spain.

  42. 42.

    Villatoro, D., Sen, S., & Sabater-Mir, J. (2011b). Exploring the dimensions of convention emergence in multiagent systems. Advances in Complex Systems, 14(02), 201–227.

    Article  MathSciNet  Google Scholar 

  43. 43.

    Ye, W., Heidemann, J., & Estrin, D. (2004). Medium access control with coordinated adaptive sleeping for wireless sensor networks. IEEE/ACM Transactions on Networking, 12(3), 493–506.

    Article  Google Scholar 

  44. 44.

    Young, H. P. (1993). The evolution of conventions. Econometrica, 61(1), 57–84.

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Mihail Mihaylov.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Mihaylov, M., Tuyls, K. & Nowé, A. A decentralized approach for convention emergence in multi-agent systems. Auton Agent Multi-Agent Syst 28, 749–778 (2014). https://doi.org/10.1007/s10458-013-9240-2

Download citation


  • Coordination games
  • Convention emergence
  • Decentralized systems