Abstract
Convention emergence studies how global convention arises from local interactions among agents. Traditionally, the studies on convention emergence are conducted by means of agent-based simulations, whereas very few studies are based on model-based approaches. In this paper, we employ model-based approach to study the convention emergence by observation with memorization in a large population under social learning. In particular, we derive the recurrence equations of the population dynamic, which is the evolution of action distribution over time, under the external majority (EM) strategy. The recurrence equations precisely predict the behaviour of the multi-agent system at any time point, which is verified with the agent-based simulations. Based on the recurrence equations, We prove the converge behavior under various situations and work out the optimal memory length under different number of actions. Finally, we show that the EM strategy outperforms other popular strategies such as Q-learning and Highest Cumulative Reward (HCR) in convergence speed under social learning, even in very large convention space.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Consider the following example. Let the number of action be 3 and memory length be 5. Consider an agent with following observation sequence (\(a_1,a_1,a_1,a_2,a_2,a_3,a_3\)). We can see that the action \(a_1\) changes from majority to minority over time. However, the “stick to the current action” strategy will choose \(a_1\) even it is a minority action, since \(a_2,a_3\) are in the tie. However, it is not a reasonable choice. In general, when an action is changing from majority to minority, and other actions become majority and in a tie at the same time, then the “stick to the current action” strategy will choose a minority action. Therefore we make the modification in the case of tie to yield a reasonable outcome.
- 2.
We measure the convergence rate based on agents’ policy, as the actual actions are affected by exploration.
- 3.
In case of tie, the agent will randomly choose one action among the actions with highest cumulative reward. The changes is made for the same reason as in Sect. 3.
References
Airiau, S., Sen, S., Villatoro, D.: Emergence of conventions through social learning. Auton. Agent. Multi-Agent Syst. 28(5), 779–804 (2014)
Brooks, L., Iba, W., Sen, S.: Modeling the emergence and convergence of norms. In: IJCAI, pp. 97–102 (2011)
Chen, B., Yu, C., Diao, Q., Liu, R., Wang, Y.: Social or individual learning? An aggregated solution for coordination in multiagent systems. J. Syst. Sci. Syst. Eng. 27(2), 180–200 (2018)
Cruise, J., Ganesh, A.: Probabilistic consensus via polling and majority rules. Queueing Syst. 78(2), 99–120 (2014)
Daskalakis, C., Kamath, G., Tzamos, C.: On the structure, covering, and learning of poisson multinomial distributions. In: 2015 IEEE 56th Annual Symposium on FOCS, pp. 1203–1217. IEEE (2015)
Delgado, J.: Emergence of social conventions in complex networks. Artif. Intell. 141(1–2), 171–185 (2002)
Delgado, J., Pujol, J.M., Sangüesa, R.: Emergence of coordination in scale-free networks. Web Intell. Agent Syst. Int. J. 1(2), 131–138 (2003)
Epstein, J.M.: Learning to be thoughtless: social norms and individual computation. Comput. Econ. 18(1), 9–24 (2001)
Gärtner, B., Zehmakan, A.N.: Majority model on random regular graphs. In: Bender, M.A., Farach-Colton, M., Mosteiro, M.A. (eds.) LATIN 2018. LNCS, vol. 10807, pp. 572–583. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-77404-6_42
Hao, J., Sun, J., Chen, G., Wang, Z., Yu, C., Ming, Z.: Efficient and robust emergence of norms through heuristic collective learning. ACM Trans. Auton. Adapt. Syst. (TAAS) 12(4), 23 (2017)
Hassin, Y., Peleg, D.: Distributed probabilistic polling and applications to proportionate agreement. Inf. Comput. 171(2), 248–268 (2001)
Hong, Y.: On computing the distribution function for the poisson binomial distribution. Comput. Stat. Data Anal. 59, 41–51 (2013)
Kittock, J.E.: Emergent conventions and the structure of multi-agent systems. In: Proceedings of the 1993 Santa Fe Institute CSSS, vol. 6, pp. 1–14. Citeseer (1993)
Mihaylov, M., Tuyls, K., Nowé, A.: A decentralized approach for convention emergence in multi-agent systems. Auton. Agent. Multi-Agent Syst. 28(5), 749–778 (2014)
Mossel, E., Neeman, J., Tamuz, O.: Majority dynamics and aggregation of information in social networks. Auton. Agent. Multi-Agent Syst. 28(3), 408–429 (2014)
Mukherjee, P., Sen, S., Airiau, S.: Norm emergence under constrained interactions in diverse societies. In: Proceedings of the 7th AAMAS, vol. 2, pp. 779–786. International Foundation for Autonomous Agents and Multiagent Systems (2008)
Parunak, H.: A mathematical analysis of collective cognitive convergence. In: Proceedings of the 8th AAMAS, vol. 1, pp. 473–480. International Foundation for Autonomous Agents and Multiagent Systems (2009)
Parunak, H., Belding, T.C., Hilscher, R., Brueckner, S.: Modeling and managing collective cognitive convergence. In: Proceedings of the 7th AAMAS, vol. 3, pp. 1505–1508. International Foundation for Autonomous Agents and Multiagent Systems (2008)
Sen, S., Airiau, S.: Emergence of norms through social learning. In: IJCAI, vol. 1507, p. 1512 (2007)
Shibusawa, R., Sugawara, T.: Norm emergence via influential weight propagation in complex networks. In: ENIC 2014 European, pp. 30–37. IEEE (2014)
Shoham, Y., Tennenholtz, M.: Emergent conventions in multi-agent systems: initial experimental results and observations. In: Proceedings of the 3rd KR, pp. 225–231 (1992)
Shoham, Y., Tennenholtz, M.: Co-learning and the evolution of social activity. Technical report, Department of Computer Science, Stanford University, CA (1994)
Shoham, Y., Tennenholtz, M.: On the emergence of social conventions: modeling, analysis, and simulations. Artif. Intell. 94(1–2), 139–166 (1997)
Urbano, P., Balsa, J., Antunes, L., Moniz, L.: Force versus majority: a comparison in convention emergence efficiency. In: Hübner, J.F., Matson, E., Boissier, O., Dignum, V. (eds.) COIN -2008. LNCS (LNAI), vol. 5428, pp. 48–63. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-00443-8_4
Urbano, P., Balsa, J., Ferreira, P., Antunes, L.: How much should agents remember? The role of memory size on convention emergence efficiency. In: Lopes, L.S., Lau, N., Mariano, P., Rocha, L.M. (eds.) EPIA 2009. LNCS (LNAI), vol. 5816, pp. 508–519. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04686-5_42
Villatoro, D., Sabater-Mir, J., Sen, S.: Social instruments for robust convention emergence. In: IJCAI, vol. 11, pp. 420–425 (2011)
Villatoro, D., Sen, S., Sabater-Mir, J.: Topology and memory effect on convention emergence. In: IEEE/WIC/ACM International Joint Conferences on WI-IAT, WI-IAT 2009, vol. 2, pp. 233–240. IEEE (2009)
Villatoro, D., Sen, S., Sabater-Mir, J.: Exploring the dimensions of convention emergence in multiagent systems. Adv. Complex Syst. 14(02), 201–227 (2011)
Walker, A., Wooldridge, M.: Understanding the emergence of conventions in multi-agent systems. In: ICMAS, vol. 95, pp. 384–389 (1995)
Watkins, C.J., Dayan, P.: Q-learning. Mach. Learn. 8(3–4), 279–292 (1992)
Yu, C., Zhang, M., Ren, F., Luo, X.: Emergence of social norms through collective learning in networked agent societies. In: Proceedings of the 2013 AAMAS, pp. 475–482. International Foundation for Autonomous Agents and Multiagent Systems (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Leung, Cw., Hu, S., Leung, Hf. (2019). Modeling Convention Emergence by Observation with Memorization. In: Nayak, A., Sharma, A. (eds) PRICAI 2019: Trends in Artificial Intelligence. PRICAI 2019. Lecture Notes in Computer Science(), vol 11670. Springer, Cham. https://doi.org/10.1007/978-3-030-29908-8_57
Download citation
DOI: https://doi.org/10.1007/978-3-030-29908-8_57
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-29907-1
Online ISBN: 978-3-030-29908-8
eBook Packages: Computer ScienceComputer Science (R0)