Social or Individual Learning? An Aggregated Solution for Coordination in Multiagent Systems

  • Bingcai Chen
  • Chao Yu
  • Qishuai Diao
  • Rui Liu
  • Yuliang Wang
Article
  • 25 Downloads

Abstract

There are mainly two different ways of learning for animals and humans: trying on yourself through interactions or imitating/copying others through communication/observation. How these two learning strategies differ and what roles they are playing in achieving coordination among individuals are two challenging problems for researchers from various disciplines. In multiagent systems, most existing work simply focuses on individual learning for achieving coordination among agents. The social learning perspective has been largely neglected. Against this background, this article contributes by proposing an integrated solution to decision making between social learning and individual learning in multiagent systems. Two integration modes have been proposed that enable agents to choose in between these two learning strategies, either in a fixed or in an adaptive manner. Experimental evaluations have shown that these two kinds of leaning strategies have different roles in maintaining efficient coordination among agents. These differences can reveal some significant insights into the manipulation and control of agent behaviors in multiagent systems, and also shed light on understanding the social factors in shaping coordinated behaviors in humans and animals.

Keywords

Individual learning social learning coordination multiagent systems 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgments

This work is supported by the NationalNatural Science Foundation of China under Grant 61771089, 61502072, 61572104 and 61403059, Fundamental Research Funds for the Central Universities of China under Grant DUT16RC(4)17, and Post-Doctoral Science Foundation of China under Grants 2014M561229 and 2015T80251.

References

  1. [1]
    Acemoglu D. and Ozdaglar A. (2011). Opinion dynamics and learning in social networks. Dynamic Games and Applications, 1(1): 3–49.MathSciNetCrossRefMATHGoogle Scholar
  2. [2]
    Barabási A. and Albert R. (2002). Statistical mechanics of complex networks. Rev. Modern Phys., 74: 47–97.MathSciNetCrossRefMATHGoogle Scholar
  3. [3]
    Castellano C., Fortunato S., and Loreto V. (2009). Statistical physics of social dynamics. Reviews of modern physics, 81(2): 591.CrossRefGoogle Scholar
  4. [4]
    Hegselmann R., Krause U. et al. (2002). Opinion dynamics and bounded confidence models, analysis, and simulation. Journal of artificial societies and social simulation, 5(3): 1–33.Google Scholar
  5. [5]
    Hethcote H. W. (2000). The mathematics of infectious diseases. SIAM review, 42(4): 599–653.MathSciNetCrossRefMATHGoogle Scholar
  6. [6]
    Hofmann L., Chakraborty N., and Sycara K. (2011). The evolution of cooperation in self-interested agent societies: a critical study. In Proc. of 10th AAMAS, pp. 685–692, Toronto, Canada, May 10-14, 2011.Google Scholar
  7. [7]
    Laland K. N. (2004). Social learning strategies. Learning & behavior, 32(1): 4–14.CrossRefGoogle Scholar
  8. [8]
    Mihaylov M., Tuyls K., and Nowé A. (2014). A decentralized approach for convention emergence in multi-agent systems. Autonomous Agents and Multi-Agent Systems, 28(5): 749–778.CrossRefGoogle Scholar
  9. [9]
    Mukherjee P., Sen S., and Airiau S. (2008). Norm emergence under constrained interactions in diverse societies. In Proc. of 7th AAMAS, pp. 779–786, Estoril, Portugal, May 12-16, 2008.Google Scholar
  10. [10]
    Rendell L., Fogarty L., Hoppitt W. J., Morgan T. J., Webster M. M., and Laland K. N. (2011). Cognitive culture: theoretical and empirical insights into social learning strategies. Trends in cognitive sciences, 15(2): 68–76.CrossRefGoogle Scholar
  11. [11]
    Savarimuthu B., Arulanandam R., and Purvis M. (2011). Aspects of active norm learning and the effect of lying on norm emergence in agent societies. Agents in Principle, Agents in Practice, LNAI, 7047: 36–50.CrossRefGoogle Scholar
  12. [12]
    Sen O. and Sen S. (2010). Effects of social network topology and options on norm emergence. Coordination, Organizations, Institutions and Norms in Agent Systems V, LNAI, 6069: 211–222.CrossRefGoogle Scholar
  13. [13]
    Sen S. and Airiau S. (2007). Emergence of norms through social learning. In Proc. of 20th IJCAI, pp. 1507–1512, Hyderabad, India, January 6-12, 2007.Google Scholar
  14. [14]
    Shibusawa R. and Sugawara T. (2014). Norm emergence via influential weight propagation in complex networks. In Network Intelligence Conference (ENIC), 2014 European. IEEE, pp. 30–37, Wroclaw, Poland, September 29-30, 2014.Google Scholar
  15. [15]
    Shoham Y. and Tennenholtz M. (1997). On the emergence of social conventions: modeling, analysis, and simulations. Artif. Intel., 94(1-2): 139–166.CrossRefMATHGoogle Scholar
  16. [16]
    Szabo G. and Fáth G. (2007). Evolutionary games on graphs. Phys. Rep., 446(4-6): 97–216.MathSciNetCrossRefGoogle Scholar
  17. [17]
    Verhagen H. (2001). Simulation of the learning of norms. Social Science Computer Review, 19(3): 296–306.CrossRefGoogle Scholar
  18. [18]
    Villatoro D., Sabater-Mir J., and Sen S. (2011). Social instruments for robust convention emergence. In Proc. of 22nd IJCAI, pp. 420–425, Barcelona, Catalonia, Spain, July 16-22, 2011, AAAI Press.Google Scholar
  19. [19]
    Villatoro D., Sen S., and Sabater-Mir J. (2009). Topology and memory effect on convention emergence. In Proc. of WIIAT’09, pp. 233–240, Milan, Italy, September 15-18, 2009.Google Scholar
  20. [20]
    Watts D. and Strogatz S. (1998). Collective dynamics of smallworld networks. Nature, 393: 440–442.CrossRefMATHGoogle Scholar
  21. [21]
    Watts D. J. and Dodds P. S. (2007). Influentials, networks, and public opinion formation. Journal of consumer research, 34(4): 441–458.CrossRefGoogle Scholar
  22. [22]
    Young H. (1996). The economics of convention. J. Econ. Pers., 10(2): 105–122.CrossRefGoogle Scholar
  23. [23]
    Yu C., Lv H., Sen S., Hao J., Ren F., and Liu R. (2016). An adaptive learning framework for efficient emergence of social norms: (extended abstract). In International Conference on Autonomous Agents Multiagent Systems, pp. 1307–1308, Singapore, May 9-13, 2016.Google Scholar
  24. [24]
    Yu C., Liu R., and Wang Y. (2016). Social learning in networked agent societies. In IEEE International Conference on Agents, pp. 57–62, Matsue, Japan, September 28-30, 2016.Google Scholar
  25. [25]
    Yu C., Lv H., Ren F., Bao H., and Hao J. (2015). Hierarchical learning for emergence of social norms in networked multiagent systems. In Australasian Joint Conference on Artificial Intelligence. pp. 630–643, Canberrra, Australia, Nov. 30-Dec. 4, 2015, Springer.Google Scholar
  26. [26]
    Yu C., Lv H., Sen S., Ren F., and Tan G. (2016). Adaptive learning for efficient emergence of social norms in networked multiagent systems. In Pacific Rim International Conference on Trends in Artificial Intelligence, pp. 805–818, Phuket, Thailand, August 22-26, 2016.Google Scholar
  27. [27]
    Yu C., Tan G., Lv H., Wang Z., Meng J., Hao J., and Ren F. (2016). Modelling adaptive learning behaviours for consensus formation in human societies. Scientific Reports, 6: 27626.CrossRefGoogle Scholar
  28. [28]
    Yu C., Wang Z., Lv H., Bao H., and Li Y. (2017). Collective Learning and Information Diffusion for Efficient Emergence of Social Norms. Springer, Singapore.CrossRefGoogle Scholar
  29. [29]
    Yu C., Y. Chen, Lv H., Ren J., Ge H., and Sun L. (2017). Neural learning for the emergence of social norms in multiagent systems. In IEEE International Conference on Agents, pp. 40–45, Beijing, China, July 6-9, 2017.Google Scholar
  30. [30]
    Yu C., Zhang M., and Ren F. (2014). Collective learning for the emergence of social norms in networked multiagent systems. IEEE Transactions on Cybernetics, 44(12): 2342–2355.CrossRefGoogle Scholar
  31. [31]
    Yu C., Zhang M., Ren F., and Luo X. (2013). Emergence of social norms through collective learning in networked agent societies. In Proc. of AAMAS2013, pp. 475–482, Saint Paul, USA, May 6-10, 2013.Google Scholar
  32. [32]
    Yu C., Zhang M., and Ren F. (2013). Emotional multiagent reinforcement learning in social dilemmas. In International Conference on Principles and Practice of Multi-Agent Systems, pp. 372–387, Dunedin, New Zealand, December 1-6, 2013.Google Scholar
  33. [33]
    Yu C., Zhang M., Ren F., and Tan G. (2015). Emotional multiagent reinforcement learning in spatial social dilemmas. IEEE Transactions on Neural Networks & Learning Systems, 26(12): 3083–3096MathSciNetCrossRefGoogle Scholar

Copyright information

© Systems Engineering Society of China and Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  • Bingcai Chen
    • 1
    • 2
  • Chao Yu
    • 1
  • Qishuai Diao
    • 1
  • Rui Liu
    • 1
  • Yuliang Wang
    • 1
  1. 1.School of Computer Science and TechnologyDalian University of TechnologyDalianChina
  2. 2.School of Computer Science and TechnologyXinjiang Normal UniversityUrumqiChina

Personalised recommendations