Artificial Intelligence and Law

, Volume 23, Issue 4, pp 367–407 | Cite as

Establishing norms with metanorms in distributed computational systems

  • Samhar MahmoudEmail author
  • Nathan Griffiths
  • Jeroen Keppens
  • Adel Taweel
  • Trevor J. M. Bench-Capon
  • Michael Luck


Norms provide a valuable mechanism for establishing coherent cooperative behaviour in decentralised systems in which there is no central authority. One of the most influential formulations of norm emergence was proposed by Axelrod (Am Political Sci Rev 80(4):1095–1111, 1986). This paper provides an empirical analysis of aspects of Axelrod’s approach, by exploring some of the key assumptions made in previous evaluations of the model. We explore the dynamics of norm emergence and the occurrence of norm collapse when applying the model over extended durations . It is this phenomenon of norm collapse that can motivate the emergence of a central authority to enforce laws and so preserve the norms, rather than relying on individuals to punish defection. Our findings identify characteristics that significantly influence norm establishment using Axelrod’s formulation, but are likely to be of importance for norm establishment more generally. Moreover, Axelrod’s model suffers from significant limitations in assuming that private strategies of individuals are available to others, and that agents are omniscient in being aware of all norm violations and punishments. Because this is an unreasonable expectation , the approach does not lend itself to modelling real-world systems such as online networks or electronic markets. In response, the paper proposes alternatives to Axelrod’s model, by replacing the evolutionary approach, enabling agents to learn, and by restricting the metapunishment of agents to cases where the original defection is observed, in order to be able to apply the model to real-world domains . This work can also help explain the formation of a “social contract” to legitimate enforcement by a central authority.


Norms Metanorms Norm emergence 



We are immensely grateful to the anonymous reviewers, whose detailed and insightful comments have improved the content and presentation of this paper. Without their valuable work, this would be a much poorer paper, and its value much diminished.


  1. Axelrod R (1984) The evolution of cooperation. Basic Books, New York Google Scholar
  2. Axelrod R (1986) An evolutionary approach to norms. Am Political Sci Rev 80(4):1095–1111CrossRefGoogle Scholar
  3. Bench-Capon TJM (2014) Analysing norms with transition systems. In: JURIX 2014: proceedings of the twenty-seventh annual conference on legal knowledge and information systems. IOS Press, pp 29–38Google Scholar
  4. Bicchieri C (2005) The grammar of society: the nature and dynamics of social norms. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  5. Binmore K (1998) Review of complexity and cooperation by Robert Axelrod. J Artif Soc Soc Situat 1(1):82Google Scholar
  6. Binmore K, Gale J, Samuleson L (1995) Learning to be imperfect: the ultimatum game. Games Econ Behav 8:56–90zbMATHCrossRefGoogle Scholar
  7. Boella G, van der Torre L, Verhagen H (2006) Introduction to normative multiagent systems. Comput Math Organ Theory 12(2–3):71–79CrossRefGoogle Scholar
  8. Boman M (1997) Norms as constraints on real-time autonomous agent action. In: Multi-agent rationality: proceedings of the 8th European workshop on modelling autonomous agents in a multi-agent world, vol 1237 of lecture notes in computer science. Springer, pp 36–44Google Scholar
  9. Borenstein E, Ruppin E (2003) Enhancing autonomous agents evolution with learning by imitation. Interdiscip J Artif Intell Simul Behav 1(4):335–348Google Scholar
  10. Bowling M, Veloso M (2001) Rational and convergent learning in stochastic games. In: IJCAI 2001: proceedings of the 17th international joint conference on artificial intelligence. pp 1021–1026Google Scholar
  11. Castelfranchi C, Conte R (1999) From conventions to prescriptions. towards a unified theory of norms. Artif Intell Law 7:323–340CrossRefGoogle Scholar
  12. Conte R, Castelfranchi C (1995) Understanding the functions of norms in social groups through simulation. In: Gilbert N, Conte R (eds) Artificial societies: the computer simulation of social life. UCL Press, London, pp 213–226Google Scholar
  13. Dautenhahn K, Nehaniv CL (eds) (2002) Imitation in animals and artifacts. MIT Press, CambridgeGoogle Scholar
  14. Delgado J (2002) Emergence of social conventions in complex networks. Artif Intell 141(1–2):171–185MathSciNetCrossRefGoogle Scholar
  15. Delgado J, Pujol JM, Sangesa R (2003) Emergence of coordination in scale-free networks. Web Intell Agent Syst 1:131–138Google Scholar
  16. Dignum F (1999) Autonomous agents with norms. Artif Intell Law 7(1):69–79CrossRefGoogle Scholar
  17. Druzin B (2010) Law without the state: The theory of high engagement and the emergence of spontaneous legal order within commercial systems. Georget J Int Law 41(3):559–620Google Scholar
  18. Durkheim V (1893) De la division du travail social. Les classiques des sciences sociales. Université due Québec à ChicoutimiGoogle Scholar
  19. Ehrlich E (1913) Fundamental principles of the sociology of law. Transaction Publishers, New JerseyGoogle Scholar
  20. Epstein JM (2001) Learning to be thoughtless: social norms and individual computation. Comput Econ 18(1):9–24zbMATHCrossRefGoogle Scholar
  21. Franks H, Griffiths N, Jhumka A (2013) Manipulating convention emergence using influencer agents. Auton Agents Multi-Agent Syst 26(3):315–353CrossRefGoogle Scholar
  22. Galan JM, Izquierdo LR (2005) Appearances can be deceiving: Lessons learned re-implementing Axelrod’s evolutionary approach to norms. J Artif Soc Soc Simul 8(3).
  23. Gibbs JP (1965) Norms: the problem of definition and classification. Am J Soc 70(5):586–594CrossRefGoogle Scholar
  24. Guerraoui R, Huguenin K, Kermarrec A, Monod M (2009) On tracking freeriders in gossip protocols. In: P2P 2009: proceedings of the 9th international conference on peer-to-peer computingGoogle Scholar
  25. Hayes G, Demiris J (1994) A robot controller using learning by imitation. In: Proceedings of the 2nd international symposium on intelligent robotic systems. pp 198–204Google Scholar
  26. Helmhout JM, Gazendam HWM, Jorna RJ (2008) Control over emergence. In: AISB 2008: proceedings of the convention: communication, interaction, and social Intelligence. pp 1–8Google Scholar
  27. Kandori M, Mailath GJ, Rob R (1993) Learning, mutation, and long run equilibria in games. Econometrica 61(1):29–56zbMATHMathSciNetCrossRefGoogle Scholar
  28. Kittock J (1995) Emergent conventions and the structure of multi–agent systems. In: Lectures in complex systems: the proceedings of the 1993 complex systems summer school, Santa Fe Institute studies in the sciences of complexity lecture vol VI. Santa Fe Institute, Addison-Wesley, pp 507–521Google Scholar
  29. Krishnan R, Smith DM, Tang Z, Telang R (2004) The impact of free-riding on peer-to-peer networks. In: HICSS ’04: proceedings of the 37th annual Hawaii international conference on system sciences. IEEE Computer SocietyGoogle Scholar
  30. Kulakowski K, Gawronski P (2009) To cooperate or to defect? altruism and reputation. Phys A Stat Mech Appl 388(17):3581–3584CrossRefGoogle Scholar
  31. Lakkaraju K, Gasser L (2008) Norm emergence in complex ambiguous situations. In: COIN 2008: proceedings of the AAAI workshop on coordination, organizations, institutions, and normsGoogle Scholar
  32. Lloyd-Kelly M, Atkinson K, Bench-Capon TJM (2014) Fostering co-operative behaviour through social intervention. In: SIMULTECH 2014: proceedings of the 4th international conference on simulation and modeling methodologies, technologies and applications. IEEE, pp 578–585Google Scholar
  33. López F, López y, Luck M (2003) Modelling norms for autonomous agents. In: Chávez E, Favela J, Mejía M, Oliart A (eds) Proceedings of the 4th Mexican conference on computer science. IEEE Computer Society, pp 238–245Google Scholar
  34. Mahmoud S, Keppens J, Luck M, Griffiths N (2011) Overcoming omniscience in Axelrod's model. In: Proceedings of the 2011 Web Intelligence/IAT Workshops. IEEE Computer Society, Washington DC, USA, pp 29–32Google Scholar
  35. Mahmoud S, Griffiths N, Keppens J, Luck, M (2012) Overcoming omniscience for norm emergence in Axelrod’s metanorm model. In: Cranefield S, Vazquez-Salceda J, van Riemsdijk B, Noriega P (eds) Coordination, organizations, institutions, and norms in agent systems VII. Lecture notes in computer science, vol 7254. Springer, Berlin, pp 186–202Google Scholar
  36. Mukherjee P, Sen S, Airiau S (2007) Emergence of norms with biased interactions in heterogeneous agent societies. In: Web intelligence and intelligent agent technology workshops, 2007 IEEE/WIC/ACM international conferences on web intelligence and intelligent agent technology. pp 512–515Google Scholar
  37. Nakamaru M, Dieckmann U (2009) Runaway selection for cooperation and strict-and-severe punishment. J Theor Biol 257(1):1–8MathSciNetCrossRefGoogle Scholar
  38. Neumann M (2012) The cognitive legacy of norm simulation. Artif Intell Law 20:339–357CrossRefGoogle Scholar
  39. Prietula MJ, Conway D (2009) The evolution of metanorms: quis custodiet ipsos custodes? Comput Math Organ Theory 15(3):147–168zbMATHCrossRefGoogle Scholar
  40. Rheinstein M (1954) Max Weber on law and economy in society. Harvard University Press, CambridgeGoogle Scholar
  41. Riolo R, Cohen M, Axelrod R (2001) Evolution of cooperation without reciprocity. Nature 414:441–443CrossRefGoogle Scholar
  42. Riveret R, Contissa G, Busquets D, Rotolo A, Pitt J, Sartor G (2013) Vicarious reinforcement and ex ante law enforcement: a study in norm-governed learning agents. In: International conference on artificial intelligence and law, ICAIL ’13, Rome, Italy, June 10–14, 2013, pp 222–226Google Scholar
  43. Riveret R, Rotolo A, Sartor G (2012) Probabilistic rule-based argumentation for norm-governed learning agents. Artif Intell Law 20(4):383–420CrossRefGoogle Scholar
  44. Salazar N, Rodriguez-Aguilar JA, Arcos JL (2010) Robust coordination in large convention spaces. AI Commun 23:357–372MathSciNetGoogle Scholar
  45. Savarimuthu BTR, Cranefield S, Purvis M, Purvis M (2007) Role model based mechanism for norm emergence in artificial agent societies. In: COIN ’07: proceedings of the international workshop on coordination, organization, institutions and norms. pp 1–12Google Scholar
  46. Savarimuthu BTR, Cranefield S (2011) Norm creation, spreading and emergence: a survey of simulation models of norms in multi-agent systems. Multiagent Grid Syst 7(1):21–54Google Scholar
  47. Sen O, Sen S (2010) Effects of social network topology and options on norm emergence. In: Padget J, Artikis A, Vasconcelos W, Stathis K, da Silva V, Matson E, Polleres A (eds) Coordination, organizations, institutions and norms in agent systems V, volume 6069 of lecture notes in computer science. Springer, Berlin, pp 211–222Google Scholar
  48. Sen S, Airiau S (2007) Emergence of norms through social learning. In: IJCAI 2007: proceedings of the 20th international joint conference on artificial intelligence. Morgan Kaufmann, pp 1507–1512Google Scholar
  49. Shoham Y, Tennenholtz M (1992) Emergent conventions in multi-agent systems: initial experimental results and observations (preliminary report). In: Proceedings of the 3rd international conference on KR&R. pp 225–232Google Scholar
  50. Shoham Y, Tennenholtz M (1994) Co-learning and the evolution of social acitivity. Technical report, Stanford, CA, USAGoogle Scholar
  51. Shoham Y, Tennenholtz M (1997) On the emergence of social conventions: modeling, analysis, and simulations. Artif Intell 94:139–166zbMATHCrossRefGoogle Scholar
  52. Slembeck T (1999) Reputations and fairness in bargaining-experimental evidence from a repeated ultimatum game with fixed opponents. Experimental, EconWPAGoogle Scholar
  53. Song M, Chen R, An J. (2007) Social conventions to promise learning convergence. In: FSKD ’07: proceedings of the 4th international conference on fuzzy systems and knowledge discovery. IEEE Computer Society, Washington, DC, USA, pp 660–662Google Scholar
  54. Ullman-Margalit E (1977) The emergence of norms. Clarendon Press, OxfordGoogle Scholar
  55. Urbano P, Balsa J, Antunes L, Moniz L (2008) Force versus majority: a comparison in convention emergence efficiency. In: Coordination, organizations, institutions and norms in agent systems IV: COIN 2008 international workshops, COIN@AAMAS 2008, Estoril, Portugal, May 12, 2008. COIN@AAAI 2008, Chicago, USA, July 14, 2008. Revised Selected Papers, pp 48–63Google Scholar
  56. Villatoro D, Andrighetto G, Sabater-Mir J, Conte R (2011) Dynamic sanctioning for robust and cost-efficient norm compliance. In: Proceedings of the 22nd international joint conference on artificial Intelligence. AAAI Press, Barcelona, pp 414–419Google Scholar
  57. Villatoro D, Sabater-Mir J, Sen S (2011) Social instruments for robust convention emergence. In: Walsh, T (ed) Proceedings of the 22nd international joint conference on artificial intelligence. AAAI Press, pp 420–425Google Scholar
  58. Villatoro D, Sen S, Sabater-Mir J (2009) Topology and memory effect on convention emergence. In: Proceedings of the 2009 IEEE/WIC/ACM international conference on web intelligence and intelligent agent technologies. IEEE, pp 233–240Google Scholar
  59. Walker A, Wooldridge M (1995) Understanding the emergence of conventions in multi-agent systems. In: Lesser V (ed) Proceedings of the first international joint conference on multi agent systems. pp 384–389Google Scholar
  60. Watkins CJCH, Dayan P (1992) Q-learning. Mach Learn 8(3–4):279–292zbMATHGoogle Scholar
  61. Yang M, Zhang Z, Li X, Dai Y (2005) An empirical study of free-riding behavior in the maze P2P file-sharing system. In: IPTPS ’05: proceedings of the 4th international workshop on peer-to-peer systems. Springer, pp 182–192Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2015

Authors and Affiliations

  • Samhar Mahmoud
    • 1
    Email author
  • Nathan Griffiths
    • 2
  • Jeroen Keppens
    • 1
  • Adel Taweel
    • 1
  • Trevor J. M. Bench-Capon
    • 3
  • Michael Luck
    • 1
  1. 1.King’s College LondonLondonUK
  2. 2.University of WarwickCoventryUK
  3. 3.University of LiverpoolLiverpoolUK

Personalised recommendations