The Norm Implementation Problem in Normative Multi-Agent Systems



The norm implementation problem consists in how to see to it that the agents in a system comply with the norms specified for that system by the system designer. It is part of the more general problem of how to synthesize or create norms for multi-agent systems, by, for example, highlighting the choice between regimentation and enforcement, or the punishment associated with a norm violation. In this paper we discuss how various ways to implement norms in a multi-agent system can be distinguished in a formal game-theoretic framework. In particular, we show how different types of norm implementation can all be uniformly specified and verified as types of transformations of extensive games. We introduce the notion of retarded preconditions to implement norms, and we illustrate the framework and the various ways to implement norms in the blocks world environment.


Multiagent System Extensive Form Normative System Formal Semantic Label Transition System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



The authors would like to thank the reviewers of the volume for their helpful comments. Davide Grossi wishes to acknowledge support by Ministère de la Culture, de L’Enseignement Supèrieur et de la Recherche, Grand-Duchè de Luxembourg (grant BFR07/123) and by Nederlandse Organisatie voor Wetenschappelijk Onderzoek (VENI grant 639.021.816).


  1. 110.
    Cohen, P.R., Levesque, H.J.: Intention is choice with commitment. Artificial Intelligence 42(2-3), 213–261 (1990)MATHCrossRefMathSciNetGoogle Scholar
  2. 209.
    Harel, D., Kozen, D., Tiuryn, J.: Dynamic logic. In: D. Gabbay, F. Guenthner (eds.) Handbook of Philosophical Logic: Volume II: Extensions of Classical Logic, pp. 497–604. Reidel, Dordrecht, The Netherlands (1984)Google Scholar
  3. 42.
    van Benthem, J., van Eijck, J., Stebletsova, V.: Modal logic, transition systems and processes. Journal of Logic and Computation 4(5), 811–855 (1994)MATHCrossRefMathSciNetGoogle Scholar
  4. 235.
    Hollis, M.: Trust within reason. Cambridge University Press, Cambridge (1998)CrossRefGoogle Scholar
  5. 313.
    Meyer, J.-J. Ch.: A different approach to deontic logic: Deontic logic viewed as a variant of dynamic logic. Notre Dame Journal of Formal Logic 29(1), 109–136 (1988)MATHCrossRefMathSciNetGoogle Scholar
  6. 400.
    Shoham, Y., Tennenholtz, M.: On social laws for artificial agent societies: Off-line design. Artificial Intelligence 73(1-2), 231–252 (1995)CrossRefGoogle Scholar
  7. 207.
    Hansen, J., Pigozzi, G., van der Torre, L.: Ten philosophical problems in deontic logic. In: G. Boella, L. van der Torre, H. Verhagen (eds.) Normative Multi-agent Systems, no. 07122 in DROPS Proceedings. Internationales Begegnungs- und Forschungszentrum fuer Informatik (IBFI), Schloss Dagstuhl, Germany (2007)Google Scholar
  8. 123.
    Dastani, M., Arbab, F., de Boer, F.S.: Coordination and composition in multi-agent systems. In: Procs. of AAMAS’05, pp. 439–446 (2005)Google Scholar
  9. 87.
    Broersen, J., Dastani, M., Hulstijn, J., van der Torre, L.: Goal generation in the BOID architecture. Cognitive Science Quarterly 2(3–4), 428–447 (2002)Google Scholar
  10. 380.
    Russell, S., Norvig, P.: Artificial Intelligence. A Modern Approach. Prentice Hall International (2001)Google Scholar
  11. 47.
    Blackburn, P., de Rijke, M., Venema, Y.: Modal Logic, Cambridge Tracts in Theoretical Computer Science, vol. 53. Cambridge University Press (2001)Google Scholar
  12. 303.
    Maskin, E.: Nash equilibrium and welfare optimality. Review of Economic Studies 66, 23–38 (1999)Google Scholar
  13. 245.
    Hurwicz, L.: Optimality and informational efficiency in resource allocation processes. In: K. Arrow, S. Karlin, P. Suppes (eds.) Mathematical Methods in the Social Sciences. Stanford University Press (1960)Google Scholar
  14. 327.
    Osborne, M.J., Rubinstein, A.: A Course in Game Theory. MIT Press (1994)Google Scholar
  15. 5.
    Alchourrón, C.E., Bulygin, E.: Normative Systems. Springer Verlag (1971)Google Scholar
  16. 420.
    van Ditmarsch, H., Kooi, B., van der Hoek, W.: Dynamic Epistemic Logic, Synthese Library Series, vol. 337. Springer (2007)Google Scholar
  17. 15.
    Anderson, A.: A reduction of deontic logic to alethic modal logic. Mind 22, 100–103 (1958)CrossRefGoogle Scholar
  18. 203.
    Grossi, D., Aldewereld, H., Dignum, F.: Ubi lex ibi poena. designing norm enforcement in electronic institutions. In: V. Dignum, N. Fornara, P. Noriega, G. Boella, O. Boissier, E. Matson, J. Vázquez-Salceda J.zquez-Salceda (eds.) Proceedings of COIN@AAMAS’06, LNCS, vol. 4386, pp. 101–114. Springer (2006)Google Scholar
  19. 40.
    van Benthem, J.: Extensive games as process models. Journal of Logic, Language and Information 11, 289–313 (2002)MATHCrossRefMathSciNetGoogle Scholar
  20. 268.
    Kanger, S.: New fondations for ethical theory. In: R. Hilpinen (ed.) Deontic Logic: Introductory and Systematic Readings, pp. 36–58. Reidel Publishing Company (1971)Google Scholar
  21. 54.
    Boella, G., van der Torre, L.: Substantive and procedural norms in normative multiagent systems. Journal of Applied Logic (In press)Google Scholar
  22. 202.
    Grossi, D.: Designing invisible handcuffs. formal investigations in institutions and organizations for multi-agent systems. Ph.D. thesis, Utrecht University, SIKS (2007)Google Scholar
  23. 206.
    Hansen, J.: Imperatives and Deontic Logic: On the Semantic Foundations of Deontic Logic. University of Leipzig (2008)Google Scholar
  24. 366.
    Ricci, A., Omicini, A., Denti, E.: Activity theory as a framework for mas coordination. In: Procs. of ESAW’02, pp. 96–110 (2002)Google Scholar
  25. 145.
    Dignum, V.: A model for organizational interaction. Ph.D. thesis, Utrecht University (2003)Google Scholar
  26. 293.
    Luck, M., McBurney, P., Preist, C.: Agent Technology: Enabling Next Generation Computing (A Roadmap for Agent Based Computing). AgentLink (2003)Google Scholar
  27. 291.
    Lopez, F., Luck, M., d’Inverno, M.: A normative framework for agent-based systems. Computational and Mathematical Organization Theory 12, 227–250 (2006)CrossRefGoogle Scholar
  28. 56.
    Boella, G., van der Torre, L.: The ontological properties of social roles in multi-agent systems: Definitional dependence, powers and roles playing roles. Artificial Intelligence and Law 15(3), 201–221 (2007)CrossRefGoogle Scholar
  29. 58.
    Boella, G., van der Torre, L., Villata, S.: Conditional dependence networks in requirements engineering. In: Proceedings of COIN’09, LNCS (2009)Google Scholar
  30. 116.
    Conte, R., Castelfranchi, C., Dignum, F.: Autonomous norm acceptance. In: J. Müller, M.P. Singh, A.S. Rao (eds.) Proceedings of the 5th International Workshop on Intelligent Agents V : Agent Theories, Architectures, and Languages (ATAL-98), vol. 1555, pp. 99–112. Springer-Verlag: Heidelberg, Germany (1999)CrossRefGoogle Scholar
  31. 50.
    Boella, G., Pigozzi, G., van der Torre, L.: Five guidelines for normative multi-agent systems. In: Proceedings of JURIX 2009 (2009)Google Scholar
  32. 188.
    Garcia-Camino, A., Noriega, P., Rodriguez-Aguilar, J.A.: Implementing norms in electronic institutions. In: Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems (AAMAS’05), pp. 667–673. ACM Press (2005)Google Scholar
  33. 260.
    Jones, A.J.I., Sergot, M.: On the characterization of law and computer systems. Deontic Logic in Computer Science pp. 275–307 (1993)Google Scholar
  34. 93.
    Bulygin, E.: Permissive norms and normative systems. In: A. Martino, F.S. Natali (eds.) Automated Analysis of Legal Texts, pp. 211–218. Publishing Company, Amsterdam (1986)Google Scholar
  35. 53.
    Boella, G., van der Torre, L.: A game-theoretic approach to normative multi-agent systems. In: G. Boella, L. van der Torre, H. Verhagen (eds.) Normative Multi-agent Systems, no. 07122 in Dagstuhl Seminar Proceedings. Internationales Begegnungs- und Forschungszentrum fuer Informatik (IBFI), Schloss Dagstuhl, Germany (2007)Google Scholar
  36. 51.
    Boella, G., Pigozzi, G., van der Torre, L.: Norms in computer science: Ten guidelines for normative multi-agent systems. In: Normative Multi-Agent Systems (NorMAS’09), no. 09121 in Dagstuhl Seminar Proceedings. Internationales Begegnungs- und Forschungszentrum fuer Informatik (IBFI) (2009)Google Scholar
  37. 287.
    Lindahl, L., Odelstad, J.: Open and closed intermediaries in normative systems. In: T. van Engers (ed.) Proceedings of the Nineteenth JURIX Conference on Legal Knowledge and Information Systems (JURIX 2006), pp. 91–100 (2006)Google Scholar
  38. 59.
    Boella, G., Verhagen, H., van der Torre, L.: Introduction to the special issue on normative multi-agent systems. Journal of Autonomous Agents and Multi Agent Systems 17(1), 1–10 (2008)CrossRefGoogle Scholar
  39. 412.
    Tennenholtz, M.: On stable social laws and qualitative equilibria. Artificial Intelligence 102(1), 1–20 (1998)MATHCrossRefMathSciNetGoogle Scholar
  40. 57.
    Boella, G., van der Torre, L., Verhagen, H.: Introduction to normative multi-agent systems. Computational and Mathematical Organization Theory 12(2-3), 71–79 (2006)CrossRefGoogle Scholar
  41. 251.
    Jackson, M.O.: A crash course in implementation theory. Social Choice and Welfare 18, 655–708 (2001)MATHCrossRefMathSciNetGoogle Scholar
  42. 88.
    Broersen, J., Dastani, M., van der Torre, L.: BDIOCTL: Obligations and the Specification of Agent Behavior. In: Proceedings of IJCAI’03, pp. 1389–1390 (2003)Google Scholar
  43. 41.
    van Benthem, J.: Logic in games. Lecture Notes of the ILLC graduate course on Logic, Language and Information, Universiteit van Amsterdam, Amsterdam, The Netherlands (2005)Google Scholar
  44. 125.
    Dastani, M., Grossi, D., Tinnemeier, N., Meyer, J.-J. Ch.: Normative multi-agent programs and their logics. In: G. Boella, G. Pigozzi, P. Noriega, H. Verhagen (eds.) Normative Multi-agent Systems, Dagstuhl Seminar Proceedings, vol. 09121 (2008)Google Scholar
  45. 242.
    Hübner, J.F., Sichman, J.S., Boissier, O.: Moise+: towards a structural, functional, and deontic model for mas organization. In: AAMAS, pp. 501–502. ACM (2002)Google Scholar
  46. 401.
    Shoham, Y., Tennenholtz, M.: On the emergence of social conventions: Modeling, analysis and simulations. Artificial Intelligence 94(1–2), 139–166 (1997)MATHCrossRefGoogle Scholar
  47. 284.
    Lewis, D.: A problem about permission. In: E. Saarinen (ed.) Essays in Honour of Jaakko Hintikka, pp. 163–175. D. Reidel, Dordrecht (1979)Google Scholar
  48. 48.
    Boella, ., van der Torre, L.: Δ: The social delegation cycle. In: Deontic Logic: 7th International Workshop on Deontic Logic in Computer Science (ΔEON’04), LNCS, vol. 3065, pp. 29–42. Springer (2004)Google Scholar
  49. 417.
    van der Torre, L., Tan, Y.: Diagnosis and decision making in normative reasoning. Artificial Intelligence and Law 7(1), 51–67 (1999)CrossRefGoogle Scholar
  50. 243.
    Hubner, J.F., Sichman, J.S., Boissier, O.: Developing organised multiagent systems using the moise+ model: programming issues at the system and agent levels. Int. J. Agent-Oriented Softw. Eng. 1(3/4), 370–395 (2007)CrossRefGoogle Scholar
  51. 197.
    Goffman, E.: Strategic interaction. Basil Blackwell, Oxford (1970)Google Scholar
  52. 52.
    Boella, G., van der Torre, L.: Enforceable social laws. In: Procs. of 4th International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS’05), pp. 682–689. ACM Press (2005)Google Scholar
  53. 55.
    Boella, G., van der Torre, L., Verhagen, H. (eds.): Normative Multi-Agent Systems, no. 07122 in Dagstuhl Seminar Proceedings. Internationales Begegnungs- und Forschungszentrum fuer Informatik (IBFI), Schloss Dagstuhl, Germany (2007)Google Scholar
  54. 167.
    Esteva, M., Rosell, B., Rodríguez-Aguilar, J.A., Arcos, J.L.: AMELI: An Agent-Based Middleware for Electronic Institutions. In: AAMAS, pp. 236–243. IEEE Computer Society (2004)Google Scholar
  55. 324.
    North, D.C.: Institutions, Institutional Change and Economic Performance. Cambridge University Press, Cambridge (1990)Google Scholar
  56. 360.
    Rao, A.S., Georgeff, M.P.: Modeling rational agents within a BDI-architecture. In: J. Allen, R. Fikes, E. Sandewall (eds.) Proceedings of the 2nd International Conference on Principles of Knowledge Representation and Reasoning (KR’91), pp. 473–484. Morgan Kaufmann publishers Inc.: San Mateo, CA, USA (1991)Google Scholar
  57. 409.
    Strauss, A.: Negotiations: Varieties, Contexts, Processes and Social Order. San Francisco, Jossey-Bass (1978)Google Scholar
  58. 375.
    Ruiter, D.: A basic classification of legal institutions. Ratio Juris 10(4), 357–371 (1997)CrossRefGoogle Scholar
  59. 252.
    Jackson, M.O.: Mechanism theory. In: U. Derigs (ed.) Encyclopedia of Life Support Systems. EOLSS Publishers (2003)Google Scholar
  60. 194.
    Giddens, A.: Social Theory and Modern Sociology. Polity Press (1984)Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.ILLC University of AmsterdamAmsterdamThe Netherlands
  2. 2.Computer Science King’s College LondonLondonUK
  3. 3.ICR University of LuxembourgLuxembourgUK

Personalised recommendations