Autonomous Agents and Multi-Agent Systems

, Volume 31, Issue 6, pp 1283–1343 | Cite as

Automated multi-level governance compliance checking

  • Thomas C. KingEmail author
  • Marina De Vos
  • Virginia Dignum
  • Catholijn M. Jonker
  • Tingting Li
  • Julian Padget
  • M. Birna van Riemsdijk


An institution typically comprises constitutive rules, which give shape and meaning to social interactions and regulative rules, which prescribe agent behaviour in the society. Regulative rules guide social interaction, in particular when they are coupled with reward and punishment regulations that are enforced for (non-)compliance. Institution examples include legislation and contracts. Formal institutional reasoning frameworks automate ascribing social meaning to agent interaction and determining whether those actions have social meanings that comprise (non-)compliant behaviour. Yet, institutions do not just govern societies. Rather, in what is called multi-level governance, institutional designs at lower governance levels (e.g., national legislation at the national level) are governed by higher level institutions (e.g., directives, human rights charters and supranational agreements). When an institution design is found to be non-compliant, punishments can be issued by annulling the legislation or imposing fines on the responsible designers (i.e., government). In order to enforce multi-level governance, higher governance levels (e.g., courts applying human rights) must check lower level institution designs (e.g., national legislation) for compliance; in order to avoid punishment, lower governance levels (e.g., national governments) must check their institution designs are compliant with higher-level institutions before enactment. However, checking non-compliance of institution designs in multi-level governance is non-trivial. In particular, because institutions in multi-level governance operate at different levels of abstraction. Lower level institutions govern with concrete regulations whilst higher level institutions typically comprise increasingly vague and abstract regulations. To address this issue, in this paper we propose a formal framework with a novel semantics that defines compliance between concrete lower level institutions and abstract higher level institutions. The formal framework is complemented by a sound and complete computational framework that automates compliance checking, which we apply to a real-world case study.


Institutions Normative reasoning Multi-level governance 



We would like to thank the anonymous reviewers of JAAMAS for helping to improve the article. Thomas C. King would like to thank John R. Searle for the correspondence on substitution-of-identicals which illuminated some formerly implicit assumptions (now explicit) made in this paper. Thomas C. King was supported by the SHINE ( project of TU Delft.


  1. 1.
    Aldewereld, H., Álvarez-Napagao, S., Dignum, F., Vázquez-Salceda, J. (2010). Making norms concrete. In Proceedings of the 9th international conference on autonomous agents and multiagent systems (AAMAS 2010) (pp. 807–814).Google Scholar
  2. 2.
    Anderson, A. R. (1958). A reduction of deontic logic to alethic modal logic. Mind, 67(265), 100–103.CrossRefGoogle Scholar
  3. 3.
    Andrighetto, G., Governatori, G., Noriega, P., van der Torre, L. (2013). Normative multi-agent systems, Vol. 4. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik.Google Scholar
  4. 4.
    Antoniou, G., Billington, D., Governatori, G., & Maher, M. J. (2001). Representation results for defeasible logic. ACM Transactions on Computational Logic, 2(2), 255–287.MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Artikis, A. (2009). Dynamic protocols for open agent systems. In 8th International conference on autonomous agents and multiagent systems (pp. 97–104).Google Scholar
  6. 6.
    Artikis, A., Pitt, J., Sergot, M. (2002). Animated specifications of computational societies. In Proceedings of the first international joint conference on Autonomous Agents and Multiagent Systems (pp. 1053 – 1061). New York, NY: ACM Press.Google Scholar
  7. 7.
    Artikis, A., Sergot, M., Pitt, J. (2003). Specifying electronic societies with the causal calculator. In Proceedings of the workshop on agent oriented software engeneering III (AOSE), volume LNCS 2585.Google Scholar
  8. 8.
    Baral, C. (2003). Knowledge representation, reasoning and declarative problem solving. Cambridge: Cambridge University Press.CrossRefzbMATHGoogle Scholar
  9. 9.
    Biagioli, C. (1997). Towards a legal rules functional micro-ontology. In Proceedings of the 1st LegOnt workshop on legal ontologies.Google Scholar
  10. 10.
    Boella, G., van der Torre, L. (2003). Permissions and obligations in hierarchical normative systems. In Proceedings of the 9th international conference on artificial intelligence and law (pp. 109–118).Google Scholar
  11. 11.
    Boella, G., van der Torre, L. (2004). Regulative and constitutive norms in normative multiagent systems. In Proceedings of 9th international conference on the principles of knowledge representation and reasoning (KR’04) (pp 255–265). AAAI Press.Google Scholar
  12. 12.
    Cliffe, O. (2007). Specifying and analysing institutions in multi-agent systems using answer set programming. PhD thesis, University of Bath.Google Scholar
  13. 13.
    Cliffe, O., De Vos, M., Padget, J. (2007). Answer set programming for representing and reasoning about virtual institutions. Computational logic in multi-agent systems (pp. 60–79).Google Scholar
  14. 14.
    Criado, N., Argente, E., Noriega, P., & Botti, V. (2013). Reasoning about constitutive norms in BDI agents. Logic Journal of the IGPL, 22(1), 66–93.MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Dastani, M., van der Torre, L., & Yorke-Smith, N. (2017). Commitments and interaction norms in organisations. Autonomous Agents and Multi-Agent Systems, 31(2), 207–249.CrossRefGoogle Scholar
  16. 16.
    Dignum, F. (2002). Abstract norms and electronic institutions. In International workshop on regulated agent-based social systems: Theories and applications (RASTA’02) (pp. 93–104).Google Scholar
  17. 17.
    Dignum, F., Weigand, H., Verharen, E. (1996) Meeting the deadline: On the formal specication of temporal deontic constraints. In International symposium on methodologies for intelligent systems (pp. 243–252).Google Scholar
  18. 18.
    Dignum, V., Vázquez-Salceda, J., & Dignum, F. (2004). OMNI: Introducing social structure, norms and ontologies into agent organizations. Programming multi-agent systems (pp. 181–198). Berlin Heidelberg: Springer.Google Scholar
  19. 19.
    D’Inverno, M., Luck, M., Noriega, P., Rodriguez-Aguilar, J., & Sierra, C. (2012). Communicating open systems. Artificial Intelligence, 186, 3146–3150.Google Scholar
  20. 20.
    Eiter, T., Faber, W., Leone, N., & Pfeifer, G. (1999). The diagnosis frontend of the dlv system. AI Communications, 12(1), 99–111.MathSciNetGoogle Scholar
  21. 21.
    European Court Reports (2014). C-293/12 Digital Rights Ireland Ltd v Minister for Communications, Marine and Natural Resources; Minister for Justice, Equality and Law Reform; Commissioner of the Garda Síochána; Ireland; and The Attorney General and Others. C-594/12 Digital Rights Irela.Google Scholar
  22. 22.
    European Parliament and the Council of the European Union (2006). Directive 2006/24/EC of the European Parliament and of the Council of 15 March 2006 on the retention of data generated or processed in connection with the provision of publicly available electronic communications services or of public communications network.Google Scholar
  23. 23.
    European Union (2000). Charter of fundamental rights of the European Union 2000/C 364/01. Official Journal of the European Communities.Google Scholar
  24. 24.
    Fichte, J. K., Hecher, M., Morak, M., Woltran, S. (2016). Counting answer sets via dynamic programming. In Informal proceedings of the first workshop on trends and applications of answer set programming, TAASP 2016, Klagenfurt, Austria, September 26, 2016.Google Scholar
  25. 25.
    Fornara, N. (2011) Specifying and monitoring obligations in open multiagent systems using semantic web technology. Semantic agent systems (pp. 25–45).Google Scholar
  26. 26.
    Fornara, N., & Colombetti, M. (2010). Representation and monitoring of commitments and norms using OWL. AI Communications, 23(4), 341–356.MathSciNetzbMATHGoogle Scholar
  27. 27.
    Fornara, N., Okouya, D., Colombetti, M. (2012). Using OWL 2 DL for expressing ACL Content and Semantics. In European workshop on multi-agent systems (pp. 97–113).Google Scholar
  28. 28.
    Gabbay, D., Horty, J., Parent, X., van der Meyden, R., van der Torre, L. (eds). (2013). Handbook of Deontic logic and normative systems, Vol. 1.Google Scholar
  29. 29.
    García-Camino, A., Noriega, P., Rodríguez-Aguilar. J.-A. (2006). An algorithm for conflict resolution in regulated compound activities. In Seventh annual international workshop engineering societies in the agents world (pp. 193–208).Google Scholar
  30. 30.
    Gebser, M., Kaufmann, B., & Kaminski, R. (2011). Potassco: The Potsdam answer set solving collection. AI Communications, 24(2), 107–124.MathSciNetzbMATHGoogle Scholar
  31. 31.
    Gelfond, M. (2008). Answer Sets. Foundations of Artificial Intelligence, 3, 285–316.CrossRefGoogle Scholar
  32. 32.
    Gelfond, M., Lifschitz, V. (1988) The stable model semantics for logic programming. In Logic programming: Proceedings of the fifth international conference and symposium (pp. 1070–1080).Google Scholar
  33. 33.
    Governatori, G., Hulstijn, J., Rotolo, A. (2007). Characterising deadlines in temporal modal defeasible logic. In Proceedings of the 20th Australian joint conference on artificial intelligence (pp. 486–496).Google Scholar
  34. 34.
    Governatori, G., Palmirani, M., Riveret, R., Rotolo, A., Sartor, G. (2005). Norm modifications in defeasible logic. In Legal knowledge and information systems (JURIX 2005) (pp. 13–22). IOS Press.Google Scholar
  35. 35.
    Governatori, G., & Rotolo, A. (2010). Changing legal systems: Legal abrogations and annulments in defeasible logic. Logic Journal of IGPL, 18, 157–194.MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Governatori, G., Rotolo, A., & Sartor, G. (2005). Temporalised normative positions in defeasible logic. In Proceedings of the 10th international conference on artificial intelligence and law (pp. 25–34). New York: ACM Press.Google Scholar
  37. 37.
    Grossi, D. (2008). Pushing Anderson’s envelope: The modal logic of ascription. In 9th International conference on deontic logic in computer science (DEON 2008) (pp. 263–277).Google Scholar
  38. 38.
    Grossi, D. (2011). Norms as ascriptions of violations: An analysis in modal logic. Journal of Applied Logic, 9(2), 95–112.MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Grossi, D., Aldewereld, H., Vázquez-Salceda, J., & Dignum, F. (2006). Ontological aspects of the implementation of norms in agent-based electronic institutions. Computational and Mathematical Organization Theory, 12, 251–275.CrossRefzbMATHGoogle Scholar
  40. 40.
    Grossi, D., Meyer, J.-J., Dignum, F. (2005).Modal logic investigations in the semantics of counts-as. In Proceedings of the 10th international conference on artificial intelligence and law (ICAIL ’05) (pp. 1–19). ACM.Google Scholar
  41. 41.
    Grossi, D., Meyer, J.-J. C., Dignum, F. (2006) Counts-as: Classification or constitution? An answer using modal logic. Lecture notes in computer science (including subseries Lecture notes in artificial intelligence and Lecture notes in bioinformatics), 4048 LNAI:115–130.Google Scholar
  42. 42.
    Grossi, D., Meyer, J. J. C., & Dignum, F. (2008). The many faces of counts-as: A formal analysis of constitutive rules. Journal of Applied Logic, 6(2), 192–217.MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Günay, A., Yolum, P. (2012) Detecting conflicts in commitments. Declarative agent languages and technologies IX (pp. 51–66).Google Scholar
  44. 44.
    Hart, H. L. A. (1961). The concept of law. Oxford: Clarendon Press.Google Scholar
  45. 45.
    Hooghe, L., & Marks, G. (2003). Unraveling the central state, but how? Types of multi-level governance. American Political Science Review, 97(2), 233–243.Google Scholar
  46. 46.
    Jiang, J. (2015). Organizational compliance: An agent-based model for designing and evaluating organizational interactions. PhD thesis, TU Delft, Delft University of Technology.Google Scholar
  47. 47.
    Jones, A. J. I., & Sergot, M. (1996). A formal characterisation of institutionalised power. Journal of IGPL, 4(3), 427–443.MathSciNetCrossRefzbMATHGoogle Scholar
  48. 48.
    King, T. C. (2016). Governing governance: A formal framework for analysing institutional design and enactment governance. PhD thesis, Delft University of Technology.Google Scholar
  49. 49.
    King, T. C., Dignum, V., Jonker, C. M. (2016). When do rule changes count-as legal rule changes? In Proceedings of the 22nd European conference on artificial intelligence (ECAI 2016). Frontiers in artificial intelligence and applications (Vol. 285. pp. 3–11).Google Scholar
  50. 50.
    King, T. C., Li, T., De Vos, M., Dignum, V., Jonker, C. M., Padget, J., Riemsdijk, M. B. V. (2015). A framework for institutions governing institutions. In Proceedings of the 2015 international conference on autonomous agents and multiagent systems (AAMAS 2015) (pp. 473–481), Istanbul, Turkey. International Foundation for Autonomous Agents and Multiagent Systems.Google Scholar
  51. 51.
    King, T. C., Li, T., Vos, M. D., Jonker, C. M., Padget, J., Riemsdijk, M. B. V., et al. (2016). International Workshops, COIN@ AAMAS, Istanbul, Turkey, May 4, 2015, COIN@ IJCAI, Buenos Aires, Argentina, July 26, 2015. Revised selected papers, 9628, (191–208)Google Scholar
  52. 52.
    King, T. C., Riemsdijk, M. B. V., Dignum, V., Jonker, C. M. (2015). Supporting request acceptance with use policies. In Coordination, organizations, institutions, and norms in agent systems X: COIN 2014 international workshops, COIN@ AAMAS, Paris, France, May 6, 2014, COIN@ PRICAI, Gold Coast, QLD, Australia, December 4, 2014, Revised Selected Papers (pp. 114 – 131). Springer.Google Scholar
  53. 53.
    Kowalski, R., & Sergot, M. (1986). A logic-based calculus of events. New Generation Computing, 4(1), 67–95.CrossRefzbMATHGoogle Scholar
  54. 54.
    Li, T. (2014) Normative conflict detection and resolution in cooperating institutions. PhD thesis, University of Bath.Google Scholar
  55. 55.
    Li, T., Balke, T., De Vos, M., Padget, J., Satoh, K. (2013). Legal conflict detection in interacting legal systems. In 1st International Jurix doctoral consortium and poster sessions, JURIX-DoCoPe 2013, in Conjunction with the 26th International conference on legal knowledge and information systems, JURIX 2013.Google Scholar
  56. 56.
    Li, T., Balke, T., De Vos, M., Satoh, K., Padget, J. (2013). Detecting conflicts in legal systems. In Y. Motomura, A. Butler, D. Bekki (Eds.), New frontiers in artificial intelligence. JSAI-isAI 2012. Lecture notes in computer science, (Vol. 7856, pp. 174–189). Springer, Berlin, Heidelberg.Google Scholar
  57. 57.
    Lopes Cardoso, H., Oliveira, E. (2008). Norm defeasibility in an institutional normative framework. In European conference on AI (ECAI ’08) (pp. 468–473).Google Scholar
  58. 58.
    Lopes Cardoso, H., & Oliveira, E. (2009). A context-based institutional normative environment. Coordination, organizations, institutions and norms in agent systems IV (pp. 140–155).Google Scholar
  59. 59.
    Lopes Cardoso, H., Oliveira, E. (2010). Directed deadline obligations in agent-based business contracts. In Coordination, organization, institutions and norms (COIN@AAMAS) (Vol. 6069, LNAI, pp. 225–240).Google Scholar
  60. 60.
    Lopes Cardoso, H., & Oliveira, E. (2010). Monitoring directed obligations with flexible deadlines: A rule-based approach. International workshop on declarative agent languages and technologies (pp. 77–92). Hungary: Budapest.Google Scholar
  61. 61.
    López, F. L. Y., Luck, M. (2003). Modelling Norms for autonomous agents. In Proceedings of The fourth Mexican conference on computer science (pp. 238–245). IEEE Computer Society.Google Scholar
  62. 62.
    López, F Ly, Luck, M., & D’Inverno, M. (2006). A normative framework for agent-based systems. Computational and Mathematical Organization Theory, 12(2–3), 227–250.CrossRefGoogle Scholar
  63. 63.
    Makinson, D., & van der Torre, L. (2003). What is input/output logic? Trends in Logic, 17, 163–174.MathSciNetCrossRefzbMATHGoogle Scholar
  64. 64.
    Nute, D. (1987). Defeasible logic. In Dov M. Gabbay, C. J. Hogger, J. A. Robinson (Eds.), Handbook of logic in artificial intelligence and logic programming (Vol. 3). New York: Oxford University Press.Google Scholar
  65. 65.
    Pieters, W., Padget, J., Dechesne, F. (2013). Obligations to enforce prohibitions: On the adequacy of security policies. In Proceedings of the 6th international conference on security of information and networks (pp. 54–61).Google Scholar
  66. 66.
    Pieters, W., Padget, J., Dechesne, F., Dignum, V., & Aldewereld, H. (2015). Effectiveness of qualitative and quantitative security obligations. Journal of Information Security and Applications, 22, 3–16.CrossRefGoogle Scholar
  67. 67.
    Pitt, J., & Artikis, A. (2015). The open agent society: Retrospective and prospective views. Artificial Intelligence and Law, 23(3), 241–270.CrossRefGoogle Scholar
  68. 68.
    Pitt, J., Diaconescu, A. (2015). Structure and governance of communities for the digital society. In IEEE International conference on autonomic computing (ICAC) (pp. 279–284).Google Scholar
  69. 69.
    Prakken, H., & Sartor, G. (1996). A dialectical model of assessing conflicting arguments in legal reasoning. Artificial Intelligence and Law, 4, 331–368.CrossRefGoogle Scholar
  70. 70.
    Searle, J. R. (1969). Speech acts: An essay in the philosophy of language. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  71. 71.
    Searle, J. R. (1983). Intentionality: An essay in the philosophy of mind. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  72. 72.
    Searle, J. R. (1995). The construction of social reality. New York: The Free Press.Google Scholar
  73. 73.
    Searle, J. R. (2005). What is an institution? Journal of Institutional Economics, 1, 1–22.CrossRefGoogle Scholar
  74. 74.
    UK (2009). The data retention (EC Directive) Regulations 2009, No. 859.Google Scholar
  75. 75.
    van der Torre, L., Tan, Y. (1988). The temporal analysis of Chisholm’s paradox. In Proceedings of the fifteenth national conference on artificial intelligence (AAAI’98) (pp. 650–655).Google Scholar
  76. 76.
    von Wright, G. (1951). Deontic logic. Mind, 60(237), 1–15.CrossRefGoogle Scholar
  77. 77.
    Yolum, P., & Singh, M. (2004). Reasoning about commitments in the event calculus: An approach for specifying and executing protocols. Annals of Mathematics and Artificial Intelligence, 42(1–3), 227–253.MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© The Author(s) 2017

Authors and Affiliations

  • Thomas C. King
    • 1
    Email author
  • Marina De Vos
    • 2
  • Virginia Dignum
    • 3
  • Catholijn M. Jonker
    • 3
  • Tingting Li
    • 4
  • Julian Padget
    • 2
  • M. Birna van Riemsdijk
    • 3
  1. 1.Lancaster UniversityLancasterUK
  2. 2.University of BathBathUK
  3. 3.Delft University of TechnologyDelftThe Netherlands
  4. 4.Imperial College LondonLondonUK

Personalised recommendations