Ten Problems of Deontic Logic and Normative Reasoning in Computer Science

  • Jan Broersen
  • Leendert van der Torre
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7388)


This tutorial presents and discusses ten problems of deontic logic and normative reasoning in computer science. Five of the problems have been taken or derived from a list of ten philosophical problems in deontic logic recently discussed by Hansen, Pigozzi and van der Torre. In what sense are obligations different from norms? How to reason about contrary-to-duty norms? How do norms change? How to relate various kinds of permissions? What is the role of constitutive norms? Hansen et al. discuss their ten philosophical problems from the viewpoint of input/output logic as developed by Makinson & van der Torre, and they argue that norms, not ideality, should take the central position in deontic semantics, and that a semantics for norms explicitly represented in the object language by using, e.g., input/output logic normative rules, provides a helpful tool for analyzing, clarifying and solving the problems of deontic logic. However, for applications in computer science and artificial intelligence we have to reconcile the input-output logic representation of norms with representations for agency, informational and motivational modalities (beliefs, intentions), time, actions and decision- and game-theoretic concepts. This leads to five more questions. What is the role of time in deontic reasoning? What is the role of action in deontic reasoning? How can we use norms to influence, solve, or control games? How do we resolve the general problem of norm compliance? How do norms interact with informational modalities such as beliefs and knowledge, and motivational modalities such as intentions and desires?


Deontic logic normative systems input/output logic agency 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ågotnes, T., Wooldridge, M., van der Hoek, W.: Normative System Games. In: Huhns, M., Shehory, O. (eds.) Proceedings of the Sixth International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2007), pp. 876–883. IFAMAAS (2007)Google Scholar
  2. 2.
    Alchourrón, C., Gärdenfors, P., Makinson, D.: On the logic of theory change: partial meet contraction and revision functions. Journal of Symbolic Logic 50, 510–530 (1985)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Alchourrón, C.E., Makinson, D.: Hierarchies of Regulations and Their Logic. In: [55], 125–148Google Scholar
  4. 4.
    Alur, R., Henzinger, T.A., Kupferman, O.: Alternating-time Temporal Logic. Journal of the ACM 49(5), 672–713 (2002)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Åqvist, L.: Good samaritans, contrary-to-duty imperatives, and epistemic obligations. NOUS 1, 361–379 (1967)CrossRefGoogle Scholar
  6. 6.
    Åqvist, L.: Combinations of tense and deontic logic. Journal of Applied Logic 3, 421–460 (2005)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Åqvist, L., Hoepelman, J.: Some theorems about a tree system of deontic tense logic. In: Hilpinen, R. (ed.) New Studies in Deontic Logic, pp. 187–221. D. Reidel Publishing Company (1981)Google Scholar
  8. 8.
    Artosi, A., Rotolo, A., Vida, S.: On the logical nature of count-as conditionals. In: Procs. of LEA 2004 Workshop (2004)Google Scholar
  9. 9.
    Bailhache, P.: Authorities and addressees in deontic logic: indexed operators and action. In: Proceedings of DEON 1991, pp. 72–88 (1991)Google Scholar
  10. 10.
    Bartha, P.: Conditional Obligation, Deontic Paradoxes, and the Logic of Agency. Annals of Mathematics and Artificial Intelligence 9(1-2), 1–23 (1993)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Belnap, N., Perloff, M., Xu, M.: Facing the future: agents and choices in our indeterminist world, Oxford (2001)Google Scholar
  12. 12.
    Boella, G., Broersen, J., van der Torre, L.: Reasoning about Constitutive Norms, Counts-As Conditionals, Institutions, Deadlines and Violations. In: Bui, T.D., Ho, T.V., Ha, Q.T. (eds.) PRIMA 2008. LNCS (LNAI), vol. 5357, pp. 86–97. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  13. 13.
    Boella, G., Governatori, G., Rotolo, A., van der Torre, L.: A Logical Understanding of Legal Interpretation. In: Lin, F., Sattler, U., Truszczynski, M. (eds.) KR. AAAI Press (2010)Google Scholar
  14. 14.
    Boella, G., Pigozzi, G., van der Torre, L.: Normative framework for normative system change. In: Sierra, C., Castelfranchi, C., Decker, K.S., Sichman, J.S. (eds.) AAMAS (1), pp. 169–176. IFAAMAS (2009)Google Scholar
  15. 15.
    Boella, G., van der Torre, L.: A Logical Architecture of a Normative System. In: [41], pp. 24–35Google Scholar
  16. 16.
    Boella, G., van der Torre, L.: Institutions with a hierarchy of authorities in distributed dynamic environments. Artif. Intell. Law 16(1), 53–71 (2008)CrossRefGoogle Scholar
  17. 17.
    Boella, G., van der Torre, L.W.N.: Regulative and Constitutive Norms in Normative Multiagent Systems. In: Dubois, D., Welty, C.A., Williams, M.-A. (eds.) KR, pp. 255–266. AAAI Press (2004)Google Scholar
  18. 18.
    Boella, G., van der Torre, L.W.N.: Enforceable social laws. In: Dignum, F., Dignum, V., Koenig, S., Kraus, S., Singh, M.P., Wooldridge, M. (eds.) AAMAS, pp. 682–689. ACM (2005)Google Scholar
  19. 19.
    Boella, G., van der Torre, L.W.N.: A Game-Theoretic Approach to Normative Multi-Agent Systems. In: Boella, et al. [20]Google Scholar
  20. 20.
    Boella, G., van der Torre, L.W.N., Verhagen, H.: Normative Multi-agent Systems, March 18-23. Dagstuhl Seminar Proceedings, vol. 07122. Internationales Begegnungs- und Forschungszentrum für Informatik (IBFI), Schloss Dagstuhl, Germany (2007)MATHGoogle Scholar
  21. 21.
    Broersen, J.: On the Logic of ‘Being Motivated to Achieve ρ, Before δ’. In: Alferes, J.J., Leite, J. (eds.) JELIA 2004. LNCS (LNAI), vol. 3229, pp. 334–346. Springer, Heidelberg (2004), doi:10.1007/b100483CrossRefGoogle Scholar
  22. 22.
    Broersen, J.: Strategic Deontic Temporal Logic as a Reduction to ATL, with an Application to Chisholm’s Scenario. In: Goble, L., Meyer, J.-J.C. (eds.) DEON 2006. LNCS (LNAI), vol. 4048, pp. 53–68. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  23. 23.
    Broersen, J.: A Logical Analysis of the Interaction between ‘Obligation-to-do’ and ‘Knowingly Doing’. In: van der Meyden, R., van der Torre, L. (eds.) DEON 2008. LNCS (LNAI), vol. 5076, pp. 140–154. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  24. 24.
    Broersen, J.: Issues in Designing Logical Models for Norm Change. In: Vouros, G., Artikis, A., Stathis, K., Pitt, J. (eds.) OAMAS 2008. LNCS, vol. 5368, pp. 1–17. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  25. 25.
    Broersen, J.: CTL.STIT: enhancing ATL to express important multi-agent system verification properties. In: AAMAS 2010: Proceedings of the Ninth International Joint Conference on Autonomous Agents and Multiagent Systems. ACM, New York (2010)Google Scholar
  26. 26.
    Broersen, J., Dastani, M., Hulstijn, J., van der Torre, L.: Goal Generation in the BOID Architecture. Cognitive Science Quarterly Journal 2(3-4), 428–447 (2002)Google Scholar
  27. 27.
    Broersen, J., Dastani, M., van der Torre, L.: Beliefs, Obligations, Intentions and Desires as Components in an Agent Architecture. International Journal of Intelligent Systems 20(9), 893–920 (2005)CrossRefMATHGoogle Scholar
  28. 28.
    Broersen, J., Dignum, F., Dignum, V., Meyer, J.-J.C.: Designing a Deontic Logic of Deadlines. In: Lomuscio, A., Nute, D. (eds.) DEON 2004. LNCS (LNAI), vol. 3065, pp. 43–56. Springer, Heidelberg (2004), doi:10.1007/b98159.CrossRefGoogle Scholar
  29. 29.
    Broersen, J., Mastop, R., Meyer, J.-J.C., Turrini, P.: A Deontic Logic for Socially Optimal Norms. In: van der Meyden, R., van der Torre, L. (eds.) DEON 2008. LNCS (LNAI), vol. 5076, pp. 218–232. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  30. 30.
    Broersen, J., van der Torre, L.: Reasoning about Norms, Obligations, Time and Agents. In: Ghose, A., Governatori, G., Sadananda, R. (eds.) PRIMA 2007. LNCS, vol. 5044, pp. 171–182. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  31. 31.
    Broersen, J., Wieringa, R., Meyer, J.-J.: A Fixed-point Characterization of a Deontic Logic of Regular ActionGoogle Scholar
  32. 32.
    Brown, M.: Doing as we ought: towards a logic of simply dischargeable obligations. In: Brown, M., Carmo, J. (eds.) Deontic Logic, Agency, and Normative Systems, Proceedings DEON 1996. Workshops in Computing, pp. 47–65. Springer (1996)Google Scholar
  33. 33.
    Bulygin, E.: Permissive Norms and Normative Concepts. In: Martino, A.A., Socci Natali, F. (eds.) Automated Analysis of Legal Texts, pp. 211–218. North Holland, Amsterdam (1986)Google Scholar
  34. 34.
    Chellas, B.: Modal Logic: An Introduction. Cambridge University Press (1980)Google Scholar
  35. 35.
    Chisholm, R.: Contrary-to-duty imperatives and deontic logic. Analysis 24, 33–36 (1963)CrossRefGoogle Scholar
  36. 36.
    Cholvy, L., Cuppens, F.: Reasoning about Norms Provided by Conflicting Regulations. In: [69], 247–264Google Scholar
  37. 37.
    Cuppens, F., Cholvy, L., Saurel, C., Carrère, J.: Merging Security Policies: Analysis of a Practical Example. In: CSFW, pp. 123–136 (1998)Google Scholar
  38. 38.
    Demolombe, R., Bretier, P., Louis, V.: Formalisation de l’obligation de faire avec délais. In: Proc. Journées Francophones sur la Modélisation Formelle de l’Interaction, Caen (2005)Google Scholar
  39. 39.
    van Eck, J.A.: A system of temporally relative modal and deontic predicate logic and its philosophical applications. Logique et Analyse 100, 339–381 (1982)MathSciNetMATHGoogle Scholar
  40. 40.
    Emerson, E.: Temporal and modal logic. In: van Leeuwen, J. (ed.) Handbook of Theoretical Computer Science, volume B: Formal Models and Semantics, ch. 14, pp. 996–1072. Elsevier Science (1990)Google Scholar
  41. 41.
    Goble, L., Meyer, J.-J.C. (eds.): DEON 2006. LNCS (LNAI), vol. 4048. Springer, Heidelberg (2006)MATHGoogle Scholar
  42. 42.
    Governatori, G.: Representing business contracts in RuleML. Int. J. Cooperative Inf. Syst. 14(2-3), 181–216 (2005)CrossRefGoogle Scholar
  43. 43.
    He, J., Yan, H., Jin, M., Liu, C.: Categorizing Software Engineering Knowledge Using a Combination of SWEBOK and Text Categorization. In: Orgun, M.A., Thornton, J. (eds.) AI 2007. LNCS (LNAI), vol. 4830, pp. 675–681. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  44. 44.
    Governatori, G., Rotolo, A.: Changing Legal Systems: Abrogation and Annulment. Part II: Temporalised Defeasible Logic. In: Proc. NorMAS 2008 (2008)Google Scholar
  45. 45.
    Governatori, G., Rotolo, A.: Changing legal systems: legal abrogations and annulments in Defeasible Logic. Logic Journal of the IGPL 18(1), 157–194 (2010)MathSciNetCrossRefMATHGoogle Scholar
  46. 46.
    Governatori, G., Sartor, G. (eds.): DEON 2010. LNCS, vol. 6181. Springer, Heidelberg (2010)MATHGoogle Scholar
  47. 47.
    Grégoire, E.: Fusing legal knowledge. In: Proceedings of the 2004 IEEE INt. Conf. on Information Reuse and Integration (IEEE-IRI 2004), pp. 522–529 (2004)Google Scholar
  48. 48.
    Hansen, J., Pigozzi, G., van der Torre, L.W.N.: Ten Philosophical Problems in Deontic Logic. In: Boella, et al. [20]Google Scholar
  49. 49.
    Hansson, B.: An Analysis of Some Deontic Logics. Nôus 3, 373–398 (1969); reprinted in [54], 121–147MathSciNetGoogle Scholar
  50. 50.
    Harel, D., Kozen, D., Tiuryn, J.: Dynamic Logic. The MIT Press (2000)Google Scholar
  51. 51.
    Herrestad, H., Krogh, C.: Obligations Directed from Bearers to Counterparts. In: Proceedings ICAIL 1995 (1995)Google Scholar
  52. 52.
    Herzig, A., Lorini, E.: A Dynamic Logic of Agency I: STIT, Capabilities and Powers. Journal of Logic, Language and Information 19(1), 89–121 (2010)MathSciNetCrossRefMATHGoogle Scholar
  53. 53.
    Hikmi, S.A.S.N., Indulwska, M., Sadiq, S. (eds.): A Study of Compliance Management in Information Systems Research (2009)Google Scholar
  54. 54.
    Hilpinen, R. (ed.): Deontic Logic: Introductory and Systematic Readings. Reidel, Dordrecht (1971)MATHGoogle Scholar
  55. 55.
    Hilpinen, R. (ed.): New Studies in Deontic Logic. Reidel, Dordrecht (1981)MATHGoogle Scholar
  56. 56.
    Horty, J.: Agency and Deontic Logic. Oxford University Press (2001)Google Scholar
  57. 57.
    Jones, A., Sergot, M.: A formal characterisation of institutionalised power. Journal of IGPL 3, 427–443 (1996)MathSciNetCrossRefMATHGoogle Scholar
  58. 58.
    Kooi, B., Tamminga, A.: Moral conflicts between groups of agents. Journal of Philosophical Logic 37(1), 1–21 (2008)MathSciNetCrossRefMATHGoogle Scholar
  59. 59.
    Lewis, D.: Counterfactuals. Basil Blackwell, Oxford (1973)MATHGoogle Scholar
  60. 60.
    Lewis, D.: A problem about permission. In: Saarinen, E. (ed.) Essays in Honour of Jaakko Hintikka, pp. 163–175. D. Reidel, Dordrecht (1979)CrossRefGoogle Scholar
  61. 61.
    Liu, F.: Changing for the Better: Preference Dynamics and Agent Diversity, Ph.D. thesis. ILLC Dissertation Series, Amsterdam (2008)Google Scholar
  62. 62.
    Loewer, B., Belzer, M.: Dyadic deontic detachment. Synthese 54, 295–318 (1983)MathSciNetCrossRefMATHGoogle Scholar
  63. 63.
    Makinson, D.: On a Fundamental Problem of Deontic Logic. In: [69], 29–53Google Scholar
  64. 64.
    Makinson, D.: On the Formal Representation of Rights Relations. Journal of Philosophical Logic 15, 403–425 (1986)MathSciNetCrossRefMATHGoogle Scholar
  65. 65.
    Makinson, D.: Five faces of minimality. Studia Logica 52, 339–379 (1993)MathSciNetCrossRefMATHGoogle Scholar
  66. 66.
    Makinson, D., van der Torre, L.: Input/Output Logics. Journal of Philosophical Logic 29, 383–408 (2000)MathSciNetCrossRefMATHGoogle Scholar
  67. 67.
    Makinson, D., van der Torre, L.: Constraints for Input/Output Logics. Journal of Philosophical Logic 30, 155–185 (2001)MathSciNetCrossRefMATHGoogle Scholar
  68. 68.
    Makinson, D., van der Torre, L.: Permissions from an Input/Output Perspective. Journal of Philosophical Logic 32, 391–416 (2003)MathSciNetCrossRefMATHGoogle Scholar
  69. 69.
    McNamara, P., Prakken, H. (eds.): Norms, Logics and Information Systems. IOS, Amsterdam (1999)Google Scholar
  70. 70.
    Meyer, J.-J.: A Different Approach to Deontic Logic: Deontic Logic Viewed as a Variant of Dynamic Logic. Notre Dame Journal of Formal Logic 29, 109–136 (1988)MathSciNetCrossRefMATHGoogle Scholar
  71. 71.
    Moses, Y., Tennenholtz, M.: Artificial social systems. Computers and AI 14, 533–562 (1995)MathSciNetGoogle Scholar
  72. 72.
    Pacuit, E., Parikh, R., Cogan, E.: The Logic of Knowledge Based Obligation. Knowledge, Rationality and Action a subjournal of Synthese 149(2), 311–341 (2006)MathSciNetMATHGoogle Scholar
  73. 73.
    Parent, X.: Remedial Interchange, Contrary-to-Duty Obligation and Commutation. Journal of Applied Non-Classical Logics 13(3-4), 345–375 (2003)CrossRefMATHGoogle Scholar
  74. 74.
    Parent, X.: Moral particularism in the light of deontic logic. Artif. Intell. Law 19(2-3), 75–98 (2011)CrossRefGoogle Scholar
  75. 75.
    Pauly, M.: A Modal Logic for Coalitional Power in Games. Journal of Logic and Computation 12(1), 149–166 (2002)MathSciNetCrossRefMATHGoogle Scholar
  76. 76.
    Prakken, H., Sergot, M.: Contrary-to-duty Obligations and Defeasible Reasoning. Studia Logica 57, 91–115 (1996)MathSciNetCrossRefMATHGoogle Scholar
  77. 77.
    Santos, F., Carmo, J.: Indirect Action, Influence and Responsibility. In: Brown, M.A., Carmo, J. (eds.) DEON, Workshops in Computing, pp. 194–215. Springer (1996)Google Scholar
  78. 78.
    Searle, J.: Speech Acts: an Essay in the Philosophy of Language. Cambridge University Press, Cambridge (1969)CrossRefGoogle Scholar
  79. 79.
    Searle, J.: The Construction of Social Reality. The Free Press, New York (1995)Google Scholar
  80. 80.
    Spohn, W.: An Analysis of Hansson’s Dyadic Deontic Logic. Journal of Philosophical Logic 4, 237–252 (1975)MathSciNetCrossRefMATHGoogle Scholar
  81. 81.
    Thomason, R.H.: Combinations of Tense and Modality. In: Gabbay, D., Guenthner, F. (eds.) Handbook of Philosophical Logic: Extensions of Classical Logic, pp. 135–165. Reidel (1984)Google Scholar
  82. 82.
    van der Torre, L.: Deontic Redundancy: A Fundamental Challenge for Deontic Logic. In: Governatori, Sartor [46], pp. 11–32Google Scholar
  83. 83.
    van der Torre, L.W.N.: Violation games: a new foundation for deontic logic. Journal of Applied Non-Classical Logics 20(4), 457–477 (2010)MathSciNetCrossRefMATHGoogle Scholar
  84. 84.
    van der Torre, L.W.N., Tan, Y.-H.: The Temporal Analysis of Chisholm’s Paradox. In: Mostow, J., Rich, C. (eds.) AAAI/IAAI, pp. 650–655. AAAI Press / The MIT Press (1998)Google Scholar
  85. 85.
    Gruber, T.R.: Ontolingua: A mechanism to Support Portable Ontologies (1990)Google Scholar
  86. 86.
    von Wright, G.H.: Deontic Logic. Mind 60, 1–15 (1951)CrossRefGoogle Scholar
  87. 87.
    von Wright, G.H.: Deontic logic. Mind 60, 1–15 (1951)CrossRefGoogle Scholar
  88. 88.
    von Wright, G.H.: Norm and action; a logical enquiry. International Library of Philosophy and Scientific Method. Routledge & Kegan Paul (1963)Google Scholar
  89. 89.
    von Wright, G.H.: On the logic of norms and actions. In: Hilpinen, R. (ed.) New Studies in Deontic Logic, pp. 3–35. D. Reidel Publishing Company (1981)Google Scholar
  90. 90.
    von Wright, G.H.: Deontic Logic - as I see it. In: McNamara, P., Prakken, H. (eds.) Norms, Logics and Information Systems. New Studies on Deontic Logic and Computer Science, pp. 15–25. IOS Press (1999)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Jan Broersen
    • 1
  • Leendert van der Torre
    • 2
  1. 1.University of UtrechtThe Netherlands
  2. 2.University of LuxembourgLuxembourg

Personalised recommendations