Autonomous Agents and Multi-Agent Systems

, Volume 31, Issue 6, pp 1236–1282 | Cite as

Detection and resolution of normative conflicts in multi-agent systems: a literature survey

  • Jéssica S. Santos
  • Jean O. Zahn
  • Eduardo A. Silvestre
  • Viviane T. Silva
  • Wamberto W. Vasconcelos
Article

Abstract

Multi-agents systems are composed of autonomous and possibly heterogeneous software agents that act according to their own interests. Some coordination mechanism must be adopted to ensure a proper functioning of the whole system. Norms can be viewed as a powerful means to regulate and influence the behaviour of the agents by specifying, for instance, obligations, permissions, or prohibitions in a given context. A critical issue that must be considered in a system governed by multiple norms is the possible existence of normative conflicts. A conflict between norms is a situation in which the fulfilment of a norm causes a violation of another one. In this paper, we present several techniques that have been proposed to detect and resolve normative conflicts in multi-agent systems. Our aim is to organize the literature, present a classification of the techniques found, and provide a means to compare alternative approaches dealing with normative conflicts.

Keywords

Norms Conflict detection Conflict resolution Multi-agent systems 

References

  1. 1.
    Aphale, M., Norman, T. J., & Şensoy, M. (2013). Goal-directed policy conflict detection and prioritisation. In Aldewereld, H., Sichman, J.S., (Eds), Coordination, organisations, institutions and norms in agent systems VIII, volume 7756 of Lecture notes in computer science (pp. 87–104). Springer.Google Scholar
  2. 2.
    Bicchieri, C. (2006). The grammar of society: The nature and dynamics of social norms. Cambridge: Cambridge University Press.Google Scholar
  3. 3.
    Boella, G., Tosatto, S. C., Garcez, A. D., Genovese, V., Perotti, A., & van der Torre, L. (2012). Learning and reasoning about norms using neural-symbolic systems. In Proceedings of the 11th international conference on autonomous agents and multiagent systems—Vol. 2. International foundation for autonomous agents and multiagent systems, pp. 1023–1030.Google Scholar
  4. 4.
    Boella, G., van der Torre, L., & Verhagen, H. (2006). Introduction to normative multiagent systems. Computational & Mathematical Organization Theory, 12(2–3), 71–79.CrossRefGoogle Scholar
  5. 5.
    Broersen, J., Dastani, M., & van der Torre, L. (2001b). Resolving conflicts between beliefs, obligations, intentions, and desires. In Proceedings of the 6th European conference on symbolic and quantitative approaches to reasoning with uncertainty, ECSQARU ’01 (pp. 568–579), London: Springer.Google Scholar
  6. 6.
    Broersen, J., Dastani, M., Hulstijn, J., Huang, Z., & van der Torre, L. (2001a). The BOID architecture: Conflicts between beliefs, obligations, intentions and desires. In Proceedings of the fifth international conference on autonomous agents, AGENTS ’01, New York, NY, USA. ACM, pp. 9–16.Google Scholar
  7. 7.
    Cholvy, L. (2006). Querying contradictory databases by taking into account their reliability and their number. In Flexible databases supporting imprecision and uncertainty (pp. 149–168). Springer.Google Scholar
  8. 8.
    Cholvy, L., & Cuppens, F. (1995). Solving normative conflicts by merging roles. In Proceedings of the 5th international conference on artificial intelligence and law, ICAIL ’95, New York, NY, USA. ACM, pp. 201–209.Google Scholar
  9. 9.
    Cholvy, L., & Cuppens, F. (1999). Reasoning about norms provided by conflicting regulations. In P. McNamara & H. Prakken (Eds.), Norms, logics and information systems: New studies in deontic logic and computer science. Frontiers in artificial intelligence and applications (Vol. 49, pp. 247–262). Washington; Amsterdam: IOS Press.Google Scholar
  10. 10.
    Cholvy, L., & Garion, C. (2004). Answering queries addressed to several databases according to a majority merging approach. Journal of Intelligent Information Systems, 22(2), 175–201.CrossRefMATHGoogle Scholar
  11. 11.
    Cliffe, O., De Vos, M., & Padget, J. (2007). Answer set programming for representing and reasoning about virtual institutions. In Inoue, K., Satoh, K., & Toni, F. (Eds), Computational logic in multi-agent systems, volume 4371 of Lecture notes in computer science (pp. 60–79). Berlin: Springer.Google Scholar
  12. 12.
    Criado, N., Argente, E., & Botti, V. (2010a). A BDI architecture for normative decision making. In Proceedings of the 9th international conference on autonomous agents and multiagent systems: volume 1. International foundation for autonomous agents and multiagent systems, pp. 1383–1384.Google Scholar
  13. 13.
    Criado, N., Argente, E., & Botti, V. J. (2010b). Normative deliberation in graded BDI agents. In Multiagent system technologies, 8th German conference, MATES 2010, Leipzig, Germany, September 27–29, 2010. Proceedings, pp. 52–63.Google Scholar
  14. 14.
    Criado, N., Argente, E., Noriega, P., & Botti, V. J. (2010c). Towards a normative BDI architecture for norm compliance. In Proceedings of the multi-agent logics, languages, and organisations federated workshops (MALLOW 2010), Lyon, France, August 30–September 2, 2010, pp. 65–81.Google Scholar
  15. 15.
    da Silva, V. T., Braga, C., & Zahn, J. O. (2015). Indirect normative conflict— conflict that depends on the application domain. In ICEIS 2015—Proceedings of the 17th international conference on enterprise information systems (Vol. 1), Barcelona, Spain, 27–30 April, 2015, pp. 452–461.Google Scholar
  16. 16.
    da Silva, V.T., Zahn, J. O. (2014). Normative conflicts that depend on the domain. In Balke, T., Dignum, F., van Riemsdijk, M. B., & Chopra, A. K. (Eds), Coordination, organizations, institutions, and norms in agent systems IX, volume 8386 of Lecture Notes in Computer Science (pp. 311–326). Springer International Publishing.Google Scholar
  17. 17.
    Diller, A. (1990). Z: An introduction to formal methods (Vol. 2). Chichester: Wiley.MATHGoogle Scholar
  18. 18.
    dos Santos Neto, B. F., da Silva, V. T., & de Lucena, C. J. P. (2010). Using Jason to develop normative agents. In A. da Rocha Costa, R. Vicari, & F. Tonidandel (Eds.), Advances in artificial intelligence—SBIA 2010, volume 6404 of Lecture notes in computer science (pp. 143–152). Berlin: Springer.Google Scholar
  19. 19.
    dos Santos Neto, B. F., da Silva, V. T., & de Lucena, C. J. P. (2012). An architectural model for autonomous normative agents. In L. Barros, M. Finger, A. Pozo, G. Gimenénez-Lugo, & M. Castilho (Eds.), Advances in artificial intelligence—SBIA 2012, Lecture notes in computer science (pp. 152–161). Berlin: Springer.Google Scholar
  20. 20.
    dos Santos Neto, B. F., da Silva, V. T., & de Lucena, C. J. P. (2013). Developing goal-oriented normative agents: The NBDI architecture. In J. Filipe, & A. Fred (Eds.), Agents and artificial intelligence, volume 271 of Communications in computer and information science (pp. 176–191). Berlin: Springer.Google Scholar
  21. 21.
    Dung, P. M. (1995). On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n-person games. Artificial Intelligence, 77(2), 321–357.MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Elhag, A. A., Breuker, J. A., & Brouwer, P. (2000). On the formal analysis of normative conflicts. Information & Communications Technology Law, 9(3), 207–217.CrossRefGoogle Scholar
  23. 23.
    Fenech, S., Pace, G. J., & Schneider, G. (2008). Detection of conflicts in electronic contracts. NWPT 2008, 34.Google Scholar
  24. 24.
    Fenech, S., Pace, G., & Schneider, G. (2009). Automatic conflict detection on contracts. In M. Leucker, C. Morgan (Eds.), Theoretical aspects of computing—ICTAC 2009, volume 5684 of Lecture notes in computer science (pp. 200–214). Berlin: Springer.Google Scholar
  25. 25.
    Fitting, M. (1990). First-order logic. In First-order logic and automated theorem proving, Texts and Monographs in Computer Science. Springer US, pp. 97–125.Google Scholar
  26. 26.
    Gaertner, D., Garcia-Camino, A., Noriega, P., Rodriguez-Aguilar, J. A., & Vasconcelos, W. W. (2007). Distributed norm management in regulated multiagent systems. In Proceedings of the 6th international joint conference on autonomous agents and multiagent systems, AAMAS ’07 (pp. 90:1–90:8). New York, NY: ACM.Google Scholar
  27. 27.
    Garcez, A. D., Broda, K., & Gabbay, D. M. (2012). Neural-symbolic learning systems: Foundations and applications. Berlin: Springer Science & Business Media.MATHGoogle Scholar
  28. 28.
    Garcia, E., Giret, A., & Botti, V. (2013). A model-driven CASE tool for developing and verifying regulated open MAS. Science of Computer Programming, 78(6), 695–704.CrossRefGoogle Scholar
  29. 29.
    García-Camino, A., Noriega, P., & Rodríguez-Aguilar, J. A. (2005). Implementing norms in electronic institutions. In Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems. ACM, pp. 667–673.Google Scholar
  30. 30.
    García-Camino, A., Noriega, P., & Rodríguez-Aguilar, J. A. (2007). An algorithm for conflict resolution in regulated compound activities. In G. O’Hare, A. Ricci, M. O’Grady, & O. Dikenelli, (Eds), Engineering societies in the agents world VII, volume 4457 of Lecture notes in computer science (pp. 193–208). Berlin: Springer.Google Scholar
  31. 31.
    Gelfond, M., & Lifschitz, V. (1991). Classical negation in logic programs and disjunctive databases. New Generation Computing, 9(3–4), 365–385.CrossRefMATHGoogle Scholar
  32. 32.
    Giannikis, G. K., & Daskalopulu, A. (2009). Normative conflicts-patterns, detection and resolution. In WEBIST, pp. 527–532.Google Scholar
  33. 33.
    Giannikis, G. K., & Daskalopulu, A. (2011). Normative conflicts in electronic contracts. Electronic Commerce Research and Applications, 10(2), 247–267.CrossRefGoogle Scholar
  34. 34.
    Grossi, D., Gabbay, D., & van der Torre, L. (2010). The norm implementation problem in normative multi-agent systems. In M. Dastani, K. V. Hindriks, & J.-J. C. Meyer (Eds.), Specification and verification of multi-agent systems (pp. 195–224). New York: Springer.CrossRefGoogle Scholar
  35. 35.
    Günay, A., & Yolum, P. (2013b). Engineering conflict-free multiagent systems. In First international workshop on engineering multiagent systems (EMAS).Google Scholar
  36. 36.
    Günay, A., & Yolum, P. (2013a). Constraint satisfaction as a tool for modeling and checking feasibility of multiagent commitments. Applied Intelligence, 39(3), 489–509.CrossRefGoogle Scholar
  37. 37.
    Hill, H. (1987). A functional taxonomy of normative conflict. Law and Philosophy, 6(2), 227–247.CrossRefGoogle Scholar
  38. 38.
    Holzmann, G. J. (2004). The SPIN model checker: Primer and reference manual (Vol. 1003). Reading: Addison-Wesley.Google Scholar
  39. 39.
    Kagal, L., & Finin, T. (2005). Modeling communicative behavior using permissions and obligations. In R. van Eijk, M.-P. Huget, & F. Dignum (Eds.), Agent communication, volume 3396 of Lecture notes in computer science (pp. 120–133). Berlin: Springer.Google Scholar
  40. 40.
    Kagal, L., & Finin, T. (2007). Modeling conversation policies using permissions and obligations. Autonomous Agents and Multi-Agent Systems, 14(2), 187–206.CrossRefGoogle Scholar
  41. 41.
    Kirkpatrick, K. (2015). The moral challenges of driverless cars. Communications of the ACM, 58(8), 19–20.CrossRefGoogle Scholar
  42. 42.
    Kollingbaum, M. J. (2005). Norm-governed practical reasoning agents. PhD thesis, University of Aberdeen.Google Scholar
  43. 43.
    Kollingbaum, M. J., & Norman, T. J. (2004). Strategies for resolving norm conflict in practical reasoning. In ECAI workshop coordination in emergent agent societies (Vol. 2004).Google Scholar
  44. 44.
    Kollingbaum, M. J., & Norman, T. J. (2006). Informed deliberation during norm-governed practical reasoning. In O. Boissier, J. Padget, V. Dignum, G. Lindemann, E. Matson, S. Ossowski, J. Sichman, & J. Vázquez-Salceda (Eds.), Coordination, organizations, institutions, and norms in multi-agent systems, volume 3913 of Lecture notes in computer science (pp. 183–197). Berlin: Springer.Google Scholar
  45. 45.
    Kollingbaum, M. J., Norman, T. J., Preece, A., & Sleeman, D. (2006). Norm refinement: Informing the re-negotiation of contracts. In ECAI 2006 workshop on coordination, organization, institutions and norms in agent systems, COIN@ ECAI (Vol. 2006), pp. 46–51.Google Scholar
  46. 46.
    Kollingbaum, M. J., Norman, T. J., Preece, A., & Sleeman, D. (2007). Norm conflicts and inconsistencies in virtual organisations. In P. Noriega, J. Vázquez-Salceda, G. Boella, O. Boissier, V. Dignum, N. Fornara, & E. Matson (Eds.), Coordination, organizations, institutions, and norms in agent systems II, volume 4386 of Lecture notes in computer science (pp. 245–258). Berlin: Springer.Google Scholar
  47. 47.
    Kollingbaum, M. J., Vasconcelos, W. W., García-Camino, A., & Norman, T. J. (2008a). Managing conflict resolution in norm-regulated environments. In A. Artikis, G. O’Hare, K. Stathis, G. Vouros (Eds.), Engineering Societies in the Agents World VIII, volume 4995 of Lecture Notes in Computer Science (pp. 55–71). Berlin: Springer.Google Scholar
  48. 48.
    Kollingbaum, M. J., Vasconcelos, W. W., García-Camino, A., & Norman, T. J. (2008b). Conflict resolution in norm-regulated environments via unification and constraints. In M. Baldoni, T. Son, M. van Riemsdijk, & M. Winikoff (Eds.), Declarative agent languages and technologies V, volume 4897 of Lecture Notes in computer science (pp. 158–174). Berlin: Springer.Google Scholar
  49. 49.
    Kyas, M., Prisacariu, C., & Schneider, G. (2008). Run-time monitoring of electronic contracts. In S. Cha, J.-Y. Choi, M. Kim, I. Lee, & M. Viswanathan (Eds.), Automated technology for verification and analysis, volume 5311 of Lecture notes in computer science (pp. 397–407). Berlin: Springer.Google Scholar
  50. 50.
    Li, T. (2013). Normative conflict detection and resolution in cooperating institutions. In Proceedings of the twenty-third international joint conference on artificial intelligence (pp. 3231–3232). AAAI Press.Google Scholar
  51. 51.
    Li, T. (2014). Normative conflict detection and resolution in cooperating institutions. PhD thesis, University of Bath.Google Scholar
  52. 52.
    Li, T., Jiang, J., Aldewereld, H., De Vos, M., Dignum, V., & Padget, J. (2014). Contextualized institutions in virtual organizations. In T. Balke, F. Dignum, van M. B. Riemsdijk, & A. K. Chopra (Eds.), Coordination, organizations, institutions, and norms in agent systems IX, volume 8386 of Lecture notes in computer science (pp. 136–154). Springer International Publishing.Google Scholar
  53. 53.
    Makinson, D., & van der Torre, L. (2000). Input/output logics. Journal of Philosophical Logic, 29(4), 383–408.MathSciNetCrossRefMATHGoogle Scholar
  54. 54.
    McNamara, P. (2006). Deontic logic. In Logic and the modalities in the twentieth century (Vol. 7). North-Holland.Google Scholar
  55. 55.
    Meneguzzi, F., & Luck, M. (2009). Norm-based behaviour modification in BDI agents. In Proceedings of The 8th international conference on autonomous agents and multiagent systems—Volume 1 (pp. 177–184). International foundation for autonomous agents and multiagent systems.Google Scholar
  56. 56.
    Meyer, J.-J. C., & Wieringa, R. (1993). Applications of deontic logic: A concise overview. In J.-J. Ch. Meyer & R. J. Wieringa (Eds.), Deontic logic in computer science: Normative system specification (pp. 3–16). Chichester: Wiley.Google Scholar
  57. 57.
    Meyer, J.-J. C., & Wieringa, R. J. (1994). Deontic logic in computer science: Normative system specification. New York: Wiley.MATHGoogle Scholar
  58. 58.
    Oren, N., Luck, M., Miles, S., & Norman, T. J. (2008). An argumentation inspired heuristic for resolving normative conflict. In 5th international workshop on coordination, organisations, institutions and norms in agent systems (COIN@AAMAS 2008).Google Scholar
  59. 59.
    Parsia, B., & Sirin, E. (2004). Pellet: An OWL DL reasoner. In Third international semantic web conference-poster (Vol. 18).Google Scholar
  60. 60.
    Prisacariu, C., & Schneider, G. (2007). A formal language for electronic contracts. In M. M. Bonsangue, & E. Johnsen (Eds.) Formal methods for open object-based distributed systems, volume 4468 of Lecture notes in computer science (pp. 174–189). Berlin: Springer.Google Scholar
  61. 61.
    Reiter, R. (1980). A logic for default reasoning. Artificial intelligence, 13(1), 81–132.MathSciNetCrossRefMATHGoogle Scholar
  62. 62.
    Russell, S., & Norvig, P. (2009). Artificial intelligence: A modern approach (3rd ed.). Upper Saddle River: Prentice Hall.MATHGoogle Scholar
  63. 63.
    Searle, J. R. (1969). Speech acts: An essay in the philosophy of language (Vol. 626). Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  64. 64.
    Şensoy, M., Norman, T. J., Vasconcelos, W. W., & Sycara, K. (2010). OWL-POLAR: Semantic policies for agent reasoning. In Patel-Schneider, P., Pan, Y., Hitzler, P., Mika, P., Zhang, L., Pan, J., Horrocks, I., & Glimm, B. (Eds), The semantic web—ISWC 2010, volume 6496 of Lecture notes in computer science (pp. 679–695). Berlin: Springer.Google Scholar
  65. 65.
    Şensoy, M., Norman, T. J., Vasconcelos, W. W., & Sycara, K. (2012). OWL-POLAR: A framework for semantic policy representation and reasoning. Web Semantics: Science, Services and Agents on the World Wide Web, 12–13, 148–160.Google Scholar
  66. 66.
    Sierra, C., Rodriguez-Aguilar, J. A., Noriega, P., Esteva, M., & Arcos, J. L. (2004). Engineering multi-agent systems as electronic institutions. European Journal for the Informatics Professional, 4(4), 33–39.Google Scholar
  67. 67.
    Vardi, M. Y. (2015). On lethal autonomous weapons. Communications of the ACM, 58(12), 5–5.CrossRefGoogle Scholar
  68. 68.
    Vasconcelos, W. W., & Norman, T. J. (2009). Contract formation through preemptive normative conflict resolution. In Proceedings of the 2009 conference on artificial intelligence research and development: Proceedings of the 12th international conference of the catalan association for artificial intelligence (pp. 179–188). Amsterdam: IOS Press.Google Scholar
  69. 69.
    Vasconcelos, W. W., Kollingbaum, M. J., & Norman, T. J. (2007b). Resolving conflict and inconsistency in norm-regulated virtual organizations. In Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems. ACM, p. 91.Google Scholar
  70. 70.
    Vasconcelos, W. W., Kollingbaum, M. J., García-Camino, A., & Norman, T. J. (2007a). Achieving conflict freedom in norm-based societies. In Workshop on coordination, organizations, institutions, and norms in agent systems. Citeseer.Google Scholar
  71. 71.
    Vasconcelos, W. W., Kollingbaum, M. J., & Norman, T. J. (2009). Normative conflict resolution in multi-agent systems. Autonomous Agents and Multi-Agent Systems, 19(2), 124–152.CrossRefGoogle Scholar
  72. 72.
    Vasconcelos, W. W., García-Camino, A., Gaertner, D., Rodríguez-Aguilar, J. A., & Noriega, P. (2012). Distributed norm management for multi-agent systems. Expert Systems with Applications, 39(5), 5990–5999.CrossRefGoogle Scholar
  73. 73.
    von Wright, G. H. (1951). Deontic logic. Mind, 60(237), 1–15.Google Scholar
  74. 74.
    Weiss, G. (1999). Multiagent systems: A modern approach to distributed artificial intelligence. Cambridge: MIT Press.Google Scholar
  75. 75.
    Wooldridge, M. J. (2009). An introduction to multiagent systems (2nd ed.). New York: Wiley.Google Scholar
  76. 76.
    Zahn, J. O. (2015). Um Mecanismo de Verificação de Conflitos Normativos Indiretos. Master’s thesis, Instituto de Computação - Universidade Federal Fluminense (IC/UFF), Niteroi, Brasil.Google Scholar
  77. 77.
    Zahn, J. O., & da Silva, V. T. (2014). On the checking of indirect normative conflicts. In Workshop Escola de Sistemas de Agentes, seus Ambientes e aplicações, 2014, Porto Alegre. Anais do Workshop-Escola de Sistemas de Agentes, seus Ambientes e aplicações, pp. 13–24.Google Scholar

Copyright information

© The Author(s) 2017

Authors and Affiliations

  • Jéssica S. Santos
    • 1
  • Jean O. Zahn
    • 1
  • Eduardo A. Silvestre
    • 1
  • Viviane T. Silva
    • 2
  • Wamberto W. Vasconcelos
    • 3
  1. 1.Computer Science Department, Institute of ComputingUniversidade Federal Fluminense (UFF)NiteróiBrazil
  2. 2.IBM ResearchRio de JaneiroBrazil
  3. 3.Department of Computing ScienceUniversity of AberdeenAberdeenUK

Personalised recommendations