Coordination with Collective and Individual Decisions
The response to a large-scale disaster, e.g. an earthquake or a terrorist incident, urges for low-cost policies that coordinate sequential decisions of multiple agents. Decisions range from collective (common good) to individual (self-interested) perspectives, intuitively shaping a two-layer decision model. However, current decision theoretic models are either purely collective or purely individual and seek optimal policies. We present a two-layer, collective versus individual (CvI) decision model and explore the tradeoff between cost reduction and loss of optimality while learning coordination skills. Experiments, in a partially observable domain, test our approach for learning a collective policy and results show near-optimal policies that exhibit coordinated behavior.
KeywordsMultiagent System Markov Decision Process Coordination Policy Individual Policy Temporal Abstraction
Unable to display preview. Download preview PDF.
- 2.Boutilier, C.: Sequential Optimality and Coordination in Multi-Agent Systems. In: Sixteenth Int. Joint Conference on Artificial Intelligence (IJCAI 1999), pp. 478–485 (1999)Google Scholar
- 3.Bradtke, S., Duff, M.: Reinforcement learning methods for continuous time Markov decision problems. Advances in Neural Inf. Processing Systems 8, 393–400 (1995)Google Scholar
- 4.Corrêa, M., Coelho, H.: Collective Mental States in Extended Mental States Framework. In: International Conference on Collective Intentionality (2004)Google Scholar
- 6.FIPA Communicative Act Library Specification (2002), http://www.fipa.org
- 7.Ghavamzadeh, M., Mahadevan, S., Makar, R.: Hierarchical Multi-Agent Reinforcement Learning. Journal of Autonomous Agents and Multi-Agent Systems (2006)Google Scholar
- 8.Jonsson, A., Barto, A.: Automated State Abstractions for Options Using the U-Tree Algorithm. Advances in Neural Inf. Processing Systems 13, 1054–1060 (2001)Google Scholar
- 9.Kitano, H., Tadokoro, S., Noda, I., Matsubara, H., Takahashi, T., Shinjou, A., Shimada, S.: RoboCup Rescue: Search and Rescue in Large-Scale Disasters as a Domain for Autonomous Agents Research. In: Conf. on Man, System and Cyb. (MSC 1999), pp. 739–743 (1999)Google Scholar
- 11.Pynadath, D., Tambe, M.: The Communicative Multiagent Team Decision Problem: Analyzing Teamwork Theories and Models. Journal of AI Research, 389–423 (2002)Google Scholar
- 12.Rohanimanesh, K., Mahadevan, S.: Learning to Take Concurrent Actions. In: Sixteenth Annual Conference on Neural Information Processing Systems, pp. 1619–1626 (2003)Google Scholar