Advertisement

Autonomous Agents and Multi-Agent Systems

, Volume 22, Issue 3, pp 415–438 | Cite as

Optimizing coalition formation for tasks with dynamically evolving rewards and nondeterministic action effects

  • Majid Ali Khan
  • Damla Turgut
  • Ladislau BölöniEmail author
Article

Abstract

We consider a problem domain where coalitions of agents are formed in order to execute tasks. Each task is assigned at most one coalition of agents, and the coalition can be reorganized during execution. Executing a task means bringing it to one of the desired terminal states, which might take several time steps. The state of the task evolves even if no coalition is assigned to its execution and depends nondeterministically on the cumulative actions of the agents in the coalition. Furthermore, we assume that the reward obtained for executing a task evolves in time: the more the execution of the task is delayed, the lesser the reward. A representative example of this class of problems is the allocation of firefighters to fires in a disaster rescue environment. We describe a practical methodology through which a problem of this class can be encoded as a Markov Decision Process. Due to the three levels of factoring in the resulting MDP (the states, actions and rewards are composites of the original features of the problem) the resulting MDP can be directly solved only for small problem instances. We describe two methods for parallel decomposition of the MDP: the MDP RSUA approach for random sampling and uniform allocation and the MDP REUSE method which reuses the lower level MDP to allocate resources to the parallel subproblems. Through an experimental study which models the problem domain using the fire simulation components of the Robocup Rescue simulator, we show that both methods significantly outperform heuristic approaches and MDP REUSE provides an overall higher performance than MDP RSUA.

Keywords

Optimization Coalition formation Agents 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Ågotnes, T., van der Hoek, W., & Wooldridge, M. (2006a). On the logic of coalitional games. In: 5th International joint conference on autonomous agents and multiagent systems (AAMAS 2006), Hakodate, Japan, May 8–12, 2006, pp. 153–160.Google Scholar
  2. 2.
    Ågotnes, T., van der Hoek, W., & Wooldridge, M. (2006b). Temporal qualitative coalitional games. In: 5th International joint conference on autonomous agents and multiagent systems (AAMAS 2006), Hakodate, Japan, May 8–12, 2006, pp. 177–184.Google Scholar
  3. 3.
    Bölöni, L., & Turgut, D. (2005). YAES—a modular simulator for mobile networks. In: Proceedings of the 8-th ACM/IEEE international symposium on modeling, analysis and simulation of wireless and mobile systems MSWIM 2005.Google Scholar
  4. 4.
    Boutilier C., Dean T., Hanks S. (1999) Decision-theoretic planning: Structural assumptions and computational leverage. Journal of Artificial Intelligence Research 11(1): 94MathSciNetGoogle Scholar
  5. 5.
    Chalkiadakis, G., & Boutilier, C. (2004). Bayesian reinforcement learning for coalition formation under uncertainty. In: Proceedings of the Third international joint conference on autonomous agents and multiagent systems, Vol. 3, pp. 1090–1097.Google Scholar
  6. 6.
    Chalkiadakis, G., & Boutilier, C. (2008). Sequential decision making in repeated coalition formation under uncertainty. In: Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems. International Foundation for Autonomous Agents and Multiagent Systems, Vol. 1, pp. 347–354.Google Scholar
  7. 7.
    Ebden, M., Briers, M., & Roberts, S. (2008). Decentralized predictive sensor allocation. In: 47th IEEE Conference on decision and control, 2008. CDC 2008, pp. 1702–1707.Google Scholar
  8. 8.
    Goranko, V. (2001). Coalition games and alternating temporal logics. In: TARK ’01: Proceedings of the 8th conference on theoretical aspects of rationality and knowledge, pp. 259–272.Google Scholar
  9. 9.
    Guestrin C., Koller D., Parr R. (2002) Multiagent planning with factored MDPs. Advances in Neural Information Processing Systems 2: 1523–1530Google Scholar
  10. 10.
    Guestrin C., Koller D., Parr R., Venkataraman S. (2003) Efficient solution algorithms for factored MDPs. Journal of Artificial Intelligence Research 19(10): 399–468zbMATHMathSciNetGoogle Scholar
  11. 11.
    van der Hoek W., Wooldridge M. (2005) On the logic of cooperation and propositional control. Artif Intell 164(1–2): 81–119zbMATHGoogle Scholar
  12. 12.
    Khan, M., Turgut, D., & Bölöni, L. (2008). Optimizing coalition formation for tasks with dynamically evolving rewards and nondeterministic action effects. In: Proceedings of international workshop on optimisation in multi-agent systems (OptMas08), in conjunction with the seventh joint conference on autonomous and multi-agent systems (AAMAS 2008), pp. 69–76.Google Scholar
  13. 13.
    Klusch M., Gerber A. (2002) Dynamic coalition formation among rational agents. Intelligent Systems 17(3): 42–47CrossRefGoogle Scholar
  14. 14.
    Meuleau, N., Hauskrecht, M., Kim, K., Peshkin, L., Kaelbling, L. P., Dean, T., et al. (1998). Solving very large weakly coupled markov decision processes. In: Proceedings of the fifteenth national conference on artificial intelligence, pp. 165–172.Google Scholar
  15. 15.
    Nüssle, T. A., Kleiner, A., & Brenner, M. (2004). Approaching urban disaster reality: The resQ firesimulator. In: Nardi D, Riedmiller M, Sammut C, Santos-Victor J (Eds.), RoboCup 2004: Robot Soccer World Cup VIII, Springer, Lecture Notes in Computer Science, Vol. 3276, pp. 474–482.Google Scholar
  16. 16.
    Osborne M., Rubinstein A. (1994) A Course in Game Theory. MIT Press.Google Scholar
  17. 17.
    Shehory O., Kraus S. (1998) Methods for task allocation via agent coalition formation. Artificial Intelligence 101(1–2): 165–200CrossRefzbMATHMathSciNetGoogle Scholar
  18. 18.
    Suijs J., Borm P., Waegenaere AD., Tijs S. (1999) Cooperative games with stochastic payoffs. European Journal of Operational Research 113(1): 193–205CrossRefzbMATHGoogle Scholar

Copyright information

© The Author(s) 2010

Authors and Affiliations

  • Majid Ali Khan
    • 1
  • Damla Turgut
    • 2
  • Ladislau Bölöni
    • 2
    Email author
  1. 1.Faculty of Computer Science and EngineeringGhulam Ishaq Khan Institute of Engineering Science and TechnologyTopi, Khyber PakhtunkhwaPakistan
  2. 2.School of Electrical Engineering and Computer ScienceUniversity of Central FloridaOrlandoUSA

Personalised recommendations