Using the Max-Plus Algorithm for Multiagent Decision Making in Coordination Graphs
Coordination graphs offer a tractable framework for cooperative multiagent decision making by decomposing the global payoff function into a sum of local terms. Each agent can in principle select an optimal individual action based on a variable elimination algorithm performed on this graph. This results in optimal behavior for the group, but its worst-case time complexity is exponential in the number of agents, and it can be slow in densely connected graphs. Moreover, variable elimination is not appropriate for real-time systems as it requires that the complete algorithm terminates before a solution can be reported. In this paper, we investigate the max-plus algorithm, an instance of the belief propagation algorithm in Bayesian networks, as an approximate alternative to variable elimination. In this method the agents exchange appropriate payoff messages over the coordination graph, and based on these messages compute their individual actions. We provide empirical evidence that this method converges to the optimal solution for tree-structured graphs (as shown by theory), and that it finds near optimal solutions in graphs with cycles, while being much faster than variable elimination.
KeywordsBayesian Network Joint Action Multiagent System Action Combination Elimination Order
Unable to display preview. Download preview PDF.
- 1.Weiss, G. (ed.): Multiagent Systems: a Modern Approach to Distributed Artificial Intelligence. MIT Press, Cambridge (1999)Google Scholar
- 2.Vlassis, N.: A concise introduction to multiagent systems and distributed AI, Informatics Institute, University of Amsterdam (2003)Google Scholar
- 3.Kitano, H., Asada, M., Kuniyoshi, Y., Noda, I., Osawa, E.: RoboCup: The Robot World Cup Initiative. In: Proc. of the IJCAI 1995 Workshop on Entertainment and AI/Alife (1995)Google Scholar
- 4.Guestrin, C., Koller, D., Parr, R.: Multiagent planning with factored MDPs. In: Advances in Neural Information Processing Systems, vol. 14. MIT Press, Cambridge (2002)Google Scholar
- 6.Vlassis, N., Elhorst, R., Kok, J.R.: Anytime algorithms for multiagent decision making using coordination graphs. In: Proc. of the International Conference on Systems, Man and Cybernetics, The Hague, The Netherlands (2004)Google Scholar
- 7.Pearl, J.: Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, San Francisco (1988)Google Scholar
- 8.Yedidia, J., Freeman, W., Weiss, Y.: Understanding belief propagation and its generalizations. In: Exploring Artificial Intelligence in the New Millennium, pp. 239–269. Morgan Kaufmann Publishers Inc., San Francisco (2003)Google Scholar
- 12.Wainwright, M., Jaakkola, T., Willsky, A.: Tree consistency and bounds on the performance of the max-product algorithm and its generalizations. Technical report, P-2554, LIDS-MIT (2002)Google Scholar
- 13.Crick, C., Pfeffer, A.: Loopy belief propagation as a basis for communication in sensor networks. In: Proc. of the 19th Conference on Uncertainty in AI (2003)Google Scholar
- 14.Murphy, K., Weiss, Y., Jordan, M.: Loopy belief propagation for approximate inference: An empirical study. In: Proc. 15th Conf. on Uncertainty in Artificial Intelligence, Stockholm, Sweden (1999)Google Scholar
- 15.Loeliger, H.A.: An introduction to factor graphs. IEEE Signal Proc. Mag., 28–41 (2004)Google Scholar
- 16.Kok, J.R., Vlassis, N.: Sparse Cooperative Q-learning. In: Greiner, R., Schuurmans, D. (eds.) Proc. of the 21st Int. Conf. on Machine Learning, pp. 481–488. ACM Press, New York (2004)Google Scholar