Using the Max-Plus Algorithm for Multiagent Decision Making in Coordination Graphs

  • Jelle R. Kok
  • Nikos Vlassis
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4020)


Coordination graphs offer a tractable framework for cooperative multiagent decision making by decomposing the global payoff function into a sum of local terms. Each agent can in principle select an optimal individual action based on a variable elimination algorithm performed on this graph. This results in optimal behavior for the group, but its worst-case time complexity is exponential in the number of agents, and it can be slow in densely connected graphs. Moreover, variable elimination is not appropriate for real-time systems as it requires that the complete algorithm terminates before a solution can be reported. In this paper, we investigate the max-plus algorithm, an instance of the belief propagation algorithm in Bayesian networks, as an approximate alternative to variable elimination. In this method the agents exchange appropriate payoff messages over the coordination graph, and based on these messages compute their individual actions. We provide empirical evidence that this method converges to the optimal solution for tree-structured graphs (as shown by theory), and that it finds near optimal solutions in graphs with cycles, while being much faster than variable elimination.


Bayesian Network Joint Action Multiagent System Action Combination Elimination Order 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Weiss, G. (ed.): Multiagent Systems: a Modern Approach to Distributed Artificial Intelligence. MIT Press, Cambridge (1999)Google Scholar
  2. 2.
    Vlassis, N.: A concise introduction to multiagent systems and distributed AI, Informatics Institute, University of Amsterdam (2003)Google Scholar
  3. 3.
    Kitano, H., Asada, M., Kuniyoshi, Y., Noda, I., Osawa, E.: RoboCup: The Robot World Cup Initiative. In: Proc. of the IJCAI 1995 Workshop on Entertainment and AI/Alife (1995)Google Scholar
  4. 4.
    Guestrin, C., Koller, D., Parr, R.: Multiagent planning with factored MDPs. In: Advances in Neural Information Processing Systems, vol. 14. MIT Press, Cambridge (2002)Google Scholar
  5. 5.
    Kok, J.R., Spaan, M.T.J., Vlassis, N.: Non-communicative multi-robot coordination in dynamic environments. Robotics and Autonomous Systems 50, 99–114 (2005)CrossRefGoogle Scholar
  6. 6.
    Vlassis, N., Elhorst, R., Kok, J.R.: Anytime algorithms for multiagent decision making using coordination graphs. In: Proc. of the International Conference on Systems, Man and Cybernetics, The Hague, The Netherlands (2004)Google Scholar
  7. 7.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, San Francisco (1988)Google Scholar
  8. 8.
    Yedidia, J., Freeman, W., Weiss, Y.: Understanding belief propagation and its generalizations. In: Exploring Artificial Intelligence in the New Millennium, pp. 239–269. Morgan Kaufmann Publishers Inc., San Francisco (2003)Google Scholar
  9. 9.
    Wainwright, M., Jaakkola, T., Willsky, A.: Tree consistency and bounds on the performance of the max-product algorithm and its generalizations. Statistics and Computing 14, 143–166 (2004)CrossRefMathSciNetGoogle Scholar
  10. 10.
    Zhang, N.L., Poole, D.: Exploiting causal independence in bayesian network inference. Journal of Artificial Intelligence Research 5, 301–328 (1996)MATHMathSciNetGoogle Scholar
  11. 11.
    Bertelé, U., Brioschir, F.: Nonserial dynamic programming. Academic Press, London (1972)MATHGoogle Scholar
  12. 12.
    Wainwright, M., Jaakkola, T., Willsky, A.: Tree consistency and bounds on the performance of the max-product algorithm and its generalizations. Technical report, P-2554, LIDS-MIT (2002)Google Scholar
  13. 13.
    Crick, C., Pfeffer, A.: Loopy belief propagation as a basis for communication in sensor networks. In: Proc. of the 19th Conference on Uncertainty in AI (2003)Google Scholar
  14. 14.
    Murphy, K., Weiss, Y., Jordan, M.: Loopy belief propagation for approximate inference: An empirical study. In: Proc. 15th Conf. on Uncertainty in Artificial Intelligence, Stockholm, Sweden (1999)Google Scholar
  15. 15.
    Loeliger, H.A.: An introduction to factor graphs. IEEE Signal Proc. Mag., 28–41 (2004)Google Scholar
  16. 16.
    Kok, J.R., Vlassis, N.: Sparse Cooperative Q-learning. In: Greiner, R., Schuurmans, D. (eds.) Proc. of the 21st Int. Conf. on Machine Learning, pp. 481–488. ACM Press, New York (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Jelle R. Kok
    • 1
  • Nikos Vlassis
    • 1
  1. 1.Informatics InstituteUniversity of AmsterdamThe Netherlands

Personalised recommendations