Using the Simulated Annealing Algorithm for Multiagent Decision Making

  • Jiang Dawei
  • Wang Shiyuan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4434)


Coordination, as a key issue in fully cooperative multiagent systems, raises a number of challenges. A crucial one among them is to efficiently find the optimal joint action in an exponential joint action space. Variable elimination offers a viable solution to this problem. Using their algorithm, each agent can choose an optimal individual action resulting in the optimal behavior for the whole agents. However, the worst-case time complexity of this algorithm grows exponentially with the number of agents. Moreover, variable elimination can only report an answer when the whole algorithm terminates. Therefore, it is unsuitable in real-time systems. In this paper, we propose an anytime algorithm, called the simulated annealing algorithm, as an approximation alternative to variable elimination. We empirically show that our algorithm can compute nearly optimal results with a small fraction of the time that variable elimination takes to find the solution to the same coordination problem.


Simulated Annealing Joint Action Multiagent System Simulated Annealing Algorithm Coordination Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Weiss, G. (ed.): Multiagent Systems: a Modern Approach to Distributed Artificial Intelligence. MIT Press, Cambridge, MA, USA (1999)Google Scholar
  2. 2.
    Woolridge, M., Wooldridge, M.J.: Introduction to Multiagent Systems. John Wiley & Sons, Inc., New York, NY, USA (2001)Google Scholar
  3. 3.
    Kitano, H., Asada, M., Kuniyoshi, Y., Noda, I., Osawa, E.: Robocup: The robot world cup initiative. In: AGENTS 1997. Proceedings of the first international conference on Autonomous agents, Marina del Rey, California, United States, pp. 340–347. ACM Press, New York, NY, USA (1997)CrossRefGoogle Scholar
  4. 4.
    Osborne, M.J., Rubinstein, A.: A Course in Game Theory. MIT Press, Cambridge (1999)Google Scholar
  5. 5.
    Carriero, N., Gelernter, D.: Linda in context. Communications of the ACM 32(4), 444–458 (1989)CrossRefGoogle Scholar
  6. 6.
    Gelernter, D.: Generative communication in Linda. ACM Transactions on Programming Languages and Systems 7(1), 80–112 (1985)zbMATHCrossRefGoogle Scholar
  7. 7.
    Boutilier, C.: Planning, learning and coordination in multiagent decision processes. In: TARK 1996. Proceedings of the 6th conference on Theoretical aspects of rationality and knowledge, The Netherlands, pp. 195–210. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (1996)Google Scholar
  8. 8.
    Tan, M.: Multi-agent reinforcement learning: Independent vs. cooperative learning. In: Huhns, M.N., Singh, M.P. (eds.) Readings in Agents, pp. 487–494. Morgan Kaufmann, San Francisco, CA, USA (1997)Google Scholar
  9. 9.
    Claus, C., Boutilier, C.: The dynamics of reinforcement learning in cooperative multiagent systems. In: AAAI/IAAI, pp. 746–752 (1998)Google Scholar
  10. 10.
    Guestrin, C., Koller, D., Parr, R.: Multiagent planning with factored MDPs. In: NIPS-14. 14th Neural Information Processing Systems (2001)Google Scholar
  11. 11.
    Guestrin, C., Venkataraman, S., Koller, D.: Context specific multiagent coordination and planning with factored MDPs. In: AAAI-2002. The Eighteenth National Conference on Artificial Intelligence, Edmonton, Canada, July 2002, pp. 253–259 (2002)Google Scholar
  12. 12.
    Guestrin, C., Koller, D., Parr, R., Venkataraman, S.: Efficient solution algorithms for factored MDPs. Accepted in Journal of Artificial Intelligence Research (JAIR) (2002)Google Scholar
  13. 13.
    Guestrin, C.: Planning Under Uncertainty in Complex Structured Environments. PhD thesis, Stanford University (2003)Google Scholar
  14. 14.
    Kok, J.R., Vlassis, N.: Using the max-plus algorithm for multiagent decision making in coordination graphs. In: Bredenfeld, A., Jacoff, A., Noda, I., Takahashi, Y. (eds.) RoboCup 2005. LNCS (LNAI), vol. 4020, Springer, Heidelberg (2006)CrossRefGoogle Scholar
  15. 15.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (1988)Google Scholar
  16. 16.
    Wainwright, M., Jaakkola, T., Willsky, A.: Tree consistency and bounds on the performance of the max-product algorithm and its generalizations. Statistics and Computing 14, 143–166 (2004)CrossRefMathSciNetGoogle Scholar
  17. 17.
    Dechter, R.: Bucket elimination: a unifying framework for reasoning. Artificial Intelligence 113(1-2), 41–85 (1999)zbMATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Arnborg, S., Corneil, D.G., Proskurowski, A.: Complexity of finding embeddings in a K-tree. SIAM J. Algebraic Discrete Methods 8(2), 277–284 (1987)zbMATHCrossRefMathSciNetGoogle Scholar
  19. 19.
    Bertelé, U., Brioschir, F.: Nonserial dynamic programming. Academic Press, London (1972)zbMATHGoogle Scholar
  20. 20.
    Michalewicz, Z., Fogel, D.B.: How to solve it: modern heuristics. Springer, New York, NY, USA (2000)zbMATHGoogle Scholar
  21. 21.
    Johnson, D.S., McGeoch, L.A.: The Traveling Salesman Problem: A Case Study in Local Optimization (Draft of November 20, 1995) In: Aarts, E.H.L., Lenstra, J.K. (eds.) To appear as a chapter in The book Local Search in Combinatorial Optimization, John Wiley & Sons, Inc., New York (1995)Google Scholar
  22. 22.
    Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220(4598), 671–680 (1983)CrossRefMathSciNetGoogle Scholar
  23. 23.
    Spears, W.M.: Simulated annealing for hard satisfiability problems. DIMACS Series in Discrete Mathematics and Theoretical Science 26, 533–558 (1996)Google Scholar
  24. 24.
    Dawei, J.: SEU_T 2005 team description (2D). In: Proceedings CD RoboCup 2005, Osaka, Japan, July 2005, Springer, Heidelberg (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Jiang Dawei
    • 1
  • Wang Shiyuan
    • 1
  1. 1.Department of Computer Science and Technology, Southeast UniversityP.R.China

Personalised recommendations