VWM: An Improvement to Multiagent Coordination in Highly Dynamic Environments

  • Seyed Hamid Hamraz
  • Behrouz Minaei-Bidgoli
  • William F. Punch
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4687)


This paper is aimed to describe a general improvement over the previous work on the cooperative multiagent coordination. The focus is on highly dynamic environments where the message transfer delay is not negligible. Therefore, the agents shall not count on communicating their intentions along the time they are making the decisions, because this will directly add the communication latencies to the decision making phase. The only way for the agents to be in touch is to communicate and share their beliefs, asynchronously with the decision making procedure. Consequently, they can share similar knowledge and make coordinated decisions based on it. However, in a very dynamic environment, the shared knowledge may not remain similar due to the communication limitations and latencies. This may lead to some inconsistencies in the team coordination performance. Addressing this issue, we propose to hold another abstraction of the environment, called Virtual World Model (VWM), for each agent in addition to its primary internal world state. The primary world state is updated as soon as a new piece of information is received while the information affects the VWM through a synchronization mechanism. The proposed idea has been implemented and tested for Iran University of Science and Technology (IUST) RoboCupRescue simulation team, the 3rd winner of the 2006 worldcup competitions.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Weiss, G. (ed.): Multiagent Systems: a Modern Approach to Distributed Artificial Intelligence. MIT Press, Cambridge, MA (1999)Google Scholar
  2. 2.
    Vlassis, N.: A concise introduction to multiagent systems and distributed AI. Informatics Institute, University of Amsterdam (2003)Google Scholar
  3. 3.
    Kitano, H., Asada, M., Kuniyoshi, Y., Noda, I., Osawa, E.: Robocup: The robot world cup initiative. In: AGENTS 1997. Proceedings of the first international conference on Autonomous agents, pp. 340–347. ACM Press, New York (1997)CrossRefGoogle Scholar
  4. 4.
    Osborne, M.J., Rubinstein, A.: A Course in Game Theory. MIT Press, Cambridge (1994)Google Scholar
  5. 5.
    Carriero, N., Gelernter, D.: Linda in context. Communications of the ACM 32(4), 444–458 (1989)CrossRefGoogle Scholar
  6. 6.
    Gelernter, D.: Generative communication in Linda. ACM Transactions on Programming Languages and Systems 7(1), 80–112 (1985)zbMATHCrossRefGoogle Scholar
  7. 7.
    Boutilier, C.: Planning, learning and coordination in multiagent decision processes. In: TARK 1996. Proceedings of the 6th conference on Theoretical aspects of rationality and knowledge, pp. 195–210. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (1996)Google Scholar
  8. 8.
    Tan, M.: Multi-agent reinforcement learning: Independent vs. cooperative learning. In: Huhns, M.N., Singh, M.P. (eds.) Readings in Agents, pp. 487–494. Morgan Kaufmann, San Francisco, CA (1997)Google Scholar
  9. 9.
    Claus, C., Boutilier, C.: The dynamics of reinforcement learning in cooperative multiagent systems. In: AAAI/IAAI, pp. 746–752 (1998)Google Scholar
  10. 10.
    Guestrin, C., Koller, D., Parr, R.: Multiagent planning with factored MDPs. In: Advances in Neural Information Processing Systems, vol. 14, The MIT Press, Cambridge (2002)Google Scholar
  11. 11.
    Vlassis, N., Elhorst, R., Kok, J.R.: Anytime algorithms for multiagent decision making using coordination graphs. In: Proc. of the International Conference on Systems, Man and Cybernetics, The Hague, The Netherlands (2004)Google Scholar
  12. 12.
    Kok, J.R., Vlassis, N.: Using the max-plus algorithm for multiagent decision making in coordination graphs. In: Bredenfeld, A., Jacoff, A., Noda, I., Takahashi, Y. (eds.) RoboCup 2005. LNCS (LNAI), vol. 4020, pp. 1–12. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  13. 13.
    Dawei, J., Shiyuan, W.: Using the simulated annealing algorithm for multiagent decision making. In: Proceedings of RoboCup International Symposium, Bremen, Germany. LNCS, vol. 4434, Springer, Heidelberg (to appear, 2007)Google Scholar
  14. 14.
    Kok, J.R., Spaan, M.T.J., Vlassis, N.: Non-communicative multi-robot coordination in dynamic environments. Robotics and Autonomous Systems 50, 99–114 (2005)CrossRefGoogle Scholar
  15. 15.
    Tews, A., Wyeth, G.: Thinking as one: Coordination of multiple mobile robots by shared representations. In: Intl. Conf. on Robotics and Systems (IROS) (2000)Google Scholar
  16. 16.
    Stone, P., Veloso, M.: Task decomposition, dynamic role assignment, and lowbandwidth communication for real-time strategic teamwork. Artificial Intelligence 110, 241–273 (1999)zbMATHCrossRefGoogle Scholar
  17. 17.
    Isik, M., Stulp, F., Mayer, G., Utz, H.: Coordination without negotiation in teams of heterogeneous robots. In: Proceedings of RoboCup International Symposium, Bremen, Germany (2006)Google Scholar
  18. 18.
    Morimoto, T.: How to develop a RoboCupRescue agent (for RoboCupRescue Simulation System version 0) (last visited 2007/07/02) (2002),
  19. 19.
    RoboCupRescue Simulation League, TC and OC: RoboCup 2006 Rescue Simulation League Rules (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Seyed Hamid Hamraz
    • 1
  • Behrouz Minaei-Bidgoli
    • 1
  • William F. Punch
    • 2
  1. 1.Department of Computer Engineering, Iran University of Science and Technology, TehranIran
  2. 2.Department of Computer Science & Engineering, East Lansing, Michigan State University, MIUSA

Personalised recommendations