Applying Reinforcement Learning to Multi-robot Team Coordination

  • Yolanda Sanz
  • Javier de Lope
  • José Antonio Martín H.
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5271)


Multi-robot systems are one of the most challenging problems in autonomous robots. Teams of homogeneous or heterogeneous robots must be able to solve complex tasks. Sometimes the tasks have a cooperative basis in which the global objective is shared by all the robots. In other situations, the robots can be different and even contradictory goals, defining a kind of competitive problems. The multi-robot systems domain is a perfect example in which the uncertainty and vagueness in sensor readings and robot odometry must be handled by using techniques which can deal with this kind of imprecise data. In this paper we introduce the use of Reinforcement Learning techniques for solving cooperative problems in teams of homogeneous robots. As an example, the problem of maintaining a mobile robots formation is studied.


Multi-robot Systems Reinforcement Learning Cooperative Behaviors Coordination 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Mataric, M.: The Robotics Primer. MIT Press, Cambridge (2007)Google Scholar
  2. 2.
    Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach, 2nd edn. Prentice Hall, New Jersey (2002)Google Scholar
  3. 3.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
  4. 4.
    Watkins, C.J., Dayan, P.: Technical Note Q-learning. Machine Learning 8, 279 (1992)zbMATHGoogle Scholar
  5. 5.
    Liu, J., Wu, J.: Multi-agent Robotic Systems. CRC Press Int., Boca Raton (2001)Google Scholar
  6. 6.
    Resenfeld, A., Kamika, G.A., Kraus, S., Shehory, O.: A study of mechanisms for improving robotic group performance. Artificial Intelligence 172(6–7), 633–655 (2008)CrossRefGoogle Scholar
  7. 7.
    Fox, D., Burgard, W., Kruppa, H., Thrun, S.: A probabilistic approach to collaborative multi-robot localization. Autonomous Robots 8(3), 325–344 (2000)CrossRefGoogle Scholar
  8. 8.
    Farinelli, A., Iocchi, L., Nardi, D.: Multi-robot systems: A classification focused on coordination. IEEE Trans. on Systems, Man, and Cybernetics, Part B 34(5), 2015–2028 (2004)CrossRefGoogle Scholar
  9. 9.
    Yang, E., Gu, D.: Multiagent Reinforcement Learning for Multi-Robot Systems: A Survey. Technical Report CSM-404, University of Essex, Department of Computer Science (2004)Google Scholar
  10. 10.
    Mataric, M.: Reinforcement Learning in the Multi-Robot Domain. Autonomous Robots 4(1), 73–83 (1997)CrossRefGoogle Scholar
  11. 11.
    Zheng, Z., et al.: Multiagent Reinforcement Learning for a Planetary Exploration Multirobot System. In: Shi, Z.-Z., Sadananda, R. (eds.) PRIMA 2006. LNCS (LNAI), vol. 4088, pp. 339–350. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  12. 12.
    Wang, Y., De Silva, C.W.: Multi-robot Box-pushing: Single-Agent Q-Learning vs. Team Q-Learning. In: IEEE/RSJ Int. Conf. Intelligent Robots and Systems, pp. 3694–3699 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Yolanda Sanz
    • 1
  • Javier de Lope
    • 1
  • José Antonio Martín H.
    • 2
  1. 1.Perception for Computers and RobotsUniversidad Politécnica de Madrid 
  2. 2.Dep. Sistemas Informáticos y ComputaciónUniversidad Complutense de Madrid 

Personalised recommendations