Distributed Learning of Cooperative Robotic Behaviors Using Particle Swarm Optimization

  • Ezequiel Di MarioEmail author
  • Iñaki Navarro
  • Alcherio Martinoli
Part of the Springer Tracts in Advanced Robotics book series (STAR, volume 109)


In this paper we study the automatic synthesis of robotic controllers for the coordinated movement of multiple mobile robots. The algorithm used to learn the controllers is a noise-resistant version of Particle Swarm Optimization, which is applied in two different settings: centralized and distributed learning. In centralized learning, every robot runs the same controller and the performance is evaluated with a global metric. In the distributed learning, robots run different controllers and the performance is evaluated independently on each robot with a local metric. Our results from learning in simulation show that it is possible to learn a cooperative task in a fully distributed way employing a local metric, and we validate the simulations with real robot experiments where the best solutions from distributed and centralized learning achieve similar performances.


Particle Swarm Optimization Real Robot Proximity Sensor Cooperative Task Overhead Camera 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This research was supported by the Swiss National Science Foundation through the National Center of Competence in Research Robotics.


  1. 1.
    Floreano, D., Mondada, F.: Evolution of homing navigation in a real mobile robot. IEEE Trans. Syst. Man Cybern. Part B Cybern. 26(3), 396–407 (1996)CrossRefGoogle Scholar
  2. 2.
    Baldassarre, G., Trianni, V., Bonani, M., Mondada, F., Dorigo, M., Nolfi, S.: Self-organized coordinated motion in groups of physically connected robots. IEEE Transactions on Systems, Man, and Cybernetics. Part B, Cybernetics: a publication of the IEEE Systems, Man, and Cybernetics Society 37(1), 224–39 (2007)Google Scholar
  3. 3.
    Gauci, M., Chen, J., Dodd, T., Groß, R.: Evolving aggregation behaviors in multi-robot systems with binary sensors. In: Distributed Autonomous Robotic Systems, Springer Tracts in Advanced Robotics, pp. 355–367 (2014)Google Scholar
  4. 4.
    Jin, Y., Branke, J.: Evolutionary optimization in uncertain environments: a survey. IEEE Trans. Evol. Comput. 9(3), 303–317 (2005)CrossRefGoogle Scholar
  5. 5.
    Pugh, J., Martinoli, A.: Distributed scalable multi-robot learning using particle swarm optimization. Swarm Intell. 3(3), 203–222 (2009)CrossRefGoogle Scholar
  6. 6.
    Balch, T., Arkin, R.: Behavior-based formation control for multi-robot teams. IEEE Trans. Robot. Autom. 14(6), 926–939 (1998)CrossRefGoogle Scholar
  7. 7.
    Fredslund, J., Mataric, M.J.: A general algorithm for robot formations using local sensing and minimal communication. IEEE Transactions on Robotics and Automation, Special Issue on Multi Robot Systems 18(5), 837–846 (2002)Google Scholar
  8. 8.
    Olfati-Saber, R.: Flocking for multi-agent dynamic systems: algorithms and theory. IEEE Trans. Autom. Control 51, 401–420 (2006)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Antonelli, G., Arrichiello, F., Chiaverini, S.: Flocking for multi-robot systems via the null-space-based behavioral control. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1409–1414 (2008)Google Scholar
  10. 10.
    Navarro, I., Matía, F.: A framework for collective movement of mobile robots based on distributed decisions. Robot. Auton. Syst. 59(10), 685–697 (2011)CrossRefGoogle Scholar
  11. 11.
    Mataric;, M.: Learning in behavior-based multi-robot systems: policies, models, and other agents. Cogn. Syst. Res. 2, 81–93 (2001)Google Scholar
  12. 12.
    Parker, L.E.: L-ALLIANCE: task-oriented multi-robot learning in behavior-based systems. In: Advanced Robotics, Special Issue on Selected Papers from IROS’96, pp. 305–322 (1997)Google Scholar
  13. 13.
    Di Mario, E., Martinoli, A.: Distributed particle swarm optimization for limited time adaptation with real robots. Robotica 32(02), 193–208 (2014)CrossRefGoogle Scholar
  14. 14.
    Reynolds, C.W.: Flocks, herds, and schools: a distributed behavioral model. Comput. Graph. 21(4), 25–34 (1987)CrossRefGoogle Scholar
  15. 15.
    Morihiro, K., Isokawa, T., Nishimura, H., Matsui, N.: Emergence of flocking behavior based on reinforcement learning. Knowl. Based Intell. Inf. Eng. Syst. 4253, 699–706 (2006)Google Scholar
  16. 16.
    Lee, S.M., Myung, H.: Particle swarm optimization-based distributed control scheme for flocking robots. Robot Intell. Technol. Appl. 208, 517–524 (2013)Google Scholar
  17. 17.
    Vatankhah, R., Etemadi, S., Honarvar, M., Alasty, A., Boroushaki, M., Vossoughi, G.: Online velocity optimization of robotic swarm flocking using particle swarm optimization (pso) method. In: International Symposium on Mechatronics and its Applications (2009)Google Scholar
  18. 18.
    Celikkanat, H.: Optimization of self-organized flocking of a robot swarm via evolutionary strategies. In: International Symposium on Computer and Information Sciences (2008)Google Scholar
  19. 19.
    Kennedy, J., Eberhart, R.: Particle swarm optimization. In: IEEE International Conference on Neural Networks, pp. 1942–1948 (1995)Google Scholar
  20. 20.
    Pugh, J., Zhang, Y., Martinoli, A.: Particle swarm optimization for unsupervised robotic learning. In: IEEE Swarm Intelligence Symposium, pp. 92–99 (2005)Google Scholar
  21. 21.
    Di Mario, E., Martinoli, A.: Distributed particle swarm optimization for limited time adaptation in autonomous robots. In: Distributed Autonomous Robotic Systems, Springer Tracts in Advanced Robotics, pp. 383–396 (2014)Google Scholar
  22. 22.
    Zhang, Y., Antonsson, E., Martinoli, A.: Evolutionary engineering design synthesis of on-board traffic monitoring sensors. Res. Eng. Des. 19(2), 113–125 (2008)CrossRefGoogle Scholar
  23. 23.
    Di Mario, E., Navarro, I., Martinoli, A.: Analysis of fitness noise in particle swarm optimization: from robotic learning to benchmark functions. In: IEEE Congress on Evolutionary Computation, pp. 2785–2792 (2014)Google Scholar
  24. 24.
    Di Mario, E., Navarro, I.n., Martinoli, A.: The role of environmental and controller complexity in the distributed optimization of multi-robot obstacle avoidance. In: IEEE International Conference on Robotics and Automation, pp. 571–577 (2014)Google Scholar
  25. 25.
    Pugh, J., Raemy, X., Favre, C., Falconi, R., Martinoli, A.: A fast on-board relative positioning module for multi-robot systems. IEEE/ASME Transactions on Mechatronics Focused Section on Mechatronics in Multi Robot Systems, pp. 151–162 (2009)Google Scholar
  26. 26.
    Michel, O.: Webots: professional mobile robot simulation. Adv. Robot. Syst. 1(1), 39–42 (2004)Google Scholar
  27. 27.
    Lochmatter, T., Roduit, P., Cianci, C., Correll, N., Jacot, J., Martinoli, A.: SwisTrack: a flexible open source tracking software for multi-agent systems. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4004–4010 (2008)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Ezequiel Di Mario
    • 1
    Email author
  • Iñaki Navarro
    • 1
  • Alcherio Martinoli
    • 1
  1. 1.Distributed Intelligent Systems and Algorithms LaboratorySchool of Architecture, Civil and Environmental Engineering, École Polytechnique Fédérale de LausanneLausanneSwitzerland

Personalised recommendations