Abstract
Designing effective behavioral controllers for mobile robots can be difficult and tedious; this process can be circumvented by using online learning techniques which allow robots to generate their own controllers online in an automated fashion. In multi-robot systems, robots operating in parallel can potentially learn at a much faster rate by sharing information amongst themselves. In this work, we use an adapted version of the Particle Swarm Optimization algorithm in order to accomplish distributed online robotic learning in groups of robots with access to only local information. The effectiveness of the learning technique on a benchmark task (generating high-performance obstacle avoidance behavior) is evaluated for robot groups of various sizes, with the maximum group size allowing each robot to individually contain and manage a single PSO particle. To increase the realism of the technique, different PSO neighborhoods based on limitations of real robotic communication are tested and compared in this scenario. We explore the effect of varying communication power for one of these communication-based PSO neighborhoods. To validate the effectiveness of these learning techniques, fully distributed online learning experiments are run using a group of 10 real robots, generating results which support the findings from our simulations.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abbeel, P., Dolgov, D., Ng, A., & Thrun, S. (2008). Apprenticeship learning for motion planning with application to parking lot navigation. In Proc. of the IEEE/RSJ int. conf. on intelligent robots and systems (pp. 1083–1090). New York: IEEE Press.
Atkeson, C. G., Moore, A. W., & Schaal, S. (1997). Locally weighted learning for control. Artificial Intelligence Review, 11, 75–113.
Balch, T. (1998). Behavioral diversity in learning robot teams. Ph.D. thesis, College of Computing, Georgia Institute of Technology, Atlanta, GA.
Billard, A., & Matarić, M. J. (2001). Learning human arm movements by imitation: evaluation of a biologically-inspired connectionist architecture. Robotics and Autonomous Systems, 37, 145–160.
Bowling, M., & Veloso, M. (2003). Simultaneous adversarial multi-robot learning. In Proc. of the int. joint conf. on artificial intelligence (pp. 699–704). Hillsdale: Erlbaum.
Di Chio, C., & Di Chio, P. (2007). EcoPS—a model of group-foraging with particle swarm systems. In LNCS: Vol. 4648. Proc. of the Euro. conf. on artificial life (pp. 685–695). Berlin: Springer.
Dorigo, M., Trianni, V., Şahin, E., Groß, R., Labella, T. H., Baldassarre, G., Nolfi, S., Deneubourg, J.-L., Mondada, F., Floreano, D., & Gambardella, L. M. (2004). Evolving self-organizing behaviors for a swarm-bot. Autonomous Robots, 17, 223–245.
Eberhart, R., & Kennedy, J. (1995). A new optimizer using particle swarm theory. In Proc. of the int. symp. on micro machine and human science (pp. 39–43). New York: IEEE Press.
Floreano, D., & Mondada, F. (1996). Evolution of homing navigation in a real mobile robot. IEEE Transactions on Systems, Man and Cybernetics, Part B, 26, 396–407.
Franklin, J. A., Mitchell, T. M., & Thrun, S. (1996). Recent advances in robot learning. Boston: Kluwer Academic.
Jatmiko, W., Sekiyama, K., & Fukuda, T. (2006). A PSO-based mobile sensor network for odor source localization in dynamic environment: theory, simulation and measurement. In Proc. of the IEEE congress on evolutionary computation (pp. 1036–1043). Los Alamitos: IEEE Computer Society.
Kaelbling, L. P. (1993). Learning in embedded systems. Cambridge: MIT Press.
Kelly, I. D., & Keating, D. A. (1998). Faster learning of control parameters through sharing experiences of autonomous mobile robots. International Journal of System Science, 29, 783–793.
Kennedy, J., & Eberhart, R. (1995). Particle swarm optimization. In Proc. of the IEEE int. conf. on neural networks (pp. 1942–1948). New York: IEEE Press.
Li, L., Martinoli, A., & Abu-Mostafa, Y. (2004). Learning and measuring specialization in collaborative swarm systems. Adaptive Behavior, Special Issue on Mathematics and Algorithms of Social Interactions, 12, 199–212.
Mahadevan, S., & Connell, J. (1991). Automatic programming of behavior-based robots using reinforcement learning. In Proc. of the natl. conf. on artificial intelligence (pp. 768–773). San Francisco: Morgan Kaufmann.
Matarić, M. J. (1994). Learning to behave socially. In Proc. of the int. conf. on the simulation of adaptive behavior (pp. 453–462). Cambridge: MIT Press.
Matarić, M. J. (2001). Learning in behavior-based multi-robot systems: Policies, models, and other agents. Cognitive Systems Research, Special Issue on Multi-Disciplinary Studies of Multi-Agent Learning, 2, 81–93.
Michel, O. (2004). Webots: professional mobile robot simulation. International Journal of Advanced Robotic Systems, 1, 39–42.
Michels, J., Saxena, A., & Ng, A. Y. (2005). High speed obstacle avoidance using monocular vision and reinforcement learning. In Proc. of the int. conf. on machine learning (pp. 593–600). New York: ACM.
Miglino, O., Lund, H. H., & Nolfi, S. (1995). Evolving mobile robots in simulated and real environments. Artificial Life, 2, 417–434.
Murciano, A., Millán, J. R., & Zamora, J. (1997). Specialization in multi-agent systems through learning. Behavioral Cybernetics, 76, 375–382.
Nehmzow, U. (2002). Learning in multi-robot scenarios through physically embedded genetic algorithms. In Proc. of the int. conf. on the simulation of adaptive behavior (pp. 391–392). Cambridge: MIT Press.
Nordin, P., & Bahnzaf, W. (1997). An on-line method to evolve behavior and to control a miniature robot in real time with genetic programming. Adaptive Behavior, 5, 107–140.
Panait, L., & Luke, S. (2005). Cooperative multi-agent learning: the state of the art. Autonomous Agents and Multi-Agent Systems, 11, 387–434.
Parker, L. E. (1997). L-ALLIANCE: task-oriented multi-robot learning in behavior-based systems. Advanced Robotics, 11, 305–322.
Pugh, J., & Martinoli, A. (2006). Multi-robot learning with particle swarm optimization. In Proc. of the int. conf. on autonomous agents and multiagent systems (pp. 441–448). New York: ACM.
Pugh, J., & Martinoli, A. (2008). Distributed adaptation in multi-robot search using particle swarm optimization. In LNCS: Vol. 5040. Proc. of the int. conf. on the simulation of adaptive behavior (pp. 393–402). Berlin: Springer.
Pugh, J., & Martinoli, A. (2009). An exploration of online parallel learning in heterogeneous multi-robot swarms. Design and control of intelligent robotic systems, SCI 177 (pp. 145–165). Berlin: Springer. Chap. 7.
Pugh, J., Zhang, Y., & Martinoli, A. (2005). Particle swarm optimization for unsupervised robotic learning. In Proc. of the IEEE swarm intelligence symposium (pp. 92–99). New York: IEEE Press.
Pugh, J., Raemy, X., Favre, C., Falconi, R., & Martinoli, A. (2009). A fast on-board relative positioning module for multi-robot systems. IEEE/ASME Transactions on Mechatronics, Focused Section on Mechatronics in Multi Robot Systems (to appear).
Smart, W. D., & Kaelbling, L. P. (2002). Effective reinforcement learning for mobile robots. In Proc. of the IEEE int. conf. on robotics and automation (pp. 3404–3410). New York: IEEE Press.
Stone, P. (1998). Layered learning in multi-agent systems. Ph.D. thesis, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA.
Stone, P., & Veloso, M. (2000). Multiagent systems: a survey from a machine learning perspective. Autonomous Robots, 8, 345–383.
Watson, R. A., Ficici, S. G., & Pollack, J. B. (2002). Embodied evolution: distributing an evolutionary algorithm in a population of robots. Robotics and Autonomous Systems, 39, 1–18.
Ye, C., Yung, N. H. C., & Wang, D. (2003). A fuzzy controller with supervised learning assisted reinforcement learning algorithm for obstacle avoidance. IEEE Transactions on Systems, Man, and Cybernetics, Part B, 33, 17–27.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Pugh, J., Martinoli, A. Distributed scalable multi-robot learning using particle swarm optimization. Swarm Intell 3, 203–222 (2009). https://doi.org/10.1007/s11721-009-0030-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11721-009-0030-z