Multi-armed Bandit Formulation of the Task Partitioning Problem in Swarm Robotics

  • Giovanni Pini
  • Arne Brutschy
  • Gianpiero Francesca
  • Marco Dorigo
  • Mauro Birattari
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7461)


Task partitioning is a way of organizing work consisting in the decomposition of a task into smaller sub-tasks that can be tackled separately. Task partitioning can be beneficial in terms of reduction of physical interference, increase of efficiency, higher parallelism, and exploitation of specialization. However, task partitioning also entails costs in terms of coordination efforts and overheads that can reduce its benefits. It is therefore important to decide when to make use of task partitioning. In this paper we show that such a decision can be formulated as a multi-armed bandit problem. This is advantageous since the theoretical properties of the multi-armed bandit problem are well understood and several algorithms have been proposed for tackling it. We carry out our study in simulation, using a swarm robotics foraging scenario as a testbed. We test an ad-hoc algorithm and two algorithms proposed in the literature for multi-armed bandit problems. The results confirm that the problem of selecting whether to partition a task can be formulated as a multi-armed bandit problem and tackled with existing algorithms.


Social Insect Online Supplementary Material Object Retrieval Bandit Problem Swarm Robotic 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agrawal, R.: Sample mean based index policies with O(log n) regret for the multi-armed bandit problem. Advances in Applied Probability 27, 1054–1078 (1995)MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine Learning 47(2), 235–256 (2002)zbMATHCrossRefGoogle Scholar
  3. 3.
    Berry, D.A., Fristedt, B.: Bandit problems: Sequential allocation of experiments. Chapman & Hall, London (1985)zbMATHCrossRefGoogle Scholar
  4. 4.
    Brutschy, A., Pini, G., Baiboun, N., Decugnière, A., Birattari, M.: The IRIDIA TAM: A device for task abstraction for the e-puck robot. Tech. Rep. TR/IRIDIA/2010-015, IRIDIA, Université Libre de Bruxelles, Brussels, Belgium (2010)Google Scholar
  5. 5.
    Dorigo, M., Şahin, E.: Guest editorial. Special Issue: Swarm robotics. Autonomous Robots 17(2-3), 111–113 (2004)CrossRefGoogle Scholar
  6. 6.
    Dorigo, M., Floreano, D., Gambardella, L.M., Mondada, F., Nolfi, S., Baaboura, T., Birattari, M., Bonani, M., Brambilla, M., Brutschy, A., Burnier, D., Campo, A., Christensen, A.L., Decugnière, A., Caro, G.D., Ducatelle, F., Ferrante, E., Förster, A., Gonzales, J.M., Guzzi, J., Longchamp, V., Magnenat, S., Mathews, N., de Oca, M.M., O’Grady, R., Pinciroli, C., Pini, G., Rétornaz, P., Roberts, J., Sperati, V., Stirling, T., Stranieri, A., Stützle, T., Trianni, V., Tuci, E., Turgut, A.E., Vaussard, F.: Swarmanoid: a novel concept for the study of heterogeneous robotic swarms. IEEE Robotics & Automation Magazine (in press, 2012)Google Scholar
  7. 7.
    Fontan, M.S., Matarić, M.J.: A study of territoriality: The role of critical mass in adaptive task division. In: Maes, P., Matarić, M.J., Meyer, J.A., Pollack, J., Wilson, S. (eds.) From Animals to Animats 4: Proceedings of the Fourth International Conference of Simulation of Adaptive Behavior, pp. 553–561. MIT Press, Cambridge (1996)Google Scholar
  8. 8.
    Frison, M., Tran, N.-L., Baiboun, N., Brutschy, A., Pini, G., Roli, A., Dorigo, M., Birattari, M.: Self-organized Task Partitioning in a Swarm of Robots. In: Dorigo, M., Birattari, M., Di Caro, G.A., Doursat, R., Engelbrecht, A.P., Floreano, D., Gambardella, L.M., Groß, R., Şahin, E., Sayama, H., Stützle, T. (eds.) ANTS 2010. LNCS, vol. 6234, pp. 287–298. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  9. 9.
    Jeanne, R.L.: The evolution of the organization of work in social insects. Monitore Zoologico Italiano 20, 119–133 (1986)Google Scholar
  10. 10.
    Lein, A., Vaughan, R.: Adaptive multi-robot bucket brigade foraging. In: Bullock, S., Noble, J., Watson, R., Bedau, M.A. (eds.) Artificial Life XI: Proceedings of the Eleventh International Conference on the Simulation and Synthesis of Living Systems, pp. 337–342. MIT Press, Cambridge (2008)Google Scholar
  11. 11.
    Lein, A., Vaughan, R.T.: Adapting to non-uniform resource distributions in robotic swarm foraging through work-site relocation. In: 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2009), pp. 601–606. IEEE Press, Piscataway (2009)CrossRefGoogle Scholar
  12. 12.
    Østergaard, E.H., Sukhatme, G.S., Matarić, M.J.: Emergent bucket brigading: A simple mechanisms for improving performance in multi-robot constrained-space foraging tasks. In: AGENTS 2001: Proceedings of the Fifth International Conference on Autonomous Agents, pp. 29–30. ACM Press, New York (2001)CrossRefGoogle Scholar
  13. 13.
    Pinciroli, C., Trianni, V., O’Grady, R., Pini, G., Brutschy, A., Brambilla, M., Mathews, N., Ferrante, E., Di Caro, G.A., Ducatelle, F., Stirling, T., Gutiérrez, A., Gambardella, L.M., Dorigo, M.: ARGoS: A modular, multi-engine simulator for heterogeneous swarm robotics. In: Proceedings of the 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2011), pp. 5027–5034. IEEE Computer Society Press, Los Alamitos (2011)CrossRefGoogle Scholar
  14. 14.
    Pini, G., Brutschy, A., Birattari, M., Dorigo, M.: Task Partitioning in Swarms of Robots: Reducing Performance Losses Due to Interference at Shared Resources. In: Cetto, J.A., Filipe, J., Ferrier, J.-L. (eds.) Informatics in Control Automation and Robotics. LNEE, vol. 85, pp. 217–228. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  15. 15.
    Pini, G., Brutschy, A., Francesca, G., Dorigo, M., Birattari, M.: Multi-armed bandit formulation of the task partitioning problem in swarm robotics – Online supplementary material (2012),
  16. 16.
    Pini, G., Brutschy, A., Frison, M., Roli, A., Birattari, M., Dorigo, M.: Task partitioning in swarms of robots: An adaptive method for strategy selection. Swarm Intelligence 5(3–4), 283–304 (2011)CrossRefGoogle Scholar
  17. 17.
    Ratnieks, F.L.W., Anderson, C.: Task partitioning in insect societies. Insectes Sociaux 46(2), 95–108 (1999)CrossRefGoogle Scholar
  18. 18.
    Shell, D.J., Matarić, M.J.: On foraging strategies for large-scale multi-robot systems. In: Proceedings of the 19th IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2717–2723. IEEE Press, Pitscataway (2006)Google Scholar
  19. 19.
    Sutton, R., Barto, A.: Reinforcement learning, an introduction. MIT Press, Cambridge (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Giovanni Pini
    • 1
  • Arne Brutschy
    • 1
  • Gianpiero Francesca
    • 1
  • Marco Dorigo
    • 1
  • Mauro Birattari
    • 1
  1. 1.IRIDIAUniversité Libre de BruxellesBrusselsBelgium

Personalised recommendations