Benchmarking Collective Perception: New Task Difficulty Metrics for Collective Decision-Making

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11804)


This paper presents nine different visual patterns for a Collective Perception scenario as new benchmark problems, which can be used for the future development of more efficient collective decision-making strategies. The experiments using isomorphism and three of the well-studied collective decision-making mechanisms are conducted to validate the performance of the new scenarios. The results on a diverse set of problems show that the real task difficulty lies not only in the quantity ratio of the features in the environment but also in their distributions and the clustering levels. Given this, two new metrics for the difficulty of the task are additionally proposed and evaluated on the provided set of benchmarks.


Collective decision making Collective perception Benchmarking Multi-agent systems Isomorphism 


  1. 1.
    Bartashevich, P., Mostaghim, S.: Positive impact of isomorphic changes in the environment on collective decision-making. In: Proceedings of the ACM Genetic and Evolutionary Computation Conference Companion (2019)Google Scholar
  2. 2.
    Behrisch, M., Bach, B., Hund, M., Delz, M., et al.: Magnostics: Image-based search of interesting matrix views for guided network exploration. IEEE Trans. Vis. Comput. Graph. 23(1), 31–40 (2017)CrossRefGoogle Scholar
  3. 3.
    Behrisch, M., Blumenschein, M., Kim, et al.: Quality metrics for information visualization. EuroVis STAR (2018)Google Scholar
  4. 4.
    Behrisch, M., Bach, B., Henry Riche, N., Schreck, T., Fekete, J.D.: Matrix reordering methods for table and network visualization. Comput. Graph. Forum 35(3), 693–716 (2016)CrossRefGoogle Scholar
  5. 5.
    Bilge, A.R., Taylor, H.A.: Framing the figure: Mental rotation revisited in light of cognitive strategies. Mem. Cognit. 45(1), 63–80 (2017)CrossRefGoogle Scholar
  6. 6.
    Ebert, J.T., Gauci, M., Nagpal, R.: Multi-feature collective decision making in robot swarms. In: Proceedings of the 17th International Conference on Autonomous Agents and Multi-Agent Systems (2018)Google Scholar
  7. 7.
    Jensen, R.: Behaviorism, latent learning, and cognitive maps: Needed revisions in introductory psychology textbooks. Behav. Anal. 29(2), 187–209 (2006)CrossRefGoogle Scholar
  8. 8.
    Morlino, G., Trianni, V., Tuci, E.: Collective perception in a swarm of autonomous robots. In: Proceedings of the International Joint Conference on Computational Intelligence, vol. 1, pp. 51–59. SciTePress (2010)Google Scholar
  9. 9.
    Passino, K.M., Seeley, T.D.: Modeling and analysis of nest-site selection by honeybee swarms: the speed and accuracy trade-off. Behav. Ecol. Sociobiol. 59, 427–442 (2005)CrossRefGoogle Scholar
  10. 10.
    Prasetyo, J., De Masi, G., Ranjan, P., Ferrante, E.: The best-of-n problem with dynamic site qualities: achieving adaptability with stubborn individuals. In: Dorigo, M., Birattari, M., Blum, C., Christensen, A.L., Reina, A., Trianni, V. (eds.) ANTS 2018. LNCS, vol. 11172, pp. 239–251. Springer, Cham (2018). Scholar
  11. 11.
    Sedig, K., Haworth, R.: Creative design of digital cognitive games: Application of cognitive toys and isomorphism. Bull. Sci. Technol. Soc. 32(5), 413–426 (2012)CrossRefGoogle Scholar
  12. 12.
    Strobel, V., Castelló Ferrer, E., Dorigo, M.: Managing byzantine robots via blockchain technology in a swarm robotics collective decision making scenario. In: Proceedings of the 17th International Conference on Autonomous Agents and Multi-Agent Systems, pp. 541–549 (2018)Google Scholar
  13. 13.
    Valentini, G.: Achieving Consensus in Robot Swarms: Design and Analysis of Strategies for the best-of-n Problem. SCI, vol. 706. Springer, Cham (2017). Scholar
  14. 14.
    Valentini, G., Brambilla, D., Hamann, H., Dorigo, M.: Collective perception of environmental features in a robot swarm. In: Dorigo, M., et al. (eds.) ANTS 2016. LNCS, vol. 9882, pp. 65–76. Springer, Cham (2016). Scholar
  15. 15.
    Valentini, G., Ferrante, E., Dorigo, M.: The best-of-n problem in robot swarms: Formalization, state of the art, and novel perspectives. Front. Robot. AI 4, 9 (2017)CrossRefGoogle Scholar
  16. 16.
    Ziemke, T., Jirenhed, D.A., Hesslow, G.: Internal simulation of perception: A minimal neuro-robotic model. Neurocomputing 68, 85–104 (2005)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Faculty of Computer ScienceOtto von Guericke UniversityMagdeburgGermany

Personalised recommendations