Advertisement

Artificial Life and Robotics

, Volume 22, Issue 3, pp 357–373 | Cite as

Quantifying the impact of communication on performance in multi-agent teams

  • Mathew ZuparicEmail author
  • Victor Jauregui
  • Mikhail Prokopenko
  • Yi Yue
Original Article
  • 272 Downloads

Abstract

In this work, we relate the extent and quality of inter-agent communication and the overall performance in teams of multiple agents. Specifically, we examine the RoboCup Soccer Simulation 2D League, and carry out multiple simulation experiments against two evenly matched teams. For each simulated run (a 2D soccer simulation game), we generate the communication efficiencies (i.e., communications sent/communications received) for each agent pair. Applying linear regression and principal component analyses, we then correlate these efficiencies with measures of performance (i.e., goals scored and goals conceded), enabling the construction of inter-agent communication networks. Analysis of these networks highlights the microscopic player-to-player and macroscopic role-to-role communications correlated with performance. The approach determines the salient pathways within inter-agent communications which globally affect the coordination and the overall performance in multi-agent teams.

Keywords

RoboCup Multi-agent Communication Regression Network 

Notes

Acknowledgements

The authors wish to thank Alexander Kalloniatis for stimulating discussions. This work was funded by Defence Science and Technology Group’s Trusted Autonomous Systems Strategic Research Initiative (Project Tyche).

References

  1. 1.
    Abreu PH, Moura J, Silva DC, Reis LP, Garganta J (2012) Performance analysis in soccer: a Cartesian coordinates based approach using RoboCup data. Soft Comput 16(1):47–61CrossRefGoogle Scholar
  2. 2.
    Aho K, Derryberry D, Peterson T (2014) Model selection for ecologists: the worldviews of aic and bic. Ecology 95(3):631–636CrossRefGoogle Scholar
  3. 3.
    Ajitha S, Suresh Kumar TV, Rajanikanth K (2013) A quantitative framework for early prediction of cooperation in multi-agent systems. ICTACT J Soft Comput 3(4):587–595CrossRefGoogle Scholar
  4. 4.
    Akaike H (1974) A new look at the statistical model identification. IEEE Trans Autom Control 19(6):716–723MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Akiyama H (2010) Agent2D Base Code. www.rctools.sourceforge.jp
  6. 6.
    Akiyama H, Nakashima T (2013) HELIOS2012: RoboCup 2012 Soccer Simulation 2D League Champion. In: Chen X, Stone P, Sucar L, van der Zant T (eds) RoboCup 2012: Robot Soccer World Cup XVI. Springer, Berlin, pp 13–19Google Scholar
  7. 7.
    Bai A, Wu F, Chen X (2015) Online planning for large markov decision processes with hierarchical decomposition. ACM Trans Intell Syst Technol 6(4):45CrossRefGoogle Scholar
  8. 8.
    Bai A, Chen X, MacAlpine P, Urieli D, Barrett, Stone P (2012) Wrighteagle and ut austin villa: RoboCup 2011 simulation league champions. In: Röfer T, Mayer N, Savage J, Saranlı U (eds) RoboCup 2011: Robot Soccer World Cup XV. Lecture Notes in Artificial Intelligence. Springer, Berlin, pp 1–12Google Scholar
  9. 9.
    Becker R, Carlin A, Lesser V, Zilberstein S (2009) Analyzing myopic approaches for multi-agent communication. Comput Intell 25(1):31–50MathSciNetCrossRefGoogle Scholar
  10. 10.
    Bernstein D, Zilberstein S, Immerman N (2002) The complexity of decentralized control of markov decision processes. Math Oper Res 27(4):819–840MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Boccaletti S, Latora V, Moreno Y, Chavez M, Hwang DU (2006) Complex networks: structure and dynamics. Phys Rep 424(4):175–308MathSciNetCrossRefGoogle Scholar
  12. 12.
    Budden D, Prokopenko M (2013) Improved particle filtering for pseudo-uniform belief distributions in robot localisation. In: Behnke S, Veloso MM, Visser A, Xiong R (eds) RoboCup 2013: Robot Soccer World Cup XVII, vol 8371. Lecture notes in artificial intelligence. Springer, BerlinGoogle Scholar
  13. 13.
    Budden D, Wang P, Obst O, Prokopenko M (2014) Simulation leagues: analysis of competition formats. In: Bianchi RAC, Akin HL, Ramamoorthy S, Sugiura K (eds) RoboCup 2014: Robot Soccer World Cup XVIII, vol 8992. Lecture notes in computer science. Springer, Berlin, pp 183–194Google Scholar
  14. 14.
    Budden D, Wang P, Obst O, Prokopenko M (2015) RoboCup simulation leagues: enabling replicable and robust investigation of complex robotic systems. IEEE Robot Autom Mag 22(3):140–146CrossRefGoogle Scholar
  15. 15.
    Butler M, Prokopenko M, Howard T (2001) Flexible synchronisation within RoboCup environment: a comparative analysis. In: Stone P, Balch T, Kraetzschmar G (eds) RoboCup 2000: Robot Soccer World Cup IV, vol 2019. Lecture notes in artificial intelligence. Springer, Berlin, pp 119–128Google Scholar
  16. 16.
    Candea C, Hu H, Iocchi L, Nardi D, Piaggio M (2001) Coordination in multi-agent robocup teams. Robot Auton Syst 36(2):67–86CrossRefzbMATHGoogle Scholar
  17. 17.
    Chen M, Foroughi E, Heintz F, Huang Z, Kapetanakis S, Kostiadis K, Kummeneje J, Noda I, Obst O, Riley P, Steffens T, Wang Y, Yin X (2001) RoboCup Soccer Server. http://wwfc.cs.virginia.edu/documentation/manual.pdf
  18. 18.
    Chickering DM (2002) Learning equivalence classes of Bayesian-network structures. J Mach Learn Res 2(Feb):445–498MathSciNetzbMATHGoogle Scholar
  19. 19.
    Chou WYJ, Marsh L, Gossink D (2009) Multi-agent coordination and optimisation in the RoboCupRescue project. In: 18th World IMACS/MODSIM Congress. Cairns, Australia (July) (Citeseer)Google Scholar
  20. 20.
    Cioppa TM, Lucas TW (2007) Efficient nearly orthogonal and space-filling latin hypercubes. Technometrics 49(1):45–55MathSciNetCrossRefGoogle Scholar
  21. 21.
    Cliff O, Prokopenko M, Fitch R (2016) An information criterion for inferring coupling of distributed dynamical systems. Front Robot AI 3:71CrossRefGoogle Scholar
  22. 22.
    Cliff OM, Lizier JT, Wang P, Wang XR, Obst O, Prokopenko M (2017) Quantifying long-range interactions and coherent structure in multi-agent dynamics. Artif Life 23(1):34–57CrossRefGoogle Scholar
  23. 23.
    Cliff OM, Lizier JT, Wang XR, Wang P, Obst O, Prokopenko M (2014) Towards quantifying interaction networks in a football match. In: Behnke S, Veloso M, Visser A, Xiong R (eds) RoboCup 2013: Robot World Cup XVII, vol 8371. Lecture notes in computer science. Springer, Berlin, pp 1–12Google Scholar
  24. 24.
    Fewell JH, Armbruster D, Ingraham J, Petersen A, Waters JS (2012) Basketball teams as strategic networks. PLoS One 7(11):e47445CrossRefGoogle Scholar
  25. 25.
    Frias-Martinez V, Marcinkiewicz M, Parsons S, Sklar E (2004) Using multiagent coordination techniques in the robocup four-legged league. In: Proceedings of the AAAI spring symposium on bridging the multi-agent and multi-robotic research gapGoogle Scholar
  26. 26.
    Gan SK, Fitch R, Sukkarieh S (2014) Online decentralized information gathering with spatial-temporal constraints. Auton Robot 37(1):1–25CrossRefGoogle Scholar
  27. 27.
    Ghahramani Z (1998) Learning dynamic bayesian networks. In: Giles C, Gori M (eds) Adaptive processing of sequences and data structures. Springer, Berlin, pp 168–197Google Scholar
  28. 28.
    Gutiérrez C, García-Magariño I (2009) A metrics suite for the communication of multi-agent systems. J Phys Agents 3(2):7–14Google Scholar
  29. 29.
    Gutiérrez C, García-Magariño I, Fuentes-Fernández R (2011) Detection of undesirable communication patterns in multi-agent systems. Eng Appl Artif Intell 24(1):103–116CrossRefGoogle Scholar
  30. 30.
    Haker M, Meyer A, Polani D, Martinetz T (2002) A method for incorporation of new evidence to improve world state estimation. In: Birk A, Coradeschi S, Tadokoro S (eds) RoboCup 2001: Robot Soccer World Cup V, vol 2377. Lecture notes in computer science. Springer, Berlin, pp 362–367Google Scholar
  31. 31.
    Hausknecht M, Mupparaju P, Subramanian S, Kalyanakrishnan S, Stone P (2016) Half field offense: an environment for multiagent learning and ad hoc teamwork. In: In AAMAS adaptive learning agents (ALA) workshopGoogle Scholar
  32. 32.
    Howard RA (1966) Information value theory. Syst Sci Cybern IEEE Trans 2(1):22–26CrossRefGoogle Scholar
  33. 33.
    Ilachinski A (2004) Articifical war: multiagent-based simulation of combat. World Scientific, SingaporeCrossRefGoogle Scholar
  34. 34.
    Jennings NR, Sycara K, Wooldridge M (1998) A roadmap of agent research and development. Auton Agents Multi-agent Syst 1(1):7–38CrossRefGoogle Scholar
  35. 35.
    Jinyi Y, Ni L, Fan Y, Yunpeng C, Zengqi S (2004) Technical solutions of TsinghuAeolus for robotic soccer. In: Polani D, Browning B, Bonarini A, Yoshida K (eds) RoboCup 2003: Robot Soccer World Cup VII. Springer, Berlin, pp 205–213CrossRefGoogle Scholar
  36. 36.
    Jolliffe IT (2002) Principal component analysis, 2nd edn. Springer, New YorkzbMATHGoogle Scholar
  37. 37.
    Kantz H, Schreiber T (2004) Nonlinear time series analysis, vol 7. Cambridge University Press, CambridgezbMATHGoogle Scholar
  38. 38.
    Kitano H, Asada M (1998) The RoboCup humanoid challenge as the millennium challenge for advanced robotics. Adv Robot 13(8):723–736CrossRefGoogle Scholar
  39. 39.
    Kitano H, Tambe M, Stone P, Veloso MM, Coradeschi S, Osawa E, Matsubara H, Noda I, Asada M (1998) The RoboCup synthetic agent challenge 97. In: Kitano M (ed) RoboCup-97: Robot Soccer World Cup I, London, UK. Springer, Berlin, pp 62–73Google Scholar
  40. 40.
    Kok JR, Spaan MTJ, Vlassis NA (2005) Non-communicative multi-robot coordination in dynamic environments. Robot Auton Syst 50(2–3):99–114CrossRefGoogle Scholar
  41. 41.
    Kok JR, Vlassis N, Groen FCA, UvA Trilearn (2003) team description. In: Polani D, Browning B, Bonarini A, Yoshida K (eds) Proceedings CD RoboCup 2003, Padua, Italy, July 2003. Springer, BerlinGoogle Scholar
  42. 42.
    Lizier J, Heinzle J, Horstmann A, Haynes J, Prokopenko M (2011) Multivariate information-theoretic measures reveal directed information structure and task relevant changes in fMRI connectivity. J Comput Neurosci 30(1):85–107MathSciNetCrossRefGoogle Scholar
  43. 43.
    Lizier J, Rubinov M (2012) Multivariate construction of effective computational networks from observational data. Max Planck Institute for Mathematics in the Sciences (preprint)Google Scholar
  44. 44.
    MacAlpine P, Barrett S, Urieli D, Vu V, Stone P (2012) Design and optimization of an omnidirectional humanoid walk: a winning approach at the RoboCup 2011 3D simulation competition. In: Proceedings of the 26th conference on artificial intelligence, AAAIGoogle Scholar
  45. 45.
    Moore D, McGabe G (1993) Introduction to the practice of statistics, 2nd edn. W.H Freedman and Company, New YorkGoogle Scholar
  46. 46.
    Mota L, Reis L, Lau N (2011) Multi-robot coordination using setplays in the middle-size and simulation leagues. Mechatronics 21(2):434–444CrossRefGoogle Scholar
  47. 47.
    Nair R, Tambe M, Marsella S (2003) Team formation for reformation in multiagent domains like RoboCupRescue. In: Kaminka G, Lima P, Rojas R (eds) RoboCup 2002: Robot Soccer World Cup VI. Lecture notes in computer science, vol. 1395. Springer Verlag, Berlin, pp 150–161Google Scholar
  48. 48.
    Niehaus C, Röfer T, Laue T (2007) Gait optimization on a humanoid robot using particle swarm optimization. In: Proceedings of the second workshop on humanoid soccer robots. IEEEGoogle Scholar
  49. 49.
    Noda I, Stone P (2003) The RoboCup soccer server and CMUnited clients: implemented infrastructure for MAS research. Auton Agents Multi-agent Syst 7(1–2):101–120CrossRefGoogle Scholar
  50. 50.
    Panait L, Luke S (2005) Cooperative multi-agent learning: the state of the art. Auton Agents Multi-agent Syst 11(3):387–434CrossRefGoogle Scholar
  51. 51.
    Park HJ, Friston K (2013) Structural and functional brain networks: from connections to cognition. Science 342(6158):1238411CrossRefGoogle Scholar
  52. 52.
    Peña JL, Touchette H (2012) A network theory analysis of football strategies. In: Proc Euromech Physics of Sports Conference, 2012 (arXiv preprint). arXiv:1206.6904
  53. 53.
    Prokopenko M, Wang P (2016) Disruptive innovations in RoboCup 2D soccer simulation league: from Cyberoos’98 to Gliders2016. In: Behnke S, Lee D, Sariel S, Sheh R (eds) RoboCup 2016: Robot Soccer World Cup XX. Lecture notes in artificial intelligence. Springer, BerlinGoogle Scholar
  54. 54.
    Prokopenko M, Wang P, Obst O, Jauregui V (2016) Gliders 2016: Integrating multi-agent approaches to tactical diversity. In: RoboCup 2016 symposium and competitions: team description papers. Germany, July, Leipzig, p 2016Google Scholar
  55. 55.
    Prokopenko M, Ay N, Obst O, Polani D (2010) Phase transitions in least-effort communications. J Stat Mech Theory Exp 2010(11):P11025CrossRefGoogle Scholar
  56. 56.
    Prokopenko M, Wang P (2003) Relating the entropy of joint beliefs to multi-agent coordination. In: Kaminka G, Lima P, Rojas R (eds) RoboCup 2002: Robot Soccer World Cup VI, vol 2752. Lecture notes in computer science. Springer, Berlin, pp 367–374Google Scholar
  57. 57.
    Prokopenko M, Wang P (2004) Evaluating team performance at the edge of chaos. In: Polani D, Browning B, Bonarini A, Yoshida K (eds) RoboCup 2003: Robot Soccer World Cup VII, vol 3020. Lecture notes in computer science. Springer, Berlin, pp 89–101Google Scholar
  58. 58.
    Pynadath DV, Tambe M (2002) The communicative multiagent team decision problem: analyzing teamwork theories and models. J Artif Intell Res 16:389–423MathSciNetzbMATHGoogle Scholar
  59. 59.
    Rein R, Memmert D (2016) Big data and tactical analysis in elite soccer: future challenges and opportunities for sports science. SpringerPlus 5(1):1410–1412CrossRefGoogle Scholar
  60. 60.
    Riedmiller M, Gabel T, Knabe J, Strasdat H (2006) Brainstormers 2D - team description 2005. In: Bredenfeld A, Jacoff A, Noda I, Takahashi Y (eds) RoboCup 2005: Robot Soccer WorldCup IX. Lecture notes in computer science, vol. 4020. Springer Verlag, Berlin, pp 219–229Google Scholar
  61. 61.
    Riedmiller M, Gabel T, Hafner R, Lange S (2009) Reinforcement learning for robot soccer. Auton Robot 27(1):55–73CrossRefGoogle Scholar
  62. 62.
    Riley P, Stone P, Veloso M (2001) Layered disclosure: revealing agents’ internals. In: Castelfranchi C, Lesperance Y (eds), Intelligent Agents VII. Agent Theories, Architectures, and Languages—7th. International Workshop, ATAL-2000, Boston, MA, USA, July 7–9, 2000, Proceedings, Lecture Notes in Artificial Intelligence. Springer, BerlinGoogle Scholar
  63. 63.
    Roth M, Simmons R, Veloso M (2005) Reasoning about joint beliefs for execution-time communication decisions. In: Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems. ACM, pp 786–793Google Scholar
  64. 64.
    Roth M, Simmons R, Veloso M (2006) What to communicate? Execution-time decision in multi-agent pomdps. Distributed autonomous robotic systems 7. Springer, Berlin, pp 177–186CrossRefGoogle Scholar
  65. 65.
    Salge C, Ay N, Polani D, Prokopenko M (2015) Zipf’s law: balancing signal usage cost and communication efficiency. PLOS One 10(10):e0139475CrossRefGoogle Scholar
  66. 66.
    Schumacher J, Wunderle T, Fries P, Jäkel F, Pipa G (2015) A statistical framework to infer delay and direction of information flow from measurements of complex systems. Neural Comput 27(8):1555–1608CrossRefGoogle Scholar
  67. 67.
    Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6(2):461–464MathSciNetCrossRefzbMATHGoogle Scholar
  68. 68.
    Shoham Y, Leyton-Brown K (2008) Multiagent systems: algorithmic, game-theoretic, and logical foundations. Cambridge University Press, CambridgeCrossRefzbMATHGoogle Scholar
  69. 69.
    Sporns O, Chialvo DR, Kaiser M, Hilgetag CC (2004) Organization, development and function of complex brain networks. Trends Cognit Sci 8(9):418–425CrossRefGoogle Scholar
  70. 70.
    Stone P, Riley P, Veloso M (2000) The CMUnited-99 champion simulator team. In: Veloso M, Pagello E, Kitano H (eds) RoboCup-99: Robot Soccer World Cup III, vol 1856. Lecture notes in artificial intelligence. Springer, Berlin, pp 35–48Google Scholar
  71. 71.
    Stone P, Riley P, Veloso M (2000) Defining and using ideal teammate and opponent models. In: Proc. of the 12th annual conf. on innovative applications of artificial intelligenceGoogle Scholar
  72. 72.
    Stone P, Veloso M (1998) Towards collaborative and adversarial learning: a case study in robotic soccer. Int J Hum Comput Stud 48(1):83–104CrossRefGoogle Scholar
  73. 73.
    Stone P, Veloso M (1999) Task decomposition, dynamic role assignment, and low-bandwidth communication for real-time strategic teamwork. Artif Intell 110(2):241–273CrossRefzbMATHGoogle Scholar
  74. 74.
    Stone P, Veloso M (2000) Multiagent systems: a survey from a machine learning perspective. Auton Robot 8(3):345–383CrossRefGoogle Scholar
  75. 75.
    Taylor ME, Stone P (2009) Transfer learning for reinforcement learning domains: a survey. J Mach Learn Res 10(1):1633–1685MathSciNetzbMATHGoogle Scholar
  76. 76.
    Venables WN, Ripley BD (2003) Modern applied statistics with S, 4th edn. Springer, New YorkzbMATHGoogle Scholar
  77. 77.
    Vicente R, Wibral M, Lindner M, Pipa G (2011) Transfer entropy—a model-free measure of effective connectivity for the neurosciences. J Comput Neurosci 30(1):45–67MathSciNetCrossRefGoogle Scholar
  78. 78.
    Vilar L, Araujo D, Davids K, Bar-Yam Y (2013) Science of winning soccer: emergent pattern-forming dynamics in association football. J Syst Sci Complex 26:73–84CrossRefGoogle Scholar
  79. 79.
    Vrieze S (2012) Model selection and psychological theory: a discussion of the differences between the Akaike information criterion (AIC) and the Bayesian information criterion (BIC). Psychol Methods 17(2):228–243CrossRefGoogle Scholar
  80. 80.
    Wasserman S, Faust K (1994) Social network analysis. Cambridge University Press, New YorkCrossRefzbMATHGoogle Scholar
  81. 81.
    Weiss G (1999) Multiagent systems: a modern approach to distributed artificial intelligence. MIT press, CambridgeGoogle Scholar
  82. 82.
    Whiteson S, Kohl N, Miikkulainen R, Stone P (2005) Evolving keepaway soccer players through task decomposition. Mach Learn 59(1):5–30CrossRefzbMATHGoogle Scholar
  83. 83.
    Wu F, Zilberstein S, Chen X (2011) Online planning for multi-agent systens with bounded communication. Artif Intell 175:487–511CrossRefzbMATHGoogle Scholar
  84. 84.
    Xu Z, Fitch R, Underwood J, Sukkarieh S (2013) Decentralized coordinated tracking with mixed discrete-continuous decisions. J Field Robot 30(5):717–740CrossRefGoogle Scholar
  85. 85.
    Zhang H, Chen X (2014) The decision-making framework of WrightEagle, the RoboCup 2013 soccer simulation 2D league champion team. In: Behnke S, Veloso M, Visser A, Xiong R (eds) RoboCup 2013: Robot Soccer World Cup XVII, vol 8371. Lecture notes in artificial intelligence. Springer, Berlin, pp 114–124Google Scholar

Copyright information

© Her Majesty the Queen in Right of Australia 2017

Authors and Affiliations

  • Mathew Zuparic
    • 1
    Email author
  • Victor Jauregui
    • 2
  • Mikhail Prokopenko
    • 2
  • Yi Yue
    • 1
  1. 1.Decision Sciences, Defence Science and Technology Group, Department of DefenceCanberraAustralia
  2. 2.Complex Systems Research Group, Faculty of Engineering and ITThe University of SydneySydneyAustralia

Personalised recommendations