Advertisement

Agent Cooperation within Adversarial Teams in Dynamic Environment – Key Issues and Development Trends

  • Bartłomiej Józef Dzieńkowski
  • Urszula Markowska-Kaczmar
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7190)

Abstract

This paper presents a survey of multi-agent systems (MAS) with adversarial teams competing in a dynamic environment. Agents within teams work together against an opposite group of agents in order to fulfill their contrary goals. The article introduces specificity of an environment and indicates fields of cooperation. It emphasizes the role of opponent analysis. Popular planning and learning methods are considered, as well. Next, possible fields of practical application are mentioned. The final part of the paper presents a summary of machine learning methods for specific problem solving and points up future development directions.

Keywords

MAS cooperation opponent strategy recognition anticipation prediction planning learning trends RoboCup battlefield combat 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Uhrmacher, A., Weyns, D.: Multi-Agent Systems: Simulation and Applications. Taylor and Francis Group, CRC Press (2009)Google Scholar
  2. 2.
    Veloso, M., Stone, P.: Individual and collaborative behaviors in a team of homogeneous robotic soccer agents. In: International Conference on Multi Agent Systems, pp. 309–316 (1998)Google Scholar
  3. 3.
    Hugel, V., Bonnin, P., Blazevic, P.: Reactive and adaptive control architecture designed for the Sony legged robots league in RoboCup 1999. In: International Conference on Intelligent Robots and Systems, vol. 2, pp. 1032–1037 (2000)Google Scholar
  4. 4.
    Bredenfeld, A., Indiveri, G.: Robot behavior engineering using DD-Designer. In: IEEE International Conference on Robotics and Automation, vol. 1, pp. 205–210 (2001)Google Scholar
  5. 5.
    Shi, L., Jiang, C., Zhen, Y., Zengqi, S.: Learning competition in robot soccer game based on an adapted neuro-fuzzy inference system. In: Proceedings of the 2001 IEEE International Symposium on Intelligent Control, pp. 195–199 (2001)Google Scholar
  6. 6.
    Zhang, B., Chen, X., Liu, L.: Agent Architecture: A Survey on RoboCup 1999 Simulator Teams. In: Proceedings of the 3rd World Congress on Intelligent Control and Automation, pp. 194–198 (2000)Google Scholar
  7. 7.
    Tan, A., Ng, G.: A Biologically-Inspired Cognitive Agent Model Integrating Declarative Knowledge and Reinforcement Learning. In: International Conference on Web Intelligence and Intelligent Agent Technology, vol. 2, pp. 248–251 (2010)Google Scholar
  8. 8.
    Tan, A., Carpenter, G., Grossberg, S.: Intelligence through interaction: Towards a unified theory for learning. In: International Symposium on Neural Networks, pp. 1098–1107 (2007)Google Scholar
  9. 9.
    Sun, Y., Bo, W.: Agent Hybrid Architecture and its Decision Processes. In: Proceedings of the Fifth International Conference on Machine Learning and Cybernetics, pp. 641–644 (2006)Google Scholar
  10. 10.
    Munoz-Hernandez, S., Wiguna, W.S.: Fuzzy Prolog as Cognitive Layer in RoboCupSoccer. In: IEEE Symposium on Computational Intelligence and Games, pp. 340–345 (2007)Google Scholar
  11. 11.
    Li, S., Ye, Z., Sun, Z.: A New Agent Architecture for RoboCup Tournament: Cognitive Architecture. In: Proceedings of the 3rd World Congress on Intelligent Control and Automation, pp. 199–202 (2000)Google Scholar
  12. 12.
    Dunin-Keplicz, B., Verbrugge, R.: Teamwork in Multi-Agent Systems – A Formal Approach. Wiley Series in Agent Technology. John Wiley and Sons Ltd. (2010)Google Scholar
  13. 13.
    Wang, P.: A Brief Survey on Cooperation in Multi-agent System. In: 2010 International Conference On Computer Design and Appliations, vol. 2, pp. 39–43 (2010)Google Scholar
  14. 14.
    Ferber, J.: Multi-Agent Systems – An Introduction to Distributed Artifical Intelligence. Addison Wesley Longman (1999)Google Scholar
  15. 15.
    Heintz, F.: FCFoo – a Short Description. In: RoboCup 1999 Team Description: Simulation League (1999)Google Scholar
  16. 16.
    Giraulf, F., Stinckwich, S.: Footux Team Description: A Hybrid recursive based agent architecture. In: RoboCup 1999 Team Description: Simulation League (1999)Google Scholar
  17. 17.
    Fathzadeh, R., Mokhtari, V., Mousakhani, M., Mahmoudi, F.: Mining Opponent Behavior: A Champion of RoboCup Coach Competition. In: IEEE 3rd Latin American Robotics Symposium, pp. 80–83 (2006)Google Scholar
  18. 18.
    Chelberg, D., Welch, L., Lakshmikumar, A., Gillen, M., Zhou, Q.: Meta-reasoning for a distributed agent architecture. In: Proceedings of the 33rd Southeastern Symposium on System Theory, pp. 377–381 (2001)Google Scholar
  19. 19.
    Sharifi, M., Mousavian, H., Aavani, A.: Predicting the future state of the RoboCup simulation environment: heuristic and neural networks approaches. In: IEEE International Conference on Systems, Man and Cybernetics, vol. 1, pp. 32–37 (2003)Google Scholar
  20. 20.
    Chohra, A., Scholl, P., Kobialka, H.-U., Hermes, J., Bredenfeld, A.: Behavior learning to predict using neural networks (NN): Towards a fast, cooperative and adversarial robot team (RoboCup). In: Proceedings of the Second International Workshop on Robot Motion and Control, pp. 79–84 (2001)Google Scholar
  21. 21.
    Camilleri, G.: A generic formal plan recognition theory. In: International Conference on Information Intelligence and Systems, pp. 540–547 (1999)Google Scholar
  22. 22.
    Weber, B.G., Mateas, M.: A data mining approach to strategy prediction. In: IEEE Symposium on Computational Intelligence and Games, pp. 140–147 (2009)Google Scholar
  23. 23.
    Han, K., Veloso, M.: Automated Robot Behavior Recognition Applied to Robotic Soccer. In: Robotics Research: 9th International Symposium, pp. 199–204 (2000)Google Scholar
  24. 24.
    Rabiner, L.: A tutorial on Hidden Markov Models and selected applications in speech recognition. Proceedings of the IEEE 77, 86–257 (1989)CrossRefGoogle Scholar
  25. 25.
    Lopez, R., Jimenez, A.: Hybridization of cognitive models using evolutionary strategies. In: IEEE Congress on Evolutionary Computation, pp. 3213–3218 (2009)Google Scholar
  26. 26.
    Javier, O., Lopez, R.: Self-Organized and Evolvable Cognitive Architecture for Intelligent Agents and Multi-agent Systems. In: Second International Conference on Computer Engineering and Applications, vol. 1, pp. 417–421 (2010)Google Scholar
  27. 27.
    Faria, B.M., Reis, L.P., Lau, N., Castillo, G.: Machine Learning algorithms applied to the classification of robotic soccer formations and opponent teams. In: IEEE Conference on Cybernetics and Intelligent Systems, pp. 344–349 (2010)Google Scholar
  28. 28.
    Devaney, M., Ram, A.: Needles in a Haystack: Plan Recognition in Large Spatial Domains Involving Multiple Agents. In: Proceedings of AAAI 1998, pp. 942–947 (1998)Google Scholar
  29. 29.
    Laird, J.E.: It knows what you’re going to do: Adding anticipation to a Quakebot. In: Proceedings of the Fifth International Conference on Autonomous Agents, pp. 385–392 (2001)Google Scholar
  30. 30.
    Gu, W., Zhou, J.: A Hostile Plan Recognition based on Plan Semantic Tree-Graph. In: IEEE International Conference on Control and Automation, pp. 2411–2416 (2007)Google Scholar
  31. 31.
    Gu, W., Ren, H., Li, B., Liu, Y., Liu, S.: Adversarial Plan Recognition and Opposition Based Tactical Plan Recognition. In: International Conference on Machine Learning and Cybernetics, pp. 499–504 (2006)Google Scholar
  32. 32.
    Li, W., Wang, W.: Multi-Agent Oriented Tactical Plan Recognition Method with Uncertainty. In: Fourth International Conference on Natural Computation, vol. 6, pp. 54–58 (2008)Google Scholar
  33. 33.
    Suzic, R., Svenson, P.: Capabilities-based plan recognition. In: 9th International Conference on Information Fusion, pp. 1–7 (2006)Google Scholar
  34. 34.
    Li, W.S., Wang, W.X.: A Hybrid Tactical Plan Recognition Method Base on Planning System. In: Third International Conference on Natural Computation, vol. 2, pp. 126–130 (2007)Google Scholar
  35. 35.
    Han, Y., Yin, M., Chen, J., Gu, W.: An Algorithm for Domain Axiom Plan Recognition based on Extended Goal Graph. In: Proceedings of 2005 International Conference on Machine Learning and Cybernetics, vol. 1, pp. 260–264 (2005)Google Scholar
  36. 36.
    Snoeck, N., van Kranenburg, H., Eertink, H.: Plan recognition in smart environments. In: 2nd International Conference on Digital Information Management, vol. 2, pp. 713–716 (2007)Google Scholar
  37. 37.
    Zhang, J., Cai, Z., Gan, Y., Zhang, B., He, L.: A New Algorithm for Predicting Future Actions in Plan Recognition. In: International Conference on Computational Intelligence and Security Workshops, pp. 140–143 (2007)Google Scholar
  38. 38.
    Gu, W., Yin, J.: The Recognition and Opposition to Multiagent Adversarial Planning. In: International Conference on Machine Learning and Cybernetics, pp. 2759–2764 (2006)Google Scholar
  39. 39.
    Ruiz, M.A., Uresti, J.R.: Team Agent Behavior Architecture in Robot Soccer. In: IEEE Latin American Robotic Symposium, pp. 20–25 (2008)Google Scholar
  40. 40.
    Riyaz, S., Basir, O.: Intelligent Planning and Execution of Tasks Using Hybrid Agents. In: International Conference on Artificial Intelligence and Computational Intelligence, vol. 1, pp. 277–282 (2009)Google Scholar
  41. 41.
    Bonissone, P., Dutta, S., Wood, N.: Merging strategic and tactical planning in dynamic and uncertain environments. IEEE Transactions on Systems, Man and Cybernetics 24(6), 841–862 (1994)CrossRefGoogle Scholar
  42. 42.
    She, Y., Grogono, P.: An Approach of Real-Time Team Behavior Control in Games. In: 21st International Conference on Tools with Artificial Intelligence, pp. 546–550 (2009)Google Scholar
  43. 43.
    Siebra, C., Tate, A.: An Investigation into the Use of Collaborative Concepts for Planning in Disaster Response Coalitions. In: IEEE Workshop on Distributed Intelligent Systems: Collective Intelligence and Its Applications, pp. 253–258 (2006)Google Scholar
  44. 44.
    Mota, L., Lau, N., Reis, L.: Co-ordination in RoboCup’s 2D simulation league: Setplays as flexible, multi-robot plans. In: IEEE Conference on Robotics Automation and Mechatronics, pp. 362–367 (2010)Google Scholar
  45. 45.
    Ehsaei, M., Heydarzadeh, Y., Aslani, S., Haghighat, A.: Pattern-Based Planning System (PBPS): A novel approach for uncertain dynamic multi-agent environments. In: 3rd International Symposium on Wireless Pervasive Computing, pp. 524–528 (2008)Google Scholar
  46. 46.
    Riedmiller, M., Gabel, T.: On Experiences in a Complex and Competitive Gaming Domain: Reinforcement Learning Meets RoboCup. In: IEEE Symposium on Computational Intelligence and Games, pp. 17–23 (2007)Google Scholar
  47. 47.
    Junyuan, T., Desheng, L.: An Optimal Strategy Learning for RoboCup in Continuous State Space. In: Proceedings of the 2006 IEEE International Conference on Mechatronics and Automation, pp. 301–305 (2006)Google Scholar
  48. 48.
    Kuo, J., Lin, H.: Cooperative RoboCup agents using genetic case-based reasoning. In: IEEE International Conference on Systems, Man and Cybernetics, pp. 613–618 (2008)Google Scholar
  49. 49.
    Kuo, J., Cheng, H.: Applying assimilation and accommodation for cooperative learning of RoboCup agent. In: International Conference on Machine Learning and Cybernetics, vol. 6, pp. 3234–3239 (2010)Google Scholar
  50. 50.
    Srinivasan, T., Aarthi, K., Aishwarya Meenakshi, S., Kausalya, M.: CBRRoboSoc: An Efficient Planning Strategy for Robotic Soccer Using Case Based Reasoning. In: International Conference on Computational Intelligence for Modelling, Control and Automation, p. 113 (2006)Google Scholar
  51. 51.
    Xiong, L., Wei, C., Jing, G., Zhenkun, Z., Zekai, H.: A New Passing Strategy Based on Q-Learning Algorithm in RoboCup. In: 2008 International Conference on Computer Science and Software Engineering, pp. 524–527 (2008)Google Scholar
  52. 52.
    Kuo, J., Ou, Y.: An Evolutionary Fuzzy Behaviour Controller using Genetic Algorithm in RoboCup Soccer Game. In: 2009 Ninth International Conference on Hybrid Intelligent Systems, pp. 281–286 (2009)Google Scholar
  53. 53.
    Shi, L., Jinyi, Y., Zhen, Y., Zengqi, S.: Multiple Rewards Fuzzy Reinforcement Learning Algorithm in RoboCup Environment. In: Proceedings of the 2001 IEEE International Conference on Control Applications, pp. 317–322 (2001)Google Scholar
  54. 54.
    Siebra, C., Tate, A.: An Investigation into the Use of Collaborative Concepts for Planning in Disaster Response Coalitions. In: IEEE Workshop on Distributed Intelligent Systems: Collective Intelligence and Its Applications, pp. 253–258 (2006)Google Scholar
  55. 55.
    Carbone, A., Finzi, A., Orlandini, A., Pirri, F., Ugazio, G.: Augmenting situation awareness via model-based control in rescue robots. In: International Conference on Intelligent Robots and Systems, pp. 3699–3705 (2005)Google Scholar
  56. 56.
    Takahashi, T., Takeuchi, I., Matsuno, F., Tadokoro, S.: Rescue simulation project and comprehensive disaster simulator architecture. In: International Conference on Intelligent Robots and Systems, vol. 3, pp. 1894–1899 (2000)Google Scholar
  57. 57.
    Tadokoro, S., Kitano, H.: The RoboCup-Rescue project: a robotic approach to the disaster mitigation problem. In: IEEE International Conference on Robotics and Automation, vol. 4, pp. 4089–4094 (2000)Google Scholar
  58. 58.
    Takahashi, T., Tadokoro, S.: Working with robots in disasters. IEEE Robotics and Automation Magazine 9(3), 34–39 (2002)CrossRefGoogle Scholar
  59. 59.
    An, A., Li, X., Xie, X.: Multi-agent interactions centric virtual battlefield simulation model. In: 2nd International Conference on Advanced Computer Control, vol. 3, pp. 315–319 (2010)Google Scholar
  60. 60.
    Li, X., Dang, S., Li, K., Liu, Q.: Multi-agent-based battlefield reconnaissance simulation by novel task decompositionand allocation. In: 5th International Conference on Computer Science and Education, pp. 1410–1414 (2010)Google Scholar
  61. 61.
    Li, Y., Wang, W., Ji, L., Zhu, J.: Research on Muti-Agent simulation and emulation in battlefield based on NETLOGO. In: International Conference on Computer Application and System Modeling, vol. 6, pp. 131–135 (2010)Google Scholar
  62. 62.
    Yu, Y., Zhao, G.: Virtual battlefield and combat simulation based on artificial life theory. In: Sixth International Conference on Natural Computation, vol. 6, pp. 2820–2825 (2010)Google Scholar
  63. 63.
    Huang, Y., Lu, C., Han, S.: Combat description framework of the naval systems based on Multi- Agent. In: IEEE International Conference on Advanced Management Science, vol. 2, pp. 591–594 (2010)Google Scholar
  64. 64.
    Parker, G.B., Probst, M.H.: Using evolution strategies for the real-time learning of controllers for autonomous agents in Xpilot-AI. In: IEEE Congress on Evolutionary Computation, pp. 1–7 (2010)Google Scholar
  65. 65.
    Parker, G.B., Parker, M.: Evolving Parameters for Xpilot Combat Agents. In: IEEE Symposium on Computational Intelligence and Games, pp. 238–243 (2007)Google Scholar
  66. 66.
    Cil, I., Mala, M.: MABSIM: A multi agent based simulation model of military unit combat. In: Second International Conference on the Applications of Digital Information and Web Technologies, pp. 731–736 (2009)Google Scholar
  67. 67.
    Liu, Y., Zhang, A.: Multi-Agent System and Its Application in Combat Simulation. In: International Symposium on Computational Intelligence and Design, vol. 1, pp. 448–452 (2008)Google Scholar
  68. 68.
    Liu, J., Zhao, C., Zhao, D., Gao, J.: Self-Organization Behaviors of Intelligent Antagonism Target Team of Air Combat Based on Pi-Calculus. In: International Workshop on Intelligent Systems and Applications, pp. 1–4 (2009)Google Scholar
  69. 69.
    Liu, J., Zhao, D., Zhao, C., Gao, J.: Study on mental attribution of decisions of multi-aircrafts cooperative combat command control. In: International Conference on Machine Learning and Cybernetics, vol. 4, pp. 2002–2005 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Bartłomiej Józef Dzieńkowski
    • 1
  • Urszula Markowska-Kaczmar
    • 1
  1. 1.Wrocław University of TechnologyWrocławPoland

Personalised recommendations