Skip to main content
SpringerLink
Log in

A Learning Invader for the “Guarding a Territory” Game

A Reinforcement Learning Problem

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

This paper explores the use of a learning algorithm in the “guarding a territory” game. The game occurs in continuous time, where a single learning invader tries to get as close as possible to a territory before being captured by a guard. Previous research has approached the problem by letting only the guard learn. We will examine the other possibility of the game, in which only the invader is going to learn. Furthermore, in our case the guard is superior (faster) to the invader. We will also consider using models with non-holonomic constraints. A control system is designed and optimized for the invader to play the game and reach Nash Equilibrium. The paper shows how the learning system is able to adapt itself. The system’s performance is evaluated through different simulations and compared to the Nash Equilibrium. Experiments with real robots were conducted and verified our simulations in a real-life environment. Our results show that our learning invader behaved rationally in different circumstances.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Berenji, H.: Fuzzy q-learning: a new approach for fuzzy dynamic programming. In: Proceedings of the Third IEEE Conference on Fuzzy Systems, 1994. IEEE World Congress on Computational Intelligence, vol. 1, pp 486–491 (1994). doi:10.1109/FUZZY.1994.343737

  2. Desouky, S., Schwartz, H.: A novel hybrid learning technique applied to a self-learning multi-robot system. In: IEEE International Conference on Systems, Man and Cybernetics, 2009. SMC 2009, pp. 2616–2623 (2009). doi:10.1109/ICSMC.2009.5346111

  3. Er, M.J., San, L.: Automatic generation of fuzzy inference systems using incremental-topological-preserving-map-based fuzzy q-learning. In: IEEE International Conference on Fuzzy Systems, 2008. FUZZ-IEEE 2008. (IEEE World Congress on Computational Intelligence), pp. 467–474 (2008). doi:10.1109/FUZZY.2008.4630410

  4. Fang, M., Li, H., Zhang, X.: A heuristic reinforcement learning based on state backtracking method. In: 2012 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology (WI-IAT), vol. 1, pp. 673–678 (2012). doi:10.1109/WI-IAT.2012.187

  5. Givigi, S., Schwartz, H.M.: Decentralized strategy selection with learning automata for multiple pursuer-evader games. Adapt. Behav. 22(4), 221–234 (2014). doi:10.1177/1059712314526261. http://adb.sagepub.com/content/22/4/221.abstract

    Article  Google Scholar 

  6. Isaacs, R.: Differential Games: A Mathematical Theory with Applications to Warfare and Pursuit, Control and Optimization (1999)

  7. Lauri, F., Koukam, A.: Robust multi-agent patrolling strategies using reinforcement learning. In: Siarry, P., Idoumghar, L., Lepagnot, J. (eds.) Swarm Intelligence Based Optimization, Lecture Notes in Computer Science, vol. 8472, pp. 157–165. Springer International Publishing (2014)

  8. Lee, Y.S., Hsia, K.H., Hsieh, J.G.: A problem of guarding a territory with two invaders and two defenders. In: 1999 IEEE International Conference on Systems, Man, and Cybernetics, 1999. IEEE SMC ’99 Conference Proceedings, vol. 3, pp. 863–868 (1999). doi:10.1109/ICSMC.1999.823341

  9. Liu, J., Liu, S., Wu, H., Zhang, Y.: A pursuit-evasion algorithm based on hierarchical reinforcement learning. In: International Conference on Measuring Technology and Mechatronics Automation, 2009. ICMTMA ’09, vol. 2, pp. 482–486 (2009). doi:10.1109/ICMTMA.2009.213

  10. Nguyen, H.T., Walker, E.: A first course in fuzzy logic. Chapman and Hall, Boca Raton (2006). www.summon.com

  11. Rzymowski, W.: A problem of guarding line segment. In: Proceedings of the 48th IEEE Conference on Decision and Control, 2009 held jointly with the 2009 28th Chinese Control Conference. CDC/CCC 2009, pp. 6444–6447 (2009). doi:10.1109/CDC.2009.5400251

  12. Schwartz, H.: Multi-Agent Machine Learning: A Reinforcement Approach. Wiley (2014)

  13. Siciliano, B., Sciavicco, L., Villani, L., Oriolo, G.: Robotics Modelling, Planning and Control. Springer (2009)

  14. Takagi, T., Sugeno, M.: Fuzzy identification of systems and its applications to modeling and control. IEEE Trans. Syst. Man Cybern. SMC-15(1), 116–132 (1985). doi:10.1109/TSMC.1985.6313399

    Article  MATH  Google Scholar 

  15. Wang, L.: A Course in Fuzzy Systems and Control. Prentice Hall PTR (1997)

  16. Wang, S., Panzica, A., Padir, T.: Motion control for intelligent ground vehicles based on the selection of paths using fuzzy inference. In: 2013 IEEE International Conference on Technologies for Practical Robot Applications (TePRA), pp. 1–6 (2013). doi:10.1109/TePRA.2013.6556354

Download references

Author information

Authors and Affiliations

  1. Carleton University, Ottawa, ON, Canada

    Hashem Raslan & Howard Schwartz

  2. Royal Military College of Canada, Kingston, ON, Canada

    Sidney Givigi

Authors
  1. Hashem Raslan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Howard Schwartz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sidney Givigi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sidney Givigi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Raslan, H., Schwartz, H. & Givigi, S. A Learning Invader for the “Guarding a Territory” Game. J Intell Robot Syst 83, 55–70 (2016). https://doi.org/10.1007/s10846-015-0317-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-015-0317-9

Keywords

Search

Navigation