Journal of Intelligent & Robotic Systems

, Volume 83, Issue 1, pp 55–70 | Cite as

A Learning Invader for the “Guarding a Territory” Game

A Reinforcement Learning Problem
  • Hashem Raslan
  • Howard Schwartz
  • Sidney GivigiEmail author


This paper explores the use of a learning algorithm in the “guarding a territory” game. The game occurs in continuous time, where a single learning invader tries to get as close as possible to a territory before being captured by a guard. Previous research has approached the problem by letting only the guard learn. We will examine the other possibility of the game, in which only the invader is going to learn. Furthermore, in our case the guard is superior (faster) to the invader. We will also consider using models with non-holonomic constraints. A control system is designed and optimized for the invader to play the game and reach Nash Equilibrium. The paper shows how the learning system is able to adapt itself. The system’s performance is evaluated through different simulations and compared to the Nash Equilibrium. Experiments with real robots were conducted and verified our simulations in a real-life environment. Our results show that our learning invader behaved rationally in different circumstances.


Reinforcement learning Machine intelligence Adaptive control Continuous time Non-holonomic Fuzzy Q-learning Nash equilibrium 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Berenji, H.: Fuzzy q-learning: a new approach for fuzzy dynamic programming. In: Proceedings of the Third IEEE Conference on Fuzzy Systems, 1994. IEEE World Congress on Computational Intelligence, vol. 1, pp 486–491 (1994). doi: 10.1109/FUZZY.1994.343737
  2. 2.
    Desouky, S., Schwartz, H.: A novel hybrid learning technique applied to a self-learning multi-robot system. In: IEEE International Conference on Systems, Man and Cybernetics, 2009. SMC 2009, pp. 2616–2623 (2009). doi: 10.1109/ICSMC.2009.5346111
  3. 3.
    Er, M.J., San, L.: Automatic generation of fuzzy inference systems using incremental-topological-preserving-map-based fuzzy q-learning. In: IEEE International Conference on Fuzzy Systems, 2008. FUZZ-IEEE 2008. (IEEE World Congress on Computational Intelligence), pp. 467–474 (2008). doi: 10.1109/FUZZY.2008.4630410
  4. 4.
    Fang, M., Li, H., Zhang, X.: A heuristic reinforcement learning based on state backtracking method. In: 2012 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology (WI-IAT), vol. 1, pp. 673–678 (2012). doi: 10.1109/WI-IAT.2012.187
  5. 5.
    Givigi, S., Schwartz, H.M.: Decentralized strategy selection with learning automata for multiple pursuer-evader games. Adapt. Behav. 22(4), 221–234 (2014). doi: 10.1177/1059712314526261. CrossRefGoogle Scholar
  6. 6.
    Isaacs, R.: Differential Games: A Mathematical Theory with Applications to Warfare and Pursuit, Control and Optimization (1999)Google Scholar
  7. 7.
    Lauri, F., Koukam, A.: Robust multi-agent patrolling strategies using reinforcement learning. In: Siarry, P., Idoumghar, L., Lepagnot, J. (eds.) Swarm Intelligence Based Optimization, Lecture Notes in Computer Science, vol. 8472, pp. 157–165. Springer International Publishing (2014)Google Scholar
  8. 8.
    Lee, Y.S., Hsia, K.H., Hsieh, J.G.: A problem of guarding a territory with two invaders and two defenders. In: 1999 IEEE International Conference on Systems, Man, and Cybernetics, 1999. IEEE SMC ’99 Conference Proceedings, vol. 3, pp. 863–868 (1999). doi: 10.1109/ICSMC.1999.823341
  9. 9.
    Liu, J., Liu, S., Wu, H., Zhang, Y.: A pursuit-evasion algorithm based on hierarchical reinforcement learning. In: International Conference on Measuring Technology and Mechatronics Automation, 2009. ICMTMA ’09, vol. 2, pp. 482–486 (2009). doi: 10.1109/ICMTMA.2009.213
  10. 10.
    Nguyen, H.T., Walker, E.: A first course in fuzzy logic. Chapman and Hall, Boca Raton (2006).
  11. 11.
    Rzymowski, W.: A problem of guarding line segment. In: Proceedings of the 48th IEEE Conference on Decision and Control, 2009 held jointly with the 2009 28th Chinese Control Conference. CDC/CCC 2009, pp. 6444–6447 (2009). doi: 10.1109/CDC.2009.5400251
  12. 12.
    Schwartz, H.: Multi-Agent Machine Learning: A Reinforcement Approach. Wiley (2014)Google Scholar
  13. 13.
    Siciliano, B., Sciavicco, L., Villani, L., Oriolo, G.: Robotics Modelling, Planning and Control. Springer (2009)Google Scholar
  14. 14.
    Takagi, T., Sugeno, M.: Fuzzy identification of systems and its applications to modeling and control. IEEE Trans. Syst. Man Cybern. SMC-15(1), 116–132 (1985). doi: 10.1109/TSMC.1985.6313399 CrossRefzbMATHGoogle Scholar
  15. 15.
    Wang, L.: A Course in Fuzzy Systems and Control. Prentice Hall PTR (1997)Google Scholar
  16. 16.
    Wang, S., Panzica, A., Padir, T.: Motion control for intelligent ground vehicles based on the selection of paths using fuzzy inference. In: 2013 IEEE International Conference on Technologies for Practical Robot Applications (TePRA), pp. 1–6 (2013). doi: 10.1109/TePRA.2013.6556354

Copyright information

© Her Majesty the Queen in Right of Canada 2016

Authors and Affiliations

  1. 1.Carleton UniversityOttawaCanada
  2. 2.Royal Military College of CanadaKingstonCanada

Personalised recommendations