Skip to main content
SpringerLink
Log in

A Decentralized Fuzzy Learning Algorithm for Pursuit-Evasion Differential Games with Superior Evaders

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

In this paper, we consider multi-pursuer single-superior-evader pursuit-evasion differential games where the evader has a speed that is similar to or higher than the speed of each pursuer. A new fuzzy reinforcement learning algorithm is proposed in this work. The proposed algorithm uses the well-known Apollonius circle mechanism to define the capture region of the learning pursuer based on its location and the location of the superior evader. The proposed algorithm uses the Apollonius circle with a developed formation control approach in the tuning mechanism of the fuzzy logic controller (FLC) of the learning pursuer so that one or some of the learning pursuers can capture the superior evader. The formation control mechanism used by the proposed algorithm guarantees that the pursuers are distributed around the superior evader in order to avoid collision between pursuers. The formation control mechanism used by the proposed algorithm also makes the Apollonius circles of each two adjacent pursuers intersect or be at least tangent to each other so that the capture of the superior evader can occur. The proposed algorithm is a decentralized algorithm as no communication among the pursuers is required. The only information the proposed algorithm requires is the position and the speed of the superior evader. The proposed algorithm is used to learn different multi-pursuer single-superior-evader pursuit-evasion differential games. The simulation results show the effectiveness of the proposed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Wang, L.X.: A Course in Fuzzy Systems and Control. Upper Saddle River, NJ: Prentice Hall (1997)

    MATH  Google Scholar 

  2. Schwartz, H.M.: Multi-agent machine learning: A reinforcement approach. John Wiley (2014)

  3. Awheda, M.D., Schwartz, H.M.: The residual gradient FACL algorithm for differential games, Proceedings of the 28th IEEE Canadian Conference on Electrical and Computer Engineering (CCECE 2015), Halifax, Nova Scotia, Canada, May, pp. 3–6 (2015)

  4. Desouky, S.F., Schwartz, H.M.: Q (λ)-learning adaptive fuzzy logic controllers for pursuit-evasion differential games. Int. J. Adapt. Control Signal Process. 25.10, 910–927 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  5. Busoniu, L., Ernst, D., Babuska, R., Schutter, B. D.: Fuzzy partition optimization for approximate fuzzy Q-iteration. In: Proceedings of the 17th IFAC World Congress (IFAC-08) (2008)

  6. Hinojosa, W., Nefti, S., Kaymak, U.: Systems control with generalized probabilistic fuzzy-reinforcement learning. IEEE Trans. Fuzzy Syst. 19.1, 51–64 (2011)

    Article  Google Scholar 

  7. Jouffe, L.: Fuzzy inference system learning by reinforcement methods. IEEE Trans. Syst. Man Cybern. C 28(3), 338–355 (1998)

    Article  Google Scholar 

  8. Abielmona, R., Petriu, E., Harb, M., Wesolkowski, S.: Mission-driven robotic intelligent sensor agents for territorial security. IEEE Comput. Intell. Mag. 6(1), 55–67 (2011)

    Article  Google Scholar 

  9. Stiffler, N.M., O’Kane, J.M.: A complete algorithm for visibility-based pursuit-evasion with multiple pursuers. In: Robotics and Automation (ICRA), IEEE International Conference on, pp. 1660–1667 (2014)

  10. Jun, C., Bhattacharya, S., Ghrist, R.: Pursuit-evasion game for normal distributions. In: Intelligent Robots and Systems (IROS 2014), IEEE/RSJ International Conference on, pp. 83–88 (2014)

  11. Festa, A., Vinter, R.B.: A decomposition technique for pursuit evasion games with many pursuers. In: Decision and Control (CDC), IEEE 52nd Annual Conference on, pp. 5797–5802 (2013)

  12. Bakolas, E.: Evasion from a group of pursuers with double integrator kinematics. In: Decision and Control (CDC), IEEE 52nd Annual Conference on, pp. 1472–1477 (2013)

  13. Pachter, M., Garcia, E., Casbeer, D.W.: Active target defense differential game. In: In Communication Control, and Computing (Allerton), 52nd Annual Allerton Conference on, pp. 46–53 (2014)

  14. Bhattacharya, S., Basar, T., Falcone, M.: Numerical approximation for a visibility based pursuit-evasion game. In: Intelligent Robots and Systems (IROS), IEEE/RSJ International Conference on, pp. 68–75 (2014)

  15. Stiffler, N.M., O’Kane, J.M.: A sampling-based algorithm for multi-robot visibility-based pursuit-evasion. In: Intelligent Robots and Systems (IROS), IEEE/RSJ International Conference on, pp. 1782–1789 (2014)

  16. Oyler, D.W., Kabamba, P.T., Girard, A.R.: Pursuit-evasion games in the presence of a line segment obstacle. In: Decision and Control (CDC), IEEE 53rd Annual Conference on, pp. 1149–1154 (2014)

  17. Exarchos, I., Tsiotras, P.: An asymmetric version of the two car pursuit-evasion game. In: Decision and Control (CDC), IEEE 53rd Annual Conference on, pp. 4272–4277 (2014)

  18. Becerra, I., Macias, V., Murrieta-Cid, R.: On the value of information in a differential pursuit-evasion game. In: Robotics and Automation (ICRA), IEEE International Conference on, pp. 4768–4774 (2015)

  19. Lin, W., Qu, Z., Simaan, M.: Nash strategies for pursuit-evasion differential games involving limited observations. IEEE Trans. Aerosp. Electron. Syst. 51(2), 1347–1356 (2015)

    Article  Google Scholar 

  20. Wang, Q., Liu, M.: Learning in hide-and-seek, IEEE/ACM transactions on networking (2015)

  21. Scott, W., Leonard, N.E.: Dynamics of pursuit and evasion in a heterogeneous herd. In: Decision and Control (CDC), IEEE 53rd Annual Conference on, pp. 2920-2925, IEEE (2014)

  22. Kumar, A., Ojha, A.: An evader-centric strategy against fast pursuer in an unknown environment with static obstacles. In: Control, Automation, Robotics and Embedded Systems (CARE), International Conference on, pp. 1-6, IEEE (2013)

  23. Dong, J., Zhang, X., Jia, X.: Strategies of Pursuit-Evasion Game Based on Improved Potential Field and Differential Game Theory for Mobile Robots. In: Instrumentation, Measurement, Computer, Communication and Control (IMCCC), Second International Conference on, pp. 1452-1456, IEEE (2012)

  24. Wang, X., Cruz Jr, J.B., Chen, G., Pham, K., Blasch, E.: Formation control in multi-player pursuit evasion game with superior evaders. In: Defense and Security Symposium, International Society for Optics and Photonics (2007)

  25. Wei, M., Chen, G., Cruz, J.B., Haynes, L.S., Chang, M.H., Blasch, E.: A decentralized approach to pursuer-evader games with multiple superior evaders in noisy environments. In: Aerospace Conference (2007)

  26. Jin, S., Qu, Z.: Pursuit-evasion games with multi-pursuer vs. one fast evader. In: Intelligent Control and Automation (WCICA), 8th World Congress (2010)

  27. Wei, M., Chen, G., Cruz, J.B., Hayes, L., Chang, M.H.: A decentralized approach to pursuer-evader games with multiple superior evaders. In: Intelligent Transportation Systems Conference, pp. 1586–1591 (2006)

  28. Liu, R., Ze-Su, C.: A novel approach based on evolutionary game theoretic model for multi-player pursuit evasion. In: Computer, Mechatronics, Control and Electronic Engineering (CMCE), International Conference on, vol. 1, pp 107–110 (2010)

  29. Givigi, S., Schwartz, H.M.: Decentralized learning in multiple pursuer-evader Markov games. In: Control & Automation (MED), 19th Mediterranean Conference on, pp. 1379–1385 (2011)

  30. Li, D., Cruz, J.B.: Better cooperative control with limited look-ahead. In: American Control Conference (2006)

  31. Li, D., Cruz, J.B., Chen, G., Kwan, C., Chang, M.H.: A hierarchical approach to multi-player pursuit-evasion differential games. In: Decision and Control, European Control Conference, CDC-ECC’05, 44th IEEE Conference on, pp. 5674–5679 (2005)

  32. Wei, M., Chen, G., Cruz, J.B., Haynes, L., Pham, K., Blasch, E..: Multi-pursuer multi-evader pursuit-evasion games with jamming confrontation. J. Aerosp. Comput. Inf. Comm. 4(3), 693–706 (2007)

  33. Cai, Z.S., Sun, L.N., Gao, H.B.: A novel hierarchical decomposition for multi-player pursuit evasion differential game with superior evaders. In: Proceedings of the first ACM/SIGEVO Summit on Genetic and Evolutionary Computation, pp. 795-798, ACM (2009)

  34. Bao-Fu, F., Qi-Shu, P., Bing-Rong, H., Lei, D., Qiu-Bo, Z., Zhaosheng, Z.: Research on high speed evader vs. multi lower speed pursuers in multi pursuit-evasion games. Inf. Technol. J. 11(8), 2012

  35. Wang, H., Yue, Q., Liu, J: Research on Pursuit-evasion games with multiple heterogeneous pursuers and a high speed evader, Control and Decision Conference (CCDC), 27th Chinese, IEEE (2015)

  36. Jin, S., Qu, Z.: A heuristic task scheduling for multi-pursuer multi-evader games, Information and Automation, (ICIA), IEEE International Conference on (2011)

  37. Kothari, M., Manathara, J.G., Postlethwaite, I: A cooperative pursuit-evasion game for non-holonomic systems. World Congress 1, 19 (2014)

    Google Scholar 

  38. Hasselt, H.V., Wiering, M.: Reinforcement learning in continuous action spaces, Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007, IEEE International Symposium on (2007)

  39. Doya, K.: Reinforcement learning in continuous time and space. Neural Comput. 12.1, 219–245 (2000)

    Article  Google Scholar 

  40. Smart, W.D., Kaelbling, L.P.: Practical reinforcement learning in continuous spaces, ICML (2000)

  41. Lazaric, A., Restelli, M., Bonarini, A.: Reinforcement learning in continuous action spaces through sequential Monte Carlo methods, Advances in neural information processing systems (2007)

  42. Desouky, S.F., Schwartz, H.M.: Self-learning fuzzy logic controllers for pursuit-evasion differential games. Robot. Auton. Syst. 59, 22–33 (2011)

    Article  MATH  Google Scholar 

  43. Takagi, T., Sugeno, M: Fuzzy identification of systems and its applications to modelling and control, IEEE Transactions on Systems, Man and Cybernetics SMC-15 (1985)

  44. Isaacs, R.: Differential Game. John Wiley (1965)

  45. LaValle, S.M: Planning Algorithms. Cambridge University Press (2006)

  46. Lim, S.H., Furukawa, T., Dissanayake, G., Whyte, H.F.D.: A time-optimal control strategy for pursuit-evasion games problems. In: International Conference on Robotics and Automation, New Orleans, LA (2004)

Download references

Author information

Authors and Affiliations

  1. Department of Systems and Computer Engineering, Carleton University, 1125 Colonel By Drive, Ottawa, ON, K1S 5B6, Canada

    Mostafa D. Awheda & Howard M. Schwartz

Authors
  1. Mostafa D. Awheda
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Howard M. Schwartz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mostafa D. Awheda.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Awheda, M.D., Schwartz, H.M. A Decentralized Fuzzy Learning Algorithm for Pursuit-Evasion Differential Games with Superior Evaders. J Intell Robot Syst 83, 35–53 (2016). https://doi.org/10.1007/s10846-015-0315-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-015-0315-y

Keywords

Search

Navigation