Advertisement

Journal of Intelligent & Robotic Systems

, Volume 83, Issue 1, pp 35–53 | Cite as

A Decentralized Fuzzy Learning Algorithm for Pursuit-Evasion Differential Games with Superior Evaders

  • Mostafa D. Awheda
  • Howard M. Schwartz
Article

Abstract

In this paper, we consider multi-pursuer single-superior-evader pursuit-evasion differential games where the evader has a speed that is similar to or higher than the speed of each pursuer. A new fuzzy reinforcement learning algorithm is proposed in this work. The proposed algorithm uses the well-known Apollonius circle mechanism to define the capture region of the learning pursuer based on its location and the location of the superior evader. The proposed algorithm uses the Apollonius circle with a developed formation control approach in the tuning mechanism of the fuzzy logic controller (FLC) of the learning pursuer so that one or some of the learning pursuers can capture the superior evader. The formation control mechanism used by the proposed algorithm guarantees that the pursuers are distributed around the superior evader in order to avoid collision between pursuers. The formation control mechanism used by the proposed algorithm also makes the Apollonius circles of each two adjacent pursuers intersect or be at least tangent to each other so that the capture of the superior evader can occur. The proposed algorithm is a decentralized algorithm as no communication among the pursuers is required. The only information the proposed algorithm requires is the position and the speed of the superior evader. The proposed algorithm is used to learn different multi-pursuer single-superior-evader pursuit-evasion differential games. The simulation results show the effectiveness of the proposed algorithm.

Keywords

Fuzzy control Reinforcement learning Pursuit-evasion differential games Apollonius circles 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Wang, L.X.: A Course in Fuzzy Systems and Control. Upper Saddle River, NJ: Prentice Hall (1997)zbMATHGoogle Scholar
  2. 2.
    Schwartz, H.M.: Multi-agent machine learning: A reinforcement approach. John Wiley (2014)Google Scholar
  3. 3.
    Awheda, M.D., Schwartz, H.M.: The residual gradient FACL algorithm for differential games, Proceedings of the 28th IEEE Canadian Conference on Electrical and Computer Engineering (CCECE 2015), Halifax, Nova Scotia, Canada, May, pp. 3–6 (2015)Google Scholar
  4. 4.
    Desouky, S.F., Schwartz, H.M.: Q (λ)-learning adaptive fuzzy logic controllers for pursuit-evasion differential games. Int. J. Adapt. Control Signal Process. 25.10, 910–927 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Busoniu, L., Ernst, D., Babuska, R., Schutter, B. D.: Fuzzy partition optimization for approximate fuzzy Q-iteration. In: Proceedings of the 17th IFAC World Congress (IFAC-08) (2008)Google Scholar
  6. 6.
    Hinojosa, W., Nefti, S., Kaymak, U.: Systems control with generalized probabilistic fuzzy-reinforcement learning. IEEE Trans. Fuzzy Syst. 19.1, 51–64 (2011)CrossRefGoogle Scholar
  7. 7.
    Jouffe, L.: Fuzzy inference system learning by reinforcement methods. IEEE Trans. Syst. Man Cybern. C 28(3), 338–355 (1998)CrossRefGoogle Scholar
  8. 8.
    Abielmona, R., Petriu, E., Harb, M., Wesolkowski, S.: Mission-driven robotic intelligent sensor agents for territorial security. IEEE Comput. Intell. Mag. 6(1), 55–67 (2011)CrossRefGoogle Scholar
  9. 9.
    Stiffler, N.M., O’Kane, J.M.: A complete algorithm for visibility-based pursuit-evasion with multiple pursuers. In: Robotics and Automation (ICRA), IEEE International Conference on, pp. 1660–1667 (2014)Google Scholar
  10. 10.
    Jun, C., Bhattacharya, S., Ghrist, R.: Pursuit-evasion game for normal distributions. In: Intelligent Robots and Systems (IROS 2014), IEEE/RSJ International Conference on, pp. 83–88 (2014)Google Scholar
  11. 11.
    Festa, A., Vinter, R.B.: A decomposition technique for pursuit evasion games with many pursuers. In: Decision and Control (CDC), IEEE 52nd Annual Conference on, pp. 5797–5802 (2013)Google Scholar
  12. 12.
    Bakolas, E.: Evasion from a group of pursuers with double integrator kinematics. In: Decision and Control (CDC), IEEE 52nd Annual Conference on, pp. 1472–1477 (2013)Google Scholar
  13. 13.
    Pachter, M., Garcia, E., Casbeer, D.W.: Active target defense differential game. In: In Communication Control, and Computing (Allerton), 52nd Annual Allerton Conference on, pp. 46–53 (2014)Google Scholar
  14. 14.
    Bhattacharya, S., Basar, T., Falcone, M.: Numerical approximation for a visibility based pursuit-evasion game. In: Intelligent Robots and Systems (IROS), IEEE/RSJ International Conference on, pp. 68–75 (2014)Google Scholar
  15. 15.
    Stiffler, N.M., O’Kane, J.M.: A sampling-based algorithm for multi-robot visibility-based pursuit-evasion. In: Intelligent Robots and Systems (IROS), IEEE/RSJ International Conference on, pp. 1782–1789 (2014)Google Scholar
  16. 16.
    Oyler, D.W., Kabamba, P.T., Girard, A.R.: Pursuit-evasion games in the presence of a line segment obstacle. In: Decision and Control (CDC), IEEE 53rd Annual Conference on, pp. 1149–1154 (2014)Google Scholar
  17. 17.
    Exarchos, I., Tsiotras, P.: An asymmetric version of the two car pursuit-evasion game. In: Decision and Control (CDC), IEEE 53rd Annual Conference on, pp. 4272–4277 (2014)Google Scholar
  18. 18.
    Becerra, I., Macias, V., Murrieta-Cid, R.: On the value of information in a differential pursuit-evasion game. In: Robotics and Automation (ICRA), IEEE International Conference on, pp. 4768–4774 (2015)Google Scholar
  19. 19.
    Lin, W., Qu, Z., Simaan, M.: Nash strategies for pursuit-evasion differential games involving limited observations. IEEE Trans. Aerosp. Electron. Syst. 51(2), 1347–1356 (2015)CrossRefGoogle Scholar
  20. 20.
    Wang, Q., Liu, M.: Learning in hide-and-seek, IEEE/ACM transactions on networking (2015)Google Scholar
  21. 21.
    Scott, W., Leonard, N.E.: Dynamics of pursuit and evasion in a heterogeneous herd. In: Decision and Control (CDC), IEEE 53rd Annual Conference on, pp. 2920-2925, IEEE (2014)Google Scholar
  22. 22.
    Kumar, A., Ojha, A.: An evader-centric strategy against fast pursuer in an unknown environment with static obstacles. In: Control, Automation, Robotics and Embedded Systems (CARE), International Conference on, pp. 1-6, IEEE (2013)Google Scholar
  23. 23.
    Dong, J., Zhang, X., Jia, X.: Strategies of Pursuit-Evasion Game Based on Improved Potential Field and Differential Game Theory for Mobile Robots. In: Instrumentation, Measurement, Computer, Communication and Control (IMCCC), Second International Conference on, pp. 1452-1456, IEEE (2012)Google Scholar
  24. 24.
    Wang, X., Cruz Jr, J.B., Chen, G., Pham, K., Blasch, E.: Formation control in multi-player pursuit evasion game with superior evaders. In: Defense and Security Symposium, International Society for Optics and Photonics (2007)Google Scholar
  25. 25.
    Wei, M., Chen, G., Cruz, J.B., Haynes, L.S., Chang, M.H., Blasch, E.: A decentralized approach to pursuer-evader games with multiple superior evaders in noisy environments. In: Aerospace Conference (2007)Google Scholar
  26. 26.
    Jin, S., Qu, Z.: Pursuit-evasion games with multi-pursuer vs. one fast evader. In: Intelligent Control and Automation (WCICA), 8th World Congress (2010)Google Scholar
  27. 27.
    Wei, M., Chen, G., Cruz, J.B., Hayes, L., Chang, M.H.: A decentralized approach to pursuer-evader games with multiple superior evaders. In: Intelligent Transportation Systems Conference, pp. 1586–1591 (2006)Google Scholar
  28. 28.
    Liu, R., Ze-Su, C.: A novel approach based on evolutionary game theoretic model for multi-player pursuit evasion. In: Computer, Mechatronics, Control and Electronic Engineering (CMCE), International Conference on, vol. 1, pp 107–110 (2010)Google Scholar
  29. 29.
    Givigi, S., Schwartz, H.M.: Decentralized learning in multiple pursuer-evader Markov games. In: Control & Automation (MED), 19th Mediterranean Conference on, pp. 1379–1385 (2011)Google Scholar
  30. 30.
    Li, D., Cruz, J.B.: Better cooperative control with limited look-ahead. In: American Control Conference (2006)Google Scholar
  31. 31.
    Li, D., Cruz, J.B., Chen, G., Kwan, C., Chang, M.H.: A hierarchical approach to multi-player pursuit-evasion differential games. In: Decision and Control, European Control Conference, CDC-ECC’05, 44th IEEE Conference on, pp. 5674–5679 (2005)Google Scholar
  32. 32.
    Wei, M., Chen, G., Cruz, J.B., Haynes, L., Pham, K., Blasch, E..: Multi-pursuer multi-evader pursuit-evasion games with jamming confrontation. J. Aerosp. Comput. Inf. Comm. 4(3), 693–706 (2007)Google Scholar
  33. 33.
    Cai, Z.S., Sun, L.N., Gao, H.B.: A novel hierarchical decomposition for multi-player pursuit evasion differential game with superior evaders. In: Proceedings of the first ACM/SIGEVO Summit on Genetic and Evolutionary Computation, pp. 795-798, ACM (2009)Google Scholar
  34. 34.
    Bao-Fu, F., Qi-Shu, P., Bing-Rong, H., Lei, D., Qiu-Bo, Z., Zhaosheng, Z.: Research on high speed evader vs. multi lower speed pursuers in multi pursuit-evasion games. Inf. Technol. J. 11(8), 2012Google Scholar
  35. 35.
    Wang, H., Yue, Q., Liu, J: Research on Pursuit-evasion games with multiple heterogeneous pursuers and a high speed evader, Control and Decision Conference (CCDC), 27th Chinese, IEEE (2015)Google Scholar
  36. 36.
    Jin, S., Qu, Z.: A heuristic task scheduling for multi-pursuer multi-evader games, Information and Automation, (ICIA), IEEE International Conference on (2011)Google Scholar
  37. 37.
    Kothari, M., Manathara, J.G., Postlethwaite, I: A cooperative pursuit-evasion game for non-holonomic systems. World Congress 1, 19 (2014)Google Scholar
  38. 38.
    Hasselt, H.V., Wiering, M.: Reinforcement learning in continuous action spaces, Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007, IEEE International Symposium on (2007)Google Scholar
  39. 39.
    Doya, K.: Reinforcement learning in continuous time and space. Neural Comput. 12.1, 219–245 (2000)CrossRefGoogle Scholar
  40. 40.
    Smart, W.D., Kaelbling, L.P.: Practical reinforcement learning in continuous spaces, ICML (2000)Google Scholar
  41. 41.
    Lazaric, A., Restelli, M., Bonarini, A.: Reinforcement learning in continuous action spaces through sequential Monte Carlo methods, Advances in neural information processing systems (2007)Google Scholar
  42. 42.
    Desouky, S.F., Schwartz, H.M.: Self-learning fuzzy logic controllers for pursuit-evasion differential games. Robot. Auton. Syst. 59, 22–33 (2011)CrossRefzbMATHGoogle Scholar
  43. 43.
    Takagi, T., Sugeno, M: Fuzzy identification of systems and its applications to modelling and control, IEEE Transactions on Systems, Man and Cybernetics SMC-15 (1985)Google Scholar
  44. 44.
    Isaacs, R.: Differential Game. John Wiley (1965)Google Scholar
  45. 45.
    LaValle, S.M: Planning Algorithms. Cambridge University Press (2006)Google Scholar
  46. 46.
    Lim, S.H., Furukawa, T., Dissanayake, G., Whyte, H.F.D.: A time-optimal control strategy for pursuit-evasion games problems. In: International Conference on Robotics and Automation, New Orleans, LA (2004)Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2015

Authors and Affiliations

  1. 1.Department of Systems and Computer EngineeringCarleton UniversityOttawaCanada

Personalised recommendations