Advertisement

An Air Combat Decision Learning System Based on a Brain-Like Cognitive Mechanism

  • Kai ZhouEmail author
  • Ruixuan Wei
  • Zhuofan Xu
  • Qirui Zhang
  • Hongxuan Lu
  • Guofeng Zhang
Article

Abstract

The unmanned aerial vehicle (UAV) has emerged unexpectedly as a new force in the recent local wars. To occupy the high ground of military technology in the future, research on autonomous air combat using UAVs is extremely important. In this paper, we propose an intelligent air combat learning system inspired by the cognitive mechanism of the human brain. The air combat capability was divided into two parts: declarative and procedural memory. Imitating the updating and storage mechanism of human knowledge, a long- and short-term hierarchical asynchronous learning principle was designed. We adopted the basic idea of using the error signal to drive learning according to neurophysiological research. Drawing lessons from the working memory mechanism, the error signal for driving the short-term learning was created in the absence of labelled data, using only the interactive information. Then, we proved that the learning mechanism could ensure that system performance advances steadily and gradually. The experiments illustrated that the learning system designed in this paper can achieve some inferior confrontation ability through self-learning without human prior knowledge. Action strategies formed by learning are analogous to the classical tactical manoeuvres of human fighter pilots. We compared our work with related works and found that our method could improve its performance continuously and finally defeat its opponent.

Keywords

Autonomous air combat Learning system Brain-like cognitive mechanism Knowledge Multi-level memory Biological neural mechanism 

Notes

Funding Information

This study was funded by the National Natural Science Foundation of China (61573373).

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

References

  1. 1.
    Virtanen K, Raivio T, Raimo PH. Decision theoretical approach to pilot simulation. J Aircr. 1999;36(4):632–41.Google Scholar
  2. 2.
    Virtanen K, Raivio T, Hamalainen RP. Modeling pilot's sequential maneuvering decisions by a multistage influence diagram. J Guid Control Dyn. 2004;27(4):655–77.Google Scholar
  3. 3.
    Virtanen K, Karelahti J, Raivio T. Modeling air combat by a moving horizon influence diagram game. J Guid Control Dyn. 2006;29(5):1080–91.  https://doi.org/10.2514/1.17168.Google Scholar
  4. 4.
    Zhong L, Tong M, Zhong W, Zhang S. Sequential maneuvering decisions based on multi-stage influence diagram in air combat. J Syst Eng Electron. 2007;18(3):551–5.Google Scholar
  5. 5.
    Austin F, Carbone G, Falco M, Hinz H. Game theory for automated maneuvering during air-to-air combat. J Guidance. 1990;13(6):1143–9.Google Scholar
  6. 6.
    Mukai H, Tanikawa A, Tunay I, Ozcan IA, Katz IN, Schättler H. Sequential linear-quadratic method for differential games with air combat applications. Comput Optim Appl. 2003;25(1–3):193–222.Google Scholar
  7. 7.
    Mauro P, Bruce AC. Numerical solution of the three dimensional orbital pursuit-evasion games. J Guid Control Dyn. 2009;32(2):474–87.Google Scholar
  8. 8.
    Poropudas J, Virtanen K. Game-theoretic validation and analysis of air combat simulation models. IEEE trans On systems, Man, and Cybernetics-Part A: Systems and Humans. 2010;40(5):1057–70.Google Scholar
  9. 9.
    Park H, Lee B, Tahk M, Yoo D. Differential game based air combat maneuver generation using scoring function matrix. Int J Aeronaut Space. 2016;17(2):204–13.  https://doi.org/10.5139/IJASS.2016.17.2.204.Google Scholar
  10. 10.
    Katz A. Tree Lookahead in air combat. J Aircr. 1994;31(4):970–3.Google Scholar
  11. 11.
    Dong Y, Ai J. Trial input method and own-aircraft state prediction in autonomous air combat. J Aircr. 2012;49(3):947–54.Google Scholar
  12. 12.
    Dong Y, Huang J, Ai J. Visual perception-based target aircraft movement prediction for autonomous air combat. J Aircr. 2015;52(2):538–52.Google Scholar
  13. 13.
    Huang C, Dong K, Huang H, Tang S, Zhang Z, Zhuoran AZ. Autonomous air combat maneuver decision using Bayesian inference and moving horizon optimization. J Syst Eng Electron. 2018;29(1):86–97.Google Scholar
  14. 14.
    Hemant RS, Shripad PM. Effect of missile turn rate on aircraft susceptibility to infrared-guided missile. J Aircr. 2013;50(2):663–6.Google Scholar
  15. 15.
    Mcgrew JS. Real-time maneuvering decisions for autonomous air combat. Cambridge: Massachusetts Institute of Technology; 2008.Google Scholar
  16. 16.
    Mcgrew JS, How JP, Bush L, Williams B, Roy N. Air Combat Strategy using Approximate Dynamic Programming. AIAA Guidance, Navigation and Control Conference and Exhibit;2008; Honolulu: American Institute of Aeronautics and Astronautics.Google Scholar
  17. 17.
    Mcgrew JS, How JP, Williams B, Roy N. Air-combat strategy using approximate dynamic programming. J Guid Control Dyn. 2010;33(5):1641–54.  https://doi.org/10.2514/1.46815.Google Scholar
  18. 18.
    Ma Y, Ma X, Song X. A case study on air combat decision using approximate dynamic programming. Math Probl Eng. 2014;2014:1–10.Google Scholar
  19. 19.
    White DA, Sofge DA. Handbook of intelligent control: neural, fuzzy, and adaptive approaches. New York: Van Nostrand Reinhold; 1992.Google Scholar
  20. 20.
    Xiao L, Sun D, Liu Y. E. G. A combined method based on expert system and BP neural network for UAV systems fault diagnosis. Artificial Intelligence and Computational Intelligence. Piscataway: IEEE Press; 2010.Google Scholar
  21. 21.
    Kaneshige J, Krishnakumar K. Artificial Immune System Approach for Air Combat Maneuvering. Intelligent Computing: Theory and Applications V;2007; Orlando: International Society for Optics and Photonics.Google Scholar
  22. 22.
    Shaw RL. Fighter combat tactics and maneuvering. Annapolis, Maryland: Naval Institute Press; 1985.Google Scholar
  23. 23.
    Etkin B, Reid LD. Dynamics of flight: stability and control. New York: Wiley; 1996.Google Scholar
  24. 24.
    Stevens BL, Lewis FL, Johnson EN. Aircraft control and simulation: dynamics, controls design, and autonomous systems: John Wiley & Sons; 2015.Google Scholar
  25. 25.
    Ullman S. Using neuroscience to develop artificial intelligence. Science (New York, NY). 2019;363(6428):692–3.  https://doi.org/10.1126/science.aau6595.Google Scholar
  26. 26.
    Zeng T, Si B. Cognitive mapping based on conjunctive representations of space and movement. Front Neurorobot. 2017;11.  https://doi.org/10.3389/fnbot.2017.00061.
  27. 27.
    Banino A, Barry C, Uria B, Blundell C, Lillicrap T, Mirowski P, et al. Vector-based navigation using grid-like representations in artificial agents. Nature. 2018;557(7705):429–33.  https://doi.org/10.1038/s41586-018-0102-6.Google Scholar
  28. 28.
    Zhao F, Zeng Y, Wang G, Bai J, Xu B. A brain-inspired decision making model based on top-down biasing of prefrontal cortex to basal ganglia and its application in autonomous UAV explorations. Cogn Comput. 2018;10(2):296–306.  https://doi.org/10.1007/s12559-017-9511-3.Google Scholar
  29. 29.
    Sun X, Peng X, Ding S. Emotional human-machine conversation generation based on long short-term memory. Cogn Comput. 2018;10(3):389–97.  https://doi.org/10.1007/s12559-017-9539-4.Google Scholar
  30. 30.
    Takac M, Knott A. A neural network model of episode representations in working memory. Cogn Comput. 2015;7(5):509–25.Google Scholar
  31. 31.
    Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997;275(5306):1593–9.Google Scholar
  32. 32.
    Miller EK, Cohen JD. An integrative theory of prefrontal cortex function. Annu Rev Neurosci. 2001;24(1):167–202.Google Scholar
  33. 33.
    Koechlin E, Summerfield C. An information theoretical approach to prefrontal executive function. Trends Cogn Sci. 2007;11(6):229–35.Google Scholar
  34. 34.
    Niv Y, Schoenbaum G. Dialogues on prediction errors. Trends Cogn Sci. 2008;12(7):265–72.Google Scholar
  35. 35.
    Garrison J, Erdeni B, Done J. Prediction error in reinforcement learning: a meta-analysis of neuroimaging studies. Neurosci Biobehav R. 2013;37(7):1297–310.Google Scholar
  36. 36.
    Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997;9(8):1735–80.Google Scholar
  37. 37.
    Gazzaniga MS, Ivry RB, Mangun GR, Steven MS. Cognitive neuroscience: the biology of the mind. W. W. Norton: New York & London; 2009.Google Scholar
  38. 38.
    Wang P, Song Q, Han H, Cheng J. Sequentially supervised long short-term memory for gesture recognition. Cogn Comput. 2016;8(5):982–91.  https://doi.org/10.1007/s12559-016-9388-6.Google Scholar
  39. 39.
    Atkinson RC, Shiffrin RM. Human memory: a proposed system and its control processes 1. Psychol Learn Motiv. 1968;2:89–195.Google Scholar
  40. 40.
    Baddeley A. Working memory. Science. 1992;255(5044):556–9.Google Scholar
  41. 41.
    Miller SP, Hudson PJ. Using evidence-based practices to build mathematics competence related to conceptual, procedural, and declarative knowledge. Learn Disabil Res Pr. 2007;22(1):47–57.Google Scholar
  42. 42.
    Laurent PA. A neural mechanism for reward discounting: insights from modeling hippocampal–striatal interactions. Cogn Comput. 2013;5(1):152–60.  https://doi.org/10.1007/s12559-012-9178-8.Google Scholar
  43. 43.
    Nielsen MA. Neural networks and deep learning. USA: Determination Press; 2015.Google Scholar
  44. 44.
    Hinton GE, Osindero S, Teh Y. Reducing the dimensionality of data with neural networks. Science. 2006;313(5786):504–7.Google Scholar
  45. 45.
    Nair V, Hinton GE. Rectified linear units improve restricted boltzmann machines. Proceedings of the 27th international conference on machine learning (ICML-10);2010; Haifa, Israel.Google Scholar
  46. 46.
    Dugas C, Bengio Y, Bélisle F, Nadeau C, Garcia R. Incorporating second-order functional knowledge for better option pricing. Proceedings of the 13th International Conference on Neural Information Processing Systems;2001: MIT Press.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Air Force Engineering UniversityXi’anChina

Personalised recommendations