Q-Learning Based Multiple Agent Reinforcement Learning Model for Air Target Threat Assessment

Abstract

Air target threat assessment is an important issue in air defense operations, which is an uncertainty process to protect the valuable assets against potential attacks of the various hostile airborne objects such as aircraft, missiles, helicopters and drones/UAV. This paper proposes a method to solve the problem of threat assessment of air targets by presenting the process of air defense scenarios in the form of Markov decision process and using reinforcement learning with Deep Q-Learning to predict most dangerous enemy actions to provide more accurate threat assessment of air attacks. On the basis of information about typical air defense combat environment, parameters of binding target trajectory (speed limit, overload limit…) and capabilities of defensive units (number of target channels, fire zone limitation, burning time,…) a simulation environment is built to train and evaluate the optimal (most dangerous) trajectory model of the target based on the given environment. This optimal trajectory can provide input information that is closer to reality, such as real time of arrival; probability of aircraft being shot down by SAM; angle of attack… for threat assessment methods (fuzzy logic, Bayes network, neural network…). The proposed model has been tested on the OpenAI Gym tool using Python programming language. It was shown that the model is suitable to calculate the level of danger of the target with the object to be protected in the context of the general air attacking environment with dynamic and complex constrains.