Research and Application of Deep Reinforcement Learning in Rotating Machinery Fault Diagnosis Under Unbalanced Samples Condition

Abstract

Due to the rotating machinery is a healthy state most of the time and it is difficult to obtain enough fault data, historical data will be highly skewed to the health state, which affects the accuracy of the intelligent fault diagnosis method based on conventional deep learning (DL). In other to improve the performance of DL algorithm under unbalanced samples, a deep reinforcement learning algorithm based on actor-critic architecture combining reinforcement learning (RL) and DL is proposed in this paper, it uses DL as a basic learner to perceive input information and uses RL as decision maker to determine the health status or fault type of rotating machinery. In proposed algorithm, reward function is improved in the actor module which increases reward when agent correctly recognizes the fault classification and encourages agents to pay attention to minority fault samples, Jensen–Shannon (JS) divergence is used to calculate the distance between agent output action distribution and target distribution to relieve the reward sparsity issue in the initial training stage. In addition, an improved exploration strategy is designed, its greedy factor decreases with epochs to explore the external environment as much as possible in the initial training stage. Finally, an advanced weighted regression is introduced as a loss function to ensure that the agent updates in a beneficial direction. The experiment on PHM2009 gearbox challenge data demonstrates that the improved actor-critic framework is helpful to guide the intelligent diagnosis model based on DL to better deal with unbalanced data.