Abstract
A dual-experience pool deep reinforcement learning (DEPDRL) model is proposed for rolling bearing fault diagnosis with unbalanced data. In this method, a dual-experience pool structure is designed to store the sample data of majority and minority classes. A parallel double residual network model is established to extract deep features of the majority and minority input samples, respectively. In the process of training, the proposed balanced cross-sampling technique is used to randomly select samples from dual-experience pool in a certain proportion to realize the training of a double residual network model. We show the effectiveness of our method on three standard data sets, and compared with Resnet18, DCNN, DQN and DQNimb methods, the results show that DEPDRL has the best performance. Finally, with wavelet time-frequency graph as input, DEPDRL is applied to rolling bearing fault diagnosis with unbalanced test data. The results show that on a variety of unbalanced data sets, both the diagnostic accuracy and the G-means value of the DEPDRL are more than 5 % higher than other algorithms, which fully indicates that the DEPDRL has a very high fault diagnosis ability of rolling bearing with unbalanced data.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Y. G. Lei, X. F. Xu and X. Cai, Research on data quality assurance for health condition monitoring of machinery, Journal of Mechanical Engineering, 57 (4) (2021) 1–9.
Y. G. Lei, B. Yang and Z. J. Du, Deep transfer diagnosis method for machinery in big data era, Journal of Mechanical Engineering, 55 (7) (2019) 1–8 (in Chinese).
L. Wen, L. Gao and X. Y. Li, A new deep transfer learning based on sparse auto-encoder for fault diagnosis, Journal of Technology, 49 (1) (2019) 136–144.
X. Y. Zhang, G. Chen and T. F. Hao, Rolling bearing fault convolutional neural network diagnosis method based on casing signal, Journal of Mechanical Science and Technology, 34 (6) (2020) 2307–2316.
X. J. Guo, L. Chen and C. Q. Shen, Hierarchical adaptive deep convolution neural network and its application to bearing fault diagnosis, Measurement, 93 (7) (2016) 490–502.
X. Zhang, H. K. Jiang and X. Zhang, Rolling bearing fault diagnosis using an optimization deep belief network, Measurement Science and Technology, 26 (11) (2015) 115002.
Y. L. Wang, H. B. Yang and X. F. Yuan, Deep learning for fault-relevant feature extraction and fault classification with stacked supervised auto-encoder, Journal of Process Control, 92 (5) (2020) 79–89.
W. Y. Huang, J. S. Cheng and Y. Yang, An improved deep convolutional neural network with multi-scale information for bearing fault diagnosis, Neurocomputing, 359 (24) (2019) 77–92.
W. Q. Hou, M. Ye and W. H. Li, Rolling element bearing fault classification using improved stacked de-noising auto-encoders, Journal of Mechanical Engineering, 54 (7) (2018) 87–96.
Z. Meng, Y. Guan and Z. Z. Pan, Fault diagnosis of rolling bearing based on secondary data enhancement and deep convolutional network, Journal of Mechanical Engineering, 57 (23) (2021) 106–115.
S. H. Khan, M. Hayat and M. Bennamoun, Cost-sensitive learning of deep feature representations from imbalanced data, IEEE Transactions on Neural Networks and Learning Systems, 29 (8) (2017) 3573–3587.
F. Zhou, S. Yang and H. J. Fu, Deep learning fault diagnosis method based on global optimization GAN for unbalanced data, Knowledge-Based Systems, 187 (2020) 104837.1–104837.19.
H. Han, W. Y. Wang and B. H. Mao, Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning, Proceedings of the 2005 International Conference on Advances in Intelligent Computing, Springer (2005) 878–887.
P. Soltanzadeh and M. Hashemzadeh, RCSMOTE: range-controlled synthetic minority over-sampling technique for handling the class imbalance problem, Information Sciences, 542 (2020) 92–111.
H. Wang, Y. Ke and G. Luo, Compressed sensing of roller bearing fault based on multiple down-sampling strategy, Measurement Science and Technology, 27 (2) (2016) 025009.
Z. Zheng, Z. Yu and Y. Wu, Generative adversarial network with multi-branch discriminator for imbalanced cross-species image-to-image translation, Neural Networks, 141 (2021) 355–371.
Z. Wu, Y. Guo, W. Lin, S. Yu and Y. Ji, A weighted deep representation learning model for imbalanced fault diagnosis in cyber-physical systems, Sensors, 18 (4) (2018) 1096.
W. Qian and S. Li, A novel class imbalance-robust network for bearing fault diagnosis utilizing raw vibration, Measurement, 156 (2020) 107567.
L. Lin, B. Wang and J. Qi, Bearing fault diagnosis considering the effect of imbalance training sample, Entropy, 21 (4) (2019) 386.
E. L. Lin, Q. Chen and X. M. Qi, Deep reinforcement learning for imbalanced classification, Applied Intelligence, 50 (2020) 2488–2502.
S. Q. Kang, Z. Liu and Y. J. Wang, A fault diagnosis method of rolling bearing based on the improved DQN network, Chinese Journal of Scientific Instrument, 42 (3) (2021) 201–212.
J. M. Johnson and T. M. Khoshgoftaar, Survey on deep learning with class imbalance, Journal of Big Data, 6 (1) (2019) 1–54.
M. Volodymyr, K. Koray and S. David, Human-level control through deep reinforcement learning, Nature, 518 (7540) (2015) 529–533.
M. A. Wiering, H. V. Hasselt and A. D. Pietersma, Reinforcement learning algorithms for solving classification problems, 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), Paris, France (2011) 91–96.
A. K. Dixit and J. J. Sherrerd, Optimization in Economic Theory, Oxford University Press (1990).
K. M. He, X. Y. Zhang, S. Ren and J. Sun, Deep residual learning for image recognition, IEEE Conference on Computer Vision and Pattern Recognition, IEEE Computer Society, Las Vegas, USA (2016) 770–778.
K. M. He, X. Y. Zhang and S. Q. Ren, Identity mappings in deep residual networks, Proceedings of the European Conference on Computer Vision, Amsterdam, Netherlands (2016) 630–645.
H. D. Shao, H. K. Jiang and H. Z. Zhang, Rolling bearing fault feature learning using improved convolutional deep belief network with compressed sensing, Mechanical Systems and Signal Processing, 100 (1) (2018) 734–765.
Y. X. Kang, G. Chen, X. K. Wei and L. Zhou, Deep residual hedging network and its application in fault diagnosis of roll-ing bearings, Acta Aeronautica et Astronautica Sinica, 43 (8) (2022) 625201, doi: https://doi.org/10.7527/S1000-6893.2021.25201.
Acknowledgments
This research is sponsored by the. National Science and Technology Major Project of China (J2019-IV-004-0071), Postgraduate Research & Practice Innovation Program of Jiangsu Province (KYCX20_0211).
Author information
Authors and Affiliations
Corresponding author
Additional information
Y. X. Kang received his Master’s from Shenyang University of Aeronautics and Astronautics, ShenYang, P. R. China, in 2018. Now he is a Ph.D. student at the College of Civil Aviation, Nanjing University of Aeronautics and Astronautics, Nanjing, P. R. China. His current research interests include rotating-machine fault diagnosis, deep learning signal analysis and processing
G. Chen received a Ph.D. in Mechanical Engineering from the Southwest Jiaotong University, Chengdu, P. R. China, in 2000. He works at the College of Civil Aviation, Nanjing University of Aeronautics and Astronautics, Nanjing, P. R. China. His current research interests include the whole aero-engine vibration, rotor-bearing dynamics, rotating machine fault diagnosis, pattern recognition and machine learning, signal analysis and processing.
Rights and permissions
About this article
Cite this article
Kang, Y., Chen, G., Pan, W. et al. A dual-experience pool deep reinforcement learning method and its application in fault diagnosis of rolling bearing with unbalanced data. J Mech Sci Technol 37, 2715–2726 (2023). https://doi.org/10.1007/s12206-023-0501-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12206-023-0501-y