Skip to main content
SpringerLink
Log in

Reinforcement learning-based cost-sensitive classifier for imbalanced fault classification

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Fault classification plays a crucial role in the industrial process monitoring domain. In the datasets collected from real-life industrial processes, the data distribution is usually imbalanced. The datasets contain a large amount of normal data (majority) and only a small amount of faulty data (minority); this phenomenon is also known as the imbalanced fault classification problem. To solve the imbalanced fault classification problem, a novel reinforcement learning (RL)-based cost-sensitive classifier (RLCC) based on policy gradient is proposed in this paper. In RLCC, a novel cost-sensitive learning strategy based on policy gradient and the actor-critic of RL is developed. The novel cost-sensitive learning strategy can adaptively learn the cost matrix and dynamically yield the sample weights. In addition, RLCC uses a newly designed reward to train the sample weight learner and classifier using an alternating iterative approach. The alternating iterative approach makes RLCC highly flexible and effective in solving the imbalanced fault classification problem. The effectiveness and practicability of the proposed RLCC method are verified through its application in a real-world dataset and an industrial process benchmark.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Kano M, Nakagawa Y. Data-based process monitoring, process control, and quality improvement: recent developments and applications in steel industry. Comput Chem Eng, 2008, 32: 12–24

    Article  Google Scholar 

  2. Ge Z, Song Z, Ding S X, et al. Data mining and analytics in the process industry: the role of machine learning. IEEE Access, 2017, 5: 20590–20616

    Article  Google Scholar 

  3. Zhu Z R, Chai Y, Yang Z M. A novel kind of sufficient conditions for safety judgement based on control barrier function. Sci China Inf Sci, 2021, 64: 199205

    Article  MathSciNet  Google Scholar 

  4. Zhang X, Wei C, Song Z. Fast locally weighted PLS modeling for large-scale industrial processes. Ind Eng Chem Res, 2020, 59: 20779–20786

    Article  Google Scholar 

  5. Khatibisepehr S, Huang B, Khare S. Design of inferential sensors in the process industry: a review of Bayesian methods. J Process Control, 2013, 23: 1575–1596

    Article  Google Scholar 

  6. Zhou D H, Qin L G, He X, et al. Distributed sensor fault diagnosis for a formation system with unknown constant time delays. Sci China Inf Sci, 2018, 61: 112205

    Article  Google Scholar 

  7. Huang D Q, Fu Y Z, Qin N, et al. Fault diagnosis of high-speed train bogie based on LSTM neural network. Sci China Inf Sci, 2021, 64: 119203

    Article  Google Scholar 

  8. Chen G, Liu Y, Ge Z. K-means Bayes algorithm for imbalanced fault classification and big data application. J Process Control, 2019, 81: 54–64

    Article  Google Scholar 

  9. Kubat M, Holte R C, Matwin S. Machine learning for the detection of oil spills in satellite radar images. Machine Learn, 1998, 30: 195–215

    Article  Google Scholar 

  10. Krawczyk B. Learning from imbalanced data: open challenges and future directions. Prog Artif Intell, 2016, 5: 221–232

    Article  Google Scholar 

  11. Wang X Y, Liu B, Cao S Y, et al. Important sampling based active learning for imbalance classification. Sci China Inf Sci, 2020, 63: 182104

    Article  MathSciNet  Google Scholar 

  12. Jiang X, Ge Z. Data augmentation classifier for imbalanced fault classification. IEEE Trans Automat Sci Eng, 2020, 18: 1206–1217

    Article  Google Scholar 

  13. Fan S, Zhang X, Song Z. Imbalanced sample selection with deep reinforcement learning for fault diagnosis. IEEE Trans Ind Inf, 2021, 18: 2518–2527

    Article  Google Scholar 

  14. Yue G, Wei P, Liu Y, et al. Automated endoscopic image classification via deep neural network with class imbalance loss. IEEE Trans Instrum Meas, 2023, 72: 1–11

    Google Scholar 

  15. Santos M S, Abreu P H, Japkowicz N, et al. On the joint-effect of class imbalance and overlap: a critical review. Artif Intell Rev, 2022, 55: 6207–6275

    Article  Google Scholar 

  16. Hoskins J C, Himmelblau D M. Artificial neural network models of knowledge representation in chemical engineering. Comput Chem Eng, 1988, 12: 881–890

    Article  Google Scholar 

  17. Hastie T, Rosset S, Zhu J, et al. Multiclass AdaBoost. Stat Its Interface, 2009, 2: 349–360

    Article  MathSciNet  MATH  Google Scholar 

  18. Krempl G, Kottke D, Lemaire V. Optimised probabilistic active learning (OPAL). Mach Learn, 2015, 100: 449–476

    Article  MathSciNet  MATH  Google Scholar 

  19. Castro C L, Braga A P. Novel cost-sensitive approach to improve the multilayer perceptron performance on imbalanced data. IEEE Trans Neural Netw Learn Syst, 2013, 24: 888–899

    Article  Google Scholar 

  20. Zheng J. Cost-sensitive boosting neural networks for software defect prediction. Expert Syst Appl, 2010, 37: 4537–4543

    Article  Google Scholar 

  21. Zhang C, Tan K C, Ren R. Training cost-sensitive deep belief networks on imbalance data problems. In: Proceedings of International Joint Conference on Neural Networks (IJCNN), 2016. 4362–4367

    Google Scholar 

  22. Sun Y, Kamel M S, Wang Y. Boosting for learning multiple classes with imbalanced class distribution. In: Proceedings of the 6th International Conference on Data Mining (ICDM’06), 2006. 592–602

    Chapter  Google Scholar 

  23. Williams R J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach Learn, 1992, 8: 229–256

    Article  MATH  Google Scholar 

  24. Chawla N V, Bowyer K W, Hall L O, et al. SMOTE: synthetic minority over-sampling technique. J Artif Intell Res, 2002, 16: 321–357

    Article  MATH  Google Scholar 

  25. Ling C X, Yang Q, Wang J, et al. Decision trees with minimal costs. In: Proceedings of the 21st International Conference on Machine Learning, 2004. 69

    Google Scholar 

  26. Zadrozny B, Elkan C. Learning and making decisions when costs and probabilities are both unknown. In: Proceedings of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2001. 204–213

    Google Scholar 

  27. Zadrozny B, Langford J, Abe N. Cost-sensitive learning by cost-proportionate example weighting. In: Proceedings of the 3rd IEEE International Conference on Data Mining, 2003. 435–442

    Chapter  Google Scholar 

  28. Sun Y, Kamel M S, Wong A K C, et al. Cost-sensitive boosting for classification of imbalanced data. Pattern Recognition, 2007, 40: 3358–3378

    Article  MATH  Google Scholar 

  29. Ting K M. A comparative study of cost-sensitive boosting algorithms. In: Proceedings of the 17th International Conference on Machine Learning, 2000. 983–990

    Google Scholar 

  30. Shawe-Taylor G K J, Karakoulas G. Optimizing classiffiers for imbalanced training sets. In: Proceedings of Advances in Neural Information Processing Systems, 1999. 11: 253

    Google Scholar 

  31. Viola P, Jones M. Fast and robust classiffication using asymmetric adaboost and a detector cascade. In: Proceedings of Advances in Neural Information Processing System, 2001. 14

    Google Scholar 

  32. Zhou Z-H, Liu X-Y. Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE Trans Knowl Data Eng, 2005, 18: 63–77

    Article  Google Scholar 

  33. Sutton R S, McAllester D A, Singh S P, et al. Policy gradient methods for reinforcement learning with function approximation. In: Proceedings of Advances in Neural Information Processing Systems, 2000. 1057–1063

    Google Scholar 

  34. Konda V R, Tsitsiklis J N. Actor-critic algorithms. In: Proceedings of Advances in Neural Information Processing Systems, 2000. 1008–1014

    Google Scholar 

  35. Bhatnagar S, Sutton R S, Ghavamzadeh M, et al. Natural actor-critic algorithms. Automatica, 2009, 45: 2471–2482

    Article  MathSciNet  MATH  Google Scholar 

  36. Chen J, Tsai C-A, Chen J J, et al. Decision threshold adjustment in class prediction. SAR QSAR Environ Res, 2006, 17: 337–352

    Article  Google Scholar 

  37. Alejo R, Sotoca J M, Casañ G A. An empirical study for the multiclass imbalance problem with neural networks. In: Proceedings of Iberoamerican Congress on Pattern Recognition, 2008. 479–486

    Google Scholar 

  38. Joshi M V, Kumar V, Agarwal R C. Evaluating boosting algorithms to classify rare classes: comparison and improvements. In: Proceedings of IEEE International Conference on Data Mining, 2001. 257–264

    Google Scholar 

  39. Brodersen K H, Ong C S, Stephan K E, et al. The balanced accuracy and its posterior distribution. In: Proceedings of the 20th International Conference on Pattern Recognition, 2010. 3121–3124

    Google Scholar 

  40. Opitz J, Burst S. Macro F1 and Macro F1. 2019. ArXiv:1911.03347

    Google Scholar 

  41. Alejo R, García V, Sotoca J M, et al. Improving the performance of the RBF neural networks trained with imbalanced samples. In: Proceedings of International Work Conference on Artificial Neural Networks, 2007. 162–169

    Google Scholar 

  42. Wu Z, Lin W, Ji Y. An integrated ensemble learning model for imbalanced fault diagnostics and prognostics. IEEE Access, 2018, 6: 8394–8402

    Article  Google Scholar 

  43. Moreno-Torres J G, Sáez J A, Herrera F. Study on the impact of partition-induced dataset shift on k-fold cross-validation. IEEE Tr ans Neural Netw Learn Syst, 2012, 23: 1304–1312

    Article  Google Scholar 

  44. van Maaten L D. Accelerating t-SNE using tree-based algorithms. J Machine Learn Res, 2014, 15: 3221–3245

    MathSciNet  MATH  Google Scholar 

  45. Lyman P R, Georgakis C. Plant-wide control of the Tennessee Eastman problem. Comput Chem Eng, 1995, 19: 321–331

    Article  Google Scholar 

  46. Yin S, Ding S X, Haghani A, et al. A comparison study of basic data-driven fault diagnosis and process monitoring methods on the benchmark Tennessee Eastman process. J Process Control, 2012, 22: 1567–1581

    Article  Google Scholar 

Download references

Acknowledgements This work was supported in part by National Natural Science Foundation of China (Grant Nos. 62003301, 61833014) and Natural Science Foundation of Zhejiang Province (Grant No. LQ21F030018).

Author information

Authors and Affiliations

  1. State Key Laboratory of Industrial Control Technology, College of Control Science and Engineering, Zhejiang University, Hangzhou, 310027, China

    Xinmin Zhang, Saite Fan & Zhihuan Song

Authors
  1. Xinmin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Saite Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhihuan Song
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Saite Fan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, X., Fan, S. & Song, Z. Reinforcement learning-based cost-sensitive classifier for imbalanced fault classification. Sci. China Inf. Sci. 66, 212201 (2023). https://doi.org/10.1007/s11432-021-3775-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-021-3775-4

Keywords

Search

Navigation