Skip to main content
SpringerLink
Log in

Few-shot transfer learning with attention for intelligent fault diagnosis of bearing

  • Original Article
  • Published:
Journal of Mechanical Science and Technology Aims and scope Submit manuscript

Abstract

The bearing is one of the key components in modern industrial equipment. In the past few years, many studies have been carried out on bearing diagnosis through data-driven methods. However, there are two practical problems. First, under actual working conditions, the lack of fault samples is a major factor that hinders the application of these methods in industrial environments. Second, there is a lack of full utilization of a priori knowledge in the current stage of methods using relational networks for fault diagnosis. It is manifested by the incompleteness of the relational network structure. To address these problems, we present a new diagnosis method based on few-shot learning, which is suitable for the environment where the data is scarce. In this method, we train the model with the data generated by the artificial damaged bearings instead of the data from the real bearing. We experimentally validate the performance improvement of the complete relational network structure. It is able to perform the few-shot learning task better. In addition, we also reduce the global feature discrepancy by introducing an attention mechanism to improve the performance of the model. And the impact of the number of layers of the attention mechanism on the model is also discussed in detail. In this paper, our model performs better under the same experimental conditions compared with other transfer learning models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. A. H. Aljemely, J. Xuan, F. K. J. Jawad, O. AlAzzawi and A. S. Alhumaima, A novel unsupervised learning method for intelligent fault diagnosis of rolling element bearing based on deep functional auto-encoder, Journal of Mechanical Science and Technology, 34 (11) (2020) 4367–4381.

    Article  Google Scholar 

  2. C. Lessmeier, J. K. Kimotho, D. Zimmer and W. Sextro, Condition monitoring of bearing damage in electromechanical drive systems by using motor current signals of electric motors: a benchmark data set for data-driven classifification, PHM Society European Conference, 3 (1) (2016) 5–8.

    Google Scholar 

  3. S. Riaz, H. Elahi, K. Javaid and T. Shahzad, Vibration feature extraction and analysis for fault diagnosis of rotating machinery-a literature survey, Asia Pacific Journal of Multidisciplinary Research, 5 (1) (2017) 103–110.

    Google Scholar 

  4. L. Liu, S. Wang, D. Liu and Y. Peng, Quantitative selection of sensor data based on improved permutation entropy for system remaining useful life prediction, Microelectronics Reliability, 75 (2017) 264–270.

    Article  Google Scholar 

  5. H. Zhang, Q. Miao, X. Zhang and Z. Liu, An improved unscented particle filter approach for lithium-ion battery remaining useful life prediction, Microelectronics Reliability, 81 (2018) 288–298.

    Article  Google Scholar 

  6. D.-S. Huang, Systematic Theory of Neural Networks For Pattern Recognitionm, Publishing House of Electronic Industry of China, Beijing, 201 (1996).

    Google Scholar 

  7. F.-F. Diego, M.-R. David, F.-R. Oscar and A.-B. Amparo, Automatic bearing fault diagnosis based on one-class ν-svm, Computers and Industrial Engineering, 64 (1) (2013) 357–365.

    Article  Google Scholar 

  8. A. Khatir and M. A. Wahab, Fast simulations for solving fracture mechanics inverse problems using pod-rbf xiga andjaya algorithm, Engineering Fracture Mechanics, 205 (2019) 285–300.

    Article  Google Scholar 

  9. S. Tiachacht, S. Khatir, C. L. Thanh, R. V. Rao, S. Mirjalili and M. A. Wahab, Inverse problem for dynamic structural health-monitoring based on slime mould algorithm, Engineering with Computers (2021) 1–24.

  10. S. Khatir, D. Boutchicha, C. Le Thanh, H. Tran-Ngoc, T. N. Nguyen and M. Abdel-Wahab, Improved ann technique combined with jaya algorithm for crack identification in plates usingxiga and experimental analysis, Theoretical and Applied Fracture Mechanics, 107 (2020) 102554.

    Article  Google Scholar 

  11. H. Tran-Ngoc, S. Khatir, H. Ho-Khac, G. De Roeck, T. Bui-Tien and M. Abdel Wahab, Efficient artificial neural networks basedon a hybrid metaheuristic optimization algorithm for damage detection in laminated composite structures, Composite Structures, 262 (2021) 113339.

    Article  Google Scholar 

  12. Z. Zhao, T. Li, J. Wu, C. Sun, S. Wang, R. Yan and X. Chen, Deep learning algorithms for rotating machinery intelligent diagnosis: an open source benchmark study, ISA Transactions, 107 (2020) 224–255.

    Article  Google Scholar 

  13. A. Khatir, M. A. Wahab, D. Boutchicha and T. Khatir, Structural health monitoring using modal strain energydamage indicator coupled with teaching-learning-based optimization algorithm and isogoemetric analysis, Journal of Sound and Vibration, 448 (2019) 230–246.

    Article  Google Scholar 

  14. X. Zhang, G. Chen, T. Hao and Z. He, Rolling bearing fault convolutional neural network diagnosis method based on casing signal, Journal of Mechanical Science and Technology, 34 (2020) 2307–2316.

    Article  Google Scholar 

  15. Y. Chen, G. Peng, C. Xie, W. Zhang, C. Li and S. Liu, Acdin: bridging the gap between artificial and real bearing damages for bearing fault diag nosis, Neurocomputing, 294 (2018) 61–71.

    Article  Google Scholar 

  16. Z. Wu, H. Jiang, K. Zhao and X. Li, An adaptive deep transfer learning method for bearing fault diagnosis, Measurement, 151 (2020) 107227.

    Article  Google Scholar 

  17. M. J. Hasan, M. M. Islam and J.-M. Kim, Acoustic spectral imaging and transfer learning for reliable bearing fault diagnosis under variable speed conditions, Measurement, 138 (2019) 620–631.

    Article  Google Scholar 

  18. J. Wu, Z. Zhao, C. Sun, R. Yan and X. Chen, Few-shot transfer learning for intelligent fault diagnosis of machine, Measurement, 166 (2020) 108202.

    Article  Google Scholar 

  19. M. Han, Y. Wu, Y. Wang and W. Liu, Roller bearing fault diagnosis based on lmd and multi-scale symbolic dynamic information entropy, Journal of Mechanical Science and Technology, 35 (5) (2021) 1993–2005.

    Article  Google Scholar 

  20. N. Bendre, H. T. Marín and P. Najafirad, Learning from few samples: a survey, arXiv.2007.15484 (2020).

  21. H. Lee, S. J. Hwang and J. Shin, Self-supervised label augmentation via input transformations, arXiv:1910.05872 (2019).

  22. A. Alfassy, L. Karlinsky, A. Aides, J. Shtok, S. Harary, R. Feris, R. Giryes and A. M. Bronstein, Laso: label-set operations networks for multi-label few-shot learning, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2019) 6548–6557.

  23. F. Sung, Y. Yang, L. Zhang, T. Xiang, P. H. S. Torr and T. M. Hospedales, Learning to compare: relation network for few-shot learning, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018) 1199–1208.

  24. S. Hochreiter and J. Schmidhuber, Long short-term memory, Neural Computation, 9 (8) (1997) 1735–1780.

    Article  Google Scholar 

  25. A. Graves, G. Wayne and I. Danihelka, Neural turing machines, arXiv:1410.5401 (2014).

  26. Z. Zhang and V. Saligrama, Zero-shot learning via semantic similarity embedding, Proceedings of the IEEE International Conference on Computer Vision (2015) 4166–4174.

  27. A. Frome, G. Corrado, J. Shlens, S. Bengio, J. Dean, M. Ranzato and T. Mikolov, Devise: a adeep visual-semantic embedding model, Advances in Neural Information Processing Systems (2013) 26.

  28. S. Ravi and H. Larochelle, Optimization as a model for few-shot learning, International Conference on Learning Representations (ICLR) (2017).

  29. C. Finn, P. Abbeel and S. Levine, Model-agnostic metalearning for fast adaptation of deep networks, International Conference on Machine Learning (2017) 1126–1135.

  30. J. Yosinski, J. Clune, Y. Bengio and H. Lipson, How transferable are features in deep neural networks?, arXiv:1411.1792 (2014).

  31. A. Li, T. Luo, Z. Lu, T. Xiang and L. Wang, Large-scale few-shot learning: knowledge transfer with class hierarchy, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2019) 7212–7220.

  32. D. Bahdanau, K. Cho and Y. Bengio, Neural machine translation by jointly learning to align and translate, arXiv:1409.0473 (2014).

  33. D. Hu, An introductory survey on attention mechanisms in nlp problems, Proceedings of SAI Intelligent Systems Conference (2019) 432–448.

  34. J. B. Lee, R. A. Rossi, S. Kim, N. K. Ahmed and E. Koh, Attention models in graphs: a survey, ACM Transactions on Knowledge Discovery from Data (TKDD), 13 (6) (2019) 1–25.

    Article  Google Scholar 

  35. S. Woo, J. Park, J.-Y. Lee and I. S. Kweon, CBAM: convolutional block attention module, Proceedings of the European Conference on Computer Vision (ECCV) (2018) 3–19.

  36. J. Hu, L. Shen and G. Sun, Squeeze-and-excitation networks, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018) 7132–7141.

  37. L. Chen, H. Zhang, J. Xiao, L. Nie, J. Shao, W. Liu and T. Chua, SCA-CNN: spatial and channel-wise attention in convolutional networks for image captioning, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017) 5659–5667.

  38. A. Santoro, D. Raposo, D. G. T. Barrett, M. Malinowski, R. Pascanu, P. Battaglia and T. Lillicrap, A simple neural network module for relational reasoning, arXiv:1706.01427 (2017).

Download references

Acknowledgments

This work was supported by the National Key R&D Program of China (Grant No. 2020YFB1712901), the Major Special Program of Chongqing Science and Technology Commission (No. CSTC 2019jscx-zdztzxX0031), Graduate Scientific Research and Innovation Foundation of Chongqing (No. CYB 19072).

Author information

Authors and Affiliations

  1. Key Laboratory of Dependable Service Computing in Cyber Physical Society, Ministry of Education, Chongqing, 400044, China

    Yao Hu, Qingyu Xiong, Qiwu Zhu, Zhengyi Yang, Zhiyuan Zhang, Dan Wu & Zihui Wu

Authors
  1. Yao Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qingyu Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiwu Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhengyi Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhiyuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Dan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Zihui Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qingyu Xiong.

Additional information

Yao Hu received the B.Sc. degree from the School of Big Data & Software Engineering, Chongqing University, Chongqing, China, in 2019. He is currently pursuing the M.Sc. degree in Big Data & Software Engineering at Chongqing University, Chongqing, China. His current research interests include deep learning and intelligent system.

Qingyu Xiong received the Ph.D. degree in Electrical and Electronic Systems Engineering from Kyushu University, Fukuoka, Japan, in 2002, and the M.Sc. degree from Chongqing University, Chongqing, China, in 1991. He is currently a Professor with the School of Big Data & Software Engineering, Chongqing University. His current research interests include intelligent system and intelligent computation, and pervasive computation and embedded system. Dr. Xiong is a member of the China Computer Federation.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, Y., Xiong, Q., Zhu, Q. et al. Few-shot transfer learning with attention for intelligent fault diagnosis of bearing. J Mech Sci Technol 36, 6181–6192 (2022). https://doi.org/10.1007/s12206-022-1132-4

Download citation

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12206-022-1132-4

Keywords

Search

Navigation