Skip to main content
SpringerLink
Log in

Tool remaining useful life prediction using deep transfer reinforcement learning based on long short-term memory networks

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

Tool wear and faults will affect the quality of machined workpiece and damage the continuity of manufacturing. The accurate prediction of remaining useful life (RUL) is significant to guarantee the processing quality and improve the productivity of automatic system. At present, the most commonly used methods for tool RUL prediction are trained by history fault data. However, when researching on new types of tools or processing high value parts, fault datasets are difficult to acquire, which leads to RUL prediction a challenge under limited fault data. To overcome the shortcomings of above prediction methods, a deep transfer reinforcement learning (DTRL) network based on long short-term memory (LSTM) network is presented in this paper. Local features are extracted from consecutive sensor data to track the tool states, and the trained network size can be dynamically adjusted by controlling time sequence length. Then in DTRL network, LSTM network is employed to construct the value function approximation for smoothly processing temporal information and mining long-term dependencies. On this basis, a novel strategy of Q-function update and transfer is presented to transfer the deep reinforcement learning (DRL) network trained by historical fault data to a new tool for RUL prediction. Finally, tool wear experiments are performed to validate effectiveness of the DTRL model. The prediction results demonstrate that the proposed method has high accuracy and generalization for similar tools and cutting conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The data sets supporting the results of this article are included within the article.

References

  1. Liao X, Zhou G, Zhang Z, Lu J, Ma J (2019) Tool wear state recognition based on GWO–SVM with feature selection of genetic algorithm. Int J Adv Manuf Technol 104(1-4):1051–1063

    Article  Google Scholar 

  2. Aghazadeh F, Tahan A, Thomas M (2018) Tool condition monitoring using spectral subtraction and convolutional neural networks in milling process. Int J Adv Manuf Technol 98(9-12):3217–3227

    Article  Google Scholar 

  3. Wang G, Qian L, Guo Z (2012) Continuous tool wear prediction based on Gaussian mixture regression model. Int J Adv Manuf Technol 66(9-12):1921–1929

    Article  Google Scholar 

  4. Si X, Wang W, Hu C (2013) A Wiener-process-based degradation model with a recursive filter algorithm for remaining useful life estimation. Mech Syst Signal Process 35(1-2):219–237

    Article  Google Scholar 

  5. Yan H, Zhou J, Pang C (2015) Gamma process with recursive MLE for wear PDF prediction in precognitive maintenance under aperiodic monitoring. Mechatronics 31:68–77

    Article  Google Scholar 

  6. Wang J, Wang P, Gao R (2015) Enhanced particle filter for tool wear prediction. J Manuf Syst 36:35–45

    Article  Google Scholar 

  7. Li C, Sanchez R, Zurita G (2016) Gearbox fault diagnosis based on deep random forest fusion of acoustic and vibratory signals. Mech Syst Signal Process 76-77:283–293

    Article  Google Scholar 

  8. Javed K, Gouriveau R, Li X, Zerhouni N (2016) Tool wear monitoring and prognostics challenges: a comparison of connectionist methods toward an adaptive ensemble model. J Intell Manuf 29(8):1873–1890

    Article  Google Scholar 

  9. Dou J, Xu C, Jiao S, Li B, Zhang J, Xu X (2019) An unsupervised online monitoring method for tool wear using a sparse auto-encoder. Int J Adv Manuf Technol 106:2493–2507

    Article  Google Scholar 

  10. Sun C, Wang P, Yan R (2019) Machine health monitoring based on locally linear embedding with kernel sparse representation for neighborhood optimization. Mech Syst Signal Process 114:25–34

    Article  Google Scholar 

  11. Widodo A, Yang B (2007) Support vector machine in machine condition monitoring and fault diagnosis. Mech Syst Signal Process 21(6):2560–2574

    Article  Google Scholar 

  12. Chen B, Chen X, Li B (2011) Reliability estimation for cutting tools based on logistic regression model using vibration signals. Mech Syst Signal Process 25(7):2526–2537

    Article  Google Scholar 

  13. Karandikar J (2019) Machine learning classification for tool life modeling using production shop-floor tool wear data. Procedia Manuf 34:446–454

    Article  Google Scholar 

  14. Kong D, Chen Y, Li N, Tan S (2016) Tool wear monitoring based on kernel principal component analysis and v-support vector regression. Int J Adv Manuf Technol 89(1-4):175–190

    Article  Google Scholar 

  15. Zhang C, Zhang H (2016) Modelling and prediction of tool wear using LS-SVM in milling operation. Int J Comput Integr Manuf 29(1):76–91

    MathSciNet  Google Scholar 

  16. Kong D, Chen Y, Li N (2018) Gaussian process regression for tool wear prediction. Mech Syst Signal Process 104:556–574

    Article  Google Scholar 

  17. Kong D, Chen Y, Li N (2017) Force-based tool wear estimation for milling process using Gaussian mixture hidden Markov models. Int J Adv Manuf Technol 92(5-8):2853–2865

    Article  Google Scholar 

  18. Zhou Y, Wang T (2018) ENN-based recognition method for tool cutting state. J Comput Sci 27:418–427

    Article  Google Scholar 

  19. Serin G, Sener B, Ozbayoglu A, Unver H (2020) Review of tool condition monitoring in machining and opportunities for deep learning. Int J Adv Manuf Technol 109:953–974

    Article  Google Scholar 

  20. Tao Z, An Q, Liu G, Chen M (2019) A novel method for tool condition monitoring based on long short-term memory and hidden Markov model hybrid framework in high-speed milling Ti-6Al-4V. Int J Adv Manuf Technol 105(7-8):3165–3182

    Article  Google Scholar 

  21. Wang J, Ma Y, Zhang L, Gao R, Wu D (2018) Deep learning for smart manufacturing: methods and applications. J Manuf Syst 48:144–156

    Article  Google Scholar 

  22. Xu X, Wang J, Ming W (2020) In-process tap tool wear monitoring and prediction using a novel model based on deep learning. Int J Adv Manuf Technol 112(1-2):453–466

    Article  Google Scholar 

  23. Jia F, Lei Y, Lin J, Zhou X, Lu N (2016) Deep neural networks: a promising tool for fault characteristic mining and intelligent diagnosis of rotating machinery with massive data. Mech Syst Signal Process 72-73:303–315

    Article  Google Scholar 

  24. Shao H, Jiang H, Zhang H, Duan W, Liang T, Wu S (2018) Rolling bearing fault feature learning using improved convolutional deep belief network with compressed sensing. Mech Syst Signal Process 100:743–765

    Article  Google Scholar 

  25. Wu X, Li J, Jin Y, Zheng S (2020) Modeling and analysis of tool wear prediction based on SVD and BiLSTM. Int J Adv Manuf Technol 106(9-10):4391–4399

    Article  Google Scholar 

  26. Zhao M, Kang M, Tang B, Pecht M (2018) Deep residual networks with dynamically weighted wavelet coefficients for fault diagnosis of planetary gearboxes. IEEE Trans Ind Electron 65:4290–4300

    Google Scholar 

  27. Arulkumaran K, Deisenroth M, Brundage M, Bharath A (2017) deep reinforcement learning: a brief survey. IEEE Signal Process Mag 34(6):26–38

    Article  Google Scholar 

  28. Silver D, Huang A, Maddison C (2016) Mastering the game of Go with deep neural networks and tree search. Nature 529:484–489

    Article  Google Scholar 

  29. Wang Y, Fang Y, Lou P, Yan J, Liu N (2020) Deep reinforcement learning based path planning for mobile robot in unknown environment. J Phys Conf Ser:1576

  30. Liu Q, Liu Z, Xu W, Tang Q (2019) Human-robot collaboration in disassembly for sustainable manufacturing. Int J Prod Res 57:4027–4044

    Article  Google Scholar 

  31. Yang H, Zhong W, Chen C (2020) Deep reinforcement learning based energy efficient resource management for social and cognitive internet of things. IEEE Internet Things J 7:5677–5689

    Article  Google Scholar 

  32. Min M, Xiao L, Chen Y (2019) Learning based computation offloading for IoT devices with energy harvesting. IEEE Trans Veh Technol 68:1930–1941

    Article  Google Scholar 

  33. Luo S (2020) Dynamic scheduling for flexible job shop with new job insertions by deep reinforcement learning. Appl Soft Comput 91:106208

    Article  Google Scholar 

  34. Hu L, Liu Z, Hu W (2020) Petri-net-based dynamic scheduling of flexible manufacturing system via deep reinforcement learning with graph convolutional network. J Manuf Syst 55:1–14

    Article  Google Scholar 

  35. Ding Y, Ma L, Ma J (2019) Intelligent fault diagnosis for rotating machinery using deep Q-network based health state classification: A deep reinforcement learning approach. Adv Eng Inform:42

  36. Mnih V, Kavukcuoglu K, Silver D (2015) Human-level control through deep reinforcement learning. Nature 518:529–533

    Article  Google Scholar 

  37. He Q, Kong F, Yan R (2007) Subspace-based gearbox condition monitoring by kernel principal component analysis. Mech Syst Signal Process 21(4):1755–1772

    Article  Google Scholar 

  38. Zhang Z, Li H, Meng G, Tu X, Cheng C (2016) Chatter detection in milling process based on the energy entropy of VMD and WPD. Int J Mach Tools Manuf 108:106–112

    Article  Google Scholar 

  39. Hong Y-S, Yoon H-S, Moon J-S, Cho Y-M, Ahn S-H (2016) Tool-wear monitoring during micro-end milling using wavelet packet transform and Fisher’s linear discriminant. Int J Precis Eng Manuf 17(7):845–855

    Article  Google Scholar 

Download references

Funding

This work was supported in part by the National Key R&D Program of China under Grant 2018YFB1308300.

Author information

Authors and Affiliations

  1. Nanjing University of Science and Technology, Nanjing, China

    Jiachen Yao, Baochun Lu & Junli Zhang

Authors
  1. Jiachen Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Baochun Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junli Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Jiachen Yao performed the data analyses and wrote the manuscript; Baochun Lu contributed to the conception of the study; Junli Zhang performed the experiment and helped perform the analysis.

Corresponding author

Correspondence to Jiachen Yao.

Ethics declarations

Ethics approval

NA.

Consent to participate

NA.

Consent to publish

Written informed consent for publication was obtained from all participants.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yao, J., Lu, B. & Zhang, J. Tool remaining useful life prediction using deep transfer reinforcement learning based on long short-term memory networks. Int J Adv Manuf Technol 118, 1077–1086 (2022). https://doi.org/10.1007/s00170-021-07950-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-021-07950-2

Keywords

Search

Navigation