Skip to main content
SpringerLink
Log in

Cutting tool prognostics enabled by hybrid CNN-LSTM with transfer learning

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

An effective strategy to predict the remaining useful life (RUL) of a cutting tool could maximise tool utilisation, optimise machining cost, and improve machining quality. In this paper, a novel approach, which is enabled by a hybrid CNN-LSTM (convolutional neural network-long short-term memory network) model with an embedded transfer learning mechanism, is designed for predicting the RUL of a cutting tool. The innovative characteristics of the approach are that the volume of datasets required for training the deep learning model for a cutting tool is alleviated by introducing the transfer learning mechanism, and the hybrid CNN-LSTM model is designed to improve the accuracy of the prediction. In specific, this approach, which takes multimodal data of a cutting tool as input, leverages a pre-trained ResNet-18 CNN model to extract features from visual inspection images of the cutting tool, the maximum mean discrepancy (MMD)-based transfer learning to adapt the trained model to the cutting tool, and a LSTM model to conduct the RUL prediction based on the image features aggregated with machining process parameters (MPPs). The performance of the approach is evaluated in terms of the root mean square error (RMS) and the mean absolute error (MAE). The results indicate the suitability of the approach for accurate wear and RUL prediction of cutting tools, enabling adaptive prognostics and health management (PHM) on cutting tools.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

It is declared that no data or materials are available for this research.

Code availability

It is declared that codes are not available for this research.

References

  1. Zhao R, Yan R, Chen Z, Mao K, Wang P, Gao RX (2019) Deep learning and its applications to machine health monitoring. Mech Syst Signal Process 115:213–237. https://doi.org/10.1016/j.ymssp.2018.05.050

    Article  Google Scholar 

  2. Johansson D, Hägglund S, Bushlya V, Ståhl JE (2017) Assessment of commonly used tool life models in metal cutting. Procedia Manuf 11:602–609. https://doi.org/10.1016/j.promfg.2017.07.154

    Article  Google Scholar 

  3. Kurada S, Bradley C (1997) A review of machine vision sensors for tool condition monitoring. Comput Ind 34:55–72. https://doi.org/10.1016/s0166-3615(96)00075-9

    Article  Google Scholar 

  4. Lee J, Lapira E, Bagheri B, Kao H an (2013) Recent advances and trends in predictive manufacturing systems in big data environment. Manuf Lett 1:38–41. https://doi.org/10.1016/j.mfglet.2013.09.005

  5. Lee J, Lapira E, Yang S, Kao A (2013) Predictive manufacturing system - trends of next-generation production systems. IFAC Proc Vol 46:150–156. https://doi.org/10.3182/20130522-3-BR-4036.00107

    Article  Google Scholar 

  6. Tao F, Qi Q, Liu A, Kusiak A (2018) Data-driven smart manufacturing. J Manuf Syst 48:157–169. https://doi.org/10.1016/j.jmsy.2018.01.006

    Article  Google Scholar 

  7. Nasir V, Sassani F (2021) A review on deep learning in machining and tool monitoring: methods, opportunities, and challenges. Int J Adv Manuf Technol 115:2683–2709. https://doi.org/10.1007/s00170-021-07325-7

    Article  Google Scholar 

  8. Zhao G, Zhang G, Ge Q, Liu X (2016) Research advances in fault diagnosis and prognostic based on deep learning. In: 2016 Prognostics and System Health Management Conference (PHM-Chengdu). IEEE, pp 1–6

  9. Kerr D, Pengilley J, Garwood R (2006) Assessment and visualisation of machine tool wear using computer vision. Int J Adv Manuf Technol 28:781–791. https://doi.org/10.1007/s00170-004-2420-0

    Article  Google Scholar 

  10. Dutta S, Pal SK, Mukhopadhyay S, Sen R (2013) Application of digital image processing in tool condition monitoring: A review. CIRP J Manuf Sci Technol 6:212–232. https://doi.org/10.1016/j.cirpj.2013.02.005

    Article  Google Scholar 

  11. Serin G, Sener B, Ozbayoglu AM, Unver HO (2020) Review of tool condition monitoring in machining and opportunities for deep learning. Int J Adv Manuf Technol 109:953–974. https://doi.org/10.1007/s00170-020-05449-w

    Article  Google Scholar 

  12. Park JJ, Ulsoy AG (1993) On-line flank wear estimation using an adaptive observer and computer vision, part 1: Theory. J Manuf Sci Eng Trans ASME 115:30–36. https://doi.org/10.1115/1.2901635

    Article  Google Scholar 

  13. ISO (2016) ISO 8688-2: Tool life testing in Milling. In: Int. Organ. Standarization. https://www.iso.org/obp/ui/#iso:std:iso:8688:-2:ed-1:v1:en. Accessed 22 Jun 2019

  14. Peng R, Liu J, Fu X, Liu C, Zhao L (2021) Application of machine vision method in tool wear monitoring. Int J Adv Manuf Technol:1–16. https://doi.org/10.1007/s00170-021-07522-4

  15. Sipos Z (1986) Investigation of cutting performance of coated HSS tools made in Hungary. NME, Miskolc Egyetem

    Google Scholar 

  16. Kong D, Chen Y, Li N (2018) Gaussian process regression for tool wear prediction. Mech Syst Signal Process 104:556–574. https://doi.org/10.1016/j.ymssp.2017.11.021

    Article  Google Scholar 

  17. Wang J, Zheng Y, Wang P, Gao RX (2017) A virtual sensing based augmented particle filter for tool condition prognosis. J Manuf Process 28:472–478. https://doi.org/10.1016/j.jmapro.2017.04.014

    Article  Google Scholar 

  18. Wang J, Wang P, Gao RX (2015) Enhanced particle filter for tool wear prediction. J Manuf Syst 36:35–45. https://doi.org/10.1016/j.jmsy.2015.03.005

    Article  Google Scholar 

  19. Sun H, Cao D, Zhao Z, Kang X (2018) A hybrid approach to cutting tool remaining useful life prediction based on the Wiener process. IEEE Trans Reliab 67:1294–1303. https://doi.org/10.1109/TR.2018.2831256

    Article  Google Scholar 

  20. Aramesh M, Attia MH, Kishawy HA, Balazinski M (2016) Estimating the remaining useful tool life of worn tools under different cutting parameters: A survival life analysis during turning of titanium metal matrix composites (Ti-MMCs). CIRP J Manuf Sci Technol 12:35–43. https://doi.org/10.1016/j.cirpj.2015.10.001

    Article  Google Scholar 

  21. Maropolous P, Alamin B (1996) Integrated tool life prediction and management for an intelligent tool selection system. J Mater Process Technol 61:225–230. https://doi.org/10.1016/0924-0136(96)02491-0

    Article  Google Scholar 

  22. Benkedjouh T, Medjaher K, Zerhouni N, Rechak S (2015) Health assessment and life prediction of cutting tools based on support vector regression. J Intell Manuf 26:213–223. https://doi.org/10.1007/s10845-013-0774-6

    Article  Google Scholar 

  23. Wu D, Jennings C, Terpenny J, Gao RX, Kumara S (2017) A comparative study on machine learning algorithms for smart manufacturing: tool wear prediction using random forests. J Manuf Sci Eng 139. https://doi.org/10.1115/1.4036350

  24. Chang WY, Wu SJ, Hsu JW (2020) Investigated iterative convergences of neural network for prediction turning tool wear. Int J Adv Manuf Technol 106:2939–2948. https://doi.org/10.1007/s00170-019-04821-9

    Article  Google Scholar 

  25. Wu X, Liu Y, Zhou X, Mou A (2019) Automatic identification of tool wear based on convolutional neural network in face milling process. Sensors (Switzerland) 19:19. https://doi.org/10.3390/s19183817

    Article  Google Scholar 

  26. Marei M, El Zaatari S, Li W (2021) Transfer learning enabled convolutional neural networks for estimating health state of cutting tools. Robot Comput Integr Manuf 71:11. https://doi.org/10.1016/j.rcim.2021.102145

    Article  Google Scholar 

  27. Lutz B, Kisskalt D, Regulin D, et al (2019) Evaluation of deep learning for semantic image segmentation in tool condition monitoring. In: Proceedings - 18th IEEE International Conference on Machine Learning and Applications, ICMLA 2019. IEEE, pp 2008–2013

  28. Ong P, Lee WK, Lau RJH (2019) Tool condition monitoring in CNC end milling using wavelet neural network based on machine vision. Int J Adv Manuf Technol 104:1369–1379. https://doi.org/10.1007/s00170-019-04020-6

    Article  Google Scholar 

  29. Zhao R, Yan R, Wang J, Mao K (2017) Learning to monitor machine health with convolutional Bi-directional LSTM networks. Sensors (Switzerland) 17:273. https://doi.org/10.3390/s17020273

    Article  Google Scholar 

  30. Wang P, Liu Z, Gao RX, Guo Y (2019) Heterogeneous data-driven hybrid machine learning for tool condition prognosis. In: CIRP Ann. https://www.sciencedirect.com/science/article/pii/S0007850619300083#fig0025. Accessed 10 Jul 2020

  31. Zhang X, Lu X, Li W, Wang S (2021) Prediction of the remaining useful life of cutting tool using the Hurst exponent and CNN-LSTM. Int J Adv Manuf Technol 112:2277–2299. https://doi.org/10.1007/s00170-020-06447-8

    Article  Google Scholar 

  32. Wang J, Yan J, Li C, Gao R.X., Zhao R. (2019) Deep heterogeneous GRU model for predictive analytics in smart manufacturing: Application to tool wear prediction. Comput Ind 111:1–14. https://doi.org/10.1016/j.compind.2019.06.001, 1

  33. Sun C, Ma M, Zhao Z, Tian S, Yan R, Chen X (2019) Deep Transfer learning based on sparse autoencoder for remaining useful life prediction of tool in manufacturing. IEEE Trans Ind Informatics 15:2416–2425. https://doi.org/10.1109/TII.2018.2881543

    Article  Google Scholar 

  34. Zou Z, Cao R, Chen W, Lei S, Gao X, Yang Y (2021) Development of a tool wear online monitoring system for dry gear hobbing machine based on new experimental approach and DAE-BPNN-integrated mathematic structure. Int J Adv Manuf Technol 1–14. https://doi.org/10.1007/s00170-021-07470-z

  35. Lei Y, Li N, Guo L, Li N, Yan T, Lin J (2018) Machinery health prognostics: a systematic review from data acquisition to RUL prediction. Mech Syst Signal Process 104:799–834. https://doi.org/10.1016/j.ymssp.2017.11.016

    Article  Google Scholar 

  36. Si XS, Wang W, Hu CH, Zhou DH (2011) Remaining useful life estimation - a review on the statistical data driven approaches. Eur J Oper Res 213:1–14. https://doi.org/10.1016/j.ejor.2010.11.018

    Article  MathSciNet  Google Scholar 

  37. Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Communications of the ACM, In, pp 84–90

    Google Scholar 

  38. Yu F, Koltun V, Funkhouser T (2017) Dilated residual networks. In: Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. IEEE, pp 636–644

  39. Li X, Jia X, Wang Y, Yang S, Zhao H, Lee J (2020) Industrial remaining useful life prediction by partial observation using deep learning with supervised attention. IEEE/ASME Trans Mechatronics 25:2241–2251. https://doi.org/10.1109/TMECH.2020.2992331

    Article  Google Scholar 

  40. Weis W (1993) Tool wear measurement on basis of optical sensors, vision systems and neuronal networks (application milling). In: Proceedings of WESCON 1993 Conference Record. IEEE, pp 134–138

  41. Wang Z, Oates T (2015) Imaging time-series to improve classification and imputation. IJCAI Int Jt Conf Artif Intell 2015:3939–3945 http://arxiv.org/abs/1506.00327

    Google Scholar 

  42. Tran M-Q, Liu M-K, Tran Q-V (2020) Milling chatter detection using scalogram and deep convolutional neural network. Int J Adv Manuf Technol 107:1505–1516. https://doi.org/10.1007/S00170-019-04807-7

    Article  Google Scholar 

  43. Ma M, Mao Z (2019) Deep recurrent convolutional neural network for remaining useful life prediction. In: 2019 IEEE International Conference on Prognostics and Health Management, ICPHM 2019. Institute of Electrical and Electronics Engineers Inc.

  44. Martínez-Arellano G, Terrazas G, Ratchev S (2019) Tool wear classification using time series imaging and deep learning. Int J Adv Manuf Technol 104:3647–3662. https://doi.org/10.1007/s00170-019-04090-6

    Article  Google Scholar 

  45. Goodfellow I, Courville A, Bengio Y (2016) Deep learning, 1st–2nd ed. MIT Press. https://www.deeplearningbook.org/.

  46. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9:1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  47. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L (2015) ImageNet large scale visual recognition challenge. Int J Comput Vis 115:211–252. https://doi.org/10.1007/s11263-015-0816-y

    Article  MathSciNet  Google Scholar 

  48. Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? Adv Neural Inf Proces Syst 4:3320–3328 http://arxiv.org/abs/1411.1792

    Google Scholar 

  49. Oquab M, Bottou L, Laptev I, Sivic J (2014) Learning and transferring mid-level image representations using convolutional neural networks. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. pp 1717–1724

  50. FarajiDavar N (2015) Transductive Transfer Learning for Computer Vision Nazli FarajiDavar. University of Surrey

    Google Scholar 

  51. Dziugaite GK, Roy DM, Ghahramani Z (2015) Training generative neural networks via maximum mean discrepancy optimization. Uncertain Artif Intell - Proc 31st Conf UAI 2015 258–267. http://arxiv.org/abs/1505.03906.

  52. Sun B, Saenko K (2016) Deep CORAL: Correlation alignment for deep domain adaptation. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). pp 443–450

  53. Pan SJ, Tsang IW, Kwok JT, Yang Q (2011) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22:199–210. https://doi.org/10.1109/TNN.2010.2091281

    Article  Google Scholar 

  54. Csurka G (2017) A comprehensive survey on domain adaptation for visual applications. Adv Comput Vis Pattern Recognit:1–35. https://doi.org/10.1007/978-3-319-58347-1_1

  55. Xiao D, Huang Y, Zhao L, Qin C, Shi H, Liu C (2019) Domain adaptive motor fault diagnosis using deep transfer learning. IEEE Access 7:80937–80949. https://doi.org/10.1109/ACCESS.2019.2921480

    Article  Google Scholar 

  56. Cai H, Jia X, Feng J, Li W, Pahren L, Lee J (2020) A similarity based methodology for machine prognostics by using kernel two sample test. ISA Trans 103:112–121. https://doi.org/10.1016/j.isatra.2020.03.007

    Article  Google Scholar 

  57. Li X, Jia XD, Zhang W, Ma H, Luo Z, Li X (2020) Intelligent cross-machine fault diagnosis approach with deep auto-encoder and domain adaptation. Neurocomputing 383:235–247. https://doi.org/10.1016/j.neucom.2019.12.033

    Article  Google Scholar 

  58. da Costa PR de O, Akçay A, Zhang Y, Kaymak U (2020) Remaining useful lifetime prediction via deep domain adaptation. Reliab Eng Syst Saf 195:1–30. https://doi.org/10.1016/j.ress.2019.106682

  59. Caires Moreira L (2020) Industry 4.0: Intelligent optimisation and control for efficient and sustainable manufacturing. Coventry University, , https://pureportal.coventry.ac.uk/en/studentthesis/industry-40(113974ee-2d36-44dd-8ebe-e102a24510b4).html.

  60. Sethian JA (2002) A review of level set and fast marching methods for image processing. Modern Methods in Scientific Computing and Applications. Springer Netherlands, In, pp 365–396

    MATH  Google Scholar 

  61. Lathuiliere S, Mesejo P, Alameda-Pineda X, Horaud R (2020) A Comprehensive Analysis of Deep Regression. IEEE Trans Pattern Anal Mach Intell 42:2065–2081. https://doi.org/10.1109/TPAMI.2019.2910523

    Article  Google Scholar 

  62. Hinton GE, Srivastava N, Krizhevsky A, et al (2012) Improving neural networks by preventing co-adaptation of feature detectors. http://arxiv.org/abs/1207.0580.

  63. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444. https://doi.org/10.1038/nature14539

    Article  Google Scholar 

  64. Goodfellow I, Bengio Y, Courville A (2016) Chapter 9 - Convolutional Networks. In: Deep Learning, 1st ed. MIT Press, Cambridge, MA, pp 326–366

  65. Szegedy C, Ioffe S, Vanhoucke V, Alemi A (2016) Inception-v4, Inception-ResNet and the impact of residual connections on learning. Pattern Recogn Lett 42:11–24. https://doi.org/10.1016/j.patrec.2014.01.008

    Article  Google Scholar 

  66. Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. 3rd Int Conf Learn Represent ICLR 2015 - Conf Track Proc. http://arxiv.org/abs/1412.6980.

  67. Chen C, Chen Z, Jiang B, Jin X (2019) Joint domain alignment and discriminative feature learning for unsupervised deep domain adaptation. 33rd AAAI Conf Artif Intell AAAI 2019, 31st Innov Appl Artif Intell Conf IAAI 2019 9th AAAI Symp Educ Adv Artif Intell EAAI 2019 33:3296–3303. https://doi.org/10.1609/aaai.v33i01.33013296

  68. Gretton A, Borgwardt KM, Rasch MJ et al (2012) A Kernel two-sample test. J Mach Learn Res 13:723–773 http://jmlr.org/papers/v13/gretton12a.html

    MathSciNet  MATH  Google Scholar 

  69. Saxena A, Celaya J, Saha B et al (2009) On applying the prognostic performance metrics. Annu Conf Progn Heal Manag Soc PHM 2009 http://papers.phmsociety.org/index.php/phmconf/article/view/1621

  70. Kim N-H, An D, Choi J-H (2017) Prognostics and Health Management of Engineering Systems. Springer International Publishing, Cham. https://doi.org/10.1007/978-3-319-44742-1.

Download references

Funding

This research is funded by Coventry University, Unipart Powertrain Application Ltd. (U.K.), Institute of Digital Engineering (U.K.), and the National Natural Science Foundation of China (Project No. 51975444).

Author information

Authors and Affiliations

  1. Faculty of Engineering, Environment and Computing, Coventry University, Coventry, UK

    Mohamed Marei & Weidong Li

  2. School of Logistics Engineering, Wuhan University of Technology, Wuhan, China

    Weidong Li

Authors
  1. Mohamed Marei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weidong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Mohamed Marei is responsible for idea and methodology development, algorithm implementation and validation and manuscript writing; Weidong Li is responsible for supervision, idea and methodology discussion, and algorithm check and manuscript refinement; apart from the above contributions, Weidong Li is also responsible for funding support and manuscript finalisation.

Corresponding author

Correspondence to Weidong Li.

Ethics declarations

Ethical approval

The authors claim that there are no ethical issues involved in this research.

Consent to participate

All the authors consent to participate in this research and contribute to the research.

Consent to publish

All the authors consent to publish the research. There are no potential copyright/plagiarism issues involved in this research.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Marei, M., Li, W. Cutting tool prognostics enabled by hybrid CNN-LSTM with transfer learning. Int J Adv Manuf Technol 118, 817–836 (2022). https://doi.org/10.1007/s00170-021-07784-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-021-07784-y

Keywords

Search

Navigation