Skip to main content
SpringerLink
Log in

Enhanced tool condition monitoring using wavelet transform-based hybrid deep learning based on sensor signal and vision system

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

A new approach for enhancing the reliability and practicality of online tool condition monitoring (TCM) is introduced in the current research. This new method is based on analyzing raw force signals and processing cutting tool images. A new vision system based on a CCD camera is used to monitor the tool condition by collecting images of the rotating cutting tool during the milling operation, making it a convenient and feasible process. Firstly, image processing wear extraction based on the projection of the rotating tool is investigated in this study. This method demonstrated a high correlation with the experimental flank wear, reaching 99.37%. Then, the tool wear prediction method was developed by combining a hybrid deep learning algorithm with a raw signal multiresolution analysis method based on the wavelet transform to improve the accuracy of identifying tool wear. This method involves combining a hybrid deep learning algorithm that consists of a Convolutional Neural Network (CNN) and Bidirectional Long-Short Term Memory (BiLSTM) with Maximal Overlap Discrete Wavelet Transform (MODWT) for preprocessing signals. Cutting experiments using different tool sizes and parameters were performed on the vertical CNC milling machine. Finally, to evaluate the performance of the proposed model, its identification accuracy was compared to that of other deep learning and machine learning models. According to the experimental result and in contrast to available TCM methods, the proposed method improves the accuracy of tool wear condition recognition. The proposed model demonstrated the highest regression coefficient R compared with common prediction methods, equal to 99.5% on average.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25

Similar content being viewed by others

Data availability

The data supporting this study's findings are available from the corresponding author upon reasonable request.

References

  1. Wang M, Zhou J, Gao J, Ziqiu Li Z, Li E (2020) Milling tool wear prediction method based on deep learning under variable working conditions. IEEE Access XX:1–9. https://doi.org/10.1109/ACCESS.2020.3010378

  2. Zheng H, Lin J (2019) A deep learning approach for high speed machining tool wear monitoring. Proc 2019 3rd IEEE Int Conf Robot Autom Sci ICRAS 2019. https://doi.org/10.1109/ICRAS.2019.8809070

  3. Mohanraj T, Yerchuru J, Krishnan H, NithinAravind RS, Yameni R (2021) Development of tool condition monitoring system in end milling process using wavelet features and Hoelder’s exponent with machine learning algorithms. Meas J Int Meas Confed 173:108671. https://doi.org/10.1016/j.measurement.2020.108671

    Article  Google Scholar 

  4. Bai L, Liu H, Zhang J, Zhao W (2023) Real-time tool breakage monitoring based on dimensionless indicators under time-varying cutting conditions. Robot Comput Integr Manuf 81:102502. https://doi.org/10.1016/j.rcim.2022.102502

    Article  Google Scholar 

  5. Wang G, Guo Z, Yang Y (2013) Force sensor based online tool wear monitoring using distributed Gaussian ARTMAP network. Sensors Actuators, A Phys 192:111–118. https://doi.org/10.1016/j.sna.2012.12.029

    Article  Google Scholar 

  6. Madhusudana CK, Kumar H, Narendranath S (2017) Face milling tool condition monitoring using sound signal. Int J Syst Assur Eng Manag 8:1643–1653. https://doi.org/10.1007/s13198-017-0637-1

    Article  Google Scholar 

  7. Anahid MJ, Heydarnia H, Niknam SA, Mehmanparast H (2020) Evaluating the sensitivity of acoustic emission signal features to the variation of cutting parameters in milling aluminum alloys: part A: frequency domain analysis. Proc Inst Mech Eng Part B J Eng Manufhttps://doi.org/10.1177/0954405420949127

  8. Zhou Y, Sun W (2020) Tool wear condition monitoring in milling process based on current sensors. IEEE Access 8:95491–95502. https://doi.org/10.1109/ACCESS.2020.2995586

    Article  Google Scholar 

  9. He Z, Shi T, Xuan J, Li T (2021) Research on tool wear prediction based on temperature signals and deep learning. Wear 478–479:203902. https://doi.org/10.1016/j.wear.2021.203902

    Article  Google Scholar 

  10. Cao K, Han J, Xu L, Shi T, Liao G, Liu Z (2022) Real-time tool condition monitoring method based on in situ temperature measurement and artificial neural network in turning. Front Mech Eng 17:1–15. https://doi.org/10.1007/s11465-021-0661-3

    Article  Google Scholar 

  11. Mannan MA, Mian Z, Kassim AA (2004) Tool wear monitoring using a fast Hough transform of images of machined surfaces. Mach Vis Appl 15:156–163. https://doi.org/10.1007/s00138-004-0137-6

    Article  Google Scholar 

  12. Vaishnav S, Agarwal A (2020) Machine learning-based instantaneous cutting force model for end milling operation. J Intell Manuf 31:1353–1366. https://doi.org/10.1007/s10845-019-01514-8

    Article  Google Scholar 

  13. You Z, Gao H, Guo L, Liu Y, Li J (2020) On-line milling cutter wear monitoring in a wide field-of-view camera. Wear 460–461:. https://doi.org/10.1016/j.wear.2020.203479

  14. Yuan J, Liu L, Yang Z, Bo J, Zhang Y (2021) Tool wear condition monitoring by combining spindle motor current signal analysis and machined surface image processing. Int J Adv Manuf Technol 116:2697–2709. https://doi.org/10.1007/s00170-021-07366-y

    Article  Google Scholar 

  15. Chen M, Li M, Zhao L, Liu J (2023) Tool wear monitoring based on the combination of machine vision and acoustic emission. Int J Adv Manuf Technol 125:3881–3897. https://doi.org/10.1007/s00170-023-11017-9

    Article  Google Scholar 

  16. Goodfellow I, Bengio Y and Ac (2016) Deep Learning. The MIT Press

  17. Tercan H, Meisen T (2022) Machine learning and deep learning based predictive quality in manufacturing : a systematic review. J Intell Manuf 33:1879–1905. https://doi.org/10.1007/s10845-022-01963-8

    Article  Google Scholar 

  18. Cao D, Sun H, Zhang J, Mo R (2020) In-process tool condition monitoring based on convolution neural network. Jisuanji Jicheng Zhizao Xitong/Computer Integr Manuf Syst CIMS 26:. https://doi.org/10.13196/j.cims.2020.01.008

  19. Nguyen VT, Pham VH (2020) Deep stacked auto-encoder network based tool wear monitoring in the face milling process. J Mech Eng 66:227–234. https://doi.org/10.5545/sv-jme.2019.6285

    Article  Google Scholar 

  20. Ma J, Luo D, Liao X, Zhang Z, Huang Y, Lu J (2021) Tool wear mechanism and prediction in milling TC18 titanium alloy using deep learning. Measurement 173:108554. https://doi.org/10.1016/j.measurement.2020.108554

    Article  Google Scholar 

  21. Xia M, Li T, Shu T, Wan J, Silva C, Wang Z (2019) A two-stage approach for the remaining useful life prediction of bearings using deep neural networks. IEEE Trans Ind Informatics 15:3703–3711

    Article  Google Scholar 

  22. Rui Zhao, Ruqiang Yan JW and KM (2017) Learning to monitor machine health with convolutional bi-directional LSTM networks. Sensors (Switzerland) 17, 273:1–18https://doi.org/10.3390/s17020273

  23. An Q, Tao Z, Xu X, El Mansori M, Chen M (2020) A data-driven model for milling tool remaining useful life prediction with convolutional and stacked LSTM network. Meas J Int Meas Confed 154:107461. https://doi.org/10.1016/j.measurement.2019.107461

    Article  Google Scholar 

  24. Li B, Lu Z, Jin X, Zhao L (2023) Tool wear prediction in milling CFRP with different fiber orientations based on multi-channel 1DCNN-LSTM. J Intell Manufhttps://doi.org/10.1007/s10845-023-02164-7

  25. Ghosh N, Ravi YB, Patra A, Mukhopadhyay S, Paul S, Mohanty AR, Chattopadhyay AB (2007) Estimation of tool wear during CNC milling using neural network-based sensor fusion. Mech Syst Signal Process 21:466–479. https://doi.org/10.1016/j.ymssp.2005.10.010

    Article  Google Scholar 

  26. Patnaik B, Mishra M, Bansal RC, Jena RK (2021) MODWT-XGBoost based smart energy solution for fault detection and classification in a smart microgrid. Appl Energy 285:116457. https://doi.org/10.1016/j.apenergy.2021.116457

    Article  Google Scholar 

  27. Costa FB, Neto CMS, Carolino SF, Ribeiro RLA, Barreto RL, Rocha TOA, Pott P (2012) Comparison between two versions of the discrete wavelet transform for real-time transient detection on synchronous machine terminals. 2012 10th IEEE/IAS Int Conf Ind Appl INDUSCON 2012 1–5. https://doi.org/10.1109/INDUSCON.2012.6453533

  28. Liu D, Dysko A, Hong Q, Tzelepis D, Booth C (2022) Transient wavelet energy-based protection scheme for inverter-dominated microgrid. IEEE Trans Smart Grid 13:2533–2546. https://doi.org/10.1109/TSG.2022.3163669

    Article  Google Scholar 

  29. Ashok V, Yadav A (2020) A real-time fault detection and classification algorithm for transmission line faults based on MODWT during power swing. Int Trans Electr Energy Syst 30:1–27. https://doi.org/10.1002/2050-7038.12164

    Article  Google Scholar 

  30. Zhu L, Wang Y, Fan Q (2014) MODWT-ARMA model for time series prediction. Appl Math Model 38:1859–1865. https://doi.org/10.1016/j.apm.2013.10.002

    Article  MathSciNet  Google Scholar 

  31. Chen HY, Lee CH (2021) Deep learning approach for vibration signals applications. Sensors 21:. https://doi.org/10.3390/s21113929

  32. Hong CW, Lee K, Ko MS, Kim JK, Oh K, Hur K (2020) Multivariate time series forecasting for remaining useful life of turbofan engine using deep-stacked neural network and correlation analysis. Proc - 2020 IEEE Int Conf Big Data Smart Comput BigComp 2020 63–70. https://doi.org/10.1109/BigComp48618.2020.00-98

  33. Wu C, Jiang P, Ding C, Feng C, Chen T (2019) Intelligent fault diagnosis of rotating machinery based on one-dimensional convolutional neural network. Comput Ind 108:53–61. https://doi.org/10.1016/j.compind.2018.12.001

    Article  Google Scholar 

  34. Javed K, Gouriveau R, Li X, Zerhouni N (2018) Tool wear monitoring and prognostics challenges : a comparison of connectionist methods toward an adaptive ensemble model. J Intell Manuf 29:1873–1890. https://doi.org/10.1007/s10845-016-1221-2

    Article  Google Scholar 

  35. Tang L, Sun Y, Li B, Shen J, Meng G (2019) Wear performance and mechanisms of PCBN tool in dry hard turning of AISI D2 hardened steel. Tribol Inthttps://doi.org/10.1016/j.triboint.2018.12.026

  36. Sun XG, Sun L, Wang EH (2014) Study on joint surface parameter identification method of shaft-toolholder and toolholder-tool for vertical CNC milling machine. Mach Tool Hydraul 42:106–109

    Google Scholar 

  37. Wang B, Sun W, Wen B (2012) The finite element modeling of high-speed spindle system dynamics with spindle-holder-tool joints. Jixie Gongcheng Xuebao(Chinese J Mech Eng 48:83–89

  38. Abdeltawab A, Zhang X, Zhang L (2023) Tool wear classification based on maximal overlap discrete wavelet transform and hybrid deep learning model. Int J Adv Manuf Technolhttps://doi.org/10.1007/s00170-023-12797-w

  39. García Plaza E, Núñez López PJ (2018) Analysis of cutting force signals by wavelet packet transform for surface roughness monitoring in CNC turning. Mech Syst Signal Process 98:634–651. https://doi.org/10.1016/j.ymssp.2017.05.006

    Article  Google Scholar 

  40. Aralikatti SS, Ravikumar KN, Kumar H, Shivananda NH, SugumaranV (2020) Comparative study on tool fault diagnosis methods using vibration signals and cutting force signals by machine learning technique. SDHM Struct Durab Heal Monit 14:127–145. https://doi.org/10.32604/SDHM.2020.07595

Download references

Acknowledgements

Thanks to Shanghai WPT Company for providing all the hardware to support this study. Thanks to the Chinese government scholarship (CSC) for financial support.

Funding

This study was funded by the Chinese government scholarship (CSC).

Author information

Authors and Affiliations

  1. Mechanical Manufacture and Automation, School of Mechatronic Engineering and Automation, Shanghai University, Shanghai, 200444, People’s Republic of China

    Ahmed Abdeltawab, Zhang Xi & Zhang Longjia

  2. Production Engineering and Mechanical Design Department, Faculty of Engineering, Mansoura University, El Mansoura, 35516, Egypt

    Ahmed Abdeltawab

Authors
  1. Ahmed Abdeltawab
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhang Xi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhang Longjia
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Ahmed Abdeltawab: writing original draft, writing review editing, idea, experimental work, signal and image processing, CNC machining and programming, data analysis investigation, methodology, validation.

Zhang Xi: project administration, idea, review editing, experimental resources support, methodology, validation, and supervision.

Zhang Longjia: vision system application, experimental work, data analysis investigation, methodology, validation.

Corresponding author

Correspondence to Zhang Xi.

Ethics declarations

Consent to participate

Not applicable

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Abdeltawab, A., Xi, Z. & Longjia, Z. Enhanced tool condition monitoring using wavelet transform-based hybrid deep learning based on sensor signal and vision system. Int J Adv Manuf Technol (2024). https://doi.org/10.1007/s00170-024-13680-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00170-024-13680-y

Keywords

Search

Navigation