Skip to main content
SpringerLink
Log in

Incremental learning of LSTM-autoencoder anomaly detection in three-axis CNC machines

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

There has been a continual effort to develop smarter, more effective CNC machines, capable of fully autonomous operation. To achieve this goal, the machines must be able to automatically detect operational and process anomalies before they cause serious damage. It has been shown that using Artificial Intelligence techniques, such as LSTM-AutoEncoders is an effective method for anomaly detection of issues such as machine chatter. Transfer learning is a valuable tool to decrease the amount of data required to implement this approach, but has lower accuracy than directly training a network on a large dataset. By implementing an incremental-ensemble of weak learners, we have been able to, not only capture changes in system dynamics over time, but incrementally improve the accuracy of a network trained through transfer learning to be comparable to a network directly trained on a large dataset. This allows us to quickly deploy networks on new systems, and obtain highly accurate anomaly estimates

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Brecher C, Chavan P, Epple A (2018) Efficient determination of stability lobe diagrams by in-process varying of spindle speed and cutting depth. Adv Manuf 6(3):272–279. https://doi.org/10.1007/s40436-018-0225-x

    Article  Google Scholar 

  2. Brecher C, Klimaschka R, Steinert A, et al (2021) Efficient determination of stability lobe diagrams deploying an automated, data-based online NC program adaption. MM Science Journal 2021-October:4830–4835. https://doi.org/10.17973/MMSJ.2021_10_2021052

  3. Burkov A (2019) The hundred-page machine learning book. Andriy Burkov

  4. Cruz YJ, Rivas M, Quiza R et al (2022) A two-step machine learning approach for dynamic model selection: a case study on a micro milling process. Comput Ind 143(103):764. https://doi.org/10.1016/J.COMPIND.2022.103764

    Article  Google Scholar 

  5. Deebak BD, Al-Turjman F (2021) Digital-twin assisted: fault diagnosis using deep transfer learning for machining tool condition. Int J Intell Syst. https://doi.org/10.1002/int.22493

    Article  Google Scholar 

  6. Elwell R, Polikar R (2011) Incremental learning of concept drift in nonstationary environments. IEEE Trans. Neural Netw 22(10):1517–1531. https://doi.org/10.1109/TNN.2011.2160459

    Article  Google Scholar 

  7. Freund Y, Schapire RE (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1):119–139. https://doi.org/10.1006/JCSS.1997.1504

    Article  MathSciNet  Google Scholar 

  8. Freund Y, Schapire RE (1999) A short introduction to boosting. J Jpn Soc Artif Intell 14(5):771–780. www.research.att.com/

  9. Gepperth A, Hammer B (2016) Incremental learning algorithms and applications. In: European symposium on artificial neural networks, bruges, Belgium. https://hal.science/hal-01418129, https://hal.science/hal-01418129/document

  10. Han X, Jin R (2020) A small sample image recognition method based on resnet and transfer learning. In: Proceedings - 2020 5th international conference on computational intelligence and applications, ICCIA 2020 pp 76–81. https://doi.org/10.1109/ICCIA49625.2020.00022

  11. Hao G, Kunpeng Z (2020) Pyramid LSTM auto-encoder for tool wear monitoring. In: 2020 IEEE 16th International Conference on automation science and engineering (CASE). IEEE, Online Zoom Meeting

  12. Hao M, Li H, Luo X et al (2020) Efficient and privacy-enhanced federated learning for industrial artificial intelligence. IEEE Trans Ind Inform 16(10):6532–6542. https://doi.org/10.1109/TII.2019.2945367

    Article  Google Scholar 

  13. Hochreiter S, Urgen Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780. http://direct.mit.edu/neco/article-pdf/9/8/1735/813796/neco.1997.9.8.1735.pdf

  14. Hsueh YW, Yang CY (2009) Tool breakage diagnosis in face milling by support vector machine. J Mater Process Technol 209(1):145–152. https://doi.org/10.1016/j.jmatprotec.2008.01.033

    Article  Google Scholar 

  15. Huong TT, Bac TP, Long DM et al (2021) Detecting cyberattacks using anomaly detection in industrial control systems: a federated learning approach. Comput Ind 132(103):509. https://doi.org/10.1016/J.COMPIND.2021.103509

    Article  Google Scholar 

  16. Jiang Y, Chen J, Zhou H, et al (2022) Contour error modeling and compensation of CNC machining based on deep learning and reinforcement learning. Int J Adv Manuf Technol. https://doi.org/10.1007/s00170-021-07895-6/Published. https://doi.org/10.1007/s00170-021-07895-6

  17. Kaushik P, Gain A, Kortylewski A, et al (2021) Understanding catastrophic forgetting and remembering in continual learning with optimal relevance mapping. In: 5th Workshop on meta-learning at NeurIPS

  18. Ke Z, Liu B, Xu H, et al (2021) CLASSIC: continual and contrastive learning of aspect sentiment classification tasks. In: EMNLP 2021 - 2021 conference on empirical methods in natural language processing, proceedings pp 6871–6883. https://doi.org/10.18653/v1/2021.emnlp-main.550. arXiv:2112.02714v1

  19. Kim YM, Shin SJ, Cho HW (2022) Predictive modeling for machining power based on multi-source transfer learning in metal cutting. Int J Precis Eng Manuf - Green Technol 9(1):107–125. https://doi.org/10.1007/s40684-021-00327-6

    Article  Google Scholar 

  20. Kounta CAKA, Arnaud L, Kamsu-Foguem B et al (2022). Review of AI-based methods for chatter detection in machining based on bibliometric analysis. https://doi.org/10.1007/s00170-022-10059-9

  21. Kuo WF, Huang BM, Lee CH (2020) Development of virtual milling system using data fusion and transfer learning. In: Proceedings - 2020 international conference on pervasive artificial intelligence, ICPAI 2020. Institute of Electrical and Electronics Engineers Inc., pp 253–257. https://doi.org/10.1109/ICPAI51961.2020.00054

  22. Li E, Bedi S, Melek W (2023) Anomaly detection in three-axis CNC machines using LSTM networks and transfer learning. Int J Adv Manuf Techol 127:5185–5198. https://doi.org/10.1007/s00170-023-11617-5

    Article  Google Scholar 

  23. Li G, Yang X, Chen D, et al (2018) Tool breakage detection using deep learning. In: Proceedings - 2018 IEEE/ACIS 3rd international conference on big data, cloud computing, data science and engineering, BCD 2018. Institute of Electrical and Electronics Engineers Inc., pp 37–42, https://doi.org/10.1109/BCD2018.2018.00014

  24. Li G, Fu Y, Chen D et al (2020) Deep anomaly detection for CNC machine cutting tool using spindle current signals. Sensors (Switzerland) 20(17):1–18. https://doi.org/10.3390/s20174896

    Article  Google Scholar 

  25. Li J, Lu J, Chen C, et al (2021) Tool wear state prediction based on feature-based transfer learning. The International Journal of Advanced Manufacturing Technology https://doi.org/10.1007/s00170-021-06780-6/Published. https://doi.org/10.1007/s00170-021-06780-6

  26. Lindemann B, Fesenmayr F, Jazdi N et al (2019) Anomaly detection in discrete manufacturing using self-learning approaches. Procedia CIRP 79:313–318. https://doi.org/10.1016/J.PROCIR.2019.02.073

    Article  Google Scholar 

  27. Liu C, Wang GF, Li ZM (2015) Incremental learning for online tool condition monitoring using Ellipsoid ARTMAP network model. Appl Soft Comput 35:186–198. https://doi.org/10.1016/J.ASOC.2015.06.023

    Article  Google Scholar 

  28. Malhotra P, Ramakrishnan A, Anand G, et al (2016) LSTM-based encoder-decoder for multi-sensor anomaly detection. 2016 Anomaly Detection Workshop. arXiv:1607.00148

  29. Mirza MJ, Masana M, Possegger H, et al (2022) An efficient domain-incremental learning approach to drive in all weather conditions. In: IEEE/CVF conference on computer vision and pattern recognition workshop, pp 3000–3010

  30. Muhlbaier MD, Topalis A, Polikar R (2009) Learn++.NC. IEEE Trans Neural Netw 20(1):152–168. https://doi.org/10.1109/TNN.2008.2008326. https://dl.acm.org/doi/10.1109/TNN.2008.2008326

  31. Narkhede P, Walambe R, Poddar S et al (2021) Incremental learning of LSTM framework for sensor fusion in attitude estimation. PeerJ Comput Sci 7:1–18. https://doi.org/10.7717/PEERJ-CS.662, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8356651/?report=abstract

  32. Park MW, Park BT, Rho HM et al (2000) Incremental Supervised learning of cutting conditons using the fuzzy ARTMAP neural network. CIRP Annals 49(1):375–378. https://doi.org/10.1016/S0007-8506(07)62968-0

    Article  Google Scholar 

  33. Polikar R, Udpa L, Udpa SS et al (2001) Learn++: an incremental learning algorithm for supervised neural networks. IEEE Trans Syst Man Cybern Part C: Appl Rev 31(4):497–508. https://doi.org/10.1109/5326.983933

    Article  Google Scholar 

  34. Postel M, Bugdayci B, Wegener K (2020) Ensemble transfer learning for refining stability predictions in milling using experimental stability states. Int J Adv Manuf Technol 107(9–10):4123–4139. https://doi.org/10.1007/s00170-020-05322-w

    Article  Google Scholar 

  35. Said Elsayed M, Le-Khac NA, Dev S, et al (2020) Network anomaly detection using LSTM based autoencoder. In: Q2SWinet 2020 - Proceedings of the 16th ACM symposium on QoS and security for wireless and mobile networks pp 37–45. https://doi.org/10.1145/3416013.3426457

  36. Tan C, Sun F, Kong T, et al (2018) A survey on deep transfer learning. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 11141 LNCS:270–279. https://doi.org/10.1007/978-3-030-01424-7_27/FIGURES/5. https://link.springer.com/chapter/10.1007/978-3-030-01424-7_27

  37. Tian C, Zhou G, Zhang J et al (2019) Optimization of cutting parameters considering tool wear conditions in low-carbon manufacturing environment. J Clean Prod 226:706–719. https://doi.org/10.1016/J.JCLEPRO.2019.04.113

    Article  Google Scholar 

  38. Unver HO, Sener B (2022) Exploring the potential of transfer learning for chatter detection. In: Procedia computer science, vol 200. Elsevier B.V., pp 151–159. https://doi.org/10.1016/j.procs.2022.01.214

  39. Van De Ven GM, Tolias AS (2019) Three scenarios for continual learning. https://github.com/GMvandeVen/continual-learning

  40. van de Ven GM, Tuytelaars T, Tolias AS (2022) Three types of incremental learning. Nature Machine Intelligence 2022 4:12 4(12):1185–1197. https://doi.org/10.1038/s42256-022-00568-3. https://www.nature.com/articles/s42256-022-00568-3

  41. Wang G, Guo Z, Qian L (2014) Online incremental learning for tool condition classification using modified Fuzzy ARTMAP network. J Intell Manuf 25(6):1403–1411. https://doi.org/10.1007/S10845-013-0738-X/TABLES/3. https://link.springer.com/article/10.1007/s10845-013-0738-x

  42. Wang H, Li M, Yue X (2021) IncLSTM: incremental ensemble LSTM model towards time series data. Comput & Electr Eng 92(107):156. https://doi.org/10.1016/J.COMPELECENG.2021.107156

    Article  Google Scholar 

  43. Wang J, Zou B, Liu M et al (2021) Milling force prediction model based on transfer learning and neural network. J Intell Manuf 32(4):947–956. https://doi.org/10.1007/S10845-020-01595-W/FIGURES/6. https://link.springer.com/article/10.1007/s10845-020-01595-w

  44. Xu G, Zhou H, Chen J (2018) CNC internal data based incremental cost-sensitive support vector machine method for tool breakage monitoring in end milling. Eng Appl Artif Intell 74:90–103. https://doi.org/10.1016/J.ENGAPPAI.2018.05.007

    Article  Google Scholar 

  45. Yang HC (2020) Roughness of milling process. https://doi.org/10.21227/rx49-xs81. https://ieee-dataport.org/open-access/roughness-milling-process

  46. Yesilli MC, Khasawneh FA, Otto A (2020) On transfer learning for chatter detection in turning using wavelet packet transform and ensemble empirical mode decomposition. CIRP J Manuf Sci Technol 28:118–135. https://doi.org/10.1016/j.cirpj.2019.11.003

    Article  Google Scholar 

  47. Yu YY, Zhang D, Zhang XM et al (2022) Online stability boundary drifting prediction in milling process: an incremental learning approach. Mech Syst Signal Process 173(109):062. https://doi.org/10.1016/J.YMSSP.2022.109062

    Article  Google Scholar 

  48. Zhao M, Yue C, Liu X (2023) Research on milling chatter identification of thin-walled parts based on incremental learning and multi-signal fusion. Int J Adv Manuf Technol 125(9–10):3925–3941. https://doi.org/10.1007/S00170-023-10944-X/FIGURES/18. https://link.springer.com/article/10.1007/s00170-023-10944-x

  49. Zhou G, Yuan S, Lu Q et al (2018) A carbon emission quantitation model and experimental evaluation for machining process considering tool wear condition. Int J Adv Manuf Technol 98:565–577. https://doi.org/10.1007/s00170-018-2281-6

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank Hurco Companies Inc for their generous support in developing this algorithm, and Perfecto Tool and Engineering for allowing your machines to be used for data collection. Your assistance was greatly appreciated.

Funding

The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.

Author information

Author notes

  1. Sanjeev Bedi and William Melek These authors contributed equally to this work.

Authors and Affiliations

  1. Department of Mechanical and Mechatronics Engineering, University of Waterloo, 200 University Avenue, Waterloo, N2L 3G1, Ontario, Canada

    Eugene Li, Yang Li, Sanjeev Bedi & William Melek

  2. R &D and Product Development, Hurco Companies, 1 Technology Way, Indianapolis, 46268, Indiana, USA

    Paul Gray

Authors
  1. Eugene Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sanjeev Bedi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. William Melek
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Paul Gray
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection and the development of the chatter indicators were performed by Yang Li. The data analysis, algorithm development and the first draft of the manuscript was written by Eugene Li and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Eugene Li.

Ethics declarations

Conflicts of interest

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, E., Li, Y., Bedi, S. et al. Incremental learning of LSTM-autoencoder anomaly detection in three-axis CNC machines. Int J Adv Manuf Technol 130, 1265–1277 (2024). https://doi.org/10.1007/s00170-023-12713-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-023-12713-2

Keywords

Search

Navigation