Skip to main content
SpringerLink
Log in

Reinforcement learning–based tool orientation optimization for five-axis machining

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

Tool orientation planning is an essential process in five-axis machining for sculptured parts with complex cavity features. Unlike traditional tool orientation optimization approaches using heuristic algorithms, which demand particularly prolonged time to achieve converged optimal result, this paper presents a novel method for tool orientation optimization based on reinforcement learning algorithm. As input to the method, the rasterized feasible region is first computed via support vector machine based on an active learning fashion that requires highly reduced sampled points. The tool orientation optimization is then converted into a reinforcement learning task, in which a soft actor-critic model is utilized and trained to obtain the optimal policy. According to preliminary testing results, the proposed method is proved to be feasible for tool orientation optimization problem, and effective to produce comparable results more efficiently compared with graph-based optimization method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

References

  1. Chu CH, Huang WN, Li YW (2012) An integrated framework of tool path planning in 5-axis machining of centrifugal impeller with split blades. J Intell Manuf 23:687–698

    Article  Google Scholar 

  2. Xu JT, Zhang DY, Sun YW (2019) Kinematics performance oriented smoothing method to plan tool orientations for 5-axis ball-end CNC machining. Int J Mech Sci 157–158:293–303

    Article  Google Scholar 

  3. Balasubramaniam M, Laxmiprasad P, Sarma S, Shaikh Z (2000) Generating 5-axis NC roughing paths directly from a tessellated representation. Comput Aided Des 32:261–277

    Article  Google Scholar 

  4. Wang N, Tang K (2007) Automatic generation of gouge-free and angular-velocity-compliant five-axis toolpath. Comput Aided Des 39:841–852

    Article  Google Scholar 

  5. Morishige K, Takeuchi Y (1997) 5-axis control rough cutting of an impeller with efficiency and accuracy. In: Proceedings of International Conference on Robotics and Automation. IEEE, pp 1241–1246

  6. Tomas LP (1983) Spatial planning: a configuration space approach. IEEE Transactions on Computers C-32:108–120

  7. Bajaj CL, Kim MS (1989) Generation of configuration space obstacles: the case of moving algebraic curves. Algorithmica 4:157–172

    Article  MathSciNet  Google Scholar 

  8. Choi BK, Kim DH, Jerard RB (1997) C-space approach to tool-path generation for die and mould machining. Comput Aided Des 29:657–669

    Article  Google Scholar 

  9. Choi BK (2001) C-space approach to tool-path generation for sculptured surface machining. In: Kimura F (ed) Geometric Modelling. Springer, US, Boston, MA, pp 85–97

    Chapter  Google Scholar 

  10. Boyse JW (1979) Interference detection among solids and surfaces. Commun ACM 22:3–9

    Article  Google Scholar 

  11. Bobrow JE, Dubowsky S, Gibson JS (1985) Time-optimal control of robotic manipulators along specified paths. Int J Rob Res 4:3–17

    Article  Google Scholar 

  12. Moore M, Wilhelms J (1988) Collision detection and response for computer animation. ACM SIGGRAPH Computer Graphics 22:289–298

    Article  Google Scholar 

  13. Chen Y, Shen LY, Yuan CM (2011) Collision and intersection detection of two ruled surfaces using bracket method. Computer Aided Geometric Design 28:114–126

    Article  MathSciNet  Google Scholar 

  14. Jun CS, Cha K, Lee YS (2003) Optimizing tool orientations for 5-axis machining by configuration-space search method. Comput Aided Des 35:549–566

    Article  Google Scholar 

  15. Plakhotnik D, Lauwers B (2014) Graph-based optimization of five-axis machine tool movements by varying tool orientation. Int J Adv Manuf Technol 74:307–318

    Article  Google Scholar 

  16. Mi ZP, Yuan CM, Ma XH, Shen LY (2017) Tool orientation optimization for 5-axis machining with C-space method. Int J Adv Manuf Technol 88:1243–1255

    Article  Google Scholar 

  17. Yuan CM, Mi ZP, Jia XH, Lin FM, Shen LY (2021) Tool orientation optimization and path planning for 5-axis machining. J Syst Sci Complex 34:83–106

    Article  Google Scholar 

  18. Wang YP, Xu JT, Sun YW (2021) Tool orientation adjustment for improving the kinematics performance of 5-axis ball-end machining via CPM method. Robotics and Computer-Integrated Manufacturing 68:102070

  19. Lu YA, Wang CY, Sui JB, Zheng LJ (2018) Smoothing rotary axes movements for ball-end milling based on the gradient-based differential Evolution Method. J Manuf Sci Eng 140

  20. Zou Q, Zhao JB (2013) Iso-parametric tool-path planning for point clouds. Comput Aided Des 45:1459–1468

    Article  Google Scholar 

  21. Suresh K, Yang DCH (1994) Constant scallop-height machining of free-form surfaces. J Ind Eng Int 116:253–259

    Article  Google Scholar 

  22. Balabokhin A, Tarbutton J (2017) Iso-scallop tool path building algorithm “based on tool performance metric” for generalized cutter and arbitrary milling zones in 3-axis CNC milling of free-form triangular meshed surfaces. J Manuf Process 28:565–572

    Article  Google Scholar 

  23. Morishige K, Kase K, Yoshimi Y (1997) Collision-free tool path generation using 2-dimensional C-space for 5-axis control machining. Int J Adv Manuf Technol 13:393–400

    Article  Google Scholar 

  24. Boser BE, Guyon IM, Vapnik VN (1992) A training algorithm for optimal margin classifiers. In: Haussler D (ed) Proceedings of the fifth annual workshop on Computational learning theory - COLT '92. ACM Press, New York, New York, USA, pp 144–152

  25. Pan J, Chitta S, Manocha D (2012) FCL: A general purpose library for collision and proximity queries. In: 2012 IEEE International Conference on Robotics and Automation. IEEE, pp 3859–3866

  26. Finkel RA, Bentley JL (1974) Quad trees a data structure for retrieval on composite keys. Acta Informatica 4:1–9

    Article  Google Scholar 

  27. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. IEEE Trans Neural Netw 9:1054

    Article  Google Scholar 

  28. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, Petersen S, Beattie C, Sadik A, Antonoglou I, King H, Kumaran D, Wierstra D, Legg S, Hassabis D (2015) Human-level control through deep reinforcement learning. Nature 518:529–533

    Article  Google Scholar 

  29. Haarnoja T, Zhou A, Abbeel P, Levine S (2018) Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor

Download references

Funding

This work was supported by the National Key R&D Programs of China (Grant No. 2020YFA0713704), the National Science Fund for Distinguished Young Scholars (Grant No. 51925505), and the National Natural Science Foundation of China (Grant No. 51805260).

Author information

Authors and Affiliations

  1. Nanjing University of Aeronautics and Astronautics, Nanjing, China

    Yu Zhang, Yingguang Li & Ke Xu

Authors
  1. Yu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yingguang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ke Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the research, writing, and reviewing of the paper.

Corresponding author

Correspondence to Yingguang Li.

Ethics declarations

Ethics approval

Not applicable.

Consent to participate

Not applicable.

Consent for publication

All authors have agreed to authorship, read and approved the manuscript, and given consent for submission and subsequent publication of the manuscript. The authors guarantee that the contribution to the work has not been previously published elsewhere.

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Li, Y. & Xu, K. Reinforcement learning–based tool orientation optimization for five-axis machining. Int J Adv Manuf Technol 119, 7311–7326 (2022). https://doi.org/10.1007/s00170-022-08668-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-022-08668-5

Keywords

Search

Navigation