Skip to main content
SpringerLink
Log in

Research on the algorithm of constant force grinding controller based on reinforcement learning PPO

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

The robot grinding process requires high real-time constant force control, but it is difficult to control the grinding force stably due to the large deformation of the robot end due to its low stiffness. To reduce the influence of low robot stiffness, positioning error and other factors on the actual grinding force instability, we proposed a constant force grinding controller algorithm based on reinforcement learning PPO. Firstly, we introduce the robot surface workpiece grinding platform and analyze the force of the grinding model. Then, a robot constant force grinding controller based on PPO was proposed to solve the grinding force instability problem of the arbitrarily curved workpiece. We described the correction of constant grinding force as a Markov decision process, and a neural network with a lightweight structure was designed to improve the response-ability of constant force control. The reward function was fitted according to prior grinding data. The actor can output displacement compensation in real-time according to the force of the sensor. Finally, we proposed a method of contour trajectory compensation method based on a single-point laser displacement sensor. We made through mobile robot sensor that scans the surface of the workpiece, using least squares to scan data fitting the polynomial can be used to represent the workpiece contour. The experimental results show that the grinding normal force is more stable and closer to the expected value, and the roughness value of the machined surface decreases. We also chose other methods for comparison; the standard deviation of grinding force is reduced by 31.9% and 58.33% respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Guo W, Zhu Y, He X (2020) A robotic grinding motion planning methodology for a novel automatic seam bead grinding robot manipulator. IEEE Access 8:75288–75302. https://doi.org/10.1109/ACCESS.2020.2987807

    Article  Google Scholar 

  2. Yang H-Y, Lian F-L (2021) Characterization and modeling of grinding speed and belt wear condition for robotic grinding process international symposium on system integration, pp 66–71. https://doi.org/10.1109/ieeeconf49454.2021.9382666

  3. Ma Z, Hong G-S, Ang MH, Poo A-N, Lin W (2018) A force control method with positive feedback for industrial finishing applications. Conference on Advanced Intelligent Mechatronics (AIM), pp 810-815. https://doi.org/10.1109/aim.2018.8452689

  4. Yang G, Zhu R, Fang Z, Chen C-Y, Zhang C (2020) Kinematic design of a 2R1T robotic end-effector with flexure joints. IEEE Access 8:57204–57213. https://doi.org/10.1109/access.2020.2982185

    Article  Google Scholar 

  5. Kuo Y-L, Huang S-Y, Lan C-C (2019) Sensorless force control of automated grinding/deburring using an adjustable force regulation mechanism. International Conference on Robotics and Automation (ICRA), pp 9489–9495. https://doi.org/10.1109/icra.2019.8794058

  6. Zhang H, Li L, Zhao J, Zhao J, Liu S, Wu J (2020) Design and implementation of hybrid force/position control for robot automation grinding aviation blade based on fuzzy PID. Int J Adv Manuf Technol 107:1741–1754. https://doi.org/10.1007/s00170-020-05061-y

    Article  Google Scholar 

  7. Jong Hyeon Park. Impedance control for biped robot locomotion. (2001) IEEE Transactions on Robotics and Automation 17:870–882.https://doi.org/10.1109/70.976014

  8. Zheng L, Rao P, Li Y, Zhao M (2019) Admittance control based humanoid robot standing balance control. IEEE Work Adv Robot Soc Impacts. https://doi.org/10.1109/arso46408.2019.8948802

    Article  Google Scholar 

  9. Zhang QW, Han LL, Xu F, Jia K (2012) Research on velocity servo-based hybrid position/force control scheme for a grinding robot. Adv Mater Res 589–593. https://doi.org/10.4028/www.scientific.net/AMR.490-495.589

  10. Xu G, Wang Z, Zhang J, Yang B, Wang Z, Xu Y (2020) Compliance control of deburring robots based on force impedance. Chinese Automation Congress (CAC):79–84. https://doi.org/10.1109/cac51589.2020.9327372

  11. He W, Ge W, Li Y, Liu Y-J, Yang C, Sun C (2017) Model identification and control design for a humanoid robot. IEEE Trans Syst, Man, Cybern: Syst 47:45–57

    Article  Google Scholar 

  12. Adachi K, Minami M, Yanou A (2013) Improvement of dynamic characteristics during the transient response of force-sensorless grinding robot by force/position control. Int Conf Mechatron Autom: 710–715. https://doi.org/10.1109/icma.2013.6618003

  13. Liu F-C, Liang L-H, Gao J-J (2014) Fuzzy PID control of space manipulator for both ground alignment and space applications. Int J Autom Comput 11:353–360. https://doi.org/10.1007/s11633-014-0800-y

    Article  Google Scholar 

  14. King MR, Haussler KK, Kawcak CE, Mcilwraith CW, Reiser RF, Frisbie DD, Werpy NM (2017) Biomechanical and histologic evaluation of the effects of underwater treadmill exercise on horses with experimentally induced osteoarthritis of the middle carpal joint. Am J Vet Res 78:558–569. https://doi.org/10.2460/ajvr.78.5.558

    Article  Google Scholar 

  15. Tian X, Huissoon JP, Xu Q, Peng B (2008) Dimensional error analysis and its intelligent pre-compensation in cnc grinding. Int J Adv Manuf Technol 36:28–33. https://doi.org/10.1007/s00170-006-0813-y

    Article  Google Scholar 

  16. Li S, Huang M, Shi Z (2020) Surface tracking system based on closed-loop force control of manipulator. Int Conf Artif Intell Electromech Autom: 292–298. https://doi.org/10.1109/aiea51086.2020.00068

  17. Solanes JE, Gracia L, Muñoz-Benavent P, Valls Miro J, Girbés V, Tornero J (2018) Human-robot cooperation for robust surface treatment using non-conventional sliding mode control. ISA Trans 80:528–541. https://doi.org/10.1016/j.isatra.2018.05.013

    Article  Google Scholar 

  18. Sun M, Guo K, Sun J (2021) Research on robot grinding force control method, in: Lecture Notes in Computer Science. Lecture Notes in Computer Science, pp 821–829. https://doi.org/10.1007/978-3-030-89098-8_77

  19. Zhou P, Zhou Y, Xie Q, Zhao H (2019) Adaptive force control for robotic grinding of complex blades. IOP Conf Ser: Mater Sci Eng 692:012008. https://doi.org/10.1088/1757-899x/692/1/012008

    Article  Google Scholar 

  20. Jia L, Wang Y, Zhang C, Zhao K, Zhou L (2019) Machine learning–based robust trajectory tracking control for FSGR. J Eng 2019:9220–9225. https://doi.org/10.1049/joe.2018.9220

    Article  Google Scholar 

  21. Song Y, Liang W, Yang Y (2012) A method for grinding removal control of a robot belt grinding system. J Intell Manuf 23:1903–1913. https://doi.org/10.1007/s10845-011-0508-6

    Article  Google Scholar 

  22. Jin X, Wang Z (2022) Proximal policy optimization based dynamic path planning algorithm for mobile robots. Electron Lett 58:13–15. https://doi.org/10.1049/ell2.12342

    Article  Google Scholar 

  23. Haiqiang G, Qi S, Pengfei W, Xinhe L (2019) Application of laser scanning in workpiece surface testing. J Appl Opt 40:686–691. https://doi.org/10.5768/JAO201940.0407002

    Article  Google Scholar 

  24. Xiao J, Zeng F, Zhang Q, Liu H (2019) Research on the forcefree control of cooperative robots based on dynamic parameters identification. Indust Robot: Int J Robot Res Appl 46:499–509. https://doi.org/10.1108/ir-01-2019-0007

    Article  Google Scholar 

  25. Tian Y, Wang B, Liu J, Shen H, Xi F, Li L (2018) Stiffness modeling and analysis of a multiple coordinated robot system. Int J Adv Manuf Technol 94:4265–4276. https://doi.org/10.1007/s00170-017-1085-4

    Article  Google Scholar 

  26. Zhang T, Yuan C, Zou YB (2022) Online optimization method of controller parameters for robot constant force grinding based on deep reinforcement learning Rainbow. J Intell Robot Syst 5:8. https://doi.org/10.1007/s10846-022-01688-z

    Article  Google Scholar 

Download references

Funding

This work was supported by the Science and Technology Planning Project of Guangdong Province [grant numbers 2020A0103010, 2021B0101420003].

Author information

Authors and Affiliations

  1. School of Mechanical and Automotive Engineering, South China University of Technology, Guangzhou, People’s Republic of China

    Tie Zhang, Chao Yuan & Yanbiao Zou

Authors
  1. Tie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chao Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanbiao Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Tie Zhang, Chao Yuan, and Yanbiao Zou. The first draft of the manuscript was written by Chao Yuan, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Tie Zhang.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, T., Yuan, C. & Zou, Y. Research on the algorithm of constant force grinding controller based on reinforcement learning PPO. Int J Adv Manuf Technol 126, 2975–2988 (2023). https://doi.org/10.1007/s00170-023-11129-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-023-11129-2

Keywords

Search

Navigation