Abstract
Shaft sleeve assembly is a common task in industrial manufacturing. The fitting approach for shaft sleeve assembly is usually interference fit, which requires significant contact forces. Conventional assembly methods, though focused on safety, often struggle to achieve high efficiency. Reinforcement learning can effectively select appropriate assembly actions through interaction with the environment, making it well-suited for shaft sleeve assembly tasks. Firstly, a comprehensive workflow for shaft sleeve assembly is formulated, including system initialization, insertion, push, and completion. Our research focuses mainly on the insertion process. Secondly, the core control algorithm adopts a deep reinforcement learning method based on the Actor-Critic architecture. The reward function includes safety reward, step length reward, and step reward. Safety reward ensures assembly security, while step length and step reward enhance assembly efficiency from different perspectives. Finally, real-world experiments on shaft sleeve assembly are conducted, including ablation experiments, parameter tuning experiments on reward function, and comparative experiments with conventional methods. The results of the ablation experiments and parameter tuning experiments indicate that after combining safety reward, step length reward, and step reward, the assembly effect achieves the best, verifying the effectiveness of the proposed reward function. Comparative experimental results demonstrate that our approach not only enhances safety compared to conventional methods but also significantly improves assembly efficiency, indicating the feasibility of this method.
Similar content being viewed by others
References
Zhu Z, Hu H (2018) Robot learning from demonstration in robotic assembly: a survey. Robotics 7(2):17
Liu S, Xing D, Li Y, Zhang J, Xu D (2019) Robust insertion control for precision assembly with passive compliance combining vision and force information. IEEE ASME Trans Mechatron 24(5):1974–1985
Haugaard RL, Buch AG, Iversen TM (2022) Self-supervised deep visual servoing for high precision peg-in-hole insertion Proceedings of 18th International Conference on Automation. Science and Engineering, IEEE, Mexico City, pp 405–410
Gai Y, Guo J, Wu D, Chen K (2021) Feature-based compliance control for precise peg-in-hole assembly. IEEE Trans Ind Electron 69(9):9309–9319
Chen Z, Xie S, Zhang X (2021) Position/force visual-sensing-based robotic sheet-like peg-in-hole assembly. IEEE Trans Instrum Meas 71:1–11
Park DI, Kim H, Park C, Choi T, Do H, Kim B, Park J (2017) Automatic assembly method with the passive compliant device Proceedings of 11th Asian Control Conference. IEEE, Gold Coast, Australia, pp 347–348
Su S, Pham DT, Ji C, Wang Y, Huang J, Zhou W, Wang H (2023) Design of a compliant device for peg-hole separation in robotic disassembly. Int J Adv Manuf 124(9):3011–3019
Park H, Bae JH, Park JH, Baeg MH, Park J (2013) Intuitive peg-in-hole assembly strategy with a compliant manipulator Proceedings of 44th International Symposium on Robotics. IEEE, Seoul, Korea, pp 1–5
Wang S, Chen G, Xu H, Wang Z (2019) A robotic peg-in-hole assembly strategy based on variable compliance center. IEEE Access 7:167534–167546
Li R, Sun C, Liu Y, Mei Y, Tan J (2023) Prediction of the parallelism error and unbalance of aero-engine rotors based on intelligent algorithm. IEEE Trans Instrum Meas 72:1–10
Su J, Li R, Qiao H, Xu J, Ai Q, Zhu J (2017) Study on dual peg-in-hole insertion using of constraints formed in the environment. Ind Robot 44(6):730–740
Jasim IF, Plapper PW, Voos H (2014) Position identification in force-guided robotic peg-in-hole assembly tasks. Procedia Cirp 23:217–222
Liu S, Xu D, Zhang D, Zhang Z (2014) High precision automatic assembly based on microscopic vision and force information. IEEE Trans Autom Sci Eng 13(1):382–393
Zhang W, Mao T, Yang R (2005) A new robotic assembly modeling and trajectory planning method using synchronized Petri nets. Int J Adv Manuf 26:420–426
Xu J, Hou Z, Wang W, Xu B, Zhang K, Chen K (2018) Feedback deep deterministic policy gradient with fuzzy reward for robotic multiple peg-in-hole assembly tasks. IEEE Trans Industr Inform 15(3):1658–1667
Thomas G, Chien M, Tamar A, Ojea JA, Abbeel P (2018) Learning robotic assembly from cad Proceedings of International Conference on Robotics and Automation. IEEE, Brisbane, Australia, pp 3524–3531
Johannink T, Bahl S, Nair A, Luo J, Kumar A, Loskyll M, Levine S (2019) Residual reinforcement learning for robot control Proceedings of International Conference on Robotics and Automation. IEEE, Montreal, Canada, pp 6023–6029
Dulac-Arnold G, Levine N, Mankowitz DJ, Li J, Paduraru C, Gowal S, Hester T (2021) Challenges of real-world reinforcement learning: definitions, benchmarks and analysis. Mach Learn 110(9):2419–2468
Hou Z, Dong H, Zhang K, Gao Q, Chen K, Xu J (2018) Knowledge-driven deep deterministic policy gradient for robotic multiple peg-in-hole assembly tasks Proceedings of International Conference on Robotics and Biomimetics. IEEE, Kuala Lumpur, Malaysia, pp 256–261
Inoue T, De Magistris G, Munawar A, Yokoya T, Tachibana R (2017) Deep reinforcement learning for high precision assembly tasks Proceedings of International Conference on Intelligent Robots and Systems. IEEE, Vancouver, pp 819–825
Sun G, Cai W, Wu Q, Zhang W, Hu F, Wang Y, Zhang Y (2020) Simulation and life analysis of impact process for directional shaft sleeve. Proceedings of International Forum on Electrical Engineering and Automation. IEEE, Hefei, pp 1021–1025
Nguyen VD, Vartanov MV, Nguyen VL (2022) Identify the position of the shaft and hole using a force-torque sensor in three-point contact assembly operations. Proceedings of International Ural Conference on Electrical Power Engineering. IEEE, Magnitogorsk, pp 295–300
Filar J, Koos V (2011) Competitive Markov decision processes. Springer, New York, pp 1–30
Funding
This work was partly supported by the National Natural Science Foundation of China (62273341).
Author information
Authors and Affiliations
Contributions
The method in this paper is firstly presented by Xumiao Ma and further improved in the way of the discussion with Professor De Xu. The manuscript was written by Xumiao Ma, and it was checked and revised by De Xu. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ma, X., Xu, D. Automated robotic assembly of shaft sleeve based on reinforcement learning. Int J Adv Manuf Technol 132, 1453–1463 (2024). https://doi.org/10.1007/s00170-024-13467-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00170-024-13467-1