Abstract
Teleoperation is vital in the construction industry, allowing safe machine manipulation from a distance. However, controlling machines at a joint level requires extensive training due to their complex degrees of freedom. Task space control offers intuitive maneuvering, but precise control often requires dynamic models, posing challenges for hydraulic machines. To address this, we use a data-driven actuator model to capture machine dynamics in real-world operations. By integrating this model into simulation and reinforcement learning, a control policy for task space control is obtained. A 3t hydraulic construction machine, Brokk 170, serves as the platform for implementing the proposed approach. Through a series of experiments, the framework’s validity is established by comparing it against a well-established Jacobian-based approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Lee, H.J., Heuer, C., Brell-Cokcan, S.: Concept of a robot assisted on-site deconstruction approach for reusing concrete walls. In: ISARC. Proceedings of the International Symposium on Automation and Robotics in Construction, vol. 39. IAARC Publications (2022)
Lee, H.J., et al.: Importance of a 5G network for construction sites: limitation of WLAN in 3D sensing applications. In: Proceedings of the International Symposium on Automation and Robotics in Construction (IAARC) (2022)
Brell-Cokcan, S., Lee, H.J.: Robotics in construction. In: Ang, M.H., Khatib, O., Siciliano, B. (eds.) Encyclopedia of Robotics. Springer, Heidelberg (2022). https://doi.org/10.1007/978-3-642-41610-1_218-1
Chen, J.Y.C., Haas, E.C., Barnes, M.J.: Human performance issues and user interface design for teleoperated robots. IEEE Trans. Syst. Man Cybernet. C (Appl. Rev) 37(6), 1231–1245 (2007)
Lee, H.J., Brell-Cokcan, S.: Towards controlled semi-autonomous deconstruction. Constr. Robot. (2023). https://doi.org/10.1007/s41693-023-00111-9
Lee, H.J., Brell-Cokcan, S.: Reinforcement learning-based virtual fixtures for teleoperation of hydraulic construction machine. arXiv preprint arXiv:2306.11897 (2023)
Khatib, O.: A unified approach for motion and force control of robot manipulators: the operational space formulation. IEEE J. Robot. Autom. 3(1), 43–53 (1987)
Nakanishi, J., Cory, R., Mistry, M., Peters, J., Schaal, S.: Comparative experiments on task space control with redundancy resolution. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Edmonton, AB (2005)
Lee, H.J., Brell-Cokcan, S.: Cartesian coordinate control for teleoperated construction machines. Constr. Robot. 5, 1–11 (2021). https://doi.org/10.1007/s41693-021-00055-y
Jud, D., et al.: HEAP - the autonomous walking excavator. In: Automation in Construction, vol. 129 (2021)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. The MIT Press, Cambridge (2018)
Levine, S., Pastor, P., Krizhevsky, A., Ibarz, J., Quillen, D.: Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. Int. J. Robot. Res. 37(4–5), 421–436 (2018)
Hwangbo, J., et al.: Learning agile and dynamic motor skills for legged robots. Sci. Robot. 4(26), eaau5872 (2019)
Makoviychuk, V., et al.: Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning (2021)
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms (2017). arXiv preprint arXiv:1707.06347
Makoviichuk, D., Makoviychuk, V.: RL-games (2021)
Buss, S.R.: Introduction to inverse kinematics with Jacobian trans- pose, pseudoinverse and damped least squares methods. IEEE J. Robot. Autom. 17(1–19), 16 (2004)
Chan, T.F., Dubey, R.V.: A weighted least-norm solution based scheme for avoiding joint limits for redundant joint manipulators. IEEE Trans. Robot. Autom. 11(2), 286–292 (1995)
Nakamura, Y., Hansafusa, H.: Inverse kinematic solutions with singularity robustness for robot manipulator control. J. Dyn. Syst. Meas. Control 108(3), 163–171 (1986)
Ahn, K.K., Nam, D.N.C., Jin, M.: Adaptive backstepping control of an electrohydraulic actuator. IEEE/ASME Trans. Mechatron. 19(3), 987–995 (2014)
Acknowledgement
This work has been supported by the North Rhine-Westphalia Ministry of Economic Affairs, Innovation, Digitalisation and Energy of the Federal Republic of Germany under the research intent 5G. NAMICO: Networked, Adaptive Mining and Construction.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lee, H.J., Brell-Cokcan, S. (2024). Task Space Control of Hydraulic Construction Machines Using Reinforcement Learning. In: Piazza, C., Capsi-Morales, P., Figueredo, L., Keppler, M., Schütze, H. (eds) Human-Friendly Robotics 2023. HFR 2023. Springer Proceedings in Advanced Robotics, vol 29. Springer, Cham. https://doi.org/10.1007/978-3-031-55000-3_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-55000-3_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-54999-1
Online ISBN: 978-3-031-55000-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)