Abstract
Legged manipulators are a prime candidate for reducing risk to human lives through completing tasks in hazardous environments. However, controlling these systems in real-world applications requires a highly functional teleoperation framework, capable of leveraging all utility of the robot to complete tasks. In this work, such a teleoperation framework is presented, where a wearable whole-body motion capture suit is integrated with a whole-body controller specialised for teleoperation and a set of teleoperation strategies that enable the control of all main frames of the robot along with additional functions. Within the whole-body controller, all tasks and constraints can be configured dynamically due to their modularity, hence enabling seamless transitions between each teleoperation strategy. As a result, this not only enables the realisation of trajectories outside the workspace without the whole-body controller but also the ability to complete tasks that would require an additional manipulator if just the gripper frames of the robot were controllable. To validate the presented framework, a set of real robot experiments have been completed to demonstrate all teleoperation strategies and analyse their proficiency.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Sleiman, J.-P., Farshidian, F., Minniti, M.V., Hutter, M.: A Unified MPC Framework for Whole-Body Dynamic Locomotion and Manipulation. IEEE Robotics and Automation Letters 6(3), 4688–4695 (2021). https://doi.org/10.1109/LRA.2021.3068908
Li, J., Gao, H., Wan, Y., Humphreys, J., Peers, C., Yu, H., Zhou, C.: Whole-body control for a torque-controlled legged mobile manipulator. Actuators 11(11) (2022). https://doi.org/10.3390/act11110304
Xin, S., Vijayakumar, S.: Online Dynamic Motion Planning and Control for Wheeled Biped Robots. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3892–3899 (2020). https://doi.org/10.1109/IROS45743.2020.9340967
You, Y., Xin, S., Zhou, C., Tsagarakis, N.: Straight Leg Walking Sxtrategy for Torque-controlled Humanoid Robots. In: IEEE International Conference on Robotics and Biomimetics, Qingdao, China, pp. 2014–2019 (2016). https://doi.org/10.1109/ROBIO.2016.7866625
Chignoli, M., Kim, D., Stanger-Jones, E., Kim, S.: The MIT Humanoid Robot: Design, Motion Planning, and Control for Acrobatic Behaviors. In: 2020 IEEE-RAS 20th International Conference on Humanoid Robots, pp. 1–8 (2021). https://doi.org/10.1109/HUMANOIDS47582.2021.9555782
Ramuzat, N., Buondonno, G., Boria, S., Stasse, O.: Comparison of Position and Torque Whole-Body Control Schemes on the Humanoid Robot TALOS. In: 2021 20th International Conference on Advanced Robotics, pp. 785–792 (2021). https://doi.org/10.1109/ICAR53236.2021.9659380
Caron, S., Kheddar, A., Tempier, O.: Stair Climbing Stabilization of the HRP-4 Humanoid Robot using Whole-body Admittance Control. In: 2019 International Conference on Robotics and Automation, pp. 277–283 (2019). https://doi.org/10.1109/ICRA.2019.8794348
Lillo, P.D., Pierri, F., Caccavale, F., Antonelli, G.: Experiments on whole-body control of a dual-arm mobile robot with the set-based task-priority inverse kinematics algorithm. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 9096–9101 (2020). https://doi.org/10.1109/IROS45743.2020.9341638
Yang, S., Chen, H., Zhang, L., Cao, Z., Wensing, P.M., Liu, Y., Pang, J., Zhang, W.: Reachability-based Push Recovery for Humanoid Robots with Variable-Height Inverted Pendulum. In: 2021 IEEE International Conference on Robotics and Automation, pp. 3054–3060 (2021). https://doi.org/10.1109/ICRA48506.2021.9561872
Castano, J.A., Hoffman, E.M., Laurenzi, A., Muratore, L., Karnedula, M., Tsagarakis, N.G.: A Whole Body Attitude Stabilizer for Hybrid Wheeled-Legged Quadruped Robots. In: IEEE International Conference on Robotics and Automation, pp. 706–712 (2018). https://doi.org/10.1109/ICRA.2018.8462875
Zhou, C., Fang, C., Wang, X., Li, Z., Tsagarakis, N.: A generic optimization-based framework for reactive collision avoidance in bipedal locomotion. In: IEEE International Conference on Automation Science and Engineering, pp. 1026–1033 (2016). https://doi.org/10.1109/COASE.2016.7743516
Murooka, M., Chappellet, K., Tanguy, A., Benallegue, M., Kumagai, I., Morisawa, M., Kanehiro, F., Kheddar, A.: Humanoid Loco-Manipulations Pattern Generation and Stabilization Control. IEEE Robotics and Automation Letters 6(3), 5597–5604 (2021). https://doi.org/10.1109/LRA.2021.3077858
Kolvenbach, H., Wisth, D., Buchanan, R., Valsecchi, G., Grandia, R., Fallon, M., Hutter, M.: Towards autonomous inspection of concrete deterioration in sewers with legged robots. J. Field Robotics 37(8), 1314–1327 (2020) https://doi.org/10.1002/rob.21964
Coelho, A., Sarkisov, Y., Wu, X., Mishra, H., Singh, H., Dietrich, A., Franchi, A., Kondak, K., Ott, C.: Whole-Body Teleoperation and Shared Control of Redundant Robots with Applications to Aerial Manipulation. Journal of Intelligent & Robotic Systems 102 (2021). https://doi.org/10.1007/s10846-021-01365-7
Xin, G., Smith, J., Rytz, D., Wolfslag, W., Lin, H.-C., Mistry, M.: Bounded haptic teleoperation of a quadruped robot’s foot posture for sensing and manipulation. In: IEEE International Conference on Robotics and Automation, pp. 1431–1437 (2020). https://doi.org/10.1109/ICRA40945.2020.9197501
Wang, S., Ramos, J.: Dynamic Locomotion Teleoperation of a Reduced Model of a Wheeled Humanoid Robot Using a Whole-Body Human-Machine Interface. IEEE Robotics and Automation Letters 7(2), 1872–1879 (2022). https://doi.org/10.1109/LRA.2021.3138521
Xing, H., Ding, L., Gao, H., Li, W., Tavakoli, M.: Dual-User Haptic Teleoperation of Complementary Motions of a Redundant Wheeled Mobile Manipulator Considering Task Priority. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 1–13 (2022). https://doi.org/10.1109/TSMC.2022.3144009
Darvish, K., Tirupachuri, Y., Romualdi, G., Rapetti, L., Ferigo, D., Chavez, F.J.A., Pucci, D.: Whole-Body Geometric Retargeting for Humanoid Robots. In: IEEE-RAS International Conference on Humanoid Robots, pp. 679–686 (2019). https://doi.org/10.1109/Humanoids43949.2019.9035059
Risiglione, M., Sleiman, J.-P., Minniti, M.V., Çizmeci, B., Dresscher, D., Hutter, M.: Passivity-based control for haptic teleoperation of a legged manipulator in presence of time-delays. In: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5276–5281 (2021). https://doi.org/10.1109/IROS51168.2021.9636642
Lin, T.-C., Krishnan, A.U., Li, Z.: Physical Fatigue Analysis of Assistive Robot Teleoperation via Whole-body Motion Mapping. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2240–2245 (2019). https://doi.org/10.1109/IROS40897.2019.8968544
Yang, C., Chen, J., Chen, F.: Neural learning enhanced teleoperation control of baxter robot using IMU based Motion Capture. In: International Conference on Automation and Computing, pp. 389–394 (2016). https://doi.org/10.1109/IConAC.2016.7604951
Wu, Y., Balatti, P., Lorenzini, M., Zhao, F., Kim, W., Ajoudani, A.: A Teleoperation Interface for Loco-Manipulation Control of Mobile Collaborative Robotic Assistant. IEEE Robotics and Automation Letters 4(4), 3593–3600 (2019). https://doi.org/10.1109/LRA.2019.2928757
Darvish, K., Tirupachuri, Y., Romualdi, G., Rapetti, L., Ferigo, D., Chavez, F.J.A., Pucci, D.: Whole-Body Geometric Retargeting for Humanoid Robots. In: IEEE-RAS International Conference on Humanoid Robots, pp. 679–686 (2019). https://doi.org/10.1109/Humanoids43949.2019.9035059
Dalin, E., Bergonzani, I., Anne, T., Ivaldi, S., Mouret, J.-B.: Whole-body Teleoperation of the Talos Humanoid Robot: Preliminary Results. In: ICRA Workshop on Teleoperation of Dynamic Legged Robots in Real Scenarios (2021)
Arduengo, M., Arduengo, A., Colomé, A., Lobo-Prat, J., Torras, C.: Human to Robot Whole-Body Motion Transfer. In: 2020 IEEE-RAS 20th International Conference on Humanoid Robots, pp. 299–305 (2021). https://doi.org/10.1109/HUMANOIDS47582.2021.9555769
Penco, L., Scianca, N., Modugno, V., Lanari, L., Oriolo, G., Ivaldi, S.: A Multimode Txeleoperation Framework for Humanoid Loco-Manipulation: an Application for the iCub Robot. IEEE Robotics Automation Magazine 26(4), 73–82 (2019). https://doi.org/10.1109/MRA.2019.2941245
Humphreys, J., Peers, C., Li, J., Wan, Y., Sun, J., Richardson, R., Zhou, C.: Teleoperating a legged manipulator through whole-body control. In: Towards Autonomous Robotic Systems, pp. 63-77 (2022). https://doi.org/10.1007/978-3-031-15908-46
Peers, C., Motawei, M., Richardson, R., Zhou, C.: Development of a Teleoperative Quadrupedal Manipulator. In: UK-RAS Conference, Hatfield, UK, pp. 17–18 (2021). https://doi.org/10.31256/Hy7Sf7G
Nakanishi, J., Cory, R., Mistry, M., Peters, J., Schaal, S.: Operational Space Control: A Theoretical and Empirical Comparison. International Journal of Robotics Research 27(6), 737–757 (2008). https://doi.org/10.1177/0278364908091463. Max-Planck-Gesellschaft
Léziart, P.-A., Flayols, T., Grimminger, F., Mansard, N., Souères, P.: Implementation of a Reactive Walking Controller for the New Open-Hardware Qxuadruped Solo-12. In: 2021 IEEE International Conference on Robotics and Automation, pp. 5007–5013 (2021). https://doi.org/10.1109/ICRA48506.2021.9561559
Li, J., Gao, H., Wan, Y., Yu, H., Zhou, C.: A real-time planning and control framework for robust and dynamic quadrupedal locomotion. J. Bionic Eng. (2023) https://doi.org/10.1007/s42235-023-00347-9
Hartley, R., Ghaffari, M., Eustice, R.M., Grizzle, J.W.: Contact-aided invariant extended kalman filtering for robot state estimation. Int. J. Robotics Res. 39(4), 402–430 (2020). https://doi.org/10.1177/0278364919894385
Funding
This work was supported by the Engineering and Physical Sciences Research Council [grant numbers EP/R513258/1-2441459, EP/V026801/2], the Advanced Machinery and Productivity Institute [Innovate UK project number 84646] and the China Scholarship Council [grant number (2020)06120186]. For the purpose of open access, the authors have applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising from this submission.
Author information
Authors and Affiliations
Contributions
Conceptualization: Joseph Humphreys, Chengxu Zhou; Methodology: Joseph Humphreys, Christopher Peers; Formal analysis and investigation: Joseph Humphreys, Christopher Peers, Jun Li, Yuhui Wan; Writing - original draft preparation: Joseph Humphreys, Christopher Peers; Writing - review and editing: Joseph Humphreys, Christopher Peers, Jun Li, Yuhui Wan, Chengxu Zhou; Funding acquisition: Chengxu Zhou; Supervision: Chengxu Zhou
Corresponding author
Ethics declarations
Competing interest
The authors have no relevant financial or non-financial interests to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Humphreys, J., Peers, C., Li, J. et al. High Utility Teleoperation Framework for Legged Manipulators Through Leveraging Whole-Body Control. J Intell Robot Syst 108, 57 (2023). https://doi.org/10.1007/s10846-023-01866-7
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10846-023-01866-7