Abstract
This paper introduces a passivity-based control framework for multi-task time-delayed bilateral teleoperation and shared control of kinematically-redundant robots. The proposed method can be seen as extension of state-of-the art hierarchical whole-body control as it allows for some of the tasks to be commanded by a remotely-located human operator through a haptic device while the others are autonomously performed. The operator is able to switch among tasks at any time without compromising the stability of the system. To enforce the passivity of the communication channel as well as to dissipate the energy generated by the null-space projectors used to enforce the hierarchy among the tasks, the Time-Domain Passivity Approach (TDPA) is applied. The efficacy of the approach is demonstrated through its application to the DLR Suspended Aerial Manipulator (SAM) in a real telemanipulation scenario with variable time delay, jitter, and package loss.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Availability of Data and Materials
The authors consent to the publication of the supplementary video, which is also available at https://youtu.be/_mXeeqcUHws.
References
Andaluz, V.H., Salinas, L., Roberti, F., Toibero, J.M., Carelli, R.: Switching Control Signal for Bilateral Tele-Operation of a Mobile Manipulator. In: 2011 9th IEEE International Conference on Control and Automation (ICCA), pp 778–783 (2011)
Anderson, R.J., Spong, M.W.: Bilateral control of teleoperators with time delay. IEEE Trans. Autom. Cont. 34(5), 494–501 (1989)
Anderson, R.J., Spong, M.W.: Asymptotic stability for force reflecting teleoperators with time delay. The Int. J. Robot. Res. 11(2), 135–149 (1992)
Artigas, J., Balachandran, R., Riecke, C., Stelzer, M., Weber, B., Ryu, J.H., Albu-Schaeffer, A.: Kontur-2: Force-Feedback Teleoperation from the International Space Station. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp 1166–1173. IEEE (2016)
Artigas, J., Ryu, J.H., Preusche, C.: Position drift compensation in time domain passivity based teleoperation. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 4250–4256. IEEE (2010)
Balachandran, R., Artigas, J., Mehmood, U., Ryu, J.H.: Performance comparison of wave variable transformation and time domain passivity approaches for Time-Delayed Teleoperation: Preliminary Results. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 410–417. IEEE (2016)
Balachandran, R., Mishra, H., Cappelli, M., Weber, B., Secchi, C., Ott, C., Albu-Schaeffer, A.: Adaptive Authority Allocation in Shared Control of Robots Using Bayesian Filters. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp 11298–11304 (2020)
Balachandran, R., Ryu, J.H., Jorda, M., Ott, C., Albu-Schaeffer, A.: Closing the force loop to enhance transparency in time-delayed teleoperation. In: 2020 IEEE International Conference on Robotics and Automation (ICRA). https://doi.org/10.1109/ICRA40945.2020.9197420, pp 10198–10204 (2020)
Bullo, F., Murray, R.M.: Proportional derivative (Pd) control on the euclidean group. In: European Control Conference, vol. 2, pp 1091–1097 (1995)
Chawda, V., Van Quang, H., O’Malley, M.K., Ryu, J.H.: Compensating position drift in time domain passivity approach based teleoperation. In: 2014 IEEE Haptics Symposium, pp 195–202. IEEE (2014)
Chopra, N., Spong, M.W.: On synchronization of networked passive systems with time delays and application to bilateral teleoperation. In: proceedings of the SICE annual conference, pp 3424–3429 (2005)
Coelho, A., Ott, C., Singh, H., Lizarralde, F., Kondak, K.: Multi-Dof time domain passivity approach based drift compensation for telemanipulation. In: 2019 19th International Conference on Advanced Robotics (ICAR), pp 695–701 (2019)
Coelho, A., Singh, H., Kondak, K., Ott, C.: Whole-body bilateral teleoperation of a redundant aerial manipulator. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp 9150–9156 (2020)
Coelho, A., Singh, H., Muskardin, T., Balachandran, R., Kondak, K.: Smoother Position-drift compensation for time domain passivity approach based teleoperation. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 5525–5532. IEEE (2018)
Colgate, J.E., Schenkel, G.G.: Passivity of a class of sampled-data systems: Application to haptic interfaces. J. Robot. Syst. 14(1), 37–47 (1997)
Dietrich, A., Ott, C.: Hierarchical impedance-based tracking control of kinematically redundant robots. IEEE Trans. Robot. 36(1), 204–221 (2020)
Dietrich, A., Ott, C., Albu-Schäffer, A.: An overview of null space projections for redundant, torque-controlled robots. The Int. J. Robot. Res. 34(11), 1385–1400 (2015)
Dietrich, A., Ott, C., Stramigioli, S.: Passivation of projection-based null space compliance control via energy tanks. IEEE Robot. Autom. Lett. 1(1), 184–191 (2016)
Dietrich, A., Wu, X., Bussmann, K., Ott, C., Albu-Schäffer, A., Stramigioli, S.: Passive hierarchical impedance control via energy tanks. IEEE Robot. Autom. Lett. 2(2), 522–529 (2016)
Farkhatdinov, I., Ryu, J.H.: Switching of control signals in teleoperation systems: Formalization and application. In: 2008 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp 353–358. IEEE (2008)
Gabellieri, C., Sarkisov, Y.S., Coelho, A., Pallottino, L., Kondak, K., Kim, M.J.: Compliance control of a cable-suspended aerial manipulator using hierarchical control framework. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (2020)
Hannaford, B., Ryu, J.H.: Time-domain passivity control of haptic interfaces. IEEE Trans. Robot. Autom. 18(1), 1–10 (2002)
Iskandar, M., Quere, G., Hagengruber, A., Dietrich, A., Vogel, J.: Employing whole-body control in assistive robotics. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 5643–5650. IEEE (2019)
Khalil, H.K., Grizzle, J.W.: Nonlinear Systems, vol. 3. Prentice Hall, Upper Saddle River (2002)
Khatib, O.: A unified approach for motion and force control of robot manipulators: The operational space formulation. IEEE J. Robot. Autom. 3(1), 43–53 (1987)
Lawrence, D.A.: Stability and transparency in bilateral teleoperation. IEEE Trans. Robot. Autom. 9(5), 624–637 (1993)
Lee, H., Ryu, J.H., Lee, J., Oh, S.: Passivity controller based on load-side damping assignment for high stiffness controlled series elastic actuators. IEEE Trans. Ind. Electron. 68(1), 871–881 (2021). https://doi.org/10.1109/TIE.2020.3013751
Lee, J., Balachandran, R., Sarkisov, Y., De Stefano, M., Coelho, A., Shinde, K., Kim, M.J., Triebel, R., Kondak, K.: Visual-inertial telepresence for aerial manipulation. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp 1222–1229 (2020)
Liu, Y.C., Chopra, N.: Semi-autonomous teleoperation in task space with redundant slave robot under communication delays. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 679–684. IEEE (2011)
Malysz, P., Sirouspour, S.: Trilateral teleoperation control of kinematically redundant robotic manipulators. Int. J. Robot. Res. 30(13), 1643–1664 (2011)
Murray, R.M.: A Mathematical Introduction to Robotic Manipulation. CRC Press, Boca Raton (1994)
Muskardin, T., Coelho, A., Noce, E.R.D., Ollero, A., Kondak, K.: Energy-based cooperative control for landing fixed-wing uavs on mobile platforms under communication delays. IEEE Robot. Autom. Lett. 5(4), 5081–5088 (2020). https://doi.org/10.1109/LRA.2020.3005374
Nakamura, Y., Hanafusa, H., Yoshikawa, T.: Task-priority based redundancy control of robot manipulators. Int. J. Robot. Res. 6(2), 3–15 (1987)
Niemeyer, G., Preusche, C., Stramigioli, S., Lee, D.: Telerobotics. In: Springer Handbook of Robotics, pp 1085–1108. Springer (2016)
Niemeyer, G., Slotine, J.J.: Stable adaptive teleoperation. IEEE J. Oceanic Eng. 16(1), 152–162 (1991)
Ollero, A., Heredia, G., Franchi, A., Antonelli, G., Kondak, K., Sanfeliu, A., Viguria, A., Martinez-de Dios, J.R., Pierri, F., Cortes, J., Santamaria-Navarro, A., Trujillo Soto, M.A., Balachandran, R., Andrade-Cetto, J., Rodriguez, A.: The aeroarms project: Aerial robots with advanced manipulation capabilities for inspection and maintenance. IEEE Robot. Autom. Magazine 25(4), 12–23 (2018)
Ott, C., Artigas, J., Preusche, C.: Subspace-oriented energy distribution for the time domain passivity approach. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 665–671. IEEE (2011)
Ott, C., Dietrich, A., Albu-Schäffer, A.: Prioritized multi-task compliance control of redundant manipulators. Automatica 53, 416–423 (2015)
Ott, C., Roa, M.A., Hirzinger, G.: Posture and balance control for biped robots based on contact force optimization. In: 2011 11th IEEE-RAS International Conference on Humanoid Robots, pp 26–33 (2011)
Panzirsch, M., Artigas, J., Ryu, J.H., Ferre, M.: Multilateral control for delayed teleoperation. In: 2013 16th International Conference on Advanced Robotics (ICAR), pp 1–6. IEEE (2013)
Park, J., Khatib, O.: Robust haptic teleoperation of a mobile manipulation platform. In: Exp Robot IX, pp 543–554. Springer (2006)
Ryu, J.H., Artigas, J., Preusche, C.: A passive bilateral control scheme for a teleoperator with time-varying communication delay. Mechatronics 20(7), 812–823 (2010)
Sarkisov, Y., Kim, M.J., Coelho, A., Tsetserukou, D., Ott, C., Kondak, K.: Optimal oscillation damping control of cable-suspended aerial manipulator with a single imu sensor. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp 5349–5355 (2020)
Sarkisov, Y.S., Kim, M.J., Bicego, D., Tsetserukou, D., Ott, C., Franchi, A., Kondak, K.: Development of sam: Cable-suspended aerial manipulator. In: 2019 IEEE International Conference on Robotics and Automation (ICRA) (2019)
Siciliano, B., Slotine, J.J.: A general framework for managing multiple tasks in highly redundant robotic systems. In: 5th International Conference on Advanced Robotics (ICAR), pp 1211–1216 (1991)
Su, H., Yang, C., Ferrigno, G., De Momi, E.: Improved human–robot collaborative control of redundant robot for teleoperated minimally invasive surgery. IEEE Robot. Autom. Lett. 4(2), 1447–1453 (2019)
Sun, D., Liao, Q., Ren, H.: Type-2 fuzzy modeling and control for bilateral teleoperation system with dynamic uncertainties and time-varying delays. IEEE Trans. Ind. Electron. 65(1), 447–459 (2018). https://doi.org/10.1109/TIE.2017.2719604
Sun, D., Naghdy, F., Du, H.: Neural network-based passivity control of teleoperation system under time-varying delays. IEEE Trans. Cybern. 47(7), 1666–1680 (2016)
Zhang, S., Fasse, E.D.: Spatial compliance modeling using a quaternion-based potential function method. Multibody Syst. Dynam. 4(1), 75–101 (2000)
Author information
Authors and Affiliations
Contributions
A. Coelho: main author, developed most of the theory presented in the paper, conducted experiments, analyzed the data, prepared the video, and wrote the paper. Y. Sarkisov: responsible for issues related to the Suspended Aerial Manipulator (SAM) and for the experimental validation. X. Wu: contributed to the development of the whole-body control section and to the one-port passivation of the local controller. H.Mishra: contributed to development of the theory related to Lie groups, especially the drift compensator and its convergence analysis. H. Singh: contributed to the multi- DoF time-domain passivity control section. A. Dietrich: contributed to the whole-body control section. A. Franchi: contributed to the final analysis of the correctness and conformity of the theory. K. Kondak: orchestrated the entire work, especially the experimental validation. C. Ott: contributed to the development of the drift compensator theory and to bringing together TDPA and whole-body control theory with a unified and mathematically sound notation.
All authors commented on previous versions of the manuscript. All authors approved the final manuscript.
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
(MP4 83.6 MB)
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Coelho, A., Sarkisov, Y., Wu, X. et al. Whole-Body Teleoperation and Shared Control of Redundant Robots with Applications to Aerial Manipulation. J Intell Robot Syst 102, 14 (2021). https://doi.org/10.1007/s10846-021-01365-7
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10846-021-01365-7