Abstract
We introduce a novel bilateral tactile telemanipulation system for dexterous robots. The system is based on a leader-follower paradigm: it integrates two leader devices, a 6D manipulandum and a 3-finger exoskeleton, which allow the human operator to directly control a robot arm and dexterous robot hand simultaneously on the follower side. Tactile sensors are mounted on the four fingertips of the robot hand to collect contact data during remote object manipulation and surface exploration; the measured tactile information is translated into haptic feedback on the leader side, enhancing the performance of the human operator by increasing transparency and reducing physical and mental fatigue during teleoperation. Thanks to its rich haptic feedback, the system is particularly interesting for blind teleoperation, where visual feedback of the workspace is not available or is limited and of poor quality, as for example in the exploration of spaces with reduced illumination and in the manipulation of objects in presence of occlusions. We report the outcomes of preliminary validation experiments, which demonstrate the effectiveness of the control methodology and provide evidence supporting the feasibility of our approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Darvish, K., et al.: Teleoperation of humanoid robots: a survey. IEEE Trans. Robot. (2023)
Vitanov, I., et al.: A suite of robotic solutions for nuclear waste decommissioning. Robotics 10(4), 112 (2021)
Handa, A., et al.: Dexpilot: vision-based teleoperation of dexterous robotic hand-arm system. In: IEEE ICRA, pp. 9164–9170 (2020)
Chi, C., Sun, X., Xue, N., Li, T., Liu, C.: Recent progress in technologies for tactile sensors. Sensors 18(4), 948 (2018)
Li, Q., Kroemer, O., Su, Z., Veiga, F.F., Kaboli, M., Ritter, H.J.: A review of tactile information: perception and action through touch. IEEE Trans. Robot. 36(6), 1619–1634 (2020)
Navarro-Guerrero, N., Toprak, S., Josifovski, J., Jamone, L.: Visuo-haptic object perception for robots: an overview. Auton. Robots 47(4), 377–403 (2023)
Omarali, B., Palermo, F., Althoefer, K., Valle, M., Farkhatdinov, I.: Tactile classification of object materials for virtual reality based robot teleoperation. In: IEEE ICRA (2022)
Omarali, B., Denoun, B., Althoefer, K., Jamone, L., Valle, M., Farkhatdinov, I.: Virtual reality based telerobotics framework with depth cameras. In: IEEE International Conference on Robot and Human Interactive Communication (2020)
Ogrinc, M., Farkhatdinov, I., Walker, R., Burdet, E.: Sensory integration of apparent motion speed and vibration magnitude. IEEE Trans. Haptics 11(3), 455–463 (2017)
Browse, R., McDonald, M.: Using tactile information in telerobotics. IEEE Trans. Syst. Man Cybern. Syst. 22(5), 1205–1210 (1992)
Fishel, J.A., et al.: Tactile telerobots for dull, dirty, dangerous, and inaccessible tasks. In: IEEE ICRA (2020)
Liu, Y., Tsai, Y.-Y., Huang, B., Guo, J.: Virtual reality based tactile sensing enhancements for bilateral teleoperation system with in-hand manipulation. IEEE Robot. Autom. Lett. 7(3), 6998–7005 (2022)
Dang, H., Weisz, J., Allen, P.K.: Blind grasping: stable robotic grasping using tactile feedback and hand kinematics. In: IEEE International Conference on Robotics and Automation 2011, pp. 5917–5922 (2011)
Lederman, S.J., Klatzky, R.L.: Hand movements: a window into haptic object recognition. Cogn. Psychol. 19(3), 342–368 (1987)
Perret, J.: Haptic trends by haption: larger, stronger, faster. In: EuroHaptics Conference (2014)
Perret, J., Parent, Q., Giudicelli, B.: Hglove: a wearable force-feedback device for the hand. In: 14th Annual EuroVR Conference (2017)
Junput, B., Wei, X., Jamone, L.: Feel it on your fingers: dataglove with vibrotactile feedback for virtual reality and telerobotics. In: Althoefer, K., Konstantinova, J., Zhang, K. (eds.) TAROS 2019. LNCS (LNAI), vol. 11649, pp. 375–385. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23807-0_31
Junput, B., Farkhatdinov, I., Jamone, L.: Touch it, rub it, feel it! haptic rendering of physical textures with a low cost wearable system. In: Mohammad, A., Dong, X., Russo, M. (eds.) TAROS 2020. LNCS (LNAI), vol. 12228, pp. 274–286. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-63486-5_29
Coppola, C., Jamone, L.: Master of puppets: multi-modal robot activity segmentation from teleoperated demonstrations. In: IEEE International Conference on Development and Learning (ICDL) (2022)
Coppola, C., Solak, G., Jamone, L.: An affordable system for the teleoperation of dexterous robotic hands using leap motion hand tracking and vibrotactile feedback. In: IEEE International Conference on Robot and Human Interactive Communication (2022)
Abbott, J.J., Okamura, A.M.: Stable forbidden-region virtual fixtures for bilateral telemanipulation. J. Dyn. Syst. Meas. Control Trans. ASME (2006)
Bonzini, A.A., Seminara, L., Macciò, S., Carfı, A., Jamone, L.: Leveraging symmetry detection to speed up haptic object exploration in robots. In: 2022 IEEE International Conference on Development and Learning (ICDL) (2022)
Rakita, D., Mutlu, B., Gleicher, M.: Relaxedik: real-time synthesis of accurate and feasible robot arm motion. In: Robotics: Science and Systems (2018)
Tomo, T.P., et al.: Covering a robot fingertip with uSkin: a soft electronic skin with distributed 3-axis force sensitive elements for robot hands. IEEE Robot. Autom. Lett. 3(1), 124–131 (2017)
Tomo, T.P., et al.: A new silicone structure for uSkin-a soft, distributed, digital 3-axis skin sensor and its integration on the humanoid robot iCub. IEEE Robot. Autom. Lett. 3(3), 2584–2591 (2018)
Kirby, E., Zenha, R., Jamone, L.: Comparing single touch to dynamic exploratory procedures for robotic tactile object recognition. IEEE Robot. Autom. Lett. 7(2), 4252–4258 (2022)
Zenha, R., Denoun, B., Coppola, C., Jamone, L.: Tactile slip detection in the wild leveraging distributed sensing of both normal and shear forces. In: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (2021)
Li, R., Wang, H., Liu, Z.: Survey on mapping human hand motion to robotic hands for teleoperation. IEEE Trans. Circuits Syst. Video Technol. 32(5), 2647–2665 (2022)
Bonzini, A.A., Seminara, L., Jamone, L.: Improving haptic exploration of object shape by discovering symmetries. In: 2022 International Conference on Robotics and Automation (ICRA), pp. 5821–5827. IEEE (2022)
Acknowledgment
This work was partially funded by EPSRC Q-Arena grant EP/V035304/1. G. Giudici is funded by the Queen Mary University of London Ph.D. studentship.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Giudici, G., Omarali, B., Bonzini, A.A., Althoefer, K., Farkhatdinov, I., Jamone, L. (2023). Feeling Good: Validation of Bilateral Tactile Telemanipulation for a Dexterous Robot. In: Iida, F., Maiolino, P., Abdulali, A., Wang, M. (eds) Towards Autonomous Robotic Systems. TAROS 2023. Lecture Notes in Computer Science(), vol 14136. Springer, Cham. https://doi.org/10.1007/978-3-031-43360-3_36
Download citation
DOI: https://doi.org/10.1007/978-3-031-43360-3_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43359-7
Online ISBN: 978-3-031-43360-3
eBook Packages: Computer ScienceComputer Science (R0)