Abstract
This paper presents an anthropomorphic soft robotic hand integrated with multiple flexible force and flex sensors in the fingers and palm. This enables the soft hand to estimate the curvature and contact forces, as it grasps different types of objects. Here, we blend the soft sensing technology with neural network to develop an in-hand object recognition ability for the soft hand. We implemented three transfer learning based neural networks to recognize objects using the force and flex sensors data. The networks are trained with 10 items and are validated with 10 unknown items of similar shape and dimensions. This approach produced a classification accuracy of up to 88.20% on unknown test items, which was comparable to previous reports in the literature. While it proves to be an efficient learning method, we also note that the accuracy may be further enhanced by incorporating piecewise compliant grasping mechanism, which would allow the soft hand to better conform to objects of angular profiles.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Deimel, R., Brock, O.: A novel type of compliant, underactuated robotic hand for dexterous grasping. Int. J. Robot. Res. 35(1–3), 161–185 (2015)
Tavakoli, M., Almeida, A.T.D.: Adaptive under-actuated anthropomorphic hand: ISR-SoftHand. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (2014)
Godfrey, S.B., Adjoudani, A., Catalano, M., Bicchi, A.: A synergy-driven approach to a myoelectric hand. In: IEEE International Conference on Rehabilitation Robotics, pp. 1–6 (2013)
Yap, H.K., Ng, H.Y., Yeow, C.-H.: High-force soft printable pneumatics for soft robotic applications. Soft Robot. 3(3), 144–158 (2016)
Galloway, K.C., et al.: Soft robotic grippers for biological sampling on deep reefs. Soft Robot. 3(1), 23–33 (2016)
She, Y., Li, C., Cleary, J., Su, H.-J.: Design and fabrication of a soft robotic hand with embedded actuators and sensors. J. Mech. Robot. 7(2), 021007 (2015)
Renda, F., Giorelli, M., Calisti, M., Cianchetti, M., Laschi, C.: Dynamic model of a multibending soft robot arm driven by cables. IEEE Trans. Robot. 30(5), 1109–1122 (2014)
Chen, Y., Guo, S., Li, C., Yang, H., Hao, L.: Size recognition and adaptive grasping using an integration of actuating and sensing soft pneumatic gripper. Robot. Auton. Syst. 104, 14–24 (2018)
Johansson, R.S., Flanagan, J.R.: Coding and use of tactile signals from the fingertips in object manipulation tasks. Nat. Rev. Neurosci. 10(5), 345–359 (2009)
Wall, V., Zoller, G., Brock, O.: A method for sensorizing soft actuators and its application to the RBO hand 2. In: 2017 IEEE International Conference on Robotics and Automation (ICRA) (2017)
Shih, B., Drotman, D., Christianson, C., Huo, Z., White, R., Christensen, H.I., Tolley, M.T.: Custom soft robotic gripper sensor skins for haptic object visualization. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2017)
Homberg, B.S., Katzschmann, R.K., Dogar, M.R., Rus, D.: Haptic identification of objects using a modular soft robotic gripper. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2015)
Zhao, H., O’Brien, K., Li, S., Shepherd, R.F.: Optoelectronically innervated soft prosthetic hand via stretchable optical waveguides. Sci. Robot. 1(1), eaai7529 (2016)
Kazemi, M., Valois, J.-S., Bagnell, J.A., Pollard, N.: Human-inspired force compliant grasping primitives. Auton. Robot. 37(2), 209–225 (2014). https://doi.org/10.1007/s10514-014-9389-9
Liarokapis, M.V., Calli, B., Spiers, A.J., Dollar, A.M.: Unplanned, model-free, single grasp object classification with underactuated hands and force sensors. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2015)
Thuruthel, T.G., Shih, B., Laschi, C., Tolley, M.T.: Soft robot perception using embedded soft sensors and recurrent neural networks. Sci. Robot. 4(26), eaav1488 (2019)
Shirafuji, S., Hosoda, K.: Detection and prevention of slip using sensors with different properties embedded in elastic artificial skin on the basis of previous experience. Robot. Auton. Syst. 61(1), 46–52 (2014)
Rocha, R.P., Lopes, P.A., Almeida, A.T.D., Tavakoli, M., Majidi, C.: Fabrication and characterization of bending and pressure sensors for a soft prosthetic hand. J. Micromech. Microeng. 28(3), 034001 (2018)
Truby, R.L., Wehner, M., Grosskopf, A.K., Vogt, D.M., Uzel, S.G.M., Wood, R.J., Lewis, J.A.: Soft robotics: soft somatosensitive actuators via embedded 3D printing. Adv. Mater. 30(15), 1870106 (2018)
Choi, C., Schwarting, W., Delpreto, J., Rus, D.: Learning object grasping for soft robot hands. IEEE Robot. Autom. Lett. 3(3), 2370–2377 (2018)
Jamone, L., Natale, L., Metta, G., Sandini, G.: Highly sensitive soft tactile sensors for an anthropomorphic robotic hand. IEEE Sens. J. 15(8), 4226–4233 (2015)
Schmitz, A., Bansho, Y., Noda, K., Iwata, H., Ogata, T., Sugano, S.: Tactile object recognition using deep learning and dropout. In: IEEE-RAS International Conference on Humanoid Robots (2014)
Hinton, G.E., Salakhutdinov, R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)
Albini, A., Denei, S., Cannata, G.: Human hand recognition from robotic skin measurements in human-robot physical interactions. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2017)
Luo, S., Mou, W., Althoefer, K., Lili, H.: Novel tactile-SIFT descriptor for object shape recognition. IEEE Sens. J. 15(9), 5001–5009 (2015)
Gandarias, J.M., Garcia-Cerezo, A.J., Gomez-De-Gabriel, J.M.: CNN-based methods for object recognition with high-resolution tactile sensors. IEEE Sens. J. 19(16), 6872–6882 (2019)
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Navarro, S.E., Gorges, N., Worn, H., Schill, J., Asfour, T., Dillmann, R.: Haptic object recognition for multi-fingered robot hands. In: 2012 IEEE Haptics Symposium (HAPTICS) (2012)
Luo, S., Mou, W., Althoefer, K., Liu, H.: Iterative closest labeled point for tactile object shape recognition. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2016)
Gandarias, J.M., Gomez-De-Gabriel, J.M., Garcia-Cerezo, A.: Human and object recognition with a high-resolution tactile sensor. In: 2017 IEEE Sensors (2017)
Acknowledgment
The authors would like to thank Andrei Nakagawa and Aaron Goh for offering valuable advice during the experiments and National Robotics Program for supporting this project. We also gratefully acknowledge the technical support of Nvidia Corporation through the Memorandum of Understanding with the Advanced Robotics Centre of the National University of Singapore on autonomous systems technologies.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Khin, P.M., Low, J.H., Ang, M.H., Yeow, CH. (2022). In-Hand Object Recognition for Sensorized Soft Hand. In: Ang Jr, M.H., Asama, H., Lin, W., Foong, S. (eds) Intelligent Autonomous Systems 16. IAS 2021. Lecture Notes in Networks and Systems, vol 412. Springer, Cham. https://doi.org/10.1007/978-3-030-95892-3_27
Download citation
DOI: https://doi.org/10.1007/978-3-030-95892-3_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-95891-6
Online ISBN: 978-3-030-95892-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)