Skip to main content
SpringerLink
Log in

Tactile object recognition in early phases of grasping using underactuated robotic hands

  • Original Research Paper
  • Published:
Intelligent Service Robotics Aims and scope Submit manuscript

Abstract

Identifying objects during the early phases of robotic grasping in unstructured environments is a crucial step toward successful dexterous robotic manipulation. Underactuated hands are versatile and quickly conform to unknown object surfaces to ensure a firm grasp. The trade-off of using such hands is that extracting information and recognizing objects is challenging due to the uncertainty introduced by the hand’s flexibility and unexpected object movements under manipulation. Combining tactile sensors and machine learning models can provide valuable information about manipulated objects to overcome such drawbacks. The present paper explores tactile object identification under two situations: single grasp, analogous to the haptic glance in humans, and through brief exploratory procedures where a robotic thumb displaces the grasped object to excite the sensors. In both scenarios, a fuzzy controller ensures that data collection occurs under approximately the same conditions in terms of forces and vibrations. Machine learning methods used for the single-grasp and short-exploratory data confirm that the former can improve object recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. de Oliveira TEA, Cretu A-M, da Fonseca VP, Petriu EM (2015) Touch sensing for humanoid robots. IEEE Instrum Meas Mag 18(5):13–19. https://doi.org/10.1109/MIM.2015.7271221

    Article  Google Scholar 

  2. Zou L, Ge C, Wang ZJ, Cretu E, Li X (2017) Novel tactile sensor technology and smart tactile sensing systems: a review. Sensors 17(11):2653

    Article  Google Scholar 

  3. Hammond FL, Weisz J, De La Llera Kurth AA, Allen PK, Howe RD (2012) Towards a design optimization method for reducing the mechanical complexity of underactuated robotic hands. In: Proceedings—IEEE international conference on robotics and automation, pp 2843–2850. https://doi.org/10.1109/ICRA.2012.6225010

  4. Lederman SJ, Klatzky RL (2009) Haptic perception: a tutorial. Atten Percept Psychophys 71(7):1439–1459. https://doi.org/10.3758/APP.71.7.1439

    Article  Google Scholar 

  5. Lederman SJ, Klatzky RL (1993) Extracting object properties through haptic exploration. Acta Physiol (Oxf) 84(1):29–40. https://doi.org/10.1016/0001-6918(93)90070-8

    Article  Google Scholar 

  6. Klatzky RL, Lederman S, Reed C (1987) There’s more to touch than meets the eye—the salience of object attributes for haptics with and without vision. J Exp Psychol-Gen 116(4):356–369

    Article  Google Scholar 

  7. Lederman SJ, Summers C, Klatzky RL (1996) Cognitive salience of haptic object properties: role of modality-encoding bias. Perception 25(8):983–998. https://doi.org/10.1068/p250983

    Article  Google Scholar 

  8. Calli B, Dollar AM, Member S (2017) Vision-based model predictive control for within-hand precision manipulation with underactuated grippers. In: 2017 IEEE international conference on robotics and automation (ICRA), pp 2839–2845. https://doi.org/10.1109/ICRA.2017.7989331

  9. Alves De Oliveira TE, Cretu AM, Petriu EM (2017) Multimodal bio-inspired tactile sensing module. IEEE Sens J 17(11):3231–3243. https://doi.org/10.1109/JSEN.2017.2690898

    Article  Google Scholar 

  10. Shintake J, Cacucciolo V, Floreano D, Shea H (2018) Soft robotic grippers. Adv Mater 30(29):1707035

    Article  Google Scholar 

  11. Spiers AJ, Liarokapis MV, Calli B, Dollar AM (2016) Single-grasp object classification and feature extraction with simple robot hands and tactile sensors. IEEE Trans Haptics 9(2):207–220. https://doi.org/10.1109/TOH.2016.2521378

    Article  Google Scholar 

  12. Eppner C, Höfer S, Jonschkowski R, Martín-Martín R, Sieverling A, Wall V, Brock O (2016) Lessons from the Amazon picking challenge: four aspects of building robotic systems. RSS. https://doi.org/10.15607/RSS.2016.XII.036

    Article  Google Scholar 

  13. Ward-Cherrier B, Cramphorn L, Lepora N (2016) Tactile manipulation with a TacThumb integrated on the open-hand M2 gripper. IEEE Robot Autom Lett 3766(c):1–1. https://doi.org/10.1109/LRA.2016.2514420

    Article  Google Scholar 

  14. Clemente F, Valle G, Controzzi M, Strauss I, Iberite F, Stieglitz T, Granata G, Rossini PM, Petrini F, Micera S (2019) Intraneural sensory feedback restores grip force control and motor coordination while using a prosthetic hand. J Neural Eng 16(2):026034

    Article  Google Scholar 

  15. Ciobanu V, Popescu N (2015) Tactile controller using fuzzy logic for robot inhand manipulation. In: 2015 19th international conference on system theory, control and computing, ICSTCC 2015—joint conference SINTES 19, SACCS 15, SIMSIS 19, pp 435–440. https://doi.org/10.1109/ICSTCC.2015.7321332

  16. Islek C, Ozdemir E (2021) Design of a fuzzy safety margin derivation system for grip force control of robotic hand in precision grasp task. Int J Adv Rob Syst 18(3):1–12. https://doi.org/10.1177/17298814211018055

    Article  Google Scholar 

  17. Mahanta GB, Deepak BBVL, Biswal BB (2021) Application of soft computing methods in robotic grasping: a state-of-the-art survey. Proc Inst Mech Eng Part E: J Process Mech Eng. https://doi.org/10.1177/09544089211039977

    Article  Google Scholar 

  18. Molchanov A, Kroemer O, Su Z, Sukhatme GS (2016) Contact localization on grasped objects using tactile sensing. In: IEEE international conference on intelligent robots and systems, 2016-Nov, pp 216–222. https://doi.org/10.1109/IROS.2016.7759058

  19. Paolini R, Rodriguez A, Srinivasa SS, Mason MT (2014) A data-driven statistical framework for post-grasp manipulation. Int J Robot Res 33(4):600–615. https://doi.org/10.1177/0278364913507756

    Article  Google Scholar 

  20. Fleer S, Moringen A, Klatzky RL, Ritter H (2020) Learning efficient haptic shape exploration with a rigid tactile sensor array. PLoS ONE 15(1):1–22. https://doi.org/10.1371/journal.pone.0226880

    Article  Google Scholar 

  21. Alves de Oliveira T, Cretu A-M, Petriu E (2017) Multimodal bio-inspired tactile sensing module for surface characterization. Sensors 17(6):1187. https://doi.org/10.3390/s17061187

  22. Prado da Fonseca V, Alves de Oliveira TE, Petriu EM (2019) Estimating the orientation of objects from tactile sensing data using machine learning methods and visual frames of reference. Sensors 19(10):2285. https://doi.org/10.3390/s19102285

    Article  Google Scholar 

  23. Robotis (2006) User’s manual dynamixel ax-12. Technical report, Robotis

  24. Lokman NAA, Ahmad H, Daud MR (2017) Three fingered gripper grasping analysis of different objects using fuzzy logic controller. Adv Sci Lett 23(6):5102–5106

    Article  Google Scholar 

  25. Zisimatos AG, Liarokapis MV, Mavrogiannis CI, Kyriakopoulos KJ (2014) Open-source, affordable, modular, light-weight, underactuated robot hands. In: 2014 IEEE/RSJ international conference on intelligent robots and systems, pp 3207–3212. https://doi.org/10.1109/IROS.2014.6943007

  26. Breiman L (2001) Random forests. Mach Learn 45(1):5–32. https://doi.org/10.1023/A:1010933404324

    Article  MATH  Google Scholar 

  27. Hoerl AE, Kennard RW (2000) Ridge regression: biased estimation for nonorthogonal problems. Technometrics 42(1):80–86. https://doi.org/10.2307/1271436

    Article  MATH  Google Scholar 

  28. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297. https://doi.org/10.1023/A:1022627411411

    Article  MATH  Google Scholar 

  29. Geurts P, Ernst D, Wehenkel L (2006) Extremely randomized trees. Mach Learn 63(1):3–42. https://doi.org/10.1007/s10994-006-6226-1

    Article  MATH  Google Scholar 

  30. Quigley M, Conley K, Gerkey B, FAust J, Foote T, Leibs J, Berger E, Wheeler R, Mg A (2009) ROS: an open-source Robot Operating System. ICRA 3(Figure 1), 5. http://www.willowgarage.com/papers/ros-open-source-robot-operating-system

  31. Luo S, Bimbo J, Dahiya R, Liu H (2017) Robotic tactile perception of object properties: a review. Mechatronics 48(November):54–67. https://doi.org/10.1016/j.mechatronics.2017.11.002arXiv:1711.03810

    Article  Google Scholar 

  32. Polic M, Krajacic I, Lepora N, Orsag M (2019) Convolutional autoencoder for feature extraction in tactile sensing. IEEE Robot Autom Lett 4(4):3671–3678

    Article  Google Scholar 

  33. Cretu A-M, de Oliveira TEA, Prado da Fonseca V, Tawbe B, Petriu EM, Groza VZ (2015) Computational intelligence and mechatronics solutions for robotic tactile object recognition. In: 2015 IEEE 9th international symposium on intelligent signal processing (WISP) proceedings, pp 1–6. https://doi.org/10.1109/WISP.2015.7139165

  34. Schmitz A, Bansho Y, Noda K, Iwata H, Ogata T, Sugano S (2015) Tactile object recognition using deep learning and dropout. https://doi.org/10.1109/HUMANOIDS.2014.7041493. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=7041493

  35. Rouhafzay G, Cretu A-M (2018) A visuo-haptic framework for object from human tactile perception \(\dagger \) 1,7

  36. da Fonseca VP, Monteiro Rocha Lima B, Alves de Oliveira TE, Zhu Q, Groza VZ, Petriu EM (2019) In-hand telemanipulation using a robotic hand and biology-inspired haptic sensing. In: 2019 IEEE international symposium on medical measurements and applications (MeMeA), pp 1–6. https://doi.org/10.1109/MeMeA.2019.8802139

Download references

Author information

Author notes

  1. Vinicius Prado da Fonseca, Xianta Jiang, Emil M. Petriu and Thiago Eustaquio Alves de Oliveira have contributed equally to this work.

Authors and Affiliations

  1. Department of Computer Science, Memorial University of Newfoundland and Labrador, 230 Elizabeth Ave, St. John’s, NL, A1C 5S7, Canada

    Vinicius Prado da Fonseca & Xianta Jiang

  2. School of Electrical Engineering and Computer Science, University of Ottawa, 800 King Edward, Ottawa, ON, K1N 6N5, Canada

    Emil M. Petriu

  3. Department of Computer Science, Lakehead University, 955 Oliver Road, Thunder Bay, ON, P7B 5E1, Canada

    Thiago Eustaquio Alves de Oliveira

Authors
  1. Vinicius Prado da Fonseca
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xianta Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Emil M. Petriu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thiago Eustaquio Alves de Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vinicius Prado da Fonseca.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

da Fonseca, V.P., Jiang, X., Petriu, E.M. et al. Tactile object recognition in early phases of grasping using underactuated robotic hands. Intel Serv Robotics 15, 513–525 (2022). https://doi.org/10.1007/s11370-022-00433-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11370-022-00433-7

Keywords

Search

Navigation