Medical & Biological Engineering & Computing

, Volume 52, Issue 4, pp 353–362 | Cite as

Object-shape recognition and 3D reconstruction from tactile sensor images

  • Anwesha Khasnobish
  • Garima Singh
  • Arindam Jati
  • Amit Konar
  • D. N. Tibarewala
Original Article


This article presents a novel approach of edged and edgeless object-shape recognition and 3D reconstruction from gradient-based analysis of tactile images. We recognize an object’s shape by visualizing a surface topology in our mind while grasping the object in our palm and also taking help from our past experience of exploring similar kind of objects. The proposed hybrid recognition strategy works in similar way in two stages. In the first stage, conventional object-shape recognition using linear support vector machine classifier is performed where regional descriptors features have been extracted from the tactile image. A 3D shape reconstruction is also performed depending upon the edged or edgeless objects classified from the tactile images. In the second stage, the hybrid recognition scheme utilizes the feature set comprising both the previously obtained regional descriptors features and some gradient-related information from the reconstructed object-shape image for the final recognition in corresponding four classes of objects viz. planar, one-edged object, two-edged object and cylindrical objects. The hybrid strategy achieves 97.62 % classification accuracy, while the conventional recognition scheme reaches only to 92.60 %. Moreover, the proposed algorithm has been proved to be less noise prone and more statistically robust.


Pattern recognition Object-shape recognition Haptic recognition Image processing Tactile image 



This study has been supported by University Grants Commission (UGC), India, University with Potential for Excellence—Phase II (UPE II) in Cognitive Science, Jadavpur University and Council of Scientific and Industrial Research (CSIR), India.


  1. 1.
    Alpaydin E (2004) Linear discrimination. Introduction to machine learning, 1st edn. MIT Press, Cambridge, pp 197–228Google Scholar
  2. 2.
    Argall BD, Billard AG (2010) A survey of tactile human-robot interactions. Rob Auton Syst 58(10):1159–1176CrossRefGoogle Scholar
  3. 3.
    Bierbaum A, Gubarev I, Dillmann R (2008) Robust shape recovery for sparse contact location and normal data from haptic exploration. In: IEEE/RSJ international conference on intelligent robots and systems (IROS 2008). IEEE, Nice, pp 3200–3205Google Scholar
  4. 4.
    Chia-Ping C, Chu-Song C, Yi-Ping H (2009) Pixel-based correspondence, and shape reconstruction for moving objects. In: Proceedings IEEE 12th international conference on computer vision workshops (ICCV workshops). IEEE, Kyoto, pp 1962–1969Google Scholar
  5. 5.
    Dahiya RS, Metta G, Valle M, Sandini G (2010) Tactile sensing: from humans to humanoids. IEEE Trans Rob 26:1–20CrossRefGoogle Scholar
  6. 6.
    Dietterich TG (1998) Approximate statistical test for comparing supervised classification learning algorithms. J Neural Comput 10(7):1895–1924CrossRefGoogle Scholar
  7. 7.
    Duda RO, Hart PE, Stork DG (2012) Unsupervised learning and clustering. In: Pattern classification. Wiley, New York, pp 517–600 Google Scholar
  8. 8.
    Gonzalez RC, Woods RE, Eddins SL (2012) Digital image processing using MATLAB, 4th edn. Tata McGraw Hill, LLC, USA, pp 551–614Google Scholar
  9. 9.
    Graupe D (2007) Principles of artificial neural networks, 2nd edn. Advanced series on circuits and systems, vol 6. World Scientific, Denver, MAGoogle Scholar
  10. 10.
    Ballesteros S, Heller MA (2008) Haptic object identification. In: Grunwald M (ed) Human haptic perception. Basics and Applications. Birkhauser Basel, Birkhauser Verlag AG, pp 207–222CrossRefGoogle Scholar
  11. 11.
    Hayward V, Astley OR, Cruz-Hernandez M, Grant D, Robles-De-La-Torre G (2004) Haptic interfaces and devices. J Sens Rev 24:16–29CrossRefGoogle Scholar
  12. 12.
    Heidemann G, Schopfer M (2004) Dynamic tactile sensing for object identification. In: Proceedings of IEEE international conference on robotics and automation (ICRA ‘04), vol 1. IEEE, New Orleans, LA, pp 813–818Google Scholar
  13. 13.
    Jain AK, Murty MN, Flynn PJ (1999) Data clustering: a review. ACM Comput Surv, N. Y., U. S. A. 31(3):264–323CrossRefGoogle Scholar
  14. 14.
    Khasnobish A, Jati A, Singh G, Bhattacharyya S, Konar A, Tibarewala DN, Kim E, Nagar AK (2012) Object-shape recognition from tactile images using a feed-forward neural network. In: Proceedings IEEE international joint conference on neural networks (IJCNN 2012). IEEE, Brisbane, QLD, pp 1–8Google Scholar
  15. 15.
    Meier M, Schopfer M, Haschke R, Ritter H (2011) A probabilistic approach to tactile shape reconstruction. IEEE Trans Rob 27:630–635CrossRefGoogle Scholar
  16. 16.
    Mitchell T (1997) Instance based learning. In: Machine learning. McGraw Hill, New York, pp 239–258Google Scholar
  17. 17.
    Murphy OJ (1990) Nearest neighbor pattern classification perceptrons. In: Proceedings of the IEEE, vol 78. IEEE, pp 1595–1598Google Scholar
  18. 18.
    Nakamoto H, Fukui W, Kobayashi F, Kojima F, Imamura N, Shirasawa H (2009) Shape classification based on tactile information by universal robot hand. In: 35th annual conference of IEEE on industrial electronics (IECON ‘09). IEEE, Porto, pp 2360–2365Google Scholar
  19. 19.
    Okamura AM, Turner ML, Cutkosky MR (1997) Haptic exploration of objects with rolling and sliding. In: Proceedings of the IEEE international conference on robotics and automation, vol 3. IEEE, Albuquerque, NM, pp 2485–2490Google Scholar
  20. 20.
    Pezzementi Z, Plaku E, Reyda C, Hager GD (2011) Tactile-object recognition from appearance information. IEEE Trans Rob 27:473–487CrossRefGoogle Scholar
  21. 21.
    Romano JM, Hsiao K, Niemeyer G, Chitta S, Kuchenbecker KJ (2011) Human-inspired robotic grasp control with tactile sensing. IEEE Trans Rob 27:1067–1079CrossRefGoogle Scholar
  22. 22.
    Shimoga KB, Goldenberg AA (1996) Soft robotic fingertips part II: modeling and impedance regulation. Int J Rob Res 15(4):335–350CrossRefGoogle Scholar
  23. 23.
    Sigiuchi H, Watanabe S, Hasegawa Y, Nornoto M (2000) A control system for multi-fingered robotic hand with distributed touch sensor. In: 26th annual conference industrial electronics society, IECON 2000, vol 1. IEEE, Nagoya, pp 434–439Google Scholar
  24. 24.
    Simpkins A (2013) Robotic tactile sensing: technologies and system (Dahiya RS, Valle M 2013) [on the shelf]. IEEE Robot Autom Mag 20(2):107–107CrossRefGoogle Scholar
  25. 25.
    Takamuku S, Fukuda A, Hosoda K (2008) Repetitive grasping with anthromorphic skin-covered hand enables robust haptic recognition. In: International conference on intelligent robots and systems (IROS 2008). IEEE, Nice, pp 3212–3217Google Scholar
  26. 26.
    Talasaz A, Patel RV (2013) Integration of force reflection with tactile sensing for minimally invasive robotics-assisted tumor localization. IEEE Trans Haptics 6:217–228CrossRefGoogle Scholar
  27. 27.
    Ting Z, Hong L, Li J, Shaowei F, Jing Y (2013) Development of a flexible 3-D tactile sensor system for anthropomorphic artificial hand. IEEE Sens J 13:510–518CrossRefGoogle Scholar
  28. 28.
    Visell Y (2009) Tactile sensory substitution: models for enaction in HCI. J Interact Comput 21:38–53CrossRefGoogle Scholar
  29. 29.
    Webb AR, Copsey KD (2011) Nonlinear discriminant analysis—kernel and projection methods. In: Statistical pattern recognition, 3rd edn. Wiley, New York, pp 274–321Google Scholar
  30. 30.
    Yamada Y, Ishiguro A, Uchikawa Y (1993) A method of 3D object reconstruction by fusing vision with touch using internal models with global and local deformations. In: Proceedings international conference on robotics and automation, vol 2. IEEE, Atlanta, GA, pp 782–787Google Scholar

Copyright information

© International Federation for Medical and Biological Engineering 2014

Authors and Affiliations

  • Anwesha Khasnobish
    • 1
  • Garima Singh
    • 2
  • Arindam Jati
    • 2
  • Amit Konar
    • 2
  • D. N. Tibarewala
    • 1
  1. 1.School of Bioscience and EngineeringJadavpur UniversityKolkataIndia
  2. 2.Department of Electronics and Telecommunication EngineeringJadavpur UniversityKolkataIndia

Personalised recommendations