Tactile Object Recognition with Semi-Supervised Learning

  • Shan Luo
  • Xiaozhou Liu
  • Kaspar Althoefer
  • Hongbin LiuEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9245)


This paper introduced a novel approach to recognize objects with tactile images by utilizing semi-supervised learning approaches. In tactile object recognition, the data are normally insufficient to build robust training models. Thus the model of Ensemble Manifold Regularization, which combines concepts of multi-view learning and semi-supervised learning, is adapted in tactile sensing to achieve better recognition accuracy. Different outputs of classic bag of words with different dictionary sizes are considered as different views to produce an optimized one based on multiple graphs learning optimization. In the experiments 12 objects were used to compare the classification performances of our proposed approach and the classic BoW model and it is proved that our proposed method outperforms the classic BoW framework and objects with similar features can be better classified.


Tactile sensors Object recognition Robot tactile systems 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Dahiya, R.S., Metta, G., Valle, M., Sandini, G.: Tactile Sensing––From Humans to Humanoids. IEEE Trans. Robot. 26(1), 1–20 (2010)CrossRefGoogle Scholar
  2. 2.
    Dahiya, R.S., Mittendorfer, P., Valle, M., Cheng, G., Lumelsky, V.J.: Directions Toward Effective Utilization of Tactile Skin: A Review. IEEE Sens. J. 13(11), 4121–4138 (2013)CrossRefGoogle Scholar
  3. 3.
    Xie, H., Liu, H., Luo, S., Seneviratne, L.D., Althoefer, K.: Fiber optics tactile array probe for tissue palpation during minimally invasive surgery. In: IEEE International Conference on Intelligent Robots and Systems (IROS), pp. 2539–2544 (2013)Google Scholar
  4. 4.
    Liu, H., Puangmali, P., Zbyszewski, D., Elhage, O., Dasgupta, P., Dai, J.S., Seneviratne, L., Althoefer, K.: An Indentation Depth-force Sensing Wheeled Probe for Abnormality Identification during Minimally Invasive Surgery. Proc. Inst. Mech. Eng. H. 224(6), 751–763 (2010)CrossRefGoogle Scholar
  5. 5.
    Althoefer, K., Zbyszewski, D., Liu, H., Puangmali, P., Seneviratne, L., Challacombe, B., Dasgupta, P., Murphy, D.: Air-cushion force-sensitive probe for soft tissue investigation during minimally invasive surgery. J. Endourol. 23(9), 1421–1424 (2009)CrossRefGoogle Scholar
  6. 6.
    Liu, H., Nguyen, K.C., Perdereau, V., Bimbo, J., Back, J., Godden, M., Seneviratne, L.D., Althoefer, K.: Finger contact sensing and the application in dexterous hand manipulation. Auton. Robots 39(1), 25–41 (2015)CrossRefGoogle Scholar
  7. 7.
    Luo, S., Mou, W., Althoefer, K., Liu, H.: Localizing the object contact through matching tactile features with visual map. In: IEEE Conference on Robotics and Automation (ICRA) (2015, to appear)Google Scholar
  8. 8.
    Song, X., Liu, H., Althoefer, K., Nanayakkara, T., Seneviratne, L.D.: Efficient Break-Away Friction Ratio and Slip Prediction Based on Haptic Surface Exploration. IEEE Trans. Robot. 30(1), 203–219 (2014)CrossRefGoogle Scholar
  9. 9.
    Bimbo, J., Seneviratne, L.D., Althoefer, K.: Combining touch and vision for the estimation of an object’s pose during manipulation. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4021–4026 (2013)Google Scholar
  10. 10.
    Meier, M., Sch, M., Haschke, R., Ritter, H.: A Probabilistic Approach to Tactile Shape Reconstruction. IEEE Trans. Robot. 27(3), 630–635 (2011)CrossRefGoogle Scholar
  11. 11.
    Martins, R., Ferreira, F., Dias, J., Member, S.: Touch attention Bayesian models for robotic active haptic exploration of heterogeneous surfaces. In: Iros, pp. 1208–1215 (2014)Google Scholar
  12. 12.
    Chathuranga, D.S., Ho, V.A., Hirai, S.: Investigation of a biomimetic fingertip’s ability to discriminate fabrics based on surface textures. 2013 IEEE/ASME Int. Conf. Adv. Intell. Mechatronics Mechatronics Hum. Wellbeing, AIM 2013, pp. 1667–1674 (2013)Google Scholar
  13. 13.
    Lederman, S.J., Klatzky, R.L.: Haptic perception : A tutorial. Attention, Perception, & Psychophys 71(7), 1439–1459 (2009)CrossRefGoogle Scholar
  14. 14.
    Kaboli, M., Mittendorfer, P., Hugel, V., Cheng, G.: Humanoids learn object properties from robust tactile feature descriptors via multi-modal artificial skin. In: IEEE-RAS Internatianl Conference on Humanoid Robots (Humanoids) (2014, to appear)Google Scholar
  15. 15.
    Decherchi, S., Gastaldo, P., Dahiya, R.S., Valle, M., Zunino, R.: Tactile-Data Classification of Contact Materials Using Computational Intelligence. IEEE Trans. Robot. 27(3), 635–639 (2011)CrossRefGoogle Scholar
  16. 16.
    Liu, H., Song, X., Bimbo, J., Seneviratne, L., Althoefer, K.: Surface material recognition through haptic exploration using an intelligent contact sensing finger. In: International Conference On Intelligent Robots And Systems, pp. 52–57 (2012)Google Scholar
  17. 17.
    Pezzementi, Z., Plaku, E., Reyda, C., Hager, G.D.: Tactile-Object Recognition From Appearance Information. IEEE Trans. Robot. 27(3), 473–487 (2011)CrossRefGoogle Scholar
  18. 18.
    Liu, H., Greco, J., Song, X., Bimbo, J., Althoefer, K.: Tactile image based contact shape recognition using neural network. In: IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), pp. 138–143 (2013)Google Scholar
  19. 19.
    Luo, S., Mou, W., Li, M., Althoefer, K., Liu, H.: Rotation and translation invariant object recognition with a tactile sensor. In: IEEE Sensors Conference, pp. 1030–1033 (2014)Google Scholar
  20. 20.
    Luo, S., Mou, W., Althoefer, K., Liu, H.: Novel Tactile - SIFT Descriptor for Object Shape Recognition. IEEE Sens. J. (2015, article in press)Google Scholar
  21. 21.
    Ji, Z., Amirabdollahian, F., Polani, D., Dautenhahn, K.: Histogram based classification of tactile patterns on periodically distributed skin sensors for a humanoid robot. In: IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp. 433–440 (2011)Google Scholar
  22. 22.
    Schneider, A., Sturm, J., Stachniss, C., Reisert, M., Burkhardt, H., Burgard, W.: Object identification with tactile sensors using bag-of-features. In: 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 243–248 (2009)Google Scholar
  23. 23.
    Madry, M., Bo, L., Kragic, D., Fox, D.: ST-HMP: unsupervised spatio-temporal feature learning for tactile data. In: IEEE Conference on Robotics and Automation (ICRA), pp. 2262–2269 (2014)Google Scholar
  24. 24.
    Allen, P.K., Michelman, P.: Acquisition and interpretation of 3-D sensor data from touch. IEEE Trans. Robot. Autom. 6(4), 397–404 (1990)CrossRefGoogle Scholar
  25. 25.
    Charlebois, M., Gupta, K., Payandeh, S.: Shape description of general, curved surfaces using tactile sensing and surface normal information. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 2819–2824, April 1997Google Scholar
  26. 26.
    Casselli, S., Magnanini, C., Zanichelli, F.: On the robustness of haptic object recognition based on polyhedral shape representations. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), vol. 2, pp. 200–206 (1995)Google Scholar
  27. 27.
    Liu, H., Song, X., Nanayakkara, T., Seneviratne, L.D., Althoefer, K.: A computationally fast algorithm for local contact shape and pose classification using a tactile array sensor. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 1410–1415 (2012)Google Scholar
  28. 28.
    Lowe, D.G.: Distinctive Image Features from Scale-Invariant Keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  29. 29.
    Pezzementi, Z., Reyda, C., Hager, G.D.: Object mapping, recognition, and localization from tactile geometry. In: 2011 IEEE International Conference on Robotics and Automation (ICRA), pp. 5942–5948 (2011)Google Scholar
  30. 30.
    Geng, B., Tao, D., Member, S.: Ensemble Manifold Regularization. IEEE Trans. Pattern Anal. Mach. Intell. 34(6), 1227–1233 (2012)CrossRefGoogle Scholar
  31. 31.
    Zhu, X., Ghahramani, Z., Lafferty, J.D.: Semi-supervised learning using Gaussian fields and harmonic functions. In: Int. Conf. Mach. Learn. - ICML 2003, vol. 20, no. 2, pp. 912–919 (2003)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Shan Luo
    • 1
  • Xiaozhou Liu
    • 1
  • Kaspar Althoefer
    • 1
  • Hongbin Liu
    • 1
    Email author
  1. 1.Department of InformaticsKing’s College LondonLondonUK

Personalised recommendations