Autonomous Robots

, Volume 43, Issue 1, pp 123–152 | Cite as

Tactile-based active object discrimination and target object search in an unknown workspace

  • Mohsen KaboliEmail author
  • Kunpeng Yao
  • Di Feng
  • Gordon Cheng


The tasks of exploring unknown workspaces and recognizing objects based on their physical properties are challenging for autonomous robots. In this paper, we present strategies solely based on tactile information to enable robots to accomplish such tasks. (1) An active exploration approach for the robot to explore unknown workspaces; (2) an active touch objects learning method that enables the robot to learn efficiently about unknown objects via their physical properties (stiffness, surface texture, and center of mass); and (3) an active object recognition strategy, based on the knowledge the robot has acquired. Furthermore, we propose a tactile-based approach for estimating the center of mass of rigid objects. Following the active touch for workspace exploration, the robotic system with the sense of touch in fingertips reduces the uncertainty of the workspace up to 65 and 70% compared respectively to uniform and random strategies, for a fixed number of samples. By means of the active touch learning method, the robot achieved 20 and 15% higher learning accuracy for the same number of training samples compared to uniform strategy and random strategy, respectively. Taking advantage of the prior knowledge obtained during the active touch learning, the robot took up to 15% fewer decision steps compared to the random method to achieve the same discrimination accuracy in active object discrimination task.


Active tactile object localization Active tactile object exploration Active tactile learning 



This work is supported by the European Commission under Grant Agreements PITN-GA-2012-317488-CONTEST.

Supplementary material

Supplementary material 1 (mp4 215115 KB)


  1. Atkeson, C. G., An C. H., & Hollerbach, J. M. (1985) Rigid body load identification for manipulators. In IEEE conference on decision and control (pp. 996–1002).Google Scholar
  2. Bhattacharya, A., & Mahajan, R. (2003). Temperature dependence of thermal conductivity of biological tissues. Physiological Measurement, 24(3), 769.CrossRefGoogle Scholar
  3. Chathuranga , D. S., Wang, Z., Ho, V. A., Mitani, A., & Hirai, S. (2013). A biomimetic soft fingertip applicable to haptic feedback systems for texture identification. In IEEE international symposium on haptic audio visual environments and games (pp. 29–33).Google Scholar
  4. Cheng, Y. (1995). Mean shift, mode seeking, and clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(8), 790–799.CrossRefGoogle Scholar
  5. Chu, V., McMahon, I., Riano, L., McDonald, C. G., He, Q., et al. (2013). Using robotic exploratory procedures to learn the meaning of haptic adjectives. In IEEE international conference on robotics and automation (pp. 3048–3055).Google Scholar
  6. Csato, L., & Opper, M. (2002). Sparse on-line gaussian processes. Neural Computation, 14(3), 641–668.CrossRefzbMATHGoogle Scholar
  7. Dahiya, R. S., Metta, G., Valle, M., Adami, A., & Lorenzelli, L. (2009). Piezoelectric oxide semiconductor field effect transistor touch sensing devices. Applied Physics Letters, 95(034), 105.Google Scholar
  8. Dahiya, R. S., Metta, G., Valle, M., & Sandini, G. (2010). Tactile sensing—From humans to humanoids. IEEE Transactions on Robotics, 26(1), 1–20.CrossRefGoogle Scholar
  9. Dallaire, P., Giguere, P., Edmond, D., & Chaib-draa, B. (2014). Autonomous tactile perception: A combined improved sensing and bayesian nonparametric approach. Robotics and Autonomous Systems, 6(4), 422–435.CrossRefGoogle Scholar
  10. Denei, S., Maiolino, P., Baglini, E., & Cannata, G. (2015). On the development of a tactile sensor for fabric manipulation and classification for industrial applications. In IEEE international conference on intelligent robots and systems (pp. 5081–5086).Google Scholar
  11. Fishel, J. A., & Loeb, G. E. (2012). Bayesian exploration for intelligent identification of textures. Frontiers in Neurorobotics, 6, 4.CrossRefGoogle Scholar
  12. Friedl, K. E., Voelker, A. R., Peer, A., & Eliasmith, C. (2016). Human-inspired neurorobotic system for classifying surface textures by touch. IEEE Robotics and Automation Letters, 1, 516–523.CrossRefGoogle Scholar
  13. Giguere, P., & Dudek, G. (2011). A simple tactile probe for surface identification by mobile robots. IEEE Transactions on Robotics, 27(3), 534–544.CrossRefGoogle Scholar
  14. Hughes, D., & Correll, N. (2015). Texture ecognition and localization in amorphous robotic skin. Bioinspiration and Biomimetics, 10(055), 002.Google Scholar
  15. Hu, H., Han, Y., Song, A., Chen, S., Wang, C., & Wang, Z. (2014). A finger-shaped tactile sensor for fabric surfaces evaluation by 2-dimensional active sliding touch. Sensors, 14, 4899–4913.CrossRefGoogle Scholar
  16. Jamali, N., Ciliberto, C., Rosasco, L., & Natale, L. (2016). Active perception: Building objects models using tactile exploration. In IEEE international conference on humanoid robots.Google Scholar
  17. Jamali, N., & Sammut, C. (2011). Majority voting: Material classification by tactile sensing using surface texture. IEEE Transactions on Robotics, 27(3), 508–521.CrossRefGoogle Scholar
  18. Jia, Y., & Tian, J. (2010). Surface patch reconstruction from ‘one-dimensional’ tactile data. IEEE Transactions on Automation Science and Engineering, 7, 400–407.CrossRefGoogle Scholar
  19. Kaboli, M., & Cheng, G. (2018). Robust tactile descriptors for discriminating objects from textural properties via artificial robotic skin. IEEE Transaction on Robotics, 34(2), 1–19.CrossRefGoogle Scholar
  20. Kaboli, M., Feng, D., & Cheng, G. (2017a). Active tactile transfer learning for object discrimination in an unstructured environment using multimodal robotic skin. International Journal of Humanoid Robotics, 15(1), 18–51.Google Scholar
  21. Kaboli, M., Feng, D., Yao, K., Lanillos, P., & Cheng, G. (2017b). A tactile-based framework for active object learning and discrimination using multimodal robotic skin. IEEE Robotics and Automation Letters, 2(4), 2143–2150.CrossRefGoogle Scholar
  22. Kaboli, M., Long, A., & Cheng, G. (2015a). Humanoids learn touch modalities identification via multi-modal robotic skin and robust tactile descriptors. Advanced Robotics, 29(21), 1411–1425.CrossRefGoogle Scholar
  23. Kaboli, M., Mittendorfer, P., Hugel, V., & Cheng, G. (2014). Humanoids learn object properties from robust tactile feature descriptors via multi-modal artificial skin. In IEEE international conference on humanoid robots (pp. 187–192).Google Scholar
  24. Kaboli, M., Rosaand, A. D. L., Walker, R., & Cheng, G. (2016a). Re-using prior tactile experience by robotic hands to discriminate in-hand objects via texture properties. In IEEE international conference on robotics and automation (pp. 2242–2247).Google Scholar
  25. Kaboli, M., Walker, R., & Cheng, G. (2015b). In-hand object recognition via texture properties with robotic hands, artificial skin, and novel tactile descriptors. In IEEE international conference on humanoid robots (pp. 2242–2247).Google Scholar
  26. Kaboli, M., Yao, K., & Cheng, G. (2016b). Tactile-based manipulation of deformable objects with dynamic center of mass. In IEEE international conference on humanoid robots.Google Scholar
  27. Kaltenbrunner, M., Sekitani, T., Reeder, J., Yokota, T., Kuribara, K., et al. (2013). An ultra-lightweight design for imperceptible plastic electronics. Nature, 499, 455–463.CrossRefGoogle Scholar
  28. Lederman, S. J. (1981). The perception of surface roughness by active and passive touch. Bulletin of the Psychonomic Society, 18, 253–255.CrossRefGoogle Scholar
  29. Lee, S., Reuveny, A., Reeder, J., Lee, S., Jin, H., et al. (2016). A transparent bending-insensitive pressure sensor. Nature Nanotechnology, 11, 472–478.CrossRefGoogle Scholar
  30. Lepora, N. F., Evans, M., Fox, C. W., Diamond, M. E., Gurney, K., & Prescott, T. J . (2010). Naive bayes texture classification applied to whisker data from a moving robot. In The international joint conference on neural networks (pp. 1–8).Google Scholar
  31. Lepora, N. F., Martinez-Hernandez, U., & Prescott, T. J. (2013). Active touch for robust perception under position uncertainty. In IEEE International Conference on Robotics and Automation (pp. 3020–3025).Google Scholar
  32. Liarokapis, M. V., Calli, B., Spiers, A., & Dollar, A. M. (2015). Unplanned, model-free, single grasp object classification with underactuated hands and force sensors. In IEEE international conference on intelligent robots and systems (pp. 5073–5080).Google Scholar
  33. Liu, H., Greco, J., Song, X., Bimbo, J., & Althoefer, K. (2013). Tactile image based contact shape recognition using neural network. In IEEE international conference on multisensor fusion and integration for intelligent systems (MFI) (pp. 138–143).Google Scholar
  34. Liu, H., Song, X., Nanayakkara, T., Seneviratne, L. D., & Althoefer, K. (2012). A computationally fast algorithm for local contact shape and pose classification using a tactile array sensor. In IEEE international conference on robotics and automation (pp. 1410–1415).Google Scholar
  35. Liu, H., Nguyen, K. C., Perdereau, V., Bimbo, J., Back, J., Godden, M., et al. (2015). Finger contact sensing and the application in dexterous hand manipulation. Autonomous Robots, 39(1), 25–41.CrossRefGoogle Scholar
  36. Martinez-Cantin, R., de Freitas, N., Doucet, A., & Castellanos, J. A. (2007). Active policy learning for robot planning and exploration under uncertainty. Robotics: Science and Systems, 3, 321–328.zbMATHGoogle Scholar
  37. Martins, R., Ferreira, J. F., & Dias, J. (2014). Touch attention bayesian models for robotic active haptic exploration of heterogeneous surfaces. In International conference on intelligent robots and systems (pp. 1208–1215).Google Scholar
  38. Mayol-Cuevas, W., Juarez-Guerrero, J., & Munoz-Gutierrez, S. (1998). A first approach to tactile texture recognition. In IEEE international conference on systems, man, and cybernetics (Vol. 5, pp. 4246–4250).Google Scholar
  39. Mohamad Hanif, N. H. H., Chappell, P. H., Cranny, A., & White, N. M . (2015). Surface texture detection with artificial fingers. In 37th annual international conference of the IEEE engineering in medicine and biology society (pp. 8018–8021).Google Scholar
  40. Nawrocki, A. R., Matsuhisa, N., Yokota, T., & Someya, T. (2016). 300-nm imperceptible, ultraflexible, and biocompatible e-skin fit with tactile sensors and organic transistors. Advanced Electronic Materials.
  41. Nguyen, K. C., & Perdereau, V. (2013). Fingertip force control based on max torque adjustment for dexterous manipulation of an anthropomorphic hand. In 2013 IEEE/RSJ international conference on intelligent robots and systems (pp. 3557–3563).Google Scholar
  42. Ohmura, Y., Kuniyosh, Y., & Nagakubo, A . (2006). Conformable and scalable tactile sensor skin for curved surfaces. In IEEE international conference on robotics and automation (pp. 1348–1353).Google Scholar
  43. Papakostas, T. V., Lima, J., & Lowe, M. (2002). A large area force sensor for smart skin applications. IEEE Sensors, 5, 1620–1624.CrossRefGoogle Scholar
  44. Rasmussen, C. E., & Williams, C. K. I. (2006). Gaussian processes for machine learning. Cambridge, MA: MIT Press.zbMATHGoogle Scholar
  45. Rebguns, A., Ford, D., & Fasel, I. R. (2011). Infomax control for acoustic exploration of objects by a mobile robot. In Lifelong learning.Google Scholar
  46. Robles-De-La-Torre, G. (2006). The importance of the sense of touch in virtual and real environments. IEEE MultiMedia, 13, 24–30.CrossRefGoogle Scholar
  47. Saal, H. P., Ting, J. A., & Vijayakumar, S. (2010). Active sequential learning with tactile feedback. In AISTATS (pp. 677–684).Google Scholar
  48. Schmitz, A., Maiolinoand, P., Maggiali, M., Natale, L., Cannata, G., & Metta, G. (2011). Methods and technologies for the implementation of large-scale robot tactile sensors. IEEE Transactions on Robotics, 27(3), 389–400.CrossRefGoogle Scholar
  49. Schneider, A., Sturm, J., Stachniss, C., Reisert, M., Burkhardt, H., & Burgard, W. (2009). Object identification with tactile sensors using bag-of-features. In IEEE international conference on intelligent robots and systems (pp. 243–248).Google Scholar
  50. Sinapov, J., Sukhoy, V., Sahai, R., & Stoytchev, A. (2011). Vibrotactile recognition and categorization of surfaces by a humanoid robot. IEEE Transactions on Robotics, 27(3), 488–497.CrossRefGoogle Scholar
  51. Song, A., Han, Y., Hu, H., & Li, J. (2014). A novel texture sensor for fabric texture measurement and classification. IEEE Transactions on Instrumentation and Measurement, 63(7), 1739–1747.CrossRefGoogle Scholar
  52. Strehl, A., & Ghosh, J. (2002). Cluster ensembles—A knowledge reuse framework for combining multiple partitions. Journal of Machine Learning Research, 3, 583–617.MathSciNetzbMATHGoogle Scholar
  53. Strohmayr, M. W., Worn, H., & Hirzinger, G . (2013). The DLR artificial skin part i: Uniting sensitivity and robustness. In IEEE international conference on robotics and automation (pp. 1012–1018).Google Scholar
  54. Tanaka, D., Matsubara, T., Ichien, K., & Sugimoto, K. (2014). Object manifold learning with action features for active tactile object recognition. In IEEE international conference on intelligent robots and systems (pp. 608–614).Google Scholar
  55. Ulmen, J, & Cutkosky, M. (2010). A robust low-cost and low-noise artificial skin for human-friendly robots. In IEEE international conference on robotics and automation (pp. 4836–4841).Google Scholar
  56. Watanabe, K., Sohgawa, M., Kanashima, T., Okuyama, M., & Norna, H. (2013). Identification of various kinds of papers using multi-axial tactile sensor with micro-cantilevers. In World haptics conference (pp. 139–144).Google Scholar
  57. Xu, D., Loeb, G. E., & Fishel, J. A. (2013). Tactile identification of objects using bayesian exploration. In IEEE international conference on robotics and automation (pp. 3056–3061).Google Scholar
  58. Yao, K., Kaboli, M., & Cheng, G. (2017). Tactile-based object center of mass exploration and discrimination. In IEEE international conference on humanoid robotics (pp. 876–881).Google Scholar
  59. Yi, Z., Calandra, R., Veiga, F., van Hoof, H., Hermans, T., Zhang, Y., et al. (2016). Active tactile object exploration with gaussian processes. In IEEE international conference on intelligent robots and systems (pp. 4925–4930).Google Scholar
  60. Yogeswaran, N., Dang, W., Navaraj, W. T., Shakthivel, D., et al. (2015). New materials and advances in making electronic skin for interactive robots. Advanced Robotics, 29(21), 1359–1373.CrossRefGoogle Scholar
  61. Yu, Y., Arima, T., & Tsujio, S. (2005). Estimation of object inertia parameters on robot pushing operation. In IEEE international conference on robotics and automation (pp. 1657–1662).Google Scholar
  62. Yu, Y., Kiyokawa, T., & Tsujio, S. (2004). Estimation of mass and center of mass of unknown and graspless cylinder-like object. International Journal of Information Acquisition, 1(01), 47–55.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Institute for Cognitive Systems, Department of Electrical and Computer EngineeringTechnische Universität MünchenMunichGermany

Personalised recommendations