Sim-to-Real Transferable Object Classification Through Touch-Based Continuum Manipulation

Conference paper
Part of the Springer Proceedings in Advanced Robotics book series (SPAR, volume 11)


It is important to investigate object perception for classification or recognition based on touch sensing, especially when robots are operating in darkness or the objects are difficult to capture by vision sensors. In this work, we present a new form of continuum manipulator equipped with sparse touch sensing, validate the effectiveness of automatic generation of the touch-based continuum wraps, and the effectiveness of object classification based on the continuum wraps. Using the indirect object shape information encoded in the robot shape, we demonstrate that a classifier trained from the simulated continuum wraps is transferable to identify the real world objects with real continuum wraps.


Continuum manipulation Tactile sensing Object perception 



This work is supported by NSF grant IIP-1439695, CMMI-1752195.


  1. 1.
    Zhang, M.M., Atanasov, N., Daniilidis, K.: Active end-effector pose selection for tactile object recognition through Monte Carlo tree search. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2017)Google Scholar
  2. 2.
    Sommer, N., Li, M., Billard, A.: Bimanual compliant tactile exploration for grasping unknown objects. In: International Conference on Robotics and Automation (ICRA). IEEE (2014)Google Scholar
  3. 3.
    Li, Q., Schürmann, C., Haschke, R., Ritter, H.J.: A control framework for tactile servoing. In: Robotics: Science and Systems (2013)Google Scholar
  4. 4.
    Yi, Z., Calandra, R., Veiga, F., van Hoof, H., Hermans, T., Zhang, Y., Peters, J.: Active tactile object exploration using Gaussian processes. In: IEEE/RSJ IROS (2016)Google Scholar
  5. 5.
    Jia, Y.-B., Tian, J.: Surface patch reconstruction from one-dimensional tactile data. IEEE Trans. Autom. Sci. Eng. 7(2), 400–407 (2010)CrossRefGoogle Scholar
  6. 6.
    Mao, H., Xiao, J.: Object shape estimation through touch-based continuum manipulation. In: International Symposium of Robotics Research (ISRR) (2017)Google Scholar
  7. 7.
    Falco, P., Lu, S., Cirillo, A., Natale, C., Pirozzi, S., Lee, D.: Cross-modal visuo-tactile object recognition using robotic active exploration. In: IEEE ICRA (2017)Google Scholar
  8. 8.
    Li, J., Xiao, J.: Progressive planning of continuum grasping in cluttered space. IEEE Trans. Robot. (TRO) 32(3), 707–716 (2016)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Mao, H., Zhang, M.M., Xiao, J., Daniilidis, K.: Shape-based object classification and recognition through continuum manipulation. In: IEEE/RSJ IROS (2017)Google Scholar
  10. 10.
    Bajo, A., Simaan, N.: Kinematics-based detection and localization of contacts along multisegment continuum robots. IEEE Trans. Robot. 28(2), 291–302 (2012)CrossRefGoogle Scholar
  11. 11.
    Santoso, J., Skorina, E.H., Luo, M., Yan, R., Onal, C.D.: Design and analysis of an origami continuum manipulation module with torsional strength. In: IEEE/RSJ IROS (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Department of Computer ScienceUNC CharlotteCharlotteUSA
  2. 2.Robotics Engineering, WPIWorcesterUSA

Personalised recommendations