Probabilistic Classification of Grasping Behaviours Using Visuo-Haptic Perception

  • S. Jafar Hosseini
  • Diego R. Faria
  • Jorge Lobo
  • Jorge Dias
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 372)


This paper presents a novel approach to visuo-haptic perception of grasping/manipulative tasks. The proposed approach is founded on a hierarchical Bayesian model which integrates the visual information with the haptic data to reach a reasonable percept of what is happening in grasping tasks. The primary goal of the approach is to identify what type of grasping behaviour is being performed by the human subject, and as a secondary goal, to simultaneously assess the quality of the respective grasping behaviour. For a simple set of grasping behaviours defined in this paper, preliminary experimental results indicate that the proposed approach could result in a robust and efficient perception of grasp behaviours.


Visuo-Haptic Perception Grasping Behaviours Hierarchical Bayesian Models (HBMs) 


  1. 1.
    Newell, F.N., Bülthoff, H.H., Ernst, M.O.: Cross-modal perception of actively explored objects. In: Eurohaptics 2003 Conference Proceedings, pp. 291–299 (2003)Google Scholar
  2. 2.
    Henriques, D.Y., Soechting, J.F.: Approaches to the study of haptic sensing. J. Neurophysiol. 93, 3036–3043 (2005)CrossRefGoogle Scholar
  3. 3.
    von Helmholtz, H.: Handbuch der physiologischen optik, vol. Bd. 9. Voss, Leipzig (1867)Google Scholar
  4. 4.
    Boshra, M., Zhang, H.: A constraint-satisfaction approach for 3D visionitouch-based object recognition. In: IEEWRSJ International Conference on Intelligent Robots and Systems, vol. 3, pp. 368–373 (1995)Google Scholar
  5. 5.
    Rucci, M., Bajcsy, R.: Learning visuo-tactile coordination in robotic systems. In: IEEE International Conference on Robotics and Automation, vol. 3, pp. 2678–2683 (1995)Google Scholar
  6. 6.
    Rafla, N.I., Merat, E.L.: Vision-taction integration for surface representation. In: IEEE International Conference on Systems Engineering, pp. 519–522 (1990)Google Scholar
  7. 7.
    Son, J.S., Howe, R. D., Wang, J., Hager, G.D.: Preliminary results on grasping with vision and touch. In: IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, IROS 1996, Osaka, Japan, vol. 3, pp. 1068–1075 (November 1996)Google Scholar
  8. 8.
    Martins, R., Faria, D.R., Dias, J.: Symbolic Level Generalization of In-hand Manipulation Tasks from Human Demonstrations using Tactile Data Information. In: Workshop on Grasping Planning and Task Learning by Imitation, Taipei, Taiwan (October 2010)Google Scholar
  9. 9.
    HANDLE project: Developmental pathway towards autonomy and dexterity in robot in-hand manipulation, EC FP7 ICT-236410 (2011),

Copyright information

© IFIP International Federation for Information Processing 2012

Authors and Affiliations

  • S. Jafar Hosseini
    • 1
  • Diego R. Faria
    • 1
  • Jorge Lobo
    • 1
  • Jorge Dias
    • 1
  1. 1.ISR - Institute of Systems and Robotics, DEEC, FCTUniversity of CoimbraPortugal

Personalised recommendations