Grasping with Vision Descriptors and Motor Primitives

  • Oliver Kroemer
  • Renaud Detry
  • Justus Piater
  • Jan Peters
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 89)


Grasping is one of the most important abilities needed for future service robots. Given the task of picking up an object from betweem clutter, traditional robotics approaches would determine a suitable grasping point and then use a movement planner to reach the goal. The planner would require precise and accurate information about the environment and long computation times, both of which may not always be available. Therefore, methods for executing grasps are required, which perform well with information gathered from only standard stereo vision, and make only a few necessary assumptions about the task environment. We propose techniques that reactively modify the robot’s learned motor primitives based on information derived from Early Cognitive Vision descriptors. The proposed techniques employ non-parametric potential fields centered on the Early Cognitive Vision descriptors to allow for curving hand trajectories around objects, and finger motions that adapt to the object’s local geometry. The methods were tested on a real robot and found to allow for easier imitation learning of human movements and give a considerable improvement to the robot’s performance in grasping tasks.


Dynamical motor primitives Early cognitive vision descriptors Grasping 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arimoto, S.: Control Theory of Multi-fingered Hands. Springer, London (2008)Google Scholar
  2. 2.
    Bard, C., Troccaz, J., Vercelli, G.: Shape analysis and hand preshaping for grasping. In: Proceedings of IROS 1991(1991)Google Scholar
  3. 3.
    Bicchi, A., Kumar, V.: Robotic grasping and contact: a Review. In: Proceedings of ICRA 2000 (2000)Google Scholar
  4. 4.
    Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Heidelberg (2006)zbMATHGoogle Scholar
  5. 5.
    Chieffi, S., Gentilucci, M.: Coordination between the transport and the grasp components during prehension movements (1993)Google Scholar
  6. 6.
    Detry, R., Kroemer, O., Popovic, M., Touati, Y., Baseski, E., Krueger, N., Peters, J., Piater, J.: Object-specific grasp affordance densities. In: ICDL (2009)Google Scholar
  7. 7.
    Detry, R., Pugeault, N., Piater, J.: Probabilistic pose recovery using learned hierarchical object models. In: International Cognitive Vision Workshop (2008)Google Scholar
  8. 8.
    Graziano, M.S.: Progress in understanding spatial coordinate systems in the primate brain. In: Neuron (2006)Google Scholar
  9. 9.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000)zbMATHGoogle Scholar
  10. 10.
    Hsiao, K., Nangeroni, P., Huber, M., Saxena, A., Ng, A.: Reactive grasping using optical proximity sensors. In: Proceedings of ICRA 2009 (2009)Google Scholar
  11. 11.
    Iberall, T.: Grasp planning for human prehension. In: Proceedings of ICAI 1987 (1987)Google Scholar
  12. 12.
    Ijspeert, A.J., Nakanishi, J., Schaal, S.: Movement imitation with nonlinear dynamical systems in humanoid robots. In: ICRA (2002)Google Scholar
  13. 13.
    Ijspeert, A.J., Nakanishi, J., Schaal, S.: Learning attractor landscapes for learning motor primitives. In: NIPS (2003)Google Scholar
  14. 14.
    Jeannerod, M.: Grasping Objects: The Hand as a Pattern Recognition Device. Perspectives of Motor Behaviour and Its Neural Basis (1997)Google Scholar
  15. 15.
    Jeannerod, M.: The study of hand movements during grasping. A historical perspective. In: Sensorimotor Control of Grasping: Physiology and Pathophysiology, Cambridge University Press, Cambridge (2009)Google Scholar
  16. 16.
    Krueger, N., Lappe, M., Woergoetter, F.: Biologically motivated multimodal processing of visual primitives. The Interdisciplinary Journal of Artificial Intelligence and the Simulation of Behaviour (2004)Google Scholar
  17. 17.
    Oztop, E., Bradley, N.S., Arbib, M.A.: Infant grasp learning: a computational model (2004)Google Scholar
  18. 18.
    Oztop, E., Kawato, M.: Models for the control of grasping. In: Sensorimotor Control of Grasping: Physiology and Pathophysiology, Cambridge University Press, Cambridge (2009)Google Scholar
  19. 19.
    Park, D.-H., Hoffmann, H., Pastor, P., Schaal, S.: Movement reproduction and obstacle avoidance with dynamic movement primitives and potential fields. In: IEEE International Conference on Humanoid Robots(HUMANOIDS) (2008)Google Scholar
  20. 20.
    Pugeault, N.: Early Cognitive Vision: Feedback Mechanisms for the Disambiguation of Early Visual Representation. Vdm Verlag Dr. Mueller (2008)Google Scholar
  21. 21.
    Saxena, A., Dreimeyer, J., Kearns, J., Osondu, C., Ng, A.: Learning to Grasp Novel Objects using Vision. In: Experimental Robotics. Springer, Berlin (2008)Google Scholar
  22. 22.
    Schaal, S., Peters, J., Nakanishi, J., Ijspeert, A.: Learning movement primitives. In: Proceedings of ISRR 2003 (2003)Google Scholar
  23. 23.
    Steffen, J., Haschke, R., Ritter, H.: Experience-based and tactile-driven dynamic grasp control. In: Proceedings of IRS 2007 (2007)Google Scholar
  24. 24.
    Wank, V., Fischer, A., Bos, K., Boesnach, I., Moldenhauer, J., Beth, T.: Similarities and varieties in human motiontrajectories of predefined grasping and disposing movements. International Journal of Humanoid Robotics (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Oliver Kroemer
    • 1
  • Renaud Detry
    • 2
  • Justus Piater
    • 2
  • Jan Peters
    • 1
  1. 1.Max Planck Institute for Biological CyberneticsTuebingenGermany
  2. 2.University of LiegeLiegeBelgium

Personalised recommendations