Learning Human Priors for Task-Constrained Grasping

  • Martin Hjelm
  • Carl Henrik Ek
  • Renaud Detry
  • Danica Kragic
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9163)


An autonomous agent using manmade objects must understand how task conditions the grasp placement. In this paper we formulate task based robotic grasping as a feature learning problem. Using a human demonstrator to provide examples of grasps associated with a specific task, we learn a representation, such that similarity in task is reflected by similarity in feature. The learned representation discards parts of the sensory input that is redundant for the task, allowing the agent to ground and reason about the relevant features for the task. Synthesized grasps for an observed task on previously unseen objects can then be filtered and ordered by matching to learned instances without the need of an analytically formulated metric. We show on a real robot how our approach is able to utilize the learned representation to synthesize and perform valid task specific grasps on novel objects.


Point Cloud Kernel Density Estimate Irrelevant Feature Task Constraint Color Quantization 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Song, D., Huebner, K., Kyrki, V., Kragic, D.: Learning task constraints for robot grasping using graphical models. In: IROS (2010)Google Scholar
  2. 2.
    Song, D., Ek, C.H., Huebner, K., Kragic, D.: Embodiment-specific representation of robot grasping using graphical models and latent-space discretization. In: IROS, pp. 980–986 (2011)Google Scholar
  3. 3.
    Saxena, A., Driemeyer, J., Ng, A.Y.: Robotic grasping of novel objects using vision. Int. J. Robot. Res. 27(2), 157–173 (2008)CrossRefGoogle Scholar
  4. 4.
    Boularias, A., Kroemer, O., Peters, J.: Learning robot grasping from 3-D images with Markov Random Fields. In: IROS, pp. 1548–1553 (2011)Google Scholar
  5. 5.
    Detry, R., Ek, C.H., Madry, M., Kragic, D.: Learning a dictionary of prototypical grasp-predicting parts from grasping experience. In: ICRA (2011)Google Scholar
  6. 6.
    Herzog, A., Pastor, P., Kalakrishnan, M., Righetti, L., Asfour, T., Schaal, S.: Template-based learning of grasp selection. In: ICRA (2012)Google Scholar
  7. 7.
    Kroemer, O., Ugur, E., Oztop, E., Peters, J.: A kernel-based approach to direct action perception. In: ICRA, pp. 2605–2610 (2012)Google Scholar
  8. 8.
    Ying, L., Fu, J.L., Pollard, N.S.: Data-driven grasp synthesis using shape matching and task-based pruning. IEEE Trans. Visual Comput. Graphics 13(4), 732–747 (2007)CrossRefGoogle Scholar
  9. 9.
    Stark, M., Lies, P., Zillich, M., Wyatt, J.C., Schiele, B.: Functional object class detection based on learned affordance cues. In: Gasteratos, A., Vincze, M., Tsotsos, J.K. (eds.) ICVS 2008. LNCS, vol. 5008, pp. 435–444. Springer, Heidelberg (2008) CrossRefGoogle Scholar
  10. 10.
    Aleotti, J., Caselli, S.: Part-based robot grasp planning from human demonstration. In: ICRA, pp. 4554–4560 (2011)Google Scholar
  11. 11.
    Hjelm, M., Detry, R., Ek, C.H., Kragic, D.: Cross-object grasp transfer. In: ICRA, Representations for Cross-task (2014)Google Scholar
  12. 12.
    Weinberger, K.Q., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. J. Mach. Learn. Res. 10, 207–244 (2009)zbMATHGoogle Scholar
  13. 13.
    Bertenthal, B.I.: Origins and early development of perception, action, and representation. Annu. Rev. Psychol. 47(1), 431–459 (1996)CrossRefGoogle Scholar
  14. 14.
    Berthier, N.E., Clifton, R.K., Gullapalli, V., McCall, D.D., Robin, D.J.: Visual information and object size in the control of reaching. J. Mot. Behav. 28(3), 187–197 (1996)CrossRefGoogle Scholar
  15. 15.
    Csurka, G., Dance, C., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: Workshop on Statistical Learning in Computer Vision, ECCV, vol. 1, pp. 1–2. Prague (2004)Google Scholar
  16. 16.
    Rusu, R.B., Blodow, N., Beetz, M.: Fast Point Feature Histograms (FPFH) for 3D registration. In: ICRA, pp. 3212–3217 (2009)Google Scholar
  17. 17.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. Int. J. Comput. Vis. 59(2), 167–181 (2004)CrossRefGoogle Scholar
  18. 18.
    Bergström, N., Bohg, J., Kragic, D.: Integration of visual cues for robotic grasping. In: Fritz, M., Schiele, B., Piater, J.H. (eds.) ICVS 2009. LNCS, vol. 5815, pp. 245–254. Springer, Heidelberg (2009) CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Martin Hjelm
    • 1
  • Carl Henrik Ek
    • 1
  • Renaud Detry
    • 2
  • Danica Kragic
    • 1
  1. 1.Autonomous Systems and the Computer Vision and Active Perception LabCSC, KTH Royal Institute of TechnologyStockholmSweden
  2. 2.University of LiègeLiègeBelgium

Personalised recommendations