Integration of Visual Cues for Robotic Grasping

  • Niklas Bergström
  • Jeannette Bohg
  • Danica Kragic
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5815)


In this paper, we propose a method that generates grasping actions for novel objects based on visual input from a stereo camera. We are integrating two methods that are advantageous either in predicting how to grasp an object or where to apply a grasp. The first one reconstructs a wire frame object model through curve matching. Elementary grasping actions can be associated to parts of this model. The second method predicts grasping points in a 2D contour image of an object. By integrating the information from the two approaches, we can generate a sparse set of full grasp configurations that are of a good quality. We demonstrate our approach integrated in a vision system for complex shaped objects as well as in cluttered scenes.


Dynamic Time Warping Stereo Camera Contour Point Real Plane Good Hypothesis 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Nguyen, V.-D.: Constructing stable grasps. Int. J. on Robotics Research 8(1), 26–37 (1989)CrossRefGoogle Scholar
  2. 2.
    Kraft, D., Pugeault, N., Baseski, E., Popovic, M., Kragic, D., Kalkan, S., Wörgötter, F., Krueger, N.: Birth of the Object: Detection of Objectness and Extraction of Object Shape through Object Action Complexes. Int. J. of Humanoid Robotics (2009)Google Scholar
  3. 3.
    Hübner, K., Kragic, D.: Selection of Robot Pre-Grasps using Box-Based Shape Approximation. In: IEEE Int. Conf. on Intelligent Robots and Systems, pp. 1765–1770 (2008)Google Scholar
  4. 4.
    Stark, M., Lies, P., Zillich, M., Wyatt, J.C., Schiele, B.: Functional Object Class Detection Based on Learned Affordance Cues. In: Gasteratos, A., Vincze, M., Tsotsos, J.K. (eds.) ICVS 2008. LNCS (LNAI), vol. 5008, pp. 435–444. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  5. 5.
    Saxena, A., Driemeyer, J., Kearns, J., Ng, A.Y.: Robotic Grasping of Novel Objects. Neural Information Processing Systems 19, 1209–1216 (2006)Google Scholar
  6. 6.
    Saxena, A., Wong, L., Ng, A.Y.: Learning Grasp Strategies with Partial Shape Information. In: AAAI Conf. on Artificial Intelligence, pp. 1491–1494 (2008)Google Scholar
  7. 7.
    Speth, J., Morales, A., Sanz, P.J.: Vision-Based Grasp Planning of 3D Objects by Extending 2D Contour Based Algorithms. In: IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (2008)Google Scholar
  8. 8.
    Detry, R., Başeski, E., Krüger, N., Popović, M., Touati, Y., Kroemer, O., Peters, J., Piater, J.: Learning object-specific grasp affordance densities. In: Int. Conf. on Development and Learning (2009)Google Scholar
  9. 9.
    Björkman, M., Eklundh, J.O.: Attending, Foveating and Recognizing Objects in Real World Scenes. In: British Machine Vision Conference (2004)Google Scholar
  10. 10.
    Bergström, N., Kragic, D.: Partial 3D Reconstruction of Objects for Early Reactive Grasping. Technical report, CAS, KTH Stockholm (2009),
  11. 11.
    Romero, J., Kragic, D., Kyrki, V., Argyros, A.: Dynamic Time Warping for Binocular Hand Tracking and Reconstruction. In: IEEE Int. Conf. on Robotics and Automation, May 2008, pp. 2289–2294 (2008)Google Scholar
  12. 12.
    Bohg, J., Kragic, D.: Grasping Familiar Objects Using Shape Context. In: Int. Conf. on Advanced Robotics (June 2009)Google Scholar
  13. 13.
    Belongie, S., Malik, J., Puzicha, J.: Shape Matching and Object Recognition Using Shape Contexts. IEEE Trans. on Pattern Analysis and Machine Intelligence 24(4), 509–522 (2002)CrossRefGoogle Scholar
  14. 14.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Niklas Bergström
    • 1
  • Jeannette Bohg
    • 1
  • Danica Kragic
    • 1
  1. 1.Computer Vision and Active Vision Laboratory, Centre for Autonomous SystemRoyal Institute of TechnologyStockholmSweden

Personalised recommendations