Early Reactive Grasping with Second Order 3D Feature Relations

  • Daniel Aarno
  • Johan Sommerfeld
  • Danica Kragic
  • Nicolas Pugeault
  • Sinan Kalkan
  • Florentin Wörgötter
  • Dirk Kraft
  • Norbert Krüger
Part of the Lecture Notes in Control and Information Sciences book series (LNCIS, volume 370)


One of the main challenges in the field of robotics is to make robots ubiquitous. To intelligently interact with the world, such robots need to understand the environment and situations around them and react appropriately, they need context-awareness. But how to equip robots with capabilities of gathering and interpreting the necessary information for novel tasks through interaction with the environment and by providing some minimal knowledge in advance? This has been a longterm question and one of the main drives in the field of cognitive system development.

The main idea behind the work presented in this paper is that the robot should, like a human infant, learn about objects by interacting with them, forming representations of the objects and their categories that are grounded in its embodiment. For this purpose, we study an early learning of object grasping process where the agent, based on a set of innate reflexes and knowledge about its embodiment. We stress out that this is not the work on grasping, it is a system that interacts with the environment based on relations of 3D visual features generated trough a stereo vision system. We show how geometry, appearance and spatial relations between the features can guide early reactive grasping which can later on be used in a more purposive manner when interacting with the environment.


Unknown Object Arbitrary Object Stereo Vision System Grasp Planning Shape Primitive 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Azad, P., Asfour, T., Dillmann, R.: Combining appearance-based and model-based methods for real-time object recognition and 6d localization. In: IEEE International Conference on Intelligent Robots and Systems (2006)Google Scholar
  2. 2.
    Bicchi, A., Kumar, V.: Robotic grasping and contact: A review. In: IEEE International Conference on Robotics and Automation, pp. 348–353 (2000)Google Scholar
  3. 3.
    Ding, D., Liu, Y.H., Wang, S.: Computing 3-d optimal formclosure grasps. In: IEEE International Conference on Robotics and Automation, pp. 3573–3578 (2000)Google Scholar
  4. 4.
    Fitzpatrick, P., Metta, G., Natale, L., Rao, S., Sandini, G.: Learning About Objects Through Action - Initial Steps Towards Artificial Cognition. In: IEEE International Conference on Robotics and Automation, pp. 3140–3145 (2003)Google Scholar
  5. 5.
    Hauck, A., Rüttinger, J., Sorg, M., Färber, G.: Visual Determination of 3D Grasping Points on Unknown Objects with a Binocular Camera System. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 272–278 (1999)Google Scholar
  6. 6.
  7. 7.
    Kamon, I., Flash, T., Edelman, S.: Learning Visually Guided Grasping: A Test Case in Sensorimotor Learning. IEEE Transactions on Systems, Man and Cybernetics 28(3), 266–276 (1998)CrossRefGoogle Scholar
  8. 8.
    Krüger, N., Felsberg, M.: An explicit and compact coding of geometric and structural information applied to stereo matching. Pattern Recognition Letters 25(8), 849–863 (2004)CrossRefGoogle Scholar
  9. 9.
    Krüger, N., Lappe, M., Wörgötter, F.: Biologically motivated multi-modal processing of visual primitives. The Interdisciplinary Journal of Artificial Intelligence and the Simulation of Behaviour 1(5), 417–428 (2004)Google Scholar
  10. 10.
    Krüger, N., Wörgötter, F.: Multi-modal primitives as functional models of hyper-columns and their use for contextual integration. In: De Gregorio, M., Di Maio, V., Frucci, M., Musio, C. (eds.) BVAI 2005. LNCS, vol. 3704, pp. 157–166. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  11. 11.
    Miller, A.T., Allen, P.: Graspit!: A versatile simulator for grasping analysis. In: ASME International Mechanical Engineering Congress and Exposition (2000)Google Scholar
  12. 12.
    Miller, A.T., Knoop, S., Allen, P.K., Christensen, H.I.: Automatic grasp planning using shape primitives. In: IEEE International Conference on Robotics and Automation, pp. 1824–1829 (2003)Google Scholar
  13. 13.
    Morales, A., Chinellato, E., Fagg, A.H., del Pobil, A.: Using experience for assessing grasp reliability. International Journal of Humanoid Robotics 1(4), 671–691 (2004)CrossRefGoogle Scholar
  14. 14.
    Morales, A., Recatalá, G., Sanz, P.J., del Pobil, Á.P.: Heuristic Vision-Based Computation of Planar Antipodal Grasps on Unknown Objects. In: IEEE International Conference on Robotics and Automation, pp. 583– 588 (2001)Google Scholar
  15. 15.
    Platt Jr., R., Fagg, A.H., Grupen, R.A.: Extending fingertip grasping to whole body grasping. In: International Conference on Robotics and Automation, pp. 2677–2682 (2003)Google Scholar
  16. 16.
    Pollard, N.S.: Parallel methods for synthesizing whole-hand grasps from generalized prototypes. PhD thesis, Dept. of Electrical Engineering and Computer Science, Massachusetts Institute of Technology (1994)Google Scholar
  17. 17.
    Pollard, N.S.: Closure and quality equivalence for efficient synthesis of grasps from examples. International Journal of Robotic Research 23(6), 595–613 (2004)CrossRefMathSciNetGoogle Scholar
  18. 18.
    Pugeault, N., Wörgötter, F., Krüger, N.: Multi-modal scene reconstruction using perceptual grouping constraints. In: Proceedings of the 5th IEEE Computer Society Workshop on Perceptual Organization in Computer Vision (in conjunction with IEEE CVPR 2006) (2006)Google Scholar
  19. 19.
    Rössler, B., Zhang, J., Knoll, A.: Visual Guided Grasping of Aggregates using Self-Valuing Learning. In: IEEE International Conference on Robotics and Automation, pp. 3912–3917 (2002)Google Scholar
  20. 20.
    Rutishauser, M., Stricker, M.: Searching for Grasping Opportunities on Unmodeled 3D Objects. In: British Machine Vision Conference, pp. 277–286 (1995)Google Scholar
  21. 21.
    Stoytchev, A.: Behavior-Grounded Representation of Tool Affordances. In: IEEE International Conference on Robotics and Automation, pp. 3060–3065 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Daniel Aarno
    • 1
  • Johan Sommerfeld
    • 1
  • Danica Kragic
    • 1
  • Nicolas Pugeault
    • 2
  • Sinan Kalkan
    • 3
  • Florentin Wörgötter
    • 3
  • Dirk Kraft
    • 4
  • Norbert Krüger
    • 4
  1. 1.Centre for Autonomous Systems, Computational Vision and Active Perception, School of Computer Science and Communication, Royal Institute of Technology, 10044 StockholmSweden
  2. 2.School of Informatics, University of Edinburgh, 2 Buccleuch Place, Edinburgh EH8 9LWUnited Kingdom
  3. 3.Bernstein Center for Computational Neuroscience, University of Goettingen, Bunsenstr. 10, 37073 GoettingenGermany
  4. 4.The Maersk Mc-Kinney Moller Institute, University of Southern Denmark, Campusvej 55, 5230 Odense MDenmark

Personalised recommendations