From Motor Learning to Interaction Learning in Robots

Volume 264 of the series Studies in Computational Intelligence pp 451-465

Learning Continuous Grasp Affordances by Sensorimotor Exploration

  • R. DetryAffiliated withUniversity of Liège
  • , E. BaşeskiAffiliated withUniversity of Southern
  • , M. PopovićAffiliated withUniversity of Southern
  • , Y. TouatiAffiliated withUniversity of Southern
  • , N. KrügerAffiliated withUniversity of Southern
  • , O. KroemerAffiliated withMPI for Biological Cybernetics
  • , J. PetersAffiliated withMPI for Biological Cybernetics
  • , J. PiaterAffiliated withUniversity of Liège

* Final gross prices may vary according to local VAT.

Get Access


We develop means of learning and representing object grasp affordances probabilistically. By grasp affordance, we refer to an entity that is able to assess whether a given relative object-gripper configuration will yield a stable grasp. These affordances are represented with grasp densities, continuous probability density functions defined on the space of 3D positions and orientations. Grasp densities are registered with a visual model of the object they characterize. They are exploited by aligning them to a target object using visual pose estimation. Grasp densities are refined through experience: A robot “plays” with an object by executing grasps drawn randomly for the object’s grasp density. The robot then uses the outcomes of these grasps to build a richer density through an importance sampling mechanism. Initial grasp densities, called hypothesis densities, are bootstrapped from grasps collected using a motion capture system, or from grasps generated from the visual model of the object. Refined densities, called empirical densities, represent affordances that have been confirmed through physical experience. The applicability of our method is demonstrated by producing empirical densities for two object with a real robot and its 3-finger hand. Hypothesis densities are created from visual cues and human demonstration.