Experimental Robotics

Volume 79 of the series Springer Tracts in Advanced Robotics pp 49-63

Mightability: A Multi-state Visuo-spatial Reasoning for Human-Robot Interaction

  • Amit Kumar PandeyAffiliated withCNRS; LAASUniversité de Toulouse; UPS, INSA, INP, ISAE; LAAS
  • , Rachid AlamiAffiliated withCNRS; LAASUniversité de Toulouse; UPS, INSA, INP, ISAE; LAAS

* Final gross prices may vary according to local VAT.

Get Access


We, the Humans, are capable of estimating various abilities of ourselves and of the person we are interacting with. Visibility and reachability are among two such abilities. Studies in neuroscience and psychology suggest that from the age of 12-15 months children start to understand the occlusion of others line-of-sight and from the age of 3 years they start to develop the ability, termed as perceived reachability for self and for others. As such capabilities evolve in the children, they start showing intuitive and proactive behavior by perceiving various abilities of the human partner.

Inspired from such studies, which suggest that visuo-spatial perception plays an important role in Human-Human interaction, we propose to equip our robot to perceive various types of abilities of the agents in the workspace. The robot perceives such abilities not only from the current state of the agent but also by virtually putting an agent into various achievable states, such as turn left, stand up, etc. As the robot estimates what an agent might be able to ‘see’ and ‘reach’ if will be in a particular state, we term such analyses as Mightability Analyses. Currently the robot is equipped to perform such Mightability analyses at two levels: cells in the 3D grid and objects in the space, which we termed as Mightability Maps (MM) and Object Oriented Mightabilities (OOM) respectively.

We have shown the applications of Mightability analyses in performing various co-operative tasks like show and make an object accessible to the human as well as competitive tasks like hide and put away an object from the human. Such Mightability analyses equip the robot for higher-level learning and decisional capabilities as well as could facilitate the robot for better verbalize interaction and proactive behavior.