International Journal of Social Robotics

, Volume 5, Issue 2, pp 215–236 | Cite as

Towards a Task-Aware Proactive Sociable Robot Based on Multi-state Perspective-Taking

  • Amit Kumar Pandey
  • Muhammad Ali
  • Rachid Alami


Robots are expected to cooperate with humans in day-to-day interaction. One aspect of such cooperation is behaving proactively. In this paper we will enable our robots, equipped with visuo-spatial perspective-taking capabilities, to behave proactively based on reasoning ‘where’ its human partner might perform a particular task with different effort levels. For this, the robot analyzes the agents’ abilities not only from the current state but also from a set of different states the agent might attain.

Depending on the task and the situation, the robot exhibits different types of proactive behaviors, such as, reaching out, suggesting a solution and providing clues by head movement, for two different tasks performed by the human partner: give and make accessible. These proactive behaviors are intended to be informative to reduce confusion of the human partner, to communicate the robot’s ability and intention and to guide the partner for better cooperation.

We have validated the behaviors by user studies, which suggest that such proactive behaviors reduce the ‘confusion’ and ‘effort’ of the users. Further, the participants reported the robot to be more ‘supportive and aware’ compared to the situations where the robot was non-proactive.

Such proactive behaviors could enrich multi-modal interaction and cooperation capabilities of the robot as well as help in developing more complex socially expected and accepted behaviors in the human centered environment.


Proactive robot Human-robot interaction Social robot Multi-state perspective taking 


  1. 1.
    Alili S, Alami R, Montreuil V (2009) A task planner for an autonomous social robot. In: Asama H, Kurokawa H, Ota J, Sekiyama K (eds) Distributed autonomous robotic systems, vol 8. Springer, Berlin, pp 335–344 Google Scholar
  2. 2.
    Berlin M, Gray J, Thomaz AI, Breazeal C (2006) Perspective taking: an organizing principle for learning in human-robot interaction. In: Proceedings of the 21st national conference on artificial intelligence (AAAI’06), vol 2. AAAI Press, Menlo Park, pp 1444–1450 Google Scholar
  3. 3.
    Breazeal C (2009) Role of expressive behaviour for robots that learn from people. Phil Trans R Soc B, Biol Sci 364:3527–3538 CrossRefGoogle Scholar
  4. 4.
    Broquère X, Sidobre D, Nguyen K (2010) From motion planning to trajectory control with bounded jerk for service manipulator robots. In: IEEE international conference on robotics and automation, pp 4505–4510 Google Scholar
  5. 5.
    Buss M, Carton D, Gonsior B, Kuehnlenz K, Landsiedel C, Mitsou N, de Nijs R, Zlotowski J, Sosnowski S, Strasser E, Tscheligi M, Weiss A, Wollherr D (2011) Towards proactive human-robot interaction in human environments. In: 2nd international conference on cognitive infocommunications (CogInfoCom), 2011, pp 1–6 Google Scholar
  6. 6.
    Cakmak M, Srinivasa S, Lee MK, Forlizzi J, Kiesler S (2011) Human preferences for robot-human hand-over configurations. In: IEEE/RSJ international conference on intelligent robots and systems (IROS), 2011, pp 1986–1993 Google Scholar
  7. 7.
    Carlson T, Demiris Y (2008) Human-wheelchair collaboration through prediction of intention and adaptive assistance. In: IEEE international conference on robotics and automation (ICRA), pp 3926–3931 Google Scholar
  8. 8.
    Cesta A, Cortellessa G, Pecora F, Rasconi R (2007) Supporting interaction in the robocare intelligent assistive environment. In: Proceedings of AAAI Spring symposium on interaction challenges for intelligent assistants. AAAI Press, Menlo Park, pp 18–25 Google Scholar
  9. 9.
    Chella A, Dindo H, Infantino I (2006) A cognitive framework for imitation learning. Robot Auton Syst 54(5):403–408 CrossRefGoogle Scholar
  10. 10.
    Choi HJ, Mark LS (2004) Scaling affordances for human reach actions. Hum Mov Sci 23(6):785–806 CrossRefGoogle Scholar
  11. 11.
    Clark HH (2003) Pointing and placing. In: Kita S (ed) Pointing: where language, culture, and cognition meet. Erlbaum, Hillsdale, pp 243–268 Google Scholar
  12. 12.
    Clodic A, Cao H, Alili S, Montreuil V, Alami R, Chatila R (2009) Shary: a supervision system adapted to human-robot interaction. In: Khatib O, Kumar V, Pappas G (eds) Experimental robotics. Springer tracts in advanced robotics, vol 54. Springer, Berlin, pp 229–238 CrossRefGoogle Scholar
  13. 13.
    Cramer HS, Kemper NA, Amin A, Evers V (2009) The effects of robot touch and proactive behaviour on perceptions of human-robot interactions. In: Proceedings of the 4th ACM/IEEE international conference on human robot interaction (HRI), pp 275–276 CrossRefGoogle Scholar
  14. 14.
    Dautenhahn K (2007) Socially intelligent robots: dimensions of human–robot interaction. Philos Trans R Soc Lond B, Biol Sci 362(1480):679–704 CrossRefGoogle Scholar
  15. 15.
    Dehais F, Sisbot EA, Alami R, Causse M (2011) Physiological and subjective evaluation of a human-robot object hand-over task. Appl Ergon 42(6):785–792 CrossRefGoogle Scholar
  16. 16.
    Duong T, Bui H, Phung D, Venkatesh S (2005) Activity recognition and abnormality detection with the switching hidden semi-Markov model. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 838–845 Google Scholar
  17. 17.
    Eimler SC, Krämer NC, Patten AMVD (2010) Prerequisites for human-agent- and human-robot interaction: towards an integrated theory. In: Trappl R (ed) European meetings on cybernetics and systems research (EMCSR) Google Scholar
  18. 18.
    Flanagan JR, Johansson RS (2003) Action plans used in action observation. Nature 6950:769–771. doi: 10.1038/nature01861 CrossRefGoogle Scholar
  19. 19.
    Gardner DL, Mark LS, Ward JA, Edkins H (2001) How do task characteristics affect the transitions between seated and standing reaches? Ecol Psychol 13(4):245–274 CrossRefGoogle Scholar
  20. 20.
    Gharbi M, Cortes J, Simeon T (2008) A sampling-based path planner for dual-arm manipulation. In: IEEE/ASME international conference on advanced intelligent mechatronics, pp 383–388 Google Scholar
  21. 21.
    Gredeback G, Kochukhova O (2010) Goal anticipation during action observation is influenced by synonymous action capabilities, a puzzling developmental study. Exp Brain Res 202:493–497 CrossRefGoogle Scholar
  22. 22.
    Kozima HY (2001) In search of otogenetic prerequisites for embodied social intelligence. In: Proceedings of the workshop on emergence and development on embodied cognition (EDEC-2001); international conference on cognitive science (ICCS-2001), pp 30–34 Google Scholar
  23. 23.
    Hoffman G (2010) Anticipation in human-robot interaction. In: Proceedings of AAAI Spring symposium Google Scholar
  24. 24.
    Holthaus P, Lutkebohle I, Hanheide M, Wachsmuth S (2010) Can I help you? In: Ge S, Li H, Cabibihan JJ, Tan Y (eds) Social robotics. Lecture notes in computer science, vol 6414. Springer, Berlin, pp 325–334 CrossRefGoogle Scholar
  25. 25.
    Holthaus P, Pitsch K, Wachsmuth S (2011) How can I help? Int J Soc Robot 3:383–393. doi: 10.1007/s12369-011-0108-9 CrossRefGoogle Scholar
  26. 26.
    Huber M, Knoll A, Brandt T, Glasauer S (2009) Handing over a cube. In: Basic and clinical aspects of vertigo and dizziness. Annals of the New York Academy of Sciences, vol 1164, pp 380–382 Google Scholar
  27. 27.
    Iio T, Shiomi M, Shinozawa K, Akimoto T, Shimohara K, Hagita N (2011) Investigating entrainment of people’s pointing gestures by robot’s gestures using a WOz method. Int J Soc Robot 3:405–414. doi: 10.1007/s12369-011-0112-0 CrossRefGoogle Scholar
  28. 28.
    Imai M, Ono T, Ishiguro H (2003) Physical relation and expression: joint attention for human-robot interaction. IEEE Trans Ind Electron 50(4):636–643 CrossRefGoogle Scholar
  29. 29.
    Jordan JS, Hunsinger M (2008) Learned patterns of action effect anticipation contribute to the spatial displacement of continuously moving stimuli. Journal of Experimental Psychology: Human Perception and Performance, 113–124 Google Scholar
  30. 30.
    Kemp C, Edsinger A, Torres-Jara E (2007) Challenges for robot manipulation in human environments. IEEE Robot Autom Mag 14(1):20–29 CrossRefGoogle Scholar
  31. 31.
    Kennedy WG, Bugajska MD, Harrison AM, Trafton JG (2009) “like-me” simulation as an effective and cognitively plausible basis for social robotics. Int J Soc Robot 1(2):181–194 CrossRefGoogle Scholar
  32. 32.
    Khatib O, Demircan E, Sapio VD, Sentis L, Besier T, Delp S (2009) Robotics-based synthesis of human motion. J Physiol 103:211–219 Google Scholar
  33. 33.
    Kockler H, Scheef L, Tepest R, David N, Bewernick B, Newen A, Schild H, May M, Vogeley K (2010) Visuospatial perspective taking in a dynamic environment: perceiving moving objects from a first-person-perspective induces a disposition to act. Conscious Cogn 19(3):690–701 CrossRefGoogle Scholar
  34. 34.
    Kwon WY, Suh IH (2010) Probabilistic temporal prediction for proactive action selection. In: IROS workshop on probabilistic graphical models in robotics, GraphBot Google Scholar
  35. 35.
    Kwon WY, Suh IH (2011) Towards proactive assistant robots for human assembly tasks. In: Proceedings of the 6th international conference on human-robot interaction, HRI’11. ACM, New York, pp 175–176 Google Scholar
  36. 36.
    L’Abbate M (2007) Modelling proactive behaviour of conversational interfaces. Ph.D. thesis, TU Darmstadt, Darmstadt Google Scholar
  37. 37.
    Lemaignan S, Ros R, Sisbot E, Alami R, Beetz M (2012) Grounding the interaction: anchoring situated discourse in everyday human-robot interaction. Int J Soc Robot 4:181–199 CrossRefGoogle Scholar
  38. 38.
    Li J, Chignell M (2011) Communication of emotion in social robots through simple head and arm movements. Int J Soc Robot 3:125–142. doi: 10.1007/s12369-010-0071-x CrossRefGoogle Scholar
  39. 39.
    Lohse M (2010) Investigating the influence of situations and expectations on user behavior-empirical analyses in human-robot interaction. Ph.D. thesis, Universität Bielefeld Google Scholar
  40. 40.
    Louwerse M, Bangerter A (2005) Focusing attention with deictic gestures and linguistic expressions. In: Proceedings of the 27th annual meeting of the cognitive science society Google Scholar
  41. 41.
    Marin-Urias L, Sisbot E, Pandey A, Tadakuma R, Alami R (2009) Towards shared attention through geometric reasoning for human robot interaction. In: 9th IEEE-RAS international conference on humanoid robots (Humanoids), pp 331–336 Google Scholar
  42. 42.
    Nicolescu M, Mataric M (2001) Learning and interacting in human-robot domains. IEEE Trans Syst Man Cybern, Part A, Syst Hum 31(5):419–430 CrossRefGoogle Scholar
  43. 43.
    Pandey A, Alami R (2010) Mightability maps: a perceptual level decisional framework for co-operative and competitive human-robot interaction. In: IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 5842–5848 Google Scholar
  44. 44.
    Pandey AK, Alami R (2011) Towards task understanding through multi-state visuo-spatial perspective taking for human-robot interaction. In: IJCAI workshop on agents learning interactively from human teachers (ALIHT-IJCAI) Google Scholar
  45. 45.
    Pardowitz M, Dillmann R (2007) Towards life-long learning in household robots: the piagetian approach. In: IEEE 6th international conference on development and learning (ICDL), pp 88–93 Google Scholar
  46. 46.
    Ros R, Lemaignan S, Sisbot E, Alami R, Steinwender J, Hamann K, Warneken F (2010) Which one? Grounding the referent based on efficient human-robot interaction. In: IEEE RO-MAN, 2010, pp 570–575 Google Scholar
  47. 47.
    Salichs M, Barber R, Khamis A, Malfaz M, Gorostiza J, Pacheco R, Rivas R, Corrales A, Delgado E, Garcia D (2006) Maggie: a robotic platform for human-robot social interaction. In: IEEE conference on robotics, automation and mechatronics, pp 1–7 Google Scholar
  48. 48.
    Sapio V, Warren J, Khatib O (2006) Predicting reaching postures using a kinematically constrained shoulder model. In: Lennarcic J, Roth B (eds) Advances in robot kinematics. Springer, Dordrecht, pp 209–218 CrossRefGoogle Scholar
  49. 49.
    Saut JP, Sahbani A, Perdereau V (2011) A generic motion planner for robot multi-fingered manipulation. Adv Robot 25(1–2):23–46 CrossRefGoogle Scholar
  50. 50.
    Saut JP, Sidobre D (2012) Efficient models for grasp planning with a multi-fingered hand. Robot Auton Syst 60(3):347–357 CrossRefGoogle Scholar
  51. 51.
    Scassellati B (2002) Theory of mind for a humanoid robot. Auton Robots 12:13–24 CrossRefzbMATHGoogle Scholar
  52. 52.
    Schmid A, Weede O, Worn H (2007) Proactive robot task selection given a human intention estimate. In: 16th IEEE international symposium on robot and human interactive communication (RO-MAN), pp 726–731 Google Scholar
  53. 53.
    Schrempf O, Hanebeck U, Schmid A, Worn H (2005) A novel approach to proactive human-robot cooperation. In: IEEE international symposium on robot and human interactive communication (Ro-MAN), pp 555–560 Google Scholar
  54. 54.
    Sciutti A, Bisio A, Nori F, Metta G, Fadiga L, Pozzo T, Sandini G (2012) Measuring human-robot interaction through motor resonance. Int J Soc Robot 4:223–234. doi: 10.1007/s12369-012-0143-1 CrossRefGoogle Scholar
  55. 55.
    Sebanz N, Knoblich G (2009) Prediction in joint action: what, when, and where. Top Cogn Sci 1(2):353–367 CrossRefGoogle Scholar
  56. 56.
    Simeon T, Laumond PJ, Lamiraux F (2001) Move3d: a generic platform for path planning. In: 4th international symposium on assembly and task planning, pp 25–30 Google Scholar
  57. 57.
    Sisbot EA, Alami R (2012) A human-aware manipulation planner. IEEE Trans Robot 28(5):1045–1057 CrossRefGoogle Scholar
  58. 58.
    Taha T, Miro JV, Dissanayake G (2008) Pomdp-based long-term user intention prediction for wheelchair navigation. In: ICRA, pp 3920–3925 Google Scholar
  59. 59.
    Tomasello M, Carpenter M, Call J, Behne T, Moll H (2005) Understanding and sharing intentions: the origins of cultural cognition. Behav Brain Sci 28(5):675–691 Google Scholar
  60. 60.
    Zwickel J (2009) Agency attribution and visuospatial perspective taking. Psychon Bull Rev 16:1089–1093 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  • Amit Kumar Pandey
    • 1
    • 2
  • Muhammad Ali
    • 1
    • 2
  • Rachid Alami
    • 1
    • 2
  1. 1.LAASCNRSToulouseFrance
  2. 2.LAASUniv de ToulouseToulouseFrance

Personalised recommendations