Manipulating Mental States through Physical Action

  • Jesse Gray
  • Cynthia Breazeal
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7621)


We present our implementation of a self-as-simulator architecture for mental state manipulation through physical action. The robot attempts to model how a human’s mental states are updated through their visual perception of the world around them. This modeling, combined with geometrically detailed, perspective correct simulations of the immediate future, allows the robot to choose actions which influence the human’s mental states through their visual perception. The system is demonstrated in a competitive game scenario, where the robot attempts to manipulate the mental states of an individual in order to win. We evaluate people’s reaction to the system, focusing on the participants’ perception of a robot with mental state manipulation capabilities.


Physical Action Mental State Real Robot Physical Robot Mental State Inference 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Blumberg, B., Downie, M., Ivanov, Y., Berlin, M., Johnson, M.P., Tomlinson, B.: Integrated learning for interactive synthetic characters. ACM Transactions on Graphics Proceedings of ACM SIGGRAPH 2002 21(3) (2002)Google Scholar
  2. 2.
    Breazeal, C., Gray, J., Berlin, M.: An embodied cognition approach to mindreading skills for socially intelligent robots. The International Journal of Robotics Research (IJHR 2009) 28(5), 656 (2009)CrossRefGoogle Scholar
  3. 3.
    Burke, R., Isla, D., Downie, M., Ivanov, Y., Blumberg, B.: CreatureSmarts: The art and architecture of a virtual brain. In: Proceedings of the Game Developers Conference, San Jose, CA, pp. 147–166 (2001)Google Scholar
  4. 4.
    Gray, J., Breazeal, C., Berlin, M., Brooks, A., Lieberman, J.: Action parsing and goal inference using self as simulator. In: 14th IEEE International Workshop on Robot and Human Interactive Communication (ROMAN), Nashville, Tennessee. IEEE (2005)Google Scholar
  5. 5.
    Gray, J., Hoffman, G., Adalgeirsson, S.O., Berlin, M., Breazeal, C.: Expressive, interactive robots: Tools, techniques, and insights based on collaborations. In: HRI 2010 Workshop: What Do Collaborations with the Arts Have to Say About HRI? (2010)Google Scholar
  6. 6.
    Johnson, M., Demiris, Y.: Perceptual perspective taking and action recognition. International Journal of Advanced Robotic Systems 2(4), 301–308 (2005)Google Scholar
  7. 7.
    Kelley, R., Tavakkoli, A., King, C., Nicolescu, M., Nicolescu, M., Bebis, G.: Understanding human intentions via hidden markov models in autonomous mobile robots. In: Proceedings of the 3rd International Conference on Human Robot Interaction, pp. 367–374 (2008)Google Scholar
  8. 8.
    Kennedy, W., Bugajska, M., Harrison, A., Trafton, J.: “like-me” simulation as an effective and cognitively plausible basis for social robotics. International Journal of Social Robotics 1(2), 181–194 (2009)CrossRefGoogle Scholar
  9. 9.
    Laird, J.E.: It knows what you’re going to do: adding anticipation to a quakebot. In: AGENTS 2001: Proceedings of the Fifth International Conference on Autonomous Agents, pp. 385–392. ACM Press, New York (2001)CrossRefGoogle Scholar
  10. 10.
    Marsella, S.C., Pynadath, D.V.: Modeling influence and theory of mind. Artificial Intelligence and the Simulation of Behavior (2005)Google Scholar
  11. 11.
    Meltzoff, A.N.: Understanding the intentions of others: re-enactment of intended acts by 18-month-old children. Developmental Psychology 31, 838–850 (1995)CrossRefGoogle Scholar
  12. 12.
    Rizzolatti, G., Fadiga, L., Gallese, V., Fogassi, L.: Premotor cortex and the recognition of motor actions. Cognitive Brain Research 3, 131–141 (1996)CrossRefGoogle Scholar
  13. 13.
    Short, E., Hart, J., Vu, M., Scassellati, B.: No fair!! an interaction with a cheating robot. In: 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 219–226. IEEE (2010)Google Scholar
  14. 14.
    Trafton, J., Schultz, A., Perznowski, D., Bugajska, M., Adams, W., Cassimatis, N., Brock, D.: Children and robots learning to play hide and seek. In: Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction, pp. 242–249. ACM (2006)Google Scholar
  15. 15.
    Trafton, J.G., Cassimatis, N.L., Bugajska, M.D., Brock, D.P., Mintz, F.E., Schultz, A.C.: Enabling effective human-robot interaction using perspective-taking in robots. IEEE Transactions on Systems, Man, and Cybernetics 35(4), 460–470 (2005)CrossRefGoogle Scholar
  16. 16.
    Wellman, H., Cross, D., Watson, J.: Meta-Analysis of Theory-of-Mind Development: The Truth about False Belief. Child Development 72(3), 655–684 (2001)CrossRefGoogle Scholar
  17. 17.
    Wimmer, H., Perner, J.: Beliefs about beliefs: Representation and constraining function on wrong beliefs in young children’s understanding of deception. Cognition 13, 103–128 (1983)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Jesse Gray
    • 1
  • Cynthia Breazeal
    • 1
  1. 1.MIT Media LabCambridgeUSA

Personalised recommendations