Training the use of theory of mind using artificial agents

  • Kim Veltman
  • Harmen de Weerd
  • Rineke Verbrugge
Original Paper


When engaging in social interaction, people rely on their ability to reason about unobservable mental content of others, which includes goals, intentions, and beliefs. This so-called theory of mind ability allows them to more easily understand, predict, and influence the behavior of others. People even use their theory of mind to reason about the theory of mind of others, which allows them to understand sentences like ‘Alice believes that Bob does not know about the surprise party’. But while the use of higher orders of theory of mind is apparent in many social interactions, empirical evidence so far suggests that people do not use this ability spontaneously when playing strategic games, even when doing so would be highly beneficial. In this paper, we attempt to encourage participants to engage in higher-order theory of mind reasoning by letting them play a game against computational agents. Since previous research suggests that competitive games may encourage the use of theory of mind, we investigate a particular competitive game, the Mod game, which can be seen as a much larger variant of the well-known rock–paper–scissors game. By using a combination of computational agents and Bayesian model selection, we simultaneously determine to what extent people make use of higher-order theory of mind reasoning, as well as to what extent computational agents can encourage the use of higher-order theory of mind in their human opponents. Our results show that participants who play the Mod game against computational theory of mind agents adjust their level of theory of mind reasoning to that of their computer opponent. Earlier experiments with other strategic games show that participants only engage in low orders of theory of mind reasoning. Surprisingly, we find that participants who knowingly play against second- and third-order theory of mind agents apply up to fourth-order theory of mind themselves, and achieve higher scores as a result.


Theory of mind Agent-based modeling Bayesian model selection Mod game Virtual training agents 



This work was supported by the Netherlands Organisation for Scientific Research (NWO) Vici Grant NWO 277-80-001, awarded to Rineke Verbrugge for the Project ‘Cognitive systems in interaction: Logical and computational models of higher-order social cognition’.


  1. 1.
    Adibsereshki N, Nesayan A, Gandomani R, Karimlou M (2015) The effectiveness of theory of mind training on the social skills of children with high functioning autism spectrum disorders. Iran J Child Neurol 9(3):40–49Google Scholar
  2. 2.
    Bruijnes M (2016) Believable suspect agents: response and interpersonal style selection for an artificial suspect. Ph.D. thesis, University of TwenteGoogle Scholar
  3. 3.
    Camerer C, Ho T, Chong J (2004) A cognitive hierarchy model of games. Quart J Econ 119(3):861–898zbMATHGoogle Scholar
  4. 4.
    Chevallier C, Noveck I, Happé F, Wilson D (2011) What’s in a voice? Prosody as a test case for the theory of mind account of autism. Neuropsychologia 49(3):507–517Google Scholar
  5. 5.
    Devaine M, Hollard G, Daunizeau J (2014) The social Bayesian brain: does mentalizing make a difference when we learn? PLoS Comput Biol 10(12):e1003992Google Scholar
  6. 6.
    Devaine M, Hollard G, Daunizeau J (2014) Theory of mind: did evolution fool us? PLoS ONE 9(2):e87619Google Scholar
  7. 7.
    de Weerd H, Verheij B (2011) The advantage of higher-order theory of mind in the game of limited bidding. In: Eijck JV, Verbrugge R (eds) Proceedings of the workshop on reasoning about other minds: logical and cognitive perspectives, CEUR workshop proceedings, pp 149–164Google Scholar
  8. 8.
    de Weerd H, Verbrugge R, Verheij B (2013) How much does it help to know what she knows you know? An agent-based simulation study. Artif Intell 199–200:67–92MathSciNetzbMATHGoogle Scholar
  9. 9.
    de Weerd H, Verbrugge R, Verheij B (2014) Theory of mind in the Mod game: an agent-based model of strategic reasoning. In: Proceedings ECSI, pp 128–136Google Scholar
  10. 10.
    de Weerd H, Verbrugge R, Verheij B (2017) Negotiating with other minds: the role of recursive theory of mind in negotiation with incomplete information. Auton Agents Multi-Agent Syst 31:250–287Google Scholar
  11. 11.
    de Weerd H, Diepgrond D, Verbrugge R (2018) Estimating the use of higher-order theory of mind using computational agents. BE J Theor Econ 18(2):1MathSciNetGoogle Scholar
  12. 12.
    Franke M, Galeazzi P (2014) On the evolution of choice principles. In: Proceedings of the second workshop reasoning about other minds: logical and cognitive perspectives, CEUR workshop proceedings, vol 1208, pp 11–15Google Scholar
  13. 13.
    Frey S (2013) Complex collective dynamics in human higher-level reasoning: a study over multiple methods. Ph.D. thesis, Indiana UniversityGoogle Scholar
  14. 14.
    Frey S, Goldstone RL (2013) Cyclic game dynamics driven by iterated reasoning. PLoS ONE 8(2):e56416Google Scholar
  15. 15.
    Goodie AS, Doshi P, Young DL (2012) Levels of theory-of-mind reasoning in competitive games. J Behav Decis Mak 25(1):95–108Google Scholar
  16. 16.
    Hedden T, Zhang J (2002) What do you think I think you think? Strategic reasoning in matrix games. Cognition 85(1):1–36Google Scholar
  17. 17.
    Herbranson WT, Schroeder J (2010) Are birds smarter than mathematicians? Pigeons (Columba livia) perform optimally on a version of the Monty Hall Dilemma. J Comp Psychol 124(1):1–13Google Scholar
  18. 18.
    Imuta K, Henry JD, Slaughter V, Selcuk B, Ruffman T (2016) Theory of mind and prosocial behavior in childhood: a meta-analytic review. Dev Psychol 52(8):1192–1205Google Scholar
  19. 19.
    Kinderman P, Dunbar RI, Bentall RP (1998) Theory-of-mind deficits and causal attributions. Br J Psychol 89(2):191–204Google Scholar
  20. 20.
    Liddle B, Nettle D (2006) Higher-order theory of mind and social competence in school-age children. J Cult Evolut Psychol 4(3–4):231–244Google Scholar
  21. 21.
    Meijering B, Taatgen NA, van Rijn H, Verbrugge R (2014) Modeling inference of mental states: as simple as possible, as complex as necessary. Interact Stud 15(3):455–477Google Scholar
  22. 22.
    Mol L, Krahmer E, Maes A, Swerts M (2009) The communicative import of gestures: evidence from a comparative analysis of human–human and human–machine interactions. Gesture 9(1):97–126Google Scholar
  23. 23.
    Mol L, Krahmer E, Maes A, Swerts M (2012) Adaptation in gesture: converging hands or converging minds? J Memory Lang 66(1):249–264Google Scholar
  24. 24.
    Premack D, Woodruff G (1978) Does the chimpanzee have a theory of mind? Behav Brain Sci 1(04):515–526Google Scholar
  25. 25.
    Rapoport A, Budescu D (1997) Randomization in individual choice behavior. Psychol Rev 104(3):603Google Scholar
  26. 26.
    Stephan KE, Penny WD, Daunizeau J, Moran RJ, Friston KJ (2009) Bayesian model selection for group studies. Neuroimage 46(4):1004–1017Google Scholar
  27. 27.
    Stiller J, Dunbar RI (2007) Perspective-taking and memory capacity predict social network size. Soc Netw 29(1):93–104Google Scholar
  28. 28.
    van der Lubbe L, Bosse T, Gerritsen C (2018) Design of an agent-based learning environment for high-risk doorstep scam victims. In: International conference on practical applications of agents and multi-agent systems, pp 335–347Google Scholar
  29. 29.
    Veltman K, de Weerd H, Verbrugge R (2017) Socially smart software agents entice people to use higher-order theory of mind in the Mod game. In: BNAIC 2017 preproceedings, pp 253–267Google Scholar
  30. 30.
    Verbrugge R, Meijering B, Wierda S, van Rijn H, Taatgen N (2018) Stepwise training supports strategic second-order theory of mind in turn-taking games. Judgm Decis Mak 13(1):79–98Google Scholar
  31. 31.
    Wagenaar W (1972) Generation of random sequences by human subjects: a critical survey of literature. Psychol Bull 77(1):65–72Google Scholar
  32. 32.
    Wright JR, Leyton-Brown K (2010) Beyond equilibrium: predicting human behavior in normal-form games. In: Proceedings of the twenty-fourth conference on artificial intelligence, pp 901–907Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Department of Artificial Intelligence, Bernoulli InstituteUniversity of GroningenGroningenThe Netherlands
  2. 2.Research Group User-Centered DesignHanze University of Applied SciencesGroningenThe Netherlands

Personalised recommendations