Cognitive Computation

, Volume 6, Issue 2, pp 182–199 | Cite as

Displaying and Regulating Different Social Response Patterns: A Computational Agent Model



Differences in social responses of individuals can often be related to differences in functioning of certain neurological mechanisms. This paper presents a computational agent model capable of showing different types of social response patterns based on such mechanisms, adopted from theories on mirror neuron systems, emotion integration, emotion regulation, and empathy. The presented agent model provides a basis for humanlike social response patterns of virtual agents in the context of simulation-based training (e.g. for training of physicians or therapists), gaming, or for agent-based generation of virtual stories.


Social response ASD Regulation Computational agent model 


  1. 1.
    Ashby WR. Design for a brain. London: Chapman & Hall; 1952. Revised edition 1960.Google Scholar
  2. 2.
    Baker AEZ, Lane AE, Angley MT, Young RL. The relationship between sensory processing patterns and behavioural responsiveness in autistic disorder: a pilot study. J Autism Dev Disord. 2008;38:867–75.PubMedCrossRefGoogle Scholar
  3. 3.
    Barringer H, Fisher M, Gabbay D, Owens R, Reynolds M. The imperative future: principles of executable temporal logic. Taunton: Research Studies Press Ltd.; 1996.Google Scholar
  4. 4.
    Bates J, Loyall AB, Reilly WS. An architecture for action, emotion, and social behavior. In: Castelfranchi C, Werner E, editors. Proceedings of the 4th European workshop on modelling autonomous agents in a multi-agent world, MAAMAW’92, selected papers. Lecture Notes in Computer Science, vol. 830, New York: Springer; 1994. p. 55–68.Google Scholar
  5. 5.
    Beer RD. On the dynamics of small continuous-time recurrent neural networks. Adapt Behav. 1995;3:469–509.CrossRefGoogle Scholar
  6. 6.
    Bickmore T, Fernando R, Ring L, Schulman D. Empathic touch by relational agents. IEEE Trans Affect Comput. 2010;1:60–71.CrossRefGoogle Scholar
  7. 7.
    Bickmore TW, Picard RW. Towards caring machines. In: Dykstra-Erickson E, Tscheligi M, editors. Proceedings of the ACM SIGCHI conference on human factors in computing systems (CHI); 2004. p. 1489–92.Google Scholar
  8. 8.
    Bonvicini KA, Perlin MJ, Bylund CL, Carroll GA, Rouse RA, Goldstein MG. Impact of communication training on physician expression of empathy in patient encounters. Patient Education Couns. 2009;75:3–10.Google Scholar
  9. 9.
    Bosse T, Jonker CM, van der Meij L, Treur J. A language and environment for analysis of dynamics by simulation. Int J Artif Intell Tools. 2007;16:435–64.CrossRefGoogle Scholar
  10. 10.
    Bosse T, Memon ZA, Treur J. A cognitive and neural model for adaptive emotion reading by mirroring preparation states and Hebbian learning. Cogn Syst Res. 2012;12:39–58.Google Scholar
  11. 11.
    Bosse T, Pontier M, Treur J. A computational model based on gross’ emotion regulation theory. Cogn Syst Res J. 2010;11:211–30.CrossRefGoogle Scholar
  12. 12.
    Boukricha H, Wachsmuth I. Empathy-based emotional alignment for a virtual human: a three step approach. Künstliche Intelligenz. 2011;25:195–204.CrossRefGoogle Scholar
  13. 13.
    Brass M, Spengler S. The inhibition of imitative behaviour and attribution of mental states. In: Striano T, Reid V, editors. Social cognition: development, neuroscience, and autism. Oxford: Wiley-Blackwell; 2009. p. 52–66.Google Scholar
  14. 14.
    Cacioppo JT, Berntson GG. Social neuroscience. New York: Psychology Press; 2005.Google Scholar
  15. 15.
    Cacioppo JT, Visser PS, Pickett CL. Social neuroscience: people thinking about thinking people. Cambridge, MA: MIT Press; 2006.Google Scholar
  16. 16.
    Crane L, Goddard L, Pring L. Sensory processing in adults with autism spectrum disorders. Autism. 2009;13:215–28.PubMedCrossRefGoogle Scholar
  17. 17.
    Corden B, Chilvers R, Skuse D. Avoidance of emotionally arousing stimuli predicts social-perceptual impairment in Asperger syndrome. Neuropsychologia. 2008;46:137–47.PubMedCrossRefGoogle Scholar
  18. 18.
    Damasio AR. The feeling of what happens. Body and emotion in the making of consciousness. New York: Harcourt Brace; 1999.Google Scholar
  19. 19.
    Damasio AR. Looking for Spinoza. New York: Harcourt; 2003.Google Scholar
  20. 20.
    Dapretto M, Davies MS, Pfeifer JH, Scott AA, Sigman M, Bookheimer SY, Iacoboni M. Understanding emotions in others: mirror neuron dysfunction in children with autism spectrum disorder. Nat Neurosci. 2006;9:28–30.PubMedCentralPubMedCrossRefGoogle Scholar
  21. 21.
    De Vignemont F, Singer T. The empathic brain: how, when and why? Trends Cogn Sci. 2006;10:437–43.CrossRefGoogle Scholar
  22. 22.
    Decety J, Cacioppo JT, editors. Handbook of social neuroscience. Oxford: Oxford University Press; 2010.Google Scholar
  23. 23.
    Fisher M. Temporal development methods for agent-based systems. J Auton Agents Multi Agent Syst. 2005;10:41–66.CrossRefGoogle Scholar
  24. 24.
    Forbus KD. Qualitative process theory. Artif Intell. 1984;24(1–3):85–168.CrossRefGoogle Scholar
  25. 25.
    Fried I, Mukamel R, Kreiman G. Internally generated preactivation of single neurons in human medial frontal cortex predicts volition. Neuron. 2011;69:548–62.Google Scholar
  26. 26.
    Frith U. Autism, explaining the enigma. New York: Blackwell; 2003.Google Scholar
  27. 27.
    Galton A. Operators vs arguments: the ins and outs of reification. Synthese. 2006;150:415–41.CrossRefGoogle Scholar
  28. 28.
    Gepner B, Féron F. Autism: a world changing too fast for a mis-wired brain? Neurosci Biobehav Rev. 2009;33:1227–42.PubMedCrossRefGoogle Scholar
  29. 29.
    Goldin PR, McRae K, Ramel W, Gross JJ. The neural bases of emotion regulation: reappraisal and suppression of negative emotion. Biol Psychiatry. 2008;63:577–86.PubMedCentralPubMedCrossRefGoogle Scholar
  30. 30.
    Gratch J, Marsella S, Petta P. Modeling the antecedents and consequences of emotion. Cogn Syst Res. 2009;10:1–5.CrossRefGoogle Scholar
  31. 31.
    Greenland S, Brumback BA. An overview of relations among causal modeling methods. Int J Epidemiol. 2002;31:1030–7.Google Scholar
  32. 32.
    Grèzes J, de Gelder B. Social perception: understanding other people’s intentions and emotions through their actions. In: Striano T, Reid V, editors. Social cognition: development, neuroscience, and autism. Oxford: Wiley-Blackwell; 2009. p. 67–78.Google Scholar
  33. 33.
    Grèzes J, Wicker B, Berthoz S, de Gelder B. A failure to grasp the affective meaning of actions in autism spectrum disorder subjects. Neuropsychologica. 2009;47:1816–25.CrossRefGoogle Scholar
  34. 34.
    Gross JJ. Antecedent- and response-focused emotion regulation: divergent consequences for experience, expression, and physiology. J Personality Social Psych. 1998;74:224–37.CrossRefGoogle Scholar
  35. 35.
    Gross JJ. Emotion regulation in adulthood: timing is everything. Curr Dir Psychol Sci. 2001;10:214–9.CrossRefGoogle Scholar
  36. 36.
    Gross JJ, editors. Handbook of emotion regulation. New York: Guilford Press; 2007.Google Scholar
  37. 37.
    Hamilton AFC, Brindley RM, Frith U. Imitation and action understanding in autistic spectrum disorders: how valid is the hypothesis of a deficit in the mirror neuron system? Neuropsychologia. 2007;45:1859–68.Google Scholar
  38. 38.
    Harmon-Jones E, Winkielman P, editors. Social neuroscience: integrating biological and psychological explanations of social behavior. New York: Guilford; 2007.Google Scholar
  39. 39.
    Hendriks M, Treur J. Modeling super mirroring functionality in action execution, imagination, mirroring, and imitation. In: Pan J-S et al., editors. Proceedings of the 2nd international conference on computational collective intelligence, ICCCI’10, part I. Lecture Notes in Artificial Intelligence, vol. 6421, New York: Springer; 2010. p. 330–42.Google Scholar
  40. 40.
    Hofsten C von, Gredebäck G. The role of looking in social cognition: perspectives from development and autism. In: Striano T, Reid V editors. Social cognition: development, neuroscience, and autism. Oxford: Wiley-Blackwell; 2009. p. 237–53.Google Scholar
  41. 41.
    Hojat M. Empathy in patient care: antecedents, development, measurement, and outcomes. New York: Springer; 2007.Google Scholar
  42. 42.
    Hojat M. Ten approaches for enhancing empathy in health and human services cultures. J Health Human Serv Adm. 2009;31:412–50.Google Scholar
  43. 43.
    Hopfield JJ. Neural networks and physical systems with emergent collective computational properties. Proc Nat Acad Sci USA. 1982;79:2554–8.PubMedCentralPubMedCrossRefGoogle Scholar
  44. 44.
    Hopfield JJ. Neurons with graded response have collective computational properties like those of two-state neurons. Proc Nat Acad Sci USA. 1984;81:3088–92.PubMedCentralPubMedCrossRefGoogle Scholar
  45. 45.
    Hutt C, Hutt SJ, Lee D, Ousted C. Arousal and childhood autism. Nature. 1964;204:908–9.PubMedCrossRefGoogle Scholar
  46. 46.
    Iacoboni M. Mirroring people: the new science of how we connect with others. New York: Farrar, Straus & Giroux; 2008 2008.Google Scholar
  47. 47.
    Iacoboni M. Mesial frontal cortex and super mirror neurons. Behav Brain Sci. 2008;31:30.CrossRefGoogle Scholar
  48. 48.
    Iacoboni M, Dapretto M. The mirror neuron system and the consequences of its dysfunction. Nat Rev Neurosci. 2006;7:942–51.PubMedCrossRefGoogle Scholar
  49. 49.
    Keysers C, Gazzola V. Social neuroscience: mirror neurons recorded in humans. Curr Biol. 2010;20:253–4.CrossRefGoogle Scholar
  50. 50.
    Kirchner JC, Hatri A, Heekeren HR, Dziobek I. Autistic symptomatology, face processing abilities, and eye fixation patterns. J Autism Dev Disord. 2011;41:158–67.PubMedCrossRefGoogle Scholar
  51. 51.
    Klein J, Moon Y, Picard R. This computer responds to user frustration: theory, design, results, and implications. Interact Comput. 2002;14:119–40.CrossRefGoogle Scholar
  52. 52.
    Kylliäinen A, Hietanen JK. Skin conductance responses to another person’s gaze in children with autism. J. Autism Dev Disord. 2006;36:517–25.Google Scholar
  53. 53.
    Laan Y van der, Treur J. An agent model for computational analysis of mirroring dysfunctioning in autism spectrum disorders. In: Mehrotra KG et al., editors. Proceedings of the 24th international conference on industrial, engineering and other applications of applied intelligent systems, IEA/AIE’11, part I. Lecture notes in artificial intelligence, vol. 6703. New York: Springer; 2011. p. 306–16.Google Scholar
  54. 54.
    Lane AE, Young RL, Baker AEZ, Angley MT. Sensory processing subtypes in autism: association with adaptive behavior. J Autism Dev Disord. 2010;40:112–22.PubMedCrossRefGoogle Scholar
  55. 55.
    Leite I, Pereira A, Castellano G, Mascarenhas S, Martinho C, Paiva A. Modelling empathy in social robotic companions. In: Ardissono L, Kuflik T, editors. UMAP 2011 workshops, LNCS 7138, 2012. p. 135–47.Google Scholar
  56. 56.
    McQuiggan S, Robison J, Phillips R, Lester J. Modeling parallel and reactive empathy in virtual agents: an inductive approach. In Proceedings of the 7th international joint conference on autonomous agents and multi-agent systems; 2008. p. 167–74.Google Scholar
  57. 57.
    Mukamel R, Ekstrom AD, Kaplan J, Iacoboni M, Fried I. Single-neuron responses in humans during execution and observation of actions. Curr Biol. 2010;20:750–6.PubMedCentralPubMedCrossRefGoogle Scholar
  58. 58.
    Neumann D, Spezio ML, Piven J, Adolphs R. Looking you in the mouth: abnormal gaze in autism resulting from impaired top-down modulation of visual attention. Soc Cogn Aff Neurosci. 2006;1:194–202.CrossRefGoogle Scholar
  59. 59.
    Ochs M, Pelachaud C, Sadek D. An empathic virtual dialog agent to improve human-machine interaction. In: Padgham, Parkes, Müller, Parsons, editors. Proceedings of 7th international conference on autonomous agents and multiagent systems, AAMAS’08; 2008. p. 89–96.Google Scholar
  60. 60.
    Paiva A. Empathy in social agents. Int J Virtual Real. 2011;10:65–8.Google Scholar
  61. 61.
    Pessoa L. On the relationship between emotion and cognition. Nat Rev Neurosci. 2008;9:148–58.PubMedCrossRefGoogle Scholar
  62. 62.
    Phelps EA. Emotion and cognition: insights from studies of the human amygdala. Annu Rev Psychol. 2006;57:27–53.PubMedCrossRefGoogle Scholar
  63. 63.
    Pineda JA, editor. Mirror neuron systems: the role of mirroring processes in social cognition. Totowa: Humana Press Inc.; 2009.Google Scholar
  64. 64.
    Port R, van Gelder TJ. Mind as motion: explorations in the dynamics of cognition. Cambridge, MA: MIT Press; 1995.Google Scholar
  65. 65.
    Richer J, Coates S, editors. Autism, the search for coherence. London: Jessica Kingsley Publishers; 2001.Google Scholar
  66. 66.
    Rizzolatti G, Sinigaglia C. Mirrors in the brain: How our minds share actions and emotions. Oxford: Oxford University Press; 2008.Google Scholar
  67. 67.
    Rodrigues SH, Mascarenhas S, Dias J, Paiva A. I can feel it too!: emergent empathic reactions between synthetic characters. 3rd International conference on affective computing and intelligent interaction and workshops (AACII), New York: IEEE Press; 2009.Google Scholar
  68. 68.
    Singer T, Leiberg S. Sharing the emotions of others: the neural bases of empathy. In: Gazzaniga MS, editor. The cognitive neurosciences, 4th ed. Cambridge: MIT Press; 2009. p. 973–86.Google Scholar
  69. 69.
    Smith A. The empathy imbalance hypothesis of autism: a theoretical approach to cognitive and emotional empathy in autistic development. Psychol Record. 2009;59:489–510.Google Scholar
  70. 70.
    Spezio ML, Adolphs R, Hurley RSE, Piven J. Analysis of face gaze in autism using ‘Bubbles’. Neuropsychologia. 2007;45:144–51.PubMedCrossRefGoogle Scholar
  71. 71.
    Suchman AL, Markakis K, Beckman HB, Frankel R. A model of empathic communication in the medical interview. J Am Med Assoc. 1997;277:678–82.CrossRefGoogle Scholar
  72. 72.
    Tinbergen EA, Tinbergen N. Early childhood autism: an ethological approach. Advances in ethology, vol. 10. Journal of Comparative Ethology Supplement. Berlin: Paul Perry; 1972.Google Scholar
  73. 73.
    Treur J. A cognitive agent model displaying and regulating different social response patterns. In: Walsh T., editor. Proceedings of the 22nd international joint conference on artificial intelligence, IJCAI’11. Menlo Park, CA: AAAI Press; 2011. p. 1743–9.Google Scholar
  74. 74.
    Treur J. A computational agent model for Hebbian learning of social interaction. In: Lu B-L, Zhang L, Kwok J., editors. Proceedings of the 18th international conference on neural information processing, ICONIP’11, part I. Lecture notes in artificial intelligence, vol. 7062. Berlin: Springer; 2011a. p. 9–19.Google Scholar
  75. 75.
    Tulsky JA, Arnold RM, Alexander SC, Olsen MK, Jeffreys AS, Rodriguez KL, Skinner CS, Farrell D, Abernethy AP, Pollak KI. Enhancing communication between oncologists and patients with a computer-based training program: a randomized trial. Ann Intern Med. 2011;155:593–602.PubMedCentralPubMedCrossRefGoogle Scholar
  76. 76.
    Williams JH, Whiten A, Suddendorf T, Perrtett DI. Imitation, mirror neurons and autism. Neurosci Biobehav Rev. 2001;25:287–95.PubMedCrossRefGoogle Scholar
  77. 77.
    Yang H, Pan Z, Zhang M, Ju C. Modeling emotional action for social characters. Knowl Eng Rev. 2008;23:321–37.CrossRefGoogle Scholar
  78. 78.
    Zimmermann C, Del Piccolo L, Benzing J, et al. Coding patient emotional cues and concerns in medical consultations: The Verona coding definitions of emotional sequences (VR-CoDES). Patient Educ Couns. 2011;82:141–8.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Agent Systems Research GroupVU University AmsterdamAmsterdamThe Netherlands

Personalised recommendations