International Journal of Social Robotics

, Volume 9, Issue 4, pp 601–613 | Cite as

A Feasibility Study Evaluating the Emotionally Expressive Robot SAM

  • Sarah A. KochEmail author
  • Carl E. Stevens
  • Christian D. Clesi
  • Jenna B. Lebersfeld
  • Alyssa G. Sellers
  • Myriah E. McNew
  • Fred J. Biasini
  • Franklin R. Amthor
  • Maria I. Hopkins


This two-part feasibility study evaluated the functionality and acceptability of Socially Animated Machine (SAM), a humanoid robotic monkey developed to elicit social interaction in children with Autism Spectrum Disorder (ASD). Socially Animated Machine was designed with an approachable, animal-like appearance, while preserving the essential features of a human face. The intent was to design a robot that would be interesting and engaging to children with ASD, yet maintain the capability to model facial expressions that convey emotional subtlety. Study 1 evaluated the accuracy of SAM’s emotional facial expressions. Typically developing children (\(N = 35\)) labeled and matched SAM’s expressions to photos of human expressions with moderate-to-substantial levels of agreement. Study 2 compared children’s level of social engagement across an interaction with SAM and an interaction with an adult experimenter. Children with ASD (\(N = 13\)) spent significantly more time attending to the partner’s face while interacting with SAM. When asked to rate their interaction with SAM, children with ASD reported high levels of happiness and comfort and requested additional interactions. These results suggest that SAM may serve as a useful tool in interventions to improve social skills, including emotion recognition, in children with ASD.


Autism spectrum disorder (ASD) Social robot Human–robot interaction Emotion recognition Engagement 



The authors thank the Social Technology for Autism Research (STAR) Lab at the University of Alabama at Birmingham (UAB) and the children and families who participated in the study and made this research possible. This study was funded in part by a grant from Civitan International Research Center. This paper is adapted from the author’s master’s thesis.

Funding This study was funded in part by a grant from Civitan International Research Center.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. 1.
    Centers for Disease Control and Prevention (2016) Prevalence and characteristics of autism spectrum disorder among children aged 8 years—autism and developmental disabilities monitoring network, 11 sites, United States, 2012. MMWR Surveill Summary 65(3):1–23CrossRefGoogle Scholar
  2. 2.
    American Psychiatric Association (2013) Diagnostic and statistical manual of mental disorders, 5th edn. American Psychiatric Publishing, ArlingtonCrossRefGoogle Scholar
  3. 3.
    Hobson RP (1986) The autistic child’s appraisal of expressions of emotion. J Child Psychol Psychiatry 27:321–342CrossRefGoogle Scholar
  4. 4.
    Paul R (2008) Interventions to improve communication in autism. Child Adolesc Psychiatr Clin N Am 17(4):835–856CrossRefGoogle Scholar
  5. 5.
    Shaked M, Yirmiya N (2003) Understanding social difficulties. In: Prior MR (ed) Learning and behavior problems in asperger syndrome. Guilford Press, New York, pp 104–125Google Scholar
  6. 6.
    Tager-Flusberg H, Joseph RM (2003) Identifying neurocognitive phenotypes in autism. Philos Trans R Soc Lond B Biol Sci 358(1430):303–314CrossRefGoogle Scholar
  7. 7.
    Tager-Flusberg H, Paul R, Lord C (2005) Language and communication in autism. In: Volkmar FR, Paul R, Klin A, Cohen D (eds) Handbook of autism and pervasive developmental disorders. Wiley, New York, pp 335–364Google Scholar
  8. 8.
    Buitelaar JK, van Engeland H, de Kogel KH, de Vries H, van Hooff JA (1991) Differences in the structure of social behaviour of autistic children and non-autistic retarded controls. J Child Psychol Psychiatry 32(6):995–1015CrossRefGoogle Scholar
  9. 9.
    Volkmar FR, Paul R, Klin A, Cohen DJ (2005) Handbook of autism and pervasive developmental disorders, diagnosis, development, neurobiology, and behavior. Wiley, New YorkCrossRefGoogle Scholar
  10. 10.
    Tantam D (2003) The challenge of adolescents and adults with asperger syndrome. Child Adolesc Psychiatr Clin N Am 12:143–163CrossRefGoogle Scholar
  11. 11.
    Chamberlain BO (2001) Isolation or involvement? The social networks of children with autism included in regular classes. Dissertation, University of California, Los AngelesGoogle Scholar
  12. 12.
    Howlin P, Goode S (1998) Outcome in adult life for people with autism, asperger syndrome. In: Volkmar FR (ed) Autism and pervasive developmental disorders. Cambridge University Press, New YorkGoogle Scholar
  13. 13.
    White SW, Keonig K, Scahill L (2007) Social skills development in children with autism spectrum disorders: a review of the intervention research. J Autism Dev Disord 37(10):1858–1868CrossRefGoogle Scholar
  14. 14.
    Koegel LK, Koegel RL, Harrower JK, Carter CM (1999) Pivotal response intervention I: overview of approach. Res Pract Persons Severe Disabil 24(3):174–185CrossRefGoogle Scholar
  15. 15.
    Dautenhahn K, Werry I (2004) Towards interactive robots in autism therapy: background, motivation and challenges. Pragmat Cogn 12:1–35CrossRefGoogle Scholar
  16. 16.
    Ricks DJ, Colton MB (2010) Trends and considerations in robot-assisted autism therapy. In: Proceedings of the IEEE international conference on robotics and automation, pp 4354–4359Google Scholar
  17. 17.
    Robins B, Dickerson P, Stribling P, Dautenhahn K (2004) Robot-mediated joint attention in children with autism: a case study in robot–human interaction. Interact Stud 5(2):161–198CrossRefGoogle Scholar
  18. 18.
    Scassellati B, Admoni H, Mataric M (2012) Robots for use in autism research. Annu Rev Biomed Eng 14:275–294CrossRefGoogle Scholar
  19. 19.
    Diehl JJ, Schmitt LM, Villano M, Crowell CR (2012) The clinical use of robots for individuals with autism spectrum disorders: a critical review. Res Autism Spect Dis 6(1):249–262CrossRefGoogle Scholar
  20. 20.
    Scassellati B (2007) How social robots will help us to diagnose, treat, and understand autism. Int J Robot Res 28:552–563CrossRefGoogle Scholar
  21. 21.
    Kozima H, Nakagawa C (2006) Interactive robots as facilitators of children’s social development. In: Lazinica A (ed) Mobile robots towards new applications. i-Tech Education and Publishing, Munich, p 784Google Scholar
  22. 22.
    Pioggia G, Igliozzi R, Ferro M, Ahluwalia A, Muratori F, De Rossi D (2005) An android for enhancing social skills and emotion recognition in people with autism. IEEE Trans Neural Syst Rehabil Eng 13(4):507–515CrossRefGoogle Scholar
  23. 23.
    Pioggia G, Igliozzi R, Sica ML, Ferro M, Muratori F, Ahluwalia A, De Rossi D (2008) Exploring emotional and imitational android-based interactions in autistic spectrum disorders. J Cyber Ther Rehabil 1(1):49–61Google Scholar
  24. 24.
    Robins B, Dautenhahn K, Te Boekhorst R, Billard A (2005) Robotic assistants in therapy and education of children with autism: can a small humanoid robot help encourage social interaction skills? Univers Access Inf 4(2):105–120CrossRefGoogle Scholar
  25. 25.
    Kozima H, Michalowski MP, Nakagawa C (2009) Keepon. Int J Soc Robot 1(1):3–18CrossRefGoogle Scholar
  26. 26.
    Kim ES, Berkovits LD, Bernier EP, Leyzberg D, Shic F, Paul R, Scassellati B (2012) Social robots as embedded reinforcers of social behavior in children with autism. J Autism Dev Disord 43(5):1038–1049CrossRefGoogle Scholar
  27. 27.
    Goris K, Saldien J, Vanderborght B, Lefeber D (2010) Probo, an intelligent huggable robot for HRI studies with children. In: Chugo D (ed) Human–robot interaction. INTECH, Rijeka, p 288Google Scholar
  28. 28.
    Simut R, Van de Perre C, Vanderborght B, Saldien J, Rusu AS, Pintea S, Vanderfaeillie J, Lefeber D, David DO (2011) The huggable social robot Probo for Social Story telling for robot assisted therapy with ASD children. In: Proceedings of the 3rd international conference on social robotics (ICSR-11), pp 97–100Google Scholar
  29. 29.
    Boccanfuso L, O’Kane JM (2011) CHARLIE: an adaptive robot with hand and face tracking for use in autism therapy. Int J Soc Robot 3:337–347CrossRefGoogle Scholar
  30. 30.
    Iacono I, Lehmann H, Marti P, Robins B, Dautenhahn K (2011) Robots as social mediators for children with autism—a preliminary analysis comparing two different robotic platforms. In: IEEE international conference on development and learning (ICDL-11), vol 2, pp 1–6Google Scholar
  31. 31.
    Feil-Seifer D, Matarić MJ (2009) Toward socially assistive robotics for augmenting interventions for children with autism spectrum disorders. In: Khatib O, Kumar V, Pappas GJ (eds) Experimental robotics. Springer, Berlin, pp 201–210Google Scholar
  32. 32.
    Dautenhahn K, Nehaniv CL, Walters M, Robins B, Kose-Bagci H, Mirza NA, Blow M (2009) Kaspar—a minimally expressive humanoid robot for human–robot interaction research. Appl Bionics Biomech 6:369–397CrossRefGoogle Scholar
  33. 33.
    Wainer J, Dautenhahn K, Robins B, Amirabdollahian F (2014) A pilot study with a novel setup for collaborative play of the humanoid robot KASPAR with children with autism. Int J Soc Robot 6:45–65CrossRefGoogle Scholar
  34. 34.
    Aldebaran Robotics (2006) NAO [robot].
  35. 35.
    Chevalier P, Martin J, Isableu B, Bazile C, Tapus A (2017) Impact of sensory preferences of individuals with autism on the recognition of emotions expressed by two robots, an avatar, and a human. Auton Robot 41(3):613–635CrossRefGoogle Scholar
  36. 36.
    Huskens B, Verschuur R, Gillesen J, Didden R, Barakova E (2013) Promoting question-asking in school-aged children with autism spectrum disorders: effectiveness of a robot intervention compared to a human–trainer intervention. Dev Neurorehabil 16(5):345–356CrossRefGoogle Scholar
  37. 37.
    Miskam MA, Masnin NF, Jamhuri MH, Shamsuddin S, Omar AR, Yussof H (2014) Encouraging children with autism to improve social and communication skills through the game-based approach. Procedia Comput Sci 42:93–98CrossRefGoogle Scholar
  38. 38.
    Tapus A, Peca A, Aly A, Pop C, Jisa L, Pintea S, Rusu A, David D (2012) Children with autism social engagement in interaction with NAO, an imitative robot—a series of single case experiments. Interact Stud 13(3):315–347CrossRefGoogle Scholar
  39. 39.
    Billard A, Robins B, Dautenhahn K, Nadel J (2006) Building robota, a mini-humanoid robot for the rehabilitation of children with autism. RESNA Assist Technol J 19(1):37–49CrossRefGoogle Scholar
  40. 40.
    Dautenhahn K, Billard A (2002) Games children with autism can play with Robota, a humanoid robotic doll. In: Keates S, Clarkson PJ, Langdon PM, Robinson P (eds) Cambridge workshop on universal access and assistive technology. Springer, London, pp 179–190CrossRefGoogle Scholar
  41. 41.
    Baron-Cohen S, Golan O, Ashwin E (2009) Can emotion recognition be taught to children with autism spectrum conditions? Philos Trans R Soc B 364:3567–3574CrossRefGoogle Scholar
  42. 42.
    Ashwin C, Chapman E, Colle L, Baron-Cohen S (2006) Impaired recognition of negative basic emotions in autism: a test of the amygdala theory. Soc Neurosci 1:349–363CrossRefGoogle Scholar
  43. 43.
    Baron-Cohen S (2000) Theory of mind and autism: a fifteen year review. In: Baron-Cohen A, Tager-Flusberg H, Cohen D (eds) Understanding other minds: perspectives from developmental cognitive neuroscience, 2nd edn. Oxford University Press, Oxford, pp 3–20Google Scholar
  44. 44.
    Mehrabian A (1968) Communication without words. Psychol Today 2(9):52–55Google Scholar
  45. 45.
    Mori M (1970) The uncanny valley. Energy 7:33–35Google Scholar
  46. 46.
    Saygin AP, Chaminade T, Ishiguro H, Driver J, Frith C (2011) The thing that should not be: predictive coding and the uncanny valley in perceiving human and humanoid robot actions. Soc Cogn Affect Neurosci 7(4):413–422CrossRefGoogle Scholar
  47. 47.
    Microsoft (2013) Surface Pro 2 [tablet].
  48. 48.
    Adobe Systems Incorporated (2017) Photoshop [computer program].
  49. 49.
    Eckman P, Friesen WV (1975) Unmasking the face: a guide to recognizing emotions from facial clues. Prentice-Hall, Englewood CliffsGoogle Scholar
  50. 50.
    Neurobehavioral Systems (2004) Presentation, version 17.2 [computer program].
  51. 51.
    Arduino LLC (2005) Arduino [computer program].
  52. 52.
    Fleiss JL (1971) Measuring nominal scale agreement among many raters. Psychol Bull 76:378–382CrossRefGoogle Scholar
  53. 53.
    Landis J, Koch G (1977) The measurement of observer agreement for categorical data. Biometrics 33:159CrossRefzbMATHGoogle Scholar
  54. 54.
    Lord C, McGee JP (2001) Educating children with autism. National Academy Press, WashingtonGoogle Scholar
  55. 55.
    Lord C, Rutter M, Goode S, Heemsbergen J, Jordan H, Mawhood L, Schopler E (1989) Autism diagnostic observation schedule: a standardized observation of communicative and social behavior. J Autism Dev Disord 19:185–212CrossRefGoogle Scholar
  56. 56.
    Kaufman AS, Kaufman NL (1990) Kaufman brief intelligence test. American Guidance Service, Circle PinesGoogle Scholar
  57. 57.
    Seeing Machines (2009) FaceLAB, version 5 [computer program].
  58. 58.
    MacDonald PM, Kirkpatrick SW, Sullivan LA (1996) Schematic drawings of facial expressions for emotion recognition and interpretation by preschool-aged children. Genet Soc Gen Psychol Monogr 122:373–388Google Scholar
  59. 59.
    Castellano G, Pereira A, Leite I, Paiva A, McOwan PW (2009) Detecting user engagement with a robot companion using task and social interaction-based features. In: Proceedings of the international conference on multimodal interfaces (ICMI-09), pp 119–126Google Scholar
  60. 60.
    Michalowski MP, Sabanovic S, Simmons R (2006) A spatial model of engagement for a social robot. In: IEEE international workshop on advanced motion control (AMC-06), pp 762–767Google Scholar
  61. 61.
    Peters C, Asteriadis S, Karpouzis K, de Sevin E (2008) Towards a real-time gaze-based shared attention for a virtual agent. In: Workshop on affective interaction in natural environments (AFFINE), ACM international conference on multimodal interfaces (ICMI-08)Google Scholar
  62. 62.
    Lahiri U, Warren Z, Sarkar N (2011) Design of a gaze-sensitive virtual social interactive system for children with autism. IEEE Trans Neural Syst Rehabil Eng 19:443–452CrossRefGoogle Scholar
  63. 63.
    Cohen J (1988) Statistical power analysis for the behavioral sciences, 2nd edn. Erlbaum, HillsdalezbMATHGoogle Scholar
  64. 64.
    Saldien J, Goris K, Vanderborght B, Vanderfaeillie J, Lefeber D (2010) Expressing emotions with the social robot probo. Int J Soc Robot 2(4):377–389CrossRefGoogle Scholar
  65. 65.
    Breazeal C (2002) Designing sociable robots. MIT Press, CambridgezbMATHGoogle Scholar
  66. 66.
    Sosnowski S, Bittermann A, Kuhnlenz K, Buss M (2006) Design and evaluation of emotion-display EDDIE. In: IEEE/RSJ international conference on intelligent robots and systems (IROS-06), pp 3113–3118Google Scholar
  67. 67.
    Canamero LD, Fredslund J (2000) How does it feel? Emotional interaction with a humanoid lego robot. In: Proceedings of the AAAI fall symposium, pp 23–28Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2017

Authors and Affiliations

  • Sarah A. Koch
    • 1
    Email author
  • Carl E. Stevens
    • 1
  • Christian D. Clesi
    • 1
  • Jenna B. Lebersfeld
    • 1
  • Alyssa G. Sellers
    • 1
  • Myriah E. McNew
    • 1
    • 2
  • Fred J. Biasini
    • 1
  • Franklin R. Amthor
    • 1
  • Maria I. Hopkins
    • 1
  1. 1.Department of PsychologyUniversity of Alabama at BirminghamBirminghamUSA
  2. 2.Department of PsychologyFlorida International UniversityMiamiUSA

Personalised recommendations