Do You Think I Approve of That? Designing Facial Expressions for a Robot

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10652)


This paper discusses initial work with a broad age range neuro-typical sample (N \(=\) 77), towards a training system in social signal recognition for high-functioning adults with an Autism Spectrum Disorder. Outlined is a principled design of four approval and four disapproval facial expressions for the EMYS robot head and the method and conduct of a pilot study ‘in the wild’, testing user recognition of these expressions. Results showed that recognition varied over the eight expressions, with some expressions perceived as ambiguous and one approval expression as disapproval. Expression type, ordering effects, age, and pre-study training are identified as important issues.


Robot expressive behaviour Autism Social signals 



This work was supported by the Research Councils UK, project EP/N034546/1. We thank the staff of the Glasgow Science Centre who provided invaluable assistance and materials to ensure successful data collection over two days.


  1. 1.
    Vinciarelli, A., Pantic, M., Bourlard, H.: Social signal processing: survey of an emerging domain. Image Vis. Comput. 27(12), 1743–1759 (2009)CrossRefGoogle Scholar
  2. 2.
    Hillier, A., Campbell, H., Mastriani, K., Izzo, M.V., Kool-Tucker, A.K., Cherry, L., Beversdorf, D.Q.: Two-year evaluation of a vocational support program for adults on the autism spectrum. Career Dev. Transit. Except. Individ. 30(1), 35–47 (2007)CrossRefGoogle Scholar
  3. 3.
    Dib, N., Sturmey, P.: Behavioral skills training and skill learning. In: Seel, N.M. (ed.) Encyclopedia of the Sciences of Learning, pp. 437–438. Springer, Boston (2012). doi: 10.1007/978-1-4419-1428-6_644 Google Scholar
  4. 4.
    Stewart, K.K., Carr, J.E., LeBlanc, L.A.: Evaluation of family-implemented behavioral skills training for teaching social skills to a child with Asperger’s disorder. Clin. Case Stud. 6(3), 252–262 (2007)CrossRefGoogle Scholar
  5. 5.
    Ekman, P., Friesen, W., Ellsworth, P.: What emotion categories or dimensions can observers judge from facial behavior? In: Ekman, P. (ed.) Emotion in the Human Face, pp. 39–55. Cambridge University Press, New York (1982)Google Scholar
  6. 6.
    Jack, R., Garrod, O., Schyns, P.: Dynamic facial expressions of emotion transmit an evolving hierarchy of signals over time. Curr. Biol. 24(2), 187–192 (2014)CrossRefGoogle Scholar
  7. 7.
    Mehrabian, A.: Pleasure-arousal-dominance: a general framework for describing and measuring individual differences in temperament. Curr. Psychol. 14(4), 261–292 (1996)CrossRefMathSciNetGoogle Scholar
  8. 8.
    Wingenbach, T.S.H., Ashwin, C., Brosnan, M.: Validation of the Amsterdam dynamic facial expression set - bath intensity variations (ADFES-BIV): a set of videos expressing low, intermediate, and high intensity emotions. PLoS ONE 11(1), e0147112 (2016). doi: 10.1371/journal.pone.0147112 CrossRefGoogle Scholar
  9. 9.
    Plaisted, K.C.: Reduced generalization in autism: an alternative to weak central coherence. In: Burack, J.A., Charman, T., Yirmiya, N., Zelazo, P.R. (eds.) The Development of Autism: Perspectives from Theory and Research, pp. 149–172. LEA (2001)Google Scholar
  10. 10.
    Snodgrass, J.: Judgment of feeling states from facial behavior: a bottom-up approach. Unpublished doctoral dissertation, University of British Columbia (1992)Google Scholar
  11. 11.
    Hadar, D.: An interrelating model between action units and emotional descriptives: pleasure, arousal and dominance. A lab research project, Hebrew University, Jerusalem (2015)Google Scholar
  12. 12.
    Grammer, K., Oberzaucher, E.: The reconstruction of facial expressions in embodied systems. ZiF: mitteilungen 2, 14–31 (2006)Google Scholar
  13. 13.
    Boukricha, H., Wachsmuth, I., Hofstätter, A., Grammer, K.: Pleasure-arousal-dominance driven facial expression simulation. In: Mühl, C., Heylen, D., Nijholt, A. (eds.) ACII 2009, pp. 119–125. IEEE (2009)Google Scholar
  14. 14.
    Kȩdzierski, J., Muszyński, R., Zoll, C., Oleksy, A., Frontkiewicz, M.: EMYS–emotive head of a social robot. Int. J. Soc. Robot. 5(2), 237–249 (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Heriot-Watt UniversityRiccartonUK

Personalised recommendations