Skip to main content
SpringerLink
Log in

Motion-Capture Patterns of Voluntarily Mimicked Dynamic Facial Expressions in Children and Adolescents With and Without ASD

  • Original Paper
  • Published:
Journal of Autism and Developmental Disorders Aims and scope Submit manuscript

Abstract

Research shows that neurotypical individuals struggle to interpret the emotional facial expressions of people with Autism Spectrum Disorder (ASD). The current study uses motion-capture to objectively quantify differences between the movement patterns of emotional facial expressions of individuals with and without ASD. Participants volitionally mimicked emotional expressions while wearing facial markers. Recorded marker movement was grouped by expression valence and intensity. We used Growth Curve Analysis to test whether movement patterns were predictable by expression type and participant group. Results show significant interactions between expression type and group, and little effect of emotion valence on ASD expressions. Together, results support perceptions that expressions of individuals with ASD are different from—and more ambiguous than—those of neurotypical individuals’.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Arsalidou, M., Morris, D., & Taylor, M. J. (2011). Converging evidence for the advantage of dynamic facial expressions. Brain Topography, 24(2), 149–163. https://doi.org/10.1007/s10548-011-0171-4.

    Article  PubMed  Google Scholar 

  • Association, A. P. (2013). Diagnostic and statistical manual of mental disorders (5th edn.). Washington DC: American Psychiatric Association.

    Book  Google Scholar 

  • Baron-Cohen, S., & Kingsley, J. (2003). Mind reading: The interactive guide to emotions. London: Jessica Kingsley Publisher

    Google Scholar 

  • Beall, P. M., Moody, E. J., McIntosh, D. N., Hepburn, S. L., & Reed, C. L. (2008). Rapid facial reactions to emotional facial expressions in typically developing children and children with autism spectrum disorder. Journal of Experimental Child Psychology, 101(3), 206–223. https://doi.org/10.1016/j.jecp.2008.04.004.

    Article  PubMed  Google Scholar 

  • Bone, D., Mertens, J., Zane, E., Lee, S., Narayanan, S., & Grossman, R. (2017). Acoustic-prosodic and physiological response to stressful interactions in children with autism spectrum disorder. In Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH (Vol. 2017). https://doi.org/10.21437/Interspeech.2017-179.

  • Bonneh, Y. S., Levanon, Y., Dean-Pardo, O., Lossos, L., & Adini, Y. (2011). Abnormal speech spectrum and increased pitch variability in young autistic children. Frontiers in Human Neuroscience, 4, 237. https://doi.org/10.3389/fnhum.2010.00237.

    Article  PubMed  PubMed Central  Google Scholar 

  • Brewer, R., Biotti, F., Catmur, C., Press, C., Happé, F., Cook, R., & Bird, G. (2016). Can neurotypical individuals read autistic facial expressions? Atypical Production of emotional facial expressions in autism spectrum disorders. Autism Research, 9(2), 262–271. https://doi.org/10.1002/aur.1508.

    Article  PubMed  Google Scholar 

  • David, F. J., Baranek, G. T., Giuliani, C. A., Mercer, V. S., Poe, M. D., & Thorpe, D. E. (2009). A pilot study: Coordination of precision grip in children and adolescents with high functioning autism. Pediatric Physical Therapy, 21(2), 205–211. https://doi.org/10.1097/PEP.0b013e3181a3afc2.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ekman, P. (2004). Emotional and conversational nonverbal signals. Language, Knowledge, and Representation. https://doi.org/10.1093/acprof.

    Article  Google Scholar 

  • Ekman, P., & Friesen, W. (1971). Constants across cultures in the face and emotion. Journal of Personality and Social Psychology, 17(2), 124.

    Article  PubMed  Google Scholar 

  • Ekman, P., & Friesen, W. (1977). Facial action coding system. Palo Alto: Consulting Pscyhologists Press, Stanford University.

    Google Scholar 

  • Ekman, P., & Friesen, W. V. (1978). The facial action coding system: A technique for the measurement of facial movement. Palo Alto, CA: Consulting Psychologists Press.

    Google Scholar 

  • Faso, D. J., Sasson, N. J., & Pinkham, A. E. (2015). Evaluating Posed and evoked facial expressions of emotion from adults with autism spectrum disorder. Journal of Autism and Developmental Disorders, 45(1), 75–89. https://doi.org/10.1007/s10803-014-2194-7.

    Article  PubMed  Google Scholar 

  • Fournier, K. A., Hass, C. J., Naik, S. K., Lodha, N., & Cauraugh, J. H. (2010). Motor coordination in autism spectrum disorders: A synthesis and meta-analysis. Journal of Autism and Developmental Disorders, 40(10), 1227–1240. https://doi.org/10.1007/s10803-010-0981-3.

    Article  PubMed  Google Scholar 

  • Grossman, R. B. (2015). Judgments of social awkwardness from brief exposure to children with and without high-functioning autism. Autism: The International Journal of Research and Practice, 19(5), 580–587. https://doi.org/10.1177/1362361314536937.

    Article  Google Scholar 

  • Grossman, R. B., Edelson, L. R., & Tager-Flusberg, H. (2013). Emotional facial and vocal expressions during story retelling by children and adolescents with high-functioning autism. Journal of Speech, Language, and Hearing Research, 56(June 2013), 1035–1044. https://doi.org/10.1044/1092-4388(2012/12-0067)Journal.

    Article  PubMed  PubMed Central  Google Scholar 

  • Guha, T., Yang, Z., Grossman, R. B., & Narayanan, S. S. (2016). A computational study of expressive facial dynamics in children with autism. IEEE Transactions on Affective Computing, 1–1. https://doi.org/10.1109/TAFFC.2016.2578316.

  • Guha, T., Yang, Z., Ramakrishna, A., Grossman, R. B., Hedley, D., Lee, S., & Narayanan, S. S. (2015). On quantifying facial expression-related atypicality of children with Autism Spectrum Disorder. In ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings 2015, 803–807. https://doi.org/10.1109/ICASSP.2015.7178080.

  • Hallett, M., Lebiedowska, M. K., Thomas, S. L., Stanhope, S. J., Denckla, M. B., & Rumsey, J. (1993). Locomotion of autistic adults. Archives of Neurology, 50(12), 1304–1308.

    Article  PubMed  Google Scholar 

  • Hauck, D. J., J (2007). House of moves high resolution facial merker-set (92 markers). Los Angeles: Digital Concepts Group, Inc. House of Moves.

    Google Scholar 

  • Izard, C. (1983). The maximally discriminative facial movement coding system. Newark: University of Delaware, Instructional Resource Center.

    Google Scholar 

  • Izard, C. E., Huebner, R. R., Risser, D., & Dougherty, L. (1980). The young infant’s ability to produce discrete emotion expressions. Developmental Psychology, 16(2), 132–140. https://doi.org/10.1037/0012-1649.16.2.132.

    Article  Google Scholar 

  • Kasari, C., Sigman, M., Mundy, P., & Yirmiya, N. (1990). Affective sharing in the context of joint attention interactions of normal, autistic, and mentally retarded children. Journal of Autism and Developmental Disorders, 20(1), 87–100.

    Article  PubMed  Google Scholar 

  • Kasari, C., Sigman, M., & Yirmiya, N. (1993). Focused and social attention of autistic children in interactions with familiar and unfamiliar adults: A comparison of autistic, mentally retarded, and normal children. Development and Psychopathology, 5(3), 403–414. https://doi.org/10.1017/S0954579400004491.

    Article  Google Scholar 

  • Kring, A. M., & Sloan, D. M. (2007). The facial expression coding system (FACES): development, validation, and utility. Psychological Assessment, 19(2), 210–224. https://doi.org/10.1037/1040-3590.19.2.210.

    Article  PubMed  Google Scholar 

  • Langdell, T. (1981). Face perception: An approach to the study of autism. London: Universtity of London.

    Google Scholar 

  • Lau, S. (1982). The effect of smiling on person perception. Journal of Social Psychology. https://doi.org/10.1080/00224545.1982.9713408.

    Article  Google Scholar 

  • Leary, M. R., & Hill, D. a. (1996). Moving on: Autism and movement disturbance. Mental Retardation, 34(1), 39–53.

    PubMed  Google Scholar 

  • Lord, C., DiLavore, P. C., Gotham, K., Guthrie, W., Luyster, R. J., Risi, S., & Rutter, M. (2012). Autism diagnostic observation schedule: ADOS-2. Los Angeles: Western Psychological Services.

    Google Scholar 

  • Lord, C., Risi, S., Lambrecht, L., Cook, E. H. J., Leventhal, B. L., DiLavore, P. C., … Rutter, M. (2000). The autism diagnostic schedule—generic: A standard measures of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders, 30(3), 205–223. https://doi.org/10.1023/A:1005592401947.

    Article  PubMed  Google Scholar 

  • Love, S. R. (1993). Recognition and Production of Facial Emotion by Autistic Children. Louisiana State University. Retrieved from http://digitalcommons.lsu.edu/gradschool_disstheses.

  • Matsugu, M., Mori, K., Mitari, Y., & Kaneda, Y. (2003). Subject independent facial expression recognition with robust face detection using a convolutional neural network. Neural Networks, 16, 555–559. https://doi.org/10.1016/S0893-6080(03)00115-1.

    Article  PubMed  Google Scholar 

  • Matsumoto, D., & Lee, M. (1993). Consciousness, Volition, and the neuropsychology of facial expressions of emotion. Consciousness and Cognition, 2, 237–254. https://doi.org/10.1006/ccog.1993.1022.

    Article  Google Scholar 

  • McIntosh, D. N., Reichmann-Decker, A., Winkielman, P., & Wilbarger, J. L. (2006). When the social mirror breaks: Deficits in automatic, but not voluntary, mimicry of emotional facial expressions in autism. Developmental Science, 9(3), 295–302. https://doi.org/10.1111/j.1467-7687.2006.00492.x.

    Article  PubMed  Google Scholar 

  • Metallinou, A., Grossman, R. B., & Narayanan, S. (2013). Quantifiying atypicality in affective facial expressions of children with autism spectrum disorders.

  • Ming, X., Brimacombe, M., & Wagner, G. C. (2007). Prevalence of motor impairment in autism spectrum disorders. Brain and Development, 29(9), 565–570. https://doi.org/10.1016/j.braindev.2007.03.002.

    Article  PubMed  Google Scholar 

  • Mirman, D. (2014). Growth Curve Analysis and Visualization Using R, 168.

  • Mirman, D., Dixon, J. A., & Magnuson, J. S. (2008). Statistical and computational models of the visual world paradigm: Growth curves and individual differences. Journal of Memory and Language, 59(4), 475–494. https://doi.org/10.1016/j.jml.2007.11.006.

    Article  PubMed  PubMed Central  Google Scholar 

  • Nadig, A., & Shaw, H. (2012). Acoustic and perceptual measurement of expressive prosody in high-functioning autism: Increased pitch range and what it means to listeners. Journal of Autism and Developmental Disorders, 42(4), 499–511.

    Article  PubMed  Google Scholar 

  • Nadig, A., Vivanti, G., & Ozonoff, S. (2009). Adaptation of object descriptions to a partner under increasing communicative demands: A comparison of children with and without autism. Autism Research, 2(6), 334–347. https://doi.org/10.1002/aur.102.

    Article  PubMed  Google Scholar 

  • Oberman, L. M., Winkielman, P., & Ramachandran, V. S. (2009). Slow echo: Facial EMG evidence for the delay of spontaneous, but not voluntary, emotional mimicry in children with autism spectrum disorders. Developmental Science, 12(4), 510–520. https://doi.org/10.1111/j.1467-7687.2008.00796.x.

    Article  PubMed  Google Scholar 

  • Otta, E., Lira, B. B. P., Delevati, N. M., Cesar, O. P., & Pires, C. S. G. (1994). The effect of smiling and of head tilting on person perception. Journal of Psychology: Interdisciplinary and Applied. https://doi.org/10.1080/00223980.1994.9712736.

    Article  Google Scholar 

  • Rinn, W. E. (1984). The neuropsychology of facial expression: A review of the neurological and psychological mechanisms for producing facial expressions. Psychological Bulletin, 95(1), 52–77. https://doi.org/10.1037/0033-2909.95.1.52.

    Article  PubMed  Google Scholar 

  • Rozga, A., King, T. Z., Vuduc, R. W., & Robins, D. L. (2013). Undifferentiated facial electromyography responses to dynamic, audio-visual emotion displays in individuals with autism spectrum disorders. Developmental Science, 16(4), 499–514. https://doi.org/10.1111/desc.12062.

    Article  PubMed  Google Scholar 

  • Sasson, N. J., Faso, D. J., Nugent, J., Lovell, S., Kennedy, D. P., & Grossman, R. B. (2017). Neurotypical peers are less willing to interact with those with autism based on thin slice judgments. London: Nature Publishing Group.

    Book  Google Scholar 

  • Scharlemann, J. P., Eckel, C. C., Kacelnik, A., & Wilson, R. K. (2001). The value of a smile: Game theory with a human face. Journal of Economic Psychology, 22, 617–640.

    Article  Google Scholar 

  • Stagg, S. D., Slavny, R., Hand, C., Cardoso, A., & Smith, P. (2014). Does facial expressivity count? How typically developing children respond initially to children with autism. Autism, 18(6), 704–711. https://doi.org/10.1177/1362361313492392.

    Article  PubMed  Google Scholar 

  • Trotman, C. A., Faraway, J. J., Silvester, K. T., Greenlee, G. M., & Johnston, L. E. (1998). Sensitivity of a method for the analysis of facial mobility. I. Vector of displacement. Cleft Palate-Craniofacial Journal, 35(2), 132–141. https://doi.org/10.1597/1545-1569(1998)035%3C0132:SOAMFT%3E2.3.CO;2

    Article  PubMed  Google Scholar 

  • Uono, S., Sato, W., & Toichi, M. (2010). Brief report: Representational momentum for dynamic facial expressions in pervasive developmental disorder. Journal of Autism and Developmental Disorders, 40(3), 371–377. https://doi.org/10.1007/s10803-009-0870-9.

    Article  PubMed  Google Scholar 

  • Yan, W. J., Wu, Q., Liang, J., Chen, Y. H., & Fu, X. (2013). How fast are the leaked facial expressions: The duration of micro-expressions. Journal of Nonverbal Behavior, 37(4), 217–230. https://doi.org/10.1007/s10919-013-0159-8.

    Article  Google Scholar 

  • Yirmiya, N., Kasari, C., Sigman, M., & Mundy, P. (1989). Facial expressions of affect in autistic, mentally retarded and normal children. Journal of Child Psychology and Psychiatry and Allied Disciplines, 30(5), 725–735. https://doi.org/10.1111/j.1469-7610.1989.tb00785.x.

    Article  Google Scholar 

  • Yoshimura, S., Sato, W., Uono, S., & Toichi, M. (2015). Impaired overt facial mimicry in response to dynamic facial expressions in high-functioning autism spectrum disorders. Journal of Autism and Developmental Disorders, 45(5), 1318–1328.

    Article  PubMed  Google Scholar 

  • Zane, E., Neumeyer, K., Mertens, J., Chugg, A., & Grossman, R. B. (2017). I think we’re alone now: Solitary social behaviors in adolescents with autism spectrum disorder. Journal of Abnormal Child Psychology. https://doi.org/10.1007/s10802-017-0351-0.

    Article  Google Scholar 

Download references

Acknowledgments

We are grateful to all the families who participated in this research. Research was supported by the NIH (NIDCD), Grant number 1R01DC012774-01, Grossman PI. We are also grateful to Dr. Darrin Rogers, of SUNY Fredonia, for offering statistical consultation.

Funding

This study was funded by the NIH (NIDCD), Grant Number 1R01DC012774-01.

Author information

Author notes

  1. Emily Zane

    Present address: Department of Communication Disorders and Sciences, SUNY Fredonia, Thompson Hall, Rm. E127, Fredonia, NY, 14963, USA

  2. Tanaya Guha

    Present address: Department of Computer Science, University of Warwick, Computer Science Building, Rm. CS3.36, Coventry, CV4 7AL, UK

Authors and Affiliations

  1. FACE Lab at Emerson College, Boston, MA, USA

    Emily Zane & Ruth Bergida Grossman

  2. Signal Analysis and Interpretation Laboratory (SAIL) at USC, 3740 McClintock Avenue, Los Angeles, CA, 90089, USA

    Zhaojun Yang, Tanaya Guha & Shrikanth Narayanan

  3. VUI, Inorporated, 15 Broad Street, Boston, MA, 02109, USA

    Lucia Pozzan

  4. Communication Sciences and Disorders at Emerson College, UBank, Rm. 803, Boston, MA, 02116, USA

    Ruth Bergida Grossman

  5. UMMS Shriver Center, Boston, MA, USA

    Ruth Bergida Grossman

Authors
  1. Emily Zane
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhaojun Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lucia Pozzan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tanaya Guha
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shrikanth Narayanan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ruth Bergida Grossman
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The first author helped to conceive and design the analysis; she collected data for the stimulus-categorization study; she performed the analysis, created the figures, and wrote the paper. The second author contributed data and analysis tools, and performed data processing, such as normalizing motion capture data and calculating distances between markers. The third author helped to conceive and design the analysis, and provided tools for performing it. The fourth author contributed analysis tools and techniques, along with data-processing techniques. The fifth author contributed to the design of the methods and oversaw data processing. The last author conceived the study design, oversaw motion-capture data collection, oversaw motion-capture pre-processing, and edited the paper.

Corresponding author

Correspondence to Emily Zane.

Ethics declarations

Conflict of interest

All authors declared that they have no conflict of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent or assent (for participants under the age of 12) was obtained from all individual participants who were included in the study, and informed consent was obtained from all participants’ caregivers.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zane, E., Yang, Z., Pozzan, L. et al. Motion-Capture Patterns of Voluntarily Mimicked Dynamic Facial Expressions in Children and Adolescents With and Without ASD. J Autism Dev Disord 49, 1062–1079 (2019). https://doi.org/10.1007/s10803-018-3811-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10803-018-3811-7

Keywords

Search

Navigation