Facial Expression Image Analysis to Classify High and Low Level ASD Kids Using Attention Mechanism Embedded Deep Learning Technique

Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 711)


One of the developmental disorder found in early childhood is Autism Spectrum Disorder (ASD). Kids suffering from ASD are affected by the way they act in society and interact with others. Usually, ASD kids are associated with excess or poor emotional facial-expressions. The primary focus of this paper is to use advanced deep learning techniques to classify ASD kids into two classes namely Low ASD kids and High ASD kids. Low and High here mentions the intensity of the ASD disorder in the kids. The proposed work aims at achieving this classification with the computer vision techniques and by learning on their facial expressions. Several videos of Low ASD kids and High ASD kids were collected. Each frame of these videos was then parsed into images to train and test an Attention based Residual Neural Network. This proposed model brings a novel method of embedding Attention mechanism on Residual Convolution Neural Networks, which results in carrying the most significant features from the initial and primary layers to the very end with very little distortion. This is done by the attention block, which weighs every parameter according to its significance. This way, the features with higher weighs are passed till the deep layers of the network without any loss of information. Thus, the proposed work is able to classify effectively Low and High ASD kids based on the videos collected successfully yielding results with a state-of-the-art accuracy of around 94%.


Attention residual networks Autism spectrum disorder Micro-expression Image processing Convolution neural networks Deep learning 



The authors would like to thank, Semiconductor device (SDL) laboratory, SENSE and research colleague of Vellore Institute technology, Chennai, India for real time dataset of facial emotion and execution of this research work.

Conflict of Interest and Funding

The authors declare that they have no conflict of interest and no funding.


  1. 1.
    S.E. Weismer, Developmental language disorders: challenges and implications of cross-group comparisons. Folia Phoniatrica et Logopaedica 65(2), 68–77 (2013)Google Scholar
  2. 2.
    G. Gillon, Y. Hyter, F.D. Fernandes, S. Ferman, Y. Hus, K. Petinou, O. Segal, T. Tumanova, I. Vogindroukas, C. Westby, et al., International survey of speech-language pathologists practices in working with children with autism spectrum disorder. Folia Phoniatrica et Logopaedica 69(1–2), 8–19 (2017)Google Scholar
  3. 3.
    P. Doehring, F.R. Volkmar, Knowledge gaps in ASD research: short and long term implications for policy (2016)Google Scholar
  4. 4.
    F. Volkmar, M. Siegel, M. Woodbury-Smith, B. King, J. McCracken, M. State et al., Practice parameter for the assessment and treatment of children and adolescents with autism spectrum disorder. J. Am. Acad. Child Adolescent Psych. 53(2), 237–257 (2014)CrossRefGoogle Scholar
  5. 5.
    C.P. Johnson, S.M. Myers, et al., Identification and evaluation of children with autism spectrum disorders. Pediatrics 120(5), 1183–1215 (2007)Google Scholar
  6. 6.
    L.J. Kirmayer, Beyond the new cross-cultural psychiatry: cultural biology, discursive psychology and the ironies of globalization. Transcult. Psych. 43(1), 126–144 (2006)Google Scholar
  7. 7.
    L.J. Kirmayer, C. Rousseau, E. Corin, D. Groleau, Training researchers in cultural psychiatry: the mcgill-cihr strategic training program. Acad. Psych. 32(4), 320–326 (2008)Google Scholar
  8. 8.
    U. Bronfenbrenner, S.J. Ceci, Nature-nuture reconceptualized in developmental perspective: a bioecological model. Psychol. Rev. 101(4), 568 (1994)Google Scholar
  9. 9.
    U. Bronfenbrenner, P.A. Morris, The bioecological model of human development, inHandbook of child psychology, vol. 1 (2007)Google Scholar
  10. 10.
    Yvette Hus, Issues in identification and assessment of children with autism and a proposed resource toolkit for speech-language pathologists. Folia Phoniatrica et Logopaedica 69(1–2), 27–37 (2017)CrossRefGoogle Scholar
  11. 11.
    T.A. Bennett, P. Szatmari, K. Georgiades, S. Hanna, M. Janus, S. Georgiades, E. Duku, S. Bryson, E. Fombonne, I.M. Smith, et al., Do reciprocal associations exist between social and language pathways in preschoolers with autism spectrum disorders? J. Child Psychol. Psych. 56(8), 874–883 (2015)Google Scholar
  12. 12.
    American Psychiatric Association et al., Dsm-5: Online assessment measures. Preuzeto s (2013)
  13. 13.
    L.B. Swineford, A. Thurm, G. Baird, A.M. Wetherby, S. Swedo, Social (pragmatic) communication disorder: a research review of this new dsm-5 diagnostic category. J. Neurodevel. Disorders 6(1), 41 (2014)Google Scholar
  14. 14.
    A.J. Esbensen, M.M. Seltzer, K.S.L. Lam, J.W. Bodfish, Age-related differences in restricted repetitive behaviors in autism spectrum disorders. J. Autism Develop. Disorders 39(1), 57–66 (2009)Google Scholar
  15. 15.
    S.E. Wolter, Relationship quality of typically developing children and their autistic siblings: a comparison by functionality and anxiety (2019)Google Scholar
  16. 16.
    C.F. Norbury, Practitioner review: social (pragmatic) communication disorder conceptualization, evidence and clinical implications. J. Child Psychol. Psychiatr. 55(3), 204–216 (2014)Google Scholar
  17. 17.
    Y. Brukner-Wertman, N. Laor, O. Golan, Social (pragmatic) communication disorder and its relation to the autism spectrum: dilemmas arising from the dsm-5 classification. J. Autism Develop. Disorders 46(8), 2821–2829 (2016)CrossRefGoogle Scholar
  18. 18.
    R. Karthik, M. Hariharan, S. Anand, P. Mathikshara, A. Johnson, R. Menaka, Attention embedded residual cnn for disease detection in tomato leaves. Appl. Soft Comput. 86, 105933 (2020)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2021

Authors and Affiliations

  1. 1.School of Electronics EngineeringVellore Institute of TechnologyChennaiIndia

Personalised recommendations