Skip to main content
SpringerLink
Log in

Automated vs Human Recognition of Emotional Facial Expressions of High-Functioning Children with Autism in a Diagnostic-Technological Context: Explorations via a Bottom-Up Approach

  • Conference paper
  • First Online:
Computers Helping People with Special Needs (ICCHP 2018)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10896))

Abstract

Early detection of autism spectrum conditions (ASC) is an important goal. Automated facial expression recognition is a promising approach and has implications for assistive and educational technologies, too. This study was an initial exploration of (1) the inter-rater reliability of human recognition of facial emotions of high functioning (HF) children with ASC; (2) the relationship between human and automated recognition of facial emotions; and (3) a ‘bottom-up’ approach on identifying ASC/typical development (TD) differences, from a screening serious game context. Thirteen HF, kindergarten-age children with ASC and 13 children with TD, matched along age and IQ, participated. Emotion recognition was administered on video-recordings from sessions of their playing with the serious game. Results showed lack of inter-rater reliability in human coding, confirming some advantages of machine coding. The simple bottom-up cross-sectional exploratory analysis did not reveal any ASC/TD difference. This is in contrast with our and others’ previous results, indicating such differences when aggregating emotion data from wider time-windows in machine-coded data-sets. This suggests that this second approach may be a more promising one to identify autism-specific emotion expression patterns.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. World Health Organization: International Classification of Diseases and Disorders, 10th edn. Author, Geneva (1993)

    Google Scholar 

  2. APA [American Psychiatric Association]: Diagnostic and Statistical Manual of Mental Disorders, 5th edn, (DSM-5). American Psychiatric Association, Washington DC (2013)

    Google Scholar 

  3. Chawarska, K., Macari, S., Volkmar, F., Kim, S.H., Shic, F.: Autism in infancy and early childhood. In: Volkmar, F., Rogers, S., Paul, R., Pelphrey, K. (eds.) Handbook of Autism and Pervasive Developmental Disorders. Willey, Hoboken (2014)

    Google Scholar 

  4. Brett, D., Warnell, F., McConachie, H., Parr, J.R.: Factors affecting age at ASD diagnosis in UK: no evidence that diagnosis age has decreased between 2004 and 2014. J. Autism Dev. Disord. 46, 1974–1984 (2016)

    Article  Google Scholar 

  5. Daniels, A.M., Mandell, D.S.: Explaining differences in age at autism spectrum disorder diagnosis: a critical review. Autism 18, 583–597 (2014)

    Article  Google Scholar 

  6. Bölte, S., Bartl-Pokorny, K.D., Jonsson, U., Berggren, S., Zhang, D., Kostrzewa, E., et al.: How can clinicians detect and treat autism early? Methodological trends of technology use in research. Acta Paediatr. 105(2), 137–144 (2016)

    Article  Google Scholar 

  7. Borsos, Z., Gyori, M.: Can automated facial expression analysis show differences between autism and typical functioning? In: Cudd, P., de Witte, L. (eds.) Harnessing the Power of Technology to Improve Lives. Studies in Health Technology and Informatics, pp. 797–804. IOS Press, Amsterdam (2017)

    Google Scholar 

  8. Owada, K., Kojima, M., Yassin, W., Kuroda, M., Kawakobo, Y., Kuwabara, H., Kano, Y., Yamasue, H.: Computer-analyzed facial expression as a surrogate marker for autism spectrum social core symptoms. PLoS ONE 13(1), e0190442 (2018)

    Article  Google Scholar 

  9. Gyori, M., Borsos, Z., Stefanik, K., Csákvári, J.: Data quality as a bottleneck in developing a social-serious-game-based multi-modal system for early screening for ‘High Functioning’ cases of autism spectrum condition. In: Miesenberger, K., Bühler, C., Penaz, P. (eds.) ICCHP 2016. LNCS, vol. 9759, pp. 358–366. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41267-2_51

    Chapter  Google Scholar 

  10. Grossard, C., Grynspan, O., Serret, S., Jouen, A.-L., Bailly, K., Cohen, D.: Serious games to teach social interactions and emotions to individuals with autism spectrum disorders (ASD). Comput. Educ. 113, 195–211 (2017)

    Article  Google Scholar 

  11. Gyori, M., Borsos, Z., Stefanik, K.: Evidence-based development and first usability testing of a social serious game based multi-modal system for early screening for atypical socio-cognitive development. In: Sik-Lányi, C., Hoogerwerf, E.J., Miesenberger, K. (eds.) Assistive Technology: Building Bridges. Studies in Health Technology and Informatics, pp. 48–54. IOS Press, Amsterdam (2015)

    Google Scholar 

  12. Rutter, M., Bailey, A., Lord, C.: The Social Communication Questionnaire: Manual. Western Psychological Services, Los Angeles (2003)

    Google Scholar 

  13. Lord, C., Rutter, M., DiLavore, P.C., Risi, S.: Autism Diagnostic Observation Schedule. Western Psychological Services, Los Angeles (1999)

    Google Scholar 

  14. Le Couteur, A., Lord, C., Rutter, M.: The Autism Diagnostic Interview-Revised. Western Psychological Services, Los Angeles (2003)

    Google Scholar 

  15. Sodian, B., Frith, U.: Deception and sabotage in autistic, retarded and normal children. J. Child Psychol. Psychiatry 33, 591–605 (1992)

    Article  Google Scholar 

  16. Kring, A.M., Sloan, D.M.: The facial expression coding system (FACES): development, validation, and utility. Psychol. Assess. 19(2), 210–224 (2007)

    Article  Google Scholar 

  17. Cohn, J.F., Ekman, P.: Measuring facial action. In: Harrigan, J.A., Rosenthal, R., Scherer, K.R. (eds.) The New Handbook of Methods in Nonverbal Behavior Research, pp. 9–64. Oxford University Press (2005)

    Google Scholar 

Download references

Ethical Approval and Acknowledgements

This research was approved by the Research Ethics Committee of the ‘Barczi Gusztav’ Faculty of Special Education, ELTE University, Budapest, Hungary. Some elements were funded by a grant within the EIT ICT Labs Hungarian Node (PI: András Lőrincz), and via a TÁMOP grant, co-financed by the European Union and the government of Hungary (TÁMOP 4.2.1./B-09/KMR-2010-0003). The work of M Gyori, Zs Borsos and K Stefanik was supported by a grant from the Hungarian Academy of Sciences within its Content Pedagogy Programme. Authors wish to thank András Lőrincz and Tibor Gregorics for their contributions and support in earlier phases of the project.

Author information

Authors and Affiliations

  1. ELTE University, Ecseri Road 3, Budapest, 1097, Hungary

    Miklos Gyori, Zsófia Borsos, Krisztina Stefanik, Zoltán Jakab, Fanni Varga & Judit Csákvári

  2. MTA-ELTE ‘Autism in Education’ Research Group, Ecseri Road 3, Budapest, 1097, Hungary

    Miklos Gyori, Zsófia Borsos & Krisztina Stefanik

Authors
  1. Miklos Gyori
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zsófia Borsos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Krisztina Stefanik
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zoltán Jakab
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fanni Varga
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Judit Csákvári
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Miklos Gyori .

Editor information

Editors and Affiliations

  1. Johannes Kepler University Linz, Linz, Austria

    Klaus Miesenberger

  2. National and Kapodistrian University of Athens, Athens, Greece

    Georgios Kouroupetroglou

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gyori, M., Borsos, Z., Stefanik, K., Jakab, Z., Varga, F., Csákvári, J. (2018). Automated vs Human Recognition of Emotional Facial Expressions of High-Functioning Children with Autism in a Diagnostic-Technological Context: Explorations via a Bottom-Up Approach. In: Miesenberger, K., Kouroupetroglou, G. (eds) Computers Helping People with Special Needs. ICCHP 2018. Lecture Notes in Computer Science(), vol 10896. Springer, Cham. https://doi.org/10.1007/978-3-319-94277-3_72

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-94277-3_72

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-94276-6

  • Online ISBN: 978-3-319-94277-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation