Skip to main content
SpringerLink
Log in

Deep Learning-Based Human Action Recognition Framework to Assess Children on the Risk of Autism or Developmental Delays

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1794))

Included in the following conference series:

  • 678 Accesses

Abstract

Automatic human action recognition of children with machine learning and deep learning methods using play-based videos can lead to developmental monitoring, early identification, and efficient management of children at risk of neurodevelopmental disorders (NDD) and Autism Spectrum Disorders (ASD). Advancements in deep learning make it feasible to develop human action recognition models with large datasets, enhance clinician capacity, and improve access, affordability, and quality of care. However, data collection is challenging due to ethical, legal, and limited datasets of children with NDD and the enormous amount of human tasks involved in video annotation. Therefore, we propose a new method to overcome these challenges by training several deep learning models using a publicly available action dataset comprising adults performing various actions. We demonstrate the effectiveness of our multiple models to recognize similar actions of children in a custom-collected video dataset of children with NDD, ASD, and Typical development. Our method assist child psychologists in intelligently detecting children at risk of NDD and measuring their progress from their videos captured in the natural environment.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Baker, S., Kandasamy, Y.: Machine learning for understanding and predicting neurodevelopmental outcomes in premature infants: a systematic review. Pediatric Res. (2022)

    Google Scholar 

  2. Carpenter, K.L., et al.: Digital behavioral phenotyping detects atypical pattern of facial expression in toddlers with autism. Autism Res. 14(3), 488–499 (2021)

    Article  Google Scholar 

  3. Carreira, J., Zisserman, A.: Quo vadis, action recognition? a new model and the kinetics dataset. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4724–4733 (2017)

    Google Scholar 

  4. Chlebowski, C., Green, J.A., Barton, M.L., Fein, D.: Using the childhood autism rating scale to diagnose autism spectrum disorders. J. Autism Dev. Disord. 40(7), 787–799 (2010)

    Article  Google Scholar 

  5. Cioni, G., Inguaggiato, E., Sgandurra, G.: Early intervention in neurodevelopmental disorders: underlying neural mechanisms. Developm. Med. Child Neurol. 58(S4), 61–66 (2016)

    Article  Google Scholar 

  6. Elbattah, M., Guerin, J.L., Carette, R., Cilia, F., Dequen, G.: Nlp-based approach to detect autism spectrum disorder in saccadic eye movement. In: 2020 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1581–1587 (2020)

    Google Scholar 

  7. Farnebäck, G.: Two-frame motion estimation based on polynomial expansion. In: Bigun, J., Gustavsson, T. (eds.) SCIA 2003. LNCS, vol. 2749, pp. 363–370. Springer, Heidelberg (2003). https://doi.org/10.1007/3-540-45103-X_50

    Chapter  Google Scholar 

  8. Feichtenhofer, C., Pinz, A., Wildes, R.P.: Spatiotemporal residual networks for video action recognition. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS 2016, pp. 3476–3484. Curran Associates Inc., Red Hook (2016)

    Google Scholar 

  9. Gupta, C., et al.: Bringing machine learning to research on intellectual and developmental disabilities: taking inspiration from neurological diseases. J. Neurodev. Disord. 14(1), 28 (2022)

    Article  Google Scholar 

  10. Hashemi, J., et al.: Computer vision analysis for quantification of autism risk behaviors. IEEE Trans. Affect. Comput. 12(1), 215–226 (2021)

    Article  Google Scholar 

  11. Hollis, C., et al.: Annual research review: Digital health interventions for children and young people with mental health problems - a systematic and meta-review. J. Child Psychol. Psychiatry 58(4), 474–503 (2017)

    Article  Google Scholar 

  12. Hollis, C., et al.: the AQUA Trial Group: The impact of a computerised test of attention and activity (QbTest) on diagnostic decision-making in children and young people with suspected attention deficit hyperactivity disorder: single-blind randomised controlled trial. J. Child Psychol. Psychiatry 59(12), 1298–1308 (2018)

    Article  Google Scholar 

  13. Kay, W., et al.: The kinetics human action video dataset. arXiv: 1705.06950 (2017)

  14. Khowaja, M., Robins, D.L., Adamson, L.B.: Utilizing two-tiered screening for early detection of autism spectrum disorder. Autism 22(7), 881–890 (2018)

    Article  Google Scholar 

  15. Klintwall, L., Eikeseth, S.: Early and intensive behavioral intervention (eibi) in autism. Comprehensive Guide Autism, 117–137 (2014)

    Google Scholar 

  16. Kohli, M., Kar, A.K., Bangalore, A., Ap, P.: Machine learning-based aba treatment recommendation and personalization for autism spectrum disorder: an exploratory study. Brain Informat. 9(1), 1–25 (2022)

    Article  Google Scholar 

  17. Kornack, J., Persicke, A., Cervantes, P., Jang, J., Dixon, D.: Economics of autism spectrum disorders: An overview of treatment and research funding. In: Handbook of Early Intervention For Autism Spectrum Disorders, pp. 165–178 (2014)

    Google Scholar 

  18. Liu, J., Shahroudy, A., Perez, M., Wang, G., Duan, L.Y., Kot, A.C.: Ntu rgb+d 120: A large-scale benchmark for 3d human activity understanding. IEEE Trans. Pattern Anal. Mach. Intell. 42(10), 2684–2701 (2020)

    Article  Google Scholar 

  19. Lord, C., et al.: The autism diagnostic observation schedule-generic: A standard measure of social and communication deficits associated with the spectrum of autism. J. Autism Dev. Disord. 30(3), 205–223 (2000)

    Article  Google Scholar 

  20. Lord, C., Storoschuk, S., Rutter, M., Pickles, A.: Using the adi-r to diagnose autism in preschool children. Infant Ment. Health J. 14(3), 234–252 (1993)

    Article  Google Scholar 

  21. Malik-Soni, N., et al.: Tackling healthcare access barriers for individuals with autism from diagnosis to adulthood. Pediatr. Res. 91(5), 1028–1035 (2021)

    Article  Google Scholar 

  22. Nabil, M.A., Akram, A., Fathalla, K.M.: Applying machine learning on home videos for remote autism diagnosis: Further study and analysis. Health Informatics J. 27(1), 1460458221991882 (2021)

    Article  Google Scholar 

  23. Ouss, L., et al.: Behavior and interaction imaging at 9 months of age predict autism/intellectual disability in high-risk infants with west syndrome. Transl. Psychiatry 10(1), 1–7 (2020)

    Article  Google Scholar 

  24. Pandey, P., Prathosh, A., Kohli, M., Pritchard, J.: Guided weak supervision for action recognition with scarce data to assess skills of children with autism. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 463–470 (2020)

    Google Scholar 

  25. Patten, E., Belardi, K., Baranek, G.T., Watson, L.R., Labban, J.D., Oller, D.K.: Vocal patterns in infants with autism spectrum disorder: Canonical babbling status and vocalization frequency. J. Autism Dev. Disord. 44(10), 2413–2428 (2014)

    Article  Google Scholar 

  26. Rahman, R., Kodesh, A., Levine, S.Z., Sandin, S., Reichenberg, A., Schlessinger, A.: Identification of newborns at risk for autism using electronic medical records and machine learning. Euro. Psych. 63(1) (2020)

    Google Scholar 

  27. Scassellati, C., Bonvicini, C., Benussi, L., Ghidoni, R., Squitti, R.: Neurodevelopmental disorders: Metallomics studies for the identification of potential biomarkers associated to diagnosis and treatment. J. Trace Elem. Med Biol. 60, 126499 (2020)

    Article  Google Scholar 

  28. Serenius, F., et al.: For the extremely preterm infants in sweden study group: neurodevelopmental outcomes among extremely preterm infants 6.5 years after active perinatal care in Sweden. JAMA Pediatrics 170(10), 954–963 (2016)

    Google Scholar 

  29. Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recognition in videos. In: NIPS (2014)

    Google Scholar 

  30. Sundberg, M.L.: VB-MAPP Verbal Behavior Milestones Assessment and Placement Program: a language and social skills assessment program for children with autism or other developmental disabilities: guide. Mark Sundberg (2008)

    Google Scholar 

  31. Tariq, Q., Daniels, J., Schwartz, J.N., Washington, P., Kalantarian, H., Wall, D.P.: Mobile detection of autism through machine learning on home video: A development and prospective validation study. PLoS Med. 15(11), e1002705 (2018)

    Article  Google Scholar 

  32. Tariq, Q., et al.: Detecting developmental delay and autism through machine learning models using home videos of bangladeshi children: Development and validation study. J. Med. Internet Res. 21(4), e13822 (2019)

    Article  Google Scholar 

  33. Uddin, M., Wang, Y., Woodbury-Smith, M.: Artificial intelligence for precision medicine in neurodevelopmental disorders. NPJ Digital Med. 2(1), 112 (2019)

    Article  Google Scholar 

  34. Valentine, A.Z., Brown, B.J., Groom, M.J., Young, E., Hollis, C., Hall, C.L.: A systematic review evaluating the implementation of technologies to assess, monitor and treat neurodevelopmental disorders: A map of the current evidence. Clin. Psychol. Rev. 80, 101870 (2020)

    Article  Google Scholar 

  35. Young, G.S., et al.: A video-based measure to identify autism risk in infancy. J. Child Psychol. Psychiatry 61(1), 88–94 (2020)

    Article  Google Scholar 

  36. Zhang, L., Wang, M., Liu, M., Zhang, D.: A survey on deep learning for neuroimaging-based brain disorder analysis. Front. Neurosci. 14, 779 (2020)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Management Studies, Indian Institute of Technology-Delhi, Delhi, India

    Manu Kohli & Arpan Kumar Kar

  2. CogniAble, Gurgaon, India

    Varun Ganjigunte Prakash

  3. Department of Electrical Communication Engineering, Indian Institute of Science, Bengaluru, India

    A. P. Prathosh

Authors
  1. Manu Kohli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arpan Kumar Kar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Varun Ganjigunte Prakash
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. A. P. Prathosh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manu Kohli .

Editor information

Editors and Affiliations

  1. Indian Institute of Technology Indore, Indore, India

    Mohammad Tanveer

  2. Indian Institute of Information Technology - Allahabad, Prayagraj, India

    Sonali Agarwal

  3. Kobe University, Kobe, Japan

    Seiichi Ozawa

  4. Indian Institute of Technology Patna, Patna, India

    Asif Ekbal

  5. University of Innsbruck, Innsbruck, Austria

    Adam Jatowt

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kohli, M., Kar, A.K., Prakash, V.G., Prathosh, A.P. (2023). Deep Learning-Based Human Action Recognition Framework to Assess Children on the Risk of Autism or Developmental Delays. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds) Neural Information Processing. ICONIP 2022. Communications in Computer and Information Science, vol 1794. Springer, Singapore. https://doi.org/10.1007/978-981-99-1648-1_38

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-1648-1_38

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-1647-4

  • Online ISBN: 978-981-99-1648-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation