Skip to main content
SpringerLink
Log in

An intelligent monitoring system for indoor safety of individuals suffering from Autism Spectrum Disorder (ASD)

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Autism Spectrum Disorder (ASD) is distinguished by a variety of behavioral and social deficits. One of the most prominent problems in an autistic child is their aggressive nature that can cause physical injuries. In order to address the following behavioral deficits, this paper is an attempt to present a novel monitoring framework to predict irregularities from the physical activities of an autistic child. The primary motive behind this study is to improve the safety measures for the child in an indoor environment by generating time sensitive-alerts for caretakers and doctors. In this study, a deep 3D CNN and LSTM based activity prediction methodology are proposed to recognize physical irregularities. The 3D CNN model extracts the spatio-temporal features from the video templates for stance prediction with subject position. The LSTM model calculates the temporal relationship in the feature maps to analyze the scale of irregularity. Moreover, in order to deal with such irregularities, a time-sensitive alert-based decision process is proposed in the present work to generate early warnings to the doctor and caretaker. The proficiency of the system is increased by storing the performed activity scores in the local database of the system which can be further utilized to provide medicinal or therapeutic assistance. In addition, the measured experimental results are validated with state-of-the-art methodologies. Hence, the calculated statistical measures justify the utility of the proposed study in the healthcare and assistive-care domain.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  • Akula A, Shah AK, Ghosh R (2018) Deep learning approach for human action recognition in infrared images. Cogn Syst Res 50:146–154. https://doi.org/10.1016/j.cogsys.2018.04.002

    Article  Google Scholar 

  • American Psychiatric Association (2013) Diagnostic and statistical manual of mental disorders: DSM-5. American Psychiatric Association, Philadelphia

    Book  Google Scholar 

  • Chen G, Zou Y, Zhang C (2019) STMP: spatial temporal multi-level proposal network for activity detection. Springer, Berlin, pp 29–41

    Google Scholar 

  • Cheron G, Laptev I, Schmid C (2015) P-CNN: pose-based CNN features for action recognition. In: Proceedings of the IEEE international conference on computer vision

  • Chu WS, Song Y, Jaimes A (2015) Video co-summarization: video summarization by visual co-occurrence. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition

  • Dunlap G, Dyer K, Koegel RL (1983) Autistic self-stimulation and intertrial interval duration. Am J Ment Defic 88:194–202

    Google Scholar 

  • Ezzahout A, Haj Thami RO (2013) Conception and development of a video surveillance system for detecting, tracking and profile analysis of a person. In: 2013 3rd international symposium ISKO-Maghreb

  • Feichtenhofer C, Pinz A, Zisserman A (2016) Convolutional two-stream network fusion for video action recognition. https://doi.org/10.1109/CVPR.2016.213

  • Feichtenhofer C, Pinz A, Wildes RP (2017) Spatiotemporal multiplier networks for video action recognition. In: Proceedings—30th IEEE conference on computer vision and pattern recognition, CVPR 2017

  • Ganz ML (2007) The lifetime distribution of the incremental societal costs of autism. Arch Pediatr Adolesc Med. https://doi.org/10.1001/archpedi.161.4.343

    Article  Google Scholar 

  • Ghasemzadeh H, Loseu V, Jafari R (2010a) Structural action recognition in body sensor networks: distributed classification based on string matching. IEEE Trans Inf Technol Biomed. https://doi.org/10.1109/TITB.2009.2036722

    Article  Google Scholar 

  • Ghasemzadeh H, Jafari R, Prabhakaran B (2010b) A body sensor network with electromyogram and inertial sensors: multimodal interpretation of muscular activities. IEEE Trans Inf Technol Biomed. https://doi.org/10.1109/TITB.2009.2035050

    Article  Google Scholar 

  • Graves A (2013) Generating sequences with recurrent neural networks. Dict Mark Commun. https://doi.org/10.4135/9781452229669.n2760

    Article  Google Scholar 

  • Griffith GM, Hastings RP, Nash S, Hill C (2010) Using matched groups to explore child behavior problems and maternal well-being in children with down syndrome and autism. J Autism Dev Disord. https://doi.org/10.1007/s10803-009-0906-1

    Article  Google Scholar 

  • Hutt C, Ounsted C (1966) The biological significance of gaze aversion with particular reference to the syndrome of infantile autism. Behav Sci. https://doi.org/10.1002/bs.3830110504

    Article  Google Scholar 

  • Hutt C, Forrest SJ, Richer J (1975) Cardiac arrhythmia and behaviour in autistic children. Acta Psychiatr Scand. https://doi.org/10.1111/j.1600-0447.1975.tb00014.x

    Article  Google Scholar 

  • Ijjina EP, Chalavadi KM (2017) Human action recognition in RGB-D videos using motion sequence information and deep learning. Pattern Recognit. https://doi.org/10.1016/j.patcog.2017.07.013

    Article  Google Scholar 

  • Karafi M, Cernock JH (2010) Recurrent neural network language modeling, pp 1045–1048. https://doi.org/10.1021/jp056727x

  • Kolda TG, Bader BW, Laboratories SN (2008) Tensor decompositions and applications. Soc Ind Appl Math 51:455–500. https://doi.org/10.1137/07070111x

    Article  MathSciNet  Google Scholar 

  • Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. Adv Neural Inf Process Syst. https://doi.org/10.1016/j.protcy.2014.09.007

    Article  Google Scholar 

  • Lindemann U, Hock A, Stuber M et al (2005) Evaluation of a fall detector based on accelerometers: a pilot study. Med Biol Eng Comput. https://doi.org/10.1007/BF02351026

    Article  Google Scholar 

  • Liu J, Shahroudy A, Xu D, Wang G (2016) Spatio-temporal LSTM with trust gates for 3D human action recognition. In: Leibe B, Matas J, Sebe N, Welling M (eds) Computer vision – ECCV 2016, vol 9907. Springer, Cham, pp 816–833

    Chapter  Google Scholar 

  • Lord C, Volkmar F, Lombroso P (2002) Genetics of childhood disorders: XLII. Autism, part 1: diagnosis and assessment in autistic spectrum disorders. J Am Acad Child Adolesc Psychiatry. https://doi.org/10.1097/00004583-200209000-00015

    Article  Google Scholar 

  • Mubashir M, Shao L, Seed L (2013) A survey on fall detection: principles and approaches. Neurocomputing. https://doi.org/10.1016/j.neucom.2011.09.037

    Article  Google Scholar 

  • Myers BJ, Mackintosh VH, Goin-Kochel RP (2009) My greatest joy and my greatest heart ache: parents own words on how having a child in the autism spectrum has affected their lives and their families lives. Res Autism Spectr Disord. https://doi.org/10.1016/j.rasd.2009.01.004

    Article  Google Scholar 

  • Ng S, Fakih A, Fourney A et al (2009) Towards a mobility diagnostic tool: tracking rollator users leg pose with a monocular vision system. In: Proceedings of the 31st annual international conference of the IEEE engineering in medicine and biology society: engineering the future of biomedicine, EMBC 2009

  • Noury N (2002) A smart sensor for the remote follow up of activity and fall detection of the elderly. In: 2nd Annual international IEEE-EMBS special topic conference on microtechnologies in medicine and biology—proceedings

  • Noury N, Rumeau P, Bourke AK et al (2008) A proposal for the classification and evaluation of fall detectors. IRBM 29:340–349

    Article  Google Scholar 

  • Nyan MN, Tay FEH, Tan AWY, Seah KHW (2006) Distinguishing fall activities from normal activities by angular rate characteristics and high-speed camera characterization. Med Eng Phys. https://doi.org/10.1016/j.medengphy.2005.11.008

    Article  Google Scholar 

  • Pascanu R, Mikolov T, Bengio Y (2012) On the difficulty of training recurrent neural networks. https://doi.org/10.1109/72.279181

  • Perry JT, Kellog S, Vaidya SM et al (2009) Survey and evaluation of real-time fall detection approaches. In: 6th International symposium on high capacity optical networks and enabling technologies, HONET 09

  • Rimminen H, Lindstrm J, Linnavuo M, Sepponen R (2010) Detection of falls among the elderly by a floor sensor using the electric near field. IEEE Trans Inf Technol Biomed. https://doi.org/10.1109/TITB.2010.2051956

    Article  Google Scholar 

  • Shahroudy A, Liu J, Ng T-T, Wang G (2016) NTU RGB+D: a large scale dataset for 3D human activity analysis. https://doi.org/10.1109/CVPR.2016.115

  • Shi G, Chan CS, Li WJ et al (2009) Mobile human airbag system for fall protection using MEMS sensors and embedded SVM classifier. IEEE Sens J. https://doi.org/10.1109/JSEN.2008.2012212

    Article  Google Scholar 

  • Singhal S, Tripathi V (2019) Action recognition framework based on normalized local binary pattern. Springer, Singapore, pp 247–255

    Google Scholar 

  • Song S, Lan C, Xing J et al (2016) An end-to-end spatio-temporal attention model for human action recognition from Skeleton data. In: Thirty-first AAAI conference on artificial intelligence, pp 4263–4270

  • Sun L, Jia K, Yeung DY, Shi BE (2015) Human action recognition using factorized spatio-temporal convolutional networks. In: Proceedings of the IEEE international conference on computer vision

  • Sutskever I, Hinton G, Krizhevsky A, Salakhutdinov RR (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res. https://doi.org/10.1214/12-AOS1000

    Article  MathSciNet  Google Scholar 

  • Tran D, Bourdev L, Fergus R et al (2015) Learning spatiotemporal features with 3D convolutional networks. In: Proceedings of the IEEE international conference on computer vision

  • Wang Y, Long M, Wang J, Yu PS (2017) Spatiotemporal pyramid network for video action recognition. In: Proceedings—30th IEEE conference on computer vision and pattern recognition, CVPR 2017

  • Xu Z, Yang Y, Hauptmann AG (2015) A discriminative CNN video representation for event detection. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition

  • Yang X, Molchanov P, Kautz J (2016) Multilayer and multimodal fusion of deep neural networks for video classification. In: Proceedings of the 24th ACM international conference on multimedia, pp 978–987

  • Yang X, Luo X, Huang T et al (2018) Towards efficient and objective work sampling: recognizing workers activities in site surveillance videos with two-stream convolutional networks. Autom Constr 94:360–370. https://doi.org/10.1016/j.autcon.2018.07.011

    Article  Google Scholar 

  • Ye Y, Ci S, Katsaggelos AK et al (2013) Wireless video surveillance: a survey. IEEE Access. https://doi.org/10.1109/ACCESS.2013.2282613

    Article  Google Scholar 

  • Ye J, Stevenson G, Dobson S (2015) KCAR: a knowledge-driven approach for concurrent activity recognition. Pervasive Mob Comput 19:4770. https://doi.org/10.1016/j.pmcj.2014.02.003

    Article  Google Scholar 

  • Zeiler MD (2012) ADADELTA: an adaptive learning rate method

Download references

Author information

Authors and Affiliations

  1. Lovely Professional University, Phagwara, Punjab, 144411, India

    Ankush Manocha & Ramandeep Singh

Authors
  1. Ankush Manocha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ramandeep Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ankush Manocha.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Manocha, A., Singh, R. An intelligent monitoring system for indoor safety of individuals suffering from Autism Spectrum Disorder (ASD). J Ambient Intell Human Comput 14, 15793–15808 (2023). https://doi.org/10.1007/s12652-019-01277-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-019-01277-3

Keywords

Search

Navigation