Skip to main content

Advertisement

SpringerLink
Log in

Recognition of emotions in autistic children using physiological signals

  • Original Paper
  • Published:
Health and Technology Aims and scope Submit manuscript

Abstract

Children with Autism Spectrum Disorder (ASD) cannot express their emotions explicitly; this makes it difficult for the parents and caretakers associated with these children to understand the child’s behavior, leading to a major setback in the child’s early developmental stages. Studies have shown that a human being’s physiological changes are directly related to his/her psychological reaction. In this paper we propose a wearable wristband for acquiring physiological signals and an algorithm, using a support vector machine (SVM) classifier, that will predict emotional states such as neutral, happy & involvement of children with autism. The psychological reactions (or emotions) are recognized based on the changes in the bodily parameters (physiological basis) such as the galvanic skin response (GSR) and heart rate variability (HRV). For this purpose, vital features extracted from the recorded physiological signals are classified into different emotional states using SVM, which resulted in an overall accuracy of 90 %. This will help the parents and the care takers to understand the emotional patterns of the child better.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Cacioppo JT, Tassinary LG, Berntson GG. Handbook of psychophysiology. 3rd ed. New York: Cambridge University Press; 2007.

    Book  Google Scholar 

  2. Picard WR. Affective computing: challenges. International Journal of Human-Computer Studies. 2003;59:55–64.

    Article  Google Scholar 

  3. Bal E, Harden E, Lamb D, Van Hecke A, Denver J, Porges S. Emotion recognition in children with autism spectrum disorders: relations to eye gaze and autonomic state. Journal of Autism and Developmental Disorders. 2009;40:358–70.

    Article  Google Scholar 

  4. Castelli F, Frith C, Happe F, Frith U. Autism, asperger syndrome and brain mechanisms for the attribution of mental states to animated shapes. Brain. 2002;125:1839–49.

    Article  Google Scholar 

  5. Carlson NR. Physiology of Behavior. Pearson Education: United States of America; 2007.

    Google Scholar 

  6. Picard RW. Future affective technology for autism and emotion communication. Phil. Trans. R. Soc. B. 2009;364:3575–84.

    Article  Google Scholar 

  7. Kaynak O, Alpaydin E, Oja E, Xu L, Raouzaiou A, Ioannou S, Karpouzis K, Tsapatsoulis N, Kollias S, Cowie R. An intelligent scheme for facial expression recognition. Kaynak, O, Eds. Artificial Neural Networks and Neural Information Processing, Lecture notes in Computer Science, Springer. 2003;2714:1109–16.

  8. Ekman P, Levenson RW, Friesen WV. Autonomic nervous system activity distinguishes among emotions. Science. 1983;221:1208–10.

    Article  Google Scholar 

  9. Busso C, Deng Z, Yildirim S, Bulut M, Lee CM, Kazemzadeh A, Lee S, Neumann U, Narayanan S. Analysis of emotion recognition using facial expressions, speech and multimodal information. Proc. Sixth ACM Int',l Conf. Multimodal Interfaces; New York. 2004;doi:10.1145/1027933.1027968

  10. Hess U, March SB. Facial mimicry and emotional contagion to dynamic emotional facial expressions and their influence on decoding accuracy. International Journal of Psychophysiology. 2001;40:129–41.

    Article  Google Scholar 

  11. Hobson RP, Ouston J, Lee A. Emotion recognition in autism: coordinating faces and voices. Psychological Medicine. 1988;18:911–23.

    Article  Google Scholar 

  12. Pomeranz B, MaCaulay RJB, Caudill MA, Kutz I, Adam D, Gordon D, Kilborn KM, Barger AC, Shannon DC, Cohen RJ, Benson H. Assessment of autonomic function in humans by heart rate spectral analysis. Amer J Physiol. 1985;248:h151–3.

    Google Scholar 

  13. Hoyera D, Friedrich H, Franka B, Pompec B, Baranowskid R, Zebrowskie JJ, Schmidtf H. Autonomic information flow improves prognostic impact of task force HRV monitoring. Computer Methods and Programs in Biomedicine. 2006;81:246–55.

    Article  Google Scholar 

  14. Nakasone A, Prendinger H, Ishizuka M. Emotion Recognition from electromyography and skin conductance. The Fifth International Workshop on Biosignal Interpretation, Tokyo, Japan. 2005;219-222.

  15. Ortony A, Clore G. Collins A. The cognitive structure of emotions: Cambridge University Press; 1988.

    Google Scholar 

  16. Levenson RW. Social psychophysiology and emotion: theory and clinical applications. John Wiley & Sons Ltd. 1988:17–42.

  17. Natarajan K, Acharya R, Alias F, Tiboleng T, Puthusserypady SK. Nonlinear analysis of EEG signals at different mental states. BioMedical Engineering OnLine. 2004. doi:10.1186/1475-925X-3-7.

    Google Scholar 

  18. Jirayucharoensak S, Pan-Ngum S, Israsena P. EEG-Based emotion recognition using deep learning network with principal component based covariate shift adaptation. The Scientific World Journal. 2014. doi:10.1155/2014/627892.

    Google Scholar 

  19. Gunes H, Pantic M. Automatic, dimensional and continuous emotion recognition. Int J Synthetic Emotions. 2010;1:68–99.

    Article  Google Scholar 

  20. Theodore P. Beauchaine, Lisa Gatzke-Kopp, Hilary K. Mead. Polyvagal theory and developmental psychopathology: Emotion dysregulation and conduct conduct problems from preschool to adolescence. Biological Psychology. 2007;74:174–84.

    Article  Google Scholar 

  21. Schupp HT, Junghofer M, Weike AI, Hamm AO. Attention and emotion: an ERP analysis of facilitated emotional stimulus processing. NeuroReport. 2003;14:1107–10. doi:10.1097/00001756-200306110-00002.

    Article  Google Scholar 

  22. Cirfaci G, Billeci L, Tartarisco G, Balocchi R, Pioggia G, Brunori E, Maestro S Morales M.A. ECG and GSR and analysis using wearable systems: Application in anorexia nervosa adolescents. Image and Signal Processing and Analysis (ISPA), 8th International Symposium. 2013; 499–504

  23. Spire technologies, https://www.spire.io/.

  24. Critchley HD, Elliott R, Mathias CJ, Dolan RJ. Neural activity relating to generation and representation of galvanic skin conductance responses: afunctional magnetic resonance imaging study. The Journal of Neuroscience. 2000;20:3033–40.

    Google Scholar 

  25. Liua C, Conna K, Sarkar N, Stone W. Physiology-based affect recognition for computer-assisted intervention of children with Autism Spectrum Disorder. Int. J. Human-Computer Studies. 2008;66:662–77.

    Article  Google Scholar 

  26. Conn K, Liu C, Sarkar N, Stone W, Warren Z. Affective Computing. Austria: ARS/I-Tech Education and Publishing; 2008. p. 365–90.

    Google Scholar 

  27. Dumas M. Emotional expression recognition using support vector machines. International conference on Multimodal Interfaces 2001

  28. Vishwanathan SVN, Narasimha MM. SSVM: a simple SVM algorithm. International Joint Conference on Neural Networks. 2002;3:2393–8. doi:10.1109/IJCNN.2002.1007516.

    Google Scholar 

  29. Burges CJC. A Tutorial on Support Vector Machines for Pattern Recognition. Data Mining and Knowledge Discovery. 1998;2:121–67. doi:10.1023/A:1009715923555.

    Article  Google Scholar 

  30. Ekman P Facial expression and emotion. American Psychologist. 1993;48:384–92. doi:10.1037/0003-066X.48.4.384.

    Article  Google Scholar 

  31. Vapnik VN. Statitical learning theory. New York: Wiley-Interscience; 1998.

    Google Scholar 

Download references

Acknowledgment

We would like to express our undying gratitude to the Center for Intelligent Systems, P E S Institute of Technology (PESIT), Bangalore, India for all their support throughout the course of the study. The project was initiated and implemented at PESIT. We also are very grateful to the Academy of Severely Handicapped and Autism (ASHA) Bangalore, India for the constant support and trust endowed upon us. We also thank Prof. Dr. Sathyaprabha, National Institute of Mental Health & Neuro Sciences (NIMHANS) Bangalore, India for her insights on interpreting the physiological signals. We are also thankful to the Department of Child & Adolescent Psychiatry at NIMHANS for providing us with all the necessary resources during the data acquisition phase.

Author information

Authors and Affiliations

  1. Department of Electronics & Communication Engineering, PES Institute of Technology, Bengaluru, India

    Niranjana Krupa, Karthik Anantharam & Sameer Datta

  2. Department of Telecommunication Engineering, PES Institute of Technology, Bengaluru, India

    Manoj Sanker

  3. Department of Child and Adolescent Psychiatry, National Institute of Mental Health and Neuro Sciences, Bengaluru, India

    John Vijay Sagar

Authors
  1. Niranjana Krupa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Karthik Anantharam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Manoj Sanker
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sameer Datta
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. John Vijay Sagar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Niranjana Krupa.

Ethics declarations

Patent

Patent filing has been done (Indian Patent), Application No.: 4808/CHE/2013.

Conflict of interest

All the four authors of this paper, Niranjana Krupa, Karthik Anantharam, Manoj Sanker, Sameer Datta and John Vijay Sagar, declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Krupa, N., Anantharam, K., Sanker, M. et al. Recognition of emotions in autistic children using physiological signals. Health Technol. 6, 137–147 (2016). https://doi.org/10.1007/s12553-016-0129-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12553-016-0129-3

Keywords

Search

Navigation