Skip to main content
SpringerLink
Log in

OpenFace Tracker and GoogleNet: To Track and Detect Emotional States for People with Asperger Syndrome

  • Conference paper
  • First Online:
Micro-Electronics and Telecommunication Engineering (ICMETE 2023)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 894))

  • 76 Accesses

Abstract

The goal of this work is to create an emotional model that can categorize emotions in real-time for persons with Asperger’s syndrome (AS). The model is based on facial expressions, head movement, and eye gaze as significant features for emotions. Developing like this an emotional model can aid other people to communicate socially with people with autism. This model overcomes the limitations of previous research that used sensitive and invasive methods to capture physiological data in order to predict emotional states. These instruments are very costly and need a controlled environment. The proposed model succeeded in classifying the emotional states of the AS using a natural spontaneous dataset without invasive tools and in an uncontrolled environment. The dataset used in this study is an available dataset and contains videos recorder in an uncontrolled environment with different facial occlusion and illumination changes. The emotions were classified as fear, disgust, joy, anticipation, and sadness. The proposed model implements a modified version of the well-known and wide-used GoogleNet machine learning model to classify emotions. The key feature of GoogleNet is the inception module, which is designed to capture different levels of spatial features and perform dimensionality reduction using parallel convolutional layers with different filter sizes. There are several metrics that were used to measure the accuracy and generality of a DL model, including accuracy, precision, recall, and F1 score, depending on the nature of the task. Overall, measuring the generality of a DL model is an important step in evaluating its performance and ensuring that it will perform well on new, unseen data. The model achieved significant performance on unseen data with accuracy (98%).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Hassan A, Pinkwart N, Shafi M (2021) Serious games to improve social and emotional intelligence in children with autism. Entertainment Comput 38:100417. https://doi.org/10.1016/j.entcom.2021.100417

    Article  Google Scholar 

  2. Sharma SR, Gonda X, Tarazi FI (2018) Autism spectrum disorder: classification, diagnosis and therapy. Pharmacol Ther 190:91–104. https://doi.org/10.1016/j.pharmthera.2018.05.007

    Article  Google Scholar 

  3. Dawood A, Turner S, Perepa P (2018) Affective computational model to extract natural affective states of students with Asperger syndrome (AS) in computer-based learning environment. IEEE Access 6:67026–67034. https://doi.org/10.1109/ACCESS.2018.2879619

    Article  Google Scholar 

  4. Faras H, Al Ateeqi N, Tidmarsh L (2010) Autism spectrum disorders. Ann Saudi Med 30(4):295–300. https://doi.org/10.4103/0256-4947.65261

  5. Lin E, Tsai SJ (2016) Genome-wide microarray analysis of gene expression profiling in major depression and antidepressant therapy. Prog Neuropsychopharmacol Biol Psychiatry 64:334–340. https://doi.org/10.1016/j.pnpbp.2015.02.008

    Article  Google Scholar 

  6. Joseph L, Pramod S, Nair LS (2017) Emotion recognition in a social robot for robot-assisted therapy to autistic treatment using deep learning. In: 2017 International conference on technological advancements in power and energy (TAP Energy). IEEE, pp 1–6. https://doi.org/10.1109/TAPENERGY.2017.8397220

  7. Perner J, Wimmer H (1985) “John thinks that Mary thinks that…” attribution of second-order beliefs by 5-to 10-year-old children. J Exp Child Psychol 39(3):437–471. https://doi.org/10.1016/0022-0965(85)90051-7

    Article  Google Scholar 

  8. McKenzie K, Russell A, Golm D, Fairchild G (2022) Empathic accuracy and cognitive and affective empathy in young adults with and without autism spectrum disorder. J Autism Dev Disord 52(5):2004–2018. https://doi.org/10.1007/s10803-021-05093-7

  9. Berggren S, Fletcher-Watson S, Milenkovic N, Marschik PB, Bölte S, Jonsson U (2018) Emotion recognition training in autism spectrum disorder: a systematic review of challenges related to generalizability. Dev Neurorehabil 21(3):141–154. https://doi.org/10.1080/17518423.2017.1305004

    Article  Google Scholar 

  10. Priyadarshini I, Kumar R, Alkhayyat A, Sharma R, Yadav K, Alkwai LM, Kumar S (2023) Survivability of industrial internet of things using machine learning and smart contracts. Comput Electr Eng 107:108617. ISSN 0045-7906. https://doi.org/10.1016/j.compeleceng.2023.108617

  11. Priyadarshini I, Mohanty P, Alkhayyat A, Sharma R, Kumar S (2023) SDN and application layer DDoS attacks detection in IoT devices by attention-based Bi-LSTM-CNN. Trans Emerging Tel Tech e4758. https://doi.org/10.1002/ett.4758

  12. Sharma R, Arya R (2023) Secured mobile IOT ecosystem using enhanced multi-level intelligent trust scheme. Comput Electr Eng 108:108715. ISSN 0045-7906. https://doi.org/10.1016/j.compeleceng.2023.108715

  13. Deng H, Hu J, Sharma R, Mo M, Ren Y (2023) NVAS: a non-interactive verifiable federated learning aggregation scheme for COVID-19 based on game theory. Comput Commun. ISSN 0140-3664. https://doi.org/10.1016/j.comcom.2023.04.026

  14. Sharma A, Rani S, Shah SH, Sharma R, Yu F, Hassan MM (2023) An efficient hybrid deep learning model for denial of service detection in cyber physical systems. IEEE Trans Netw Sci Eng. https://doi.org/10.1109/TNSE.2023.3273301

  15. Gupta U, Sharma R (2023) Analysis of criminal spatial events in India using exploratory data analysis and regression. Comput Electr Eng 109(Part A):108761. ISSN 0045-7906. https://doi.org/10.1016/j.compeleceng.2023.108761

  16. Goyal B et al (2023) Detection of fake accounts on social media using multimodal data with deep learning. IEEE Trans Comput Soc Syst. https://doi.org/10.1109/TCSS.2023.3296837

  17. Sneha PM, Sharma R, Ghosh U, Alnumay WS (2023) Internet of things and long-range antenna’s; challenges, solutions and comparison in next generation systems. Microprocess Microsyst 104934. ISSN 0141-9331. https://doi.org/10.1016/j.micpro.2023.104934

  18. Vohnout R et al (2023) Living lab long-term sustainability in hybrid access positive energy districts—a prosumager smart fog computing perspective. IEEE Internet Things J. https://doi.org/10.1109/JIOT.2023.3280594

  19. Yu X, Li W, Zhou X et al (2023) Deep learning personalized recommendation-based construction method of hybrid blockchain model. Sci Rep 13:17915. https://doi.org/10.1038/s41598-023-39564-x

    Article  Google Scholar 

  20. Yadav S et al (2023) Video object detection from compressed formats for modern lightweight consumer electronics. IEEE Trans Consum Electron. https://doi.org/10.1109/TCE.2023.3325480

  21. Singh A, Dewan S (2020) AutisMitr: emotion recognition assistive tool for autistic children. Open Comput Sci 10(1):259–269. https://doi.org/10.1515/comp-2020-0006

    Article  Google Scholar 

  22. Drimalla H, Baskow I, Behnia B, Roepke S, Dziobek I (2021) Imitation and recognition of facial emotions in autism: a computer vision approach. Mol Autism 12:1–15. https://doi.org/10.1186/s13229-021-00430-0

    Article  Google Scholar 

  23. Silva V, Soares F, Esteves JS, Santos CP, Pereira AP (2021) Fostering emotion recognition in children with autism spectrum disorder. Multimodal Technol Interact 5(10):57. https://doi.org/10.3390/mti5100057

    Article  Google Scholar 

  24. Dawood A, Turner S, Perepa P (2019) Natural-Spontaneous affective-cognitive dataset for adult students with and without Asperger syndrome. IEEE Access 7:77990–77999. https://doi.org/10.1109/ACCESS.2019.2921914

    Article  Google Scholar 

  25. Zhou X (ed) (2015) Proceedings of the 23rd ACM international conference on multimedia. Association for Computing Machinery.

    Google Scholar 

  26. Xu X, Jiang X, Ma C, Du P, Li X, Lv S, Li L (2020) A deep learning system to screen novel coronavirus disease 2019 pneumonia. Engineering 6:1122–1129. https://doi.org/10.1016/j.eng.2020.04.010

    Article  Google Scholar 

  27. Cook J, Ramadas V (2020) When to consult precision-recall curves. Stand Genomic Sci 20(1):131–148. https://doi.org/10.1177/1536867X20909693

    Article  Google Scholar 

  28. Sasank TS et al (2021) J Phys: Conf Ser 1879:032124

    Google Scholar 

  29. Shahab S, Agarwal P, Mufti T, Obaid AJ (2022) SIoT (Social Internet of Things): a review. In Fong S, Dey N, Joshi A (eds) ICT analysis and applications. Lecture notes in networks and systems, vol 314. Springer, Singapore. https://doi.org/10.1007/978-981-16-5655-2_28

Download references

Author information

Authors and Affiliations

  1. Computer Science Department, College of Science for Women, University of Babylon, Babylon, Iraq

    Mays Ali Shaker & Amina Atiya Dawood

Authors
  1. Mays Ali Shaker
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amina Atiya Dawood
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mays Ali Shaker .

Editor information

Editors and Affiliations

  1. Department of Electronics and Communication Engineering, ABES Engineering College, Ghaziabad, Uttar Pradesh, India

    Devendra Kumar Sharma

  2. National Taipei University of Business, Taoyuan, Taiwan

    Sheng-Lung Peng

  3. Department of Electronics and Communication Engineering, ABES Engineering College, Ghaziabad, Uttar Pradesh, India

    Rohit Sharma

  4. Department of Embedded Systems Engineering, Incheon National University, Incheon, Korea (Republic of)

    Gwanggil Jeon

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Shaker, M.A., Dawood, A.A. (2024). OpenFace Tracker and GoogleNet: To Track and Detect Emotional States for People with Asperger Syndrome. In: Sharma, D.K., Peng, SL., Sharma, R., Jeon, G. (eds) Micro-Electronics and Telecommunication Engineering. ICMETE 2023. Lecture Notes in Networks and Systems, vol 894. Springer, Singapore. https://doi.org/10.1007/978-981-99-9562-2_4

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-9562-2_4

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-9561-5

  • Online ISBN: 978-981-99-9562-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation