Skip to main content
SpringerLink
Log in

The use of artificial intelligence for detecting the duration of autistic students' emotions in social interaction with the NAO robot: a case study

  • Original Research
  • Published:
International Journal of Information Technology Aims and scope Submit manuscript

Abstract

Within the field of education, technology is a fundamental element in responding to the diversity of students in the classroom. In this sense, robotics is the tool that can best help the demands of autistic students. Therefore, the aim of the research is to explore the application of robotics by analysing the emotions of autistic children to promote communication and social interaction. To this end, an automatic system based on neural networks has been designed to identify the emotions expressed by four autistic children throughout the process of interaction with the NAO robot where imitation, play and social interaction activities were developed. The results show that the emotions of sadness and anger are those expressed by the students throughout the activity for the greatest amount of time. Future lines of research include the possibility of designing other types of activities with the robot to analyse the influence they have on the children's moods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Salari N, Rasoulpoor S, Rasoulpoor S, Shohaimi S, Jafarpour S, Abdoli N, Khaledi-Paveh B, Mohammadi M (2022) The global prevalence of autism spectrum disorder: a comprehensive systematic review and meta-analysis. Int J Pediatr 48:1–16. https://doi.org/10.1186/s13052-022-01310-w

    Article  Google Scholar 

  2. American Psychiatric Association (2013) Diagnostic and statistical manual of mental disorders, 5th edn. American Psychiatric Publishing, Arington

    Book  Google Scholar 

  3. Premack D, Woodruff G (1978) Does the chimpanzee have a theory of mind? Behav Brain Sci. 4:515–526. https://doi.org/10.1017/S0140525X00076512

    Article  Google Scholar 

  4. Hopkins I, Gower M, Perex T, Smith D, Amthor F, Casey Wimsatt F, Biasini F (2011) Avatar assistant: improving social skills in students with an ASD through a computer-based intervention. J Autism Dev Disoders 41:1542–1555. https://doi.org/10.1007/s10803-011-1179-z

    Article  Google Scholar 

  5. Abu-Amara F, Bensefia A, Mohammad H, Tamimi H (2021) Robot and virtual reality-based intervention in autism: a comprehensive review. Int J Inf Technol 13:1879–1891. https://doi.org/10.1007/s41870-021-00740-9

    Article  Google Scholar 

  6. Khan S, Al Shafee R, Huda R, Khaliluzzaman M, Chowdhury F (2023) Predicting the level of autism and improvement rate from assessment dataset using machine learning techniques. Int J Inf Technol 15:1647–1652. https://doi.org/10.1007/s41870-023-01212-y

    Article  Google Scholar 

  7. Al-Nafjan A, Alhakbani N, Alabdulkareem A (2023) Measuring engagement in robot-assisted therapy for Autistic children. Behav Sci 13:618–634. https://doi.org/10.3390/bs13080618

    Article  PubMed  PubMed Central  Google Scholar 

  8. Robins B, Dautenhahn K (2014) Tactile interactions with Humanoid robot: Novel Play Scenario Implementations with Children with Autism. Int J Soc Robot 6:397–415. https://doi.org/10.1007/s12369-014-0228-0

    Article  Google Scholar 

  9. Syriopoulou-Deli C, GkioInta E (2022) Review of assistive technology in the training of children with autism spectrum disorders. Int J Dev Disabilities 68:73–85. https://doi.org/10.1080/20473869.2019.1706333

    Article  Google Scholar 

  10. Cabibihan J, Javed H, Ang M, Aljunied S (2013) Why robots? A survey on the roles and benefits of social robots in the therapy of children with autism. Int J Soc Robot 5:593–618. https://doi.org/10.1007/s12369-013-0202-2

    Article  Google Scholar 

  11. Fuentes-Alvarez R, Morfín-Santana A, Ibañez K, Chairez I, Salazar S (2023) Energetic optimization of an autonomous mobile socially assistive robot for autism spectrum disorder. Front Robotics AI 9:1–12. https://doi.org/10.3389/frobt.2022.1053115

    Article  Google Scholar 

  12. Wallbridge C, McGregor C, Drozdz N, Von dem Hagen E, Jones C (2023) A systematic review of familiarisation methods used in human-robot interactions for Autistic participants. Int J Soc Robot. https://doi.org/10.1007/s12369-023-01015-y

    Article  Google Scholar 

  13. Scassellati B, Admoni H, Matarić M (2012) Robots for use in Autism research. Ann Rev Biomed Eng 14:275–294. https://doi.org/10.1146/annurev-bioeng-071811-150036

    Article  CAS  Google Scholar 

  14. Podpečan V (2023) Can you dance? A study of child-robot interaction and emotional response using the NAO robot. Multimodal Technol Interact 7:1–14. https://doi.org/10.3390/mti7090085

    Article  Google Scholar 

  15. Ismail L, Verhoeven T, Dambre J, Wyffels F (2019) Leveraging robotics research for children with autism: a review. Int J Soc Robot 11:3890–4410. https://doi.org/10.1007/s12369-018-0508-1

    Article  Google Scholar 

  16. Arent K, Brown D, Kruk-Lasocka J, Lukasz T, Pasieczna A, Standen P, Szczepanowski R (2022) The use of social robots in the diagnosis of autism in preschool children. Appl Sci 12:1–16. https://doi.org/10.3390/app12178399

    Article  CAS  Google Scholar 

  17. Giullian N, Ricks D, Atherton A, Colton M, Goodrich M, Brinton B (2010) Detailed requirements for robots in autism therapy. In: Proc of the IEEE international conference on systems man and cybernetics. p 2595–2602

  18. Puglisi A, Capri T, Pignolo L, Gismondo S, Chila P, Minutoli R, Marino F, Failla C, Arnao A, Tartarisco G, Cerasa A, Pioggia G (2022) Social Humanoid robot for children with autism spectrum disorders: a review of modalities, indicators and Pitfalls. Children 9:1–14. https://doi.org/10.3390/children9070953

    Article  Google Scholar 

  19. Robins B, Otero N, Ferrari E, Dautenhahn K (2007) Eliciting requirements for a robotic toy for children with autism—results from user panels. In: Proc of the 16th IEEE international symposium on robot and human interactive communication (RO-MAN). p 101–106

  20. Michaud F, Duquette A, Nadeau I (2003) Characteristics of mobile robotic toys for children with pervasive developmental disorders. In: Proc of the IEEE international conference on systems, man and cybernetics. p 2938–2943

  21. Soori M, Arezoo B, Dastres R (2023) Artificial intelligence, machine learning and deep learning in advanced robotics, a review. Cogn Robot. 3:54–70. https://doi.org/10.1016/j.cogr.2023.04.001

    Article  Google Scholar 

  22. Kaplan A, Kressler T, Brill J, Hancock P (2023) Trust in artificial intelligence: meta-analytic findings. Hum Factors 65:337–359. https://doi.org/10.1177/00187208211013988

    Article  PubMed  Google Scholar 

  23. Verdú E, Regueras L, Gal E, Castro J, Verdú M, Kohen-Vacs D (2017) Integration of an intelligent tutoring system in a course of computer network design. Educ Tech Res Dev 65:653–677. https://doi.org/10.1007/s11423-016-9503-0

    Article  Google Scholar 

  24. Droit-Volet S, Brunot S, Niedenthal P (2010) Brief report. Perception of the duration of emotional events. Cogn Emot 18:849–858. https://doi.org/10.1080/02699930341000194

    Article  Google Scholar 

  25. Codispoti M, Mazzeti M, Bradley M (2009) Unmasking emotion: exposure duration and emotional engagement. Psychophysiology 46:731–738. https://doi.org/10.1111/j.1469-8986.2009.00804.x

    Article  PubMed  Google Scholar 

  26. Jahromi L, Meek S, Ober-Reynolds S (2012) Emotion regulation in the context of frustration in children with high functioning autism and their typical peers. J Child Psychol Psychiatr 53:1250–1258. https://doi.org/10.1111/j.1469-7610.2012.02560.x

    Article  Google Scholar 

  27. Guo Y, Garfin D, Ly A, Goldberg W (2017) Emotion coregulation in mother-child dyads: a dynamic system analysis of children with and without autism spectrum disorder. J Abnormal Child Psychol. 45:1369–1383. https://doi.org/10.1007/s10802-016-0234-9

    Article  Google Scholar 

  28. Abdul A, Mislan F, Ismail A (2015) Autistic children’s Kansei responses towards humanoid-robot as teaching mediator. Proc Comput Sci 76:488–493

    Article  Google Scholar 

  29. So W, Wong M, Lam W, Cheng C, Ku S, Lam K, Huang Y, Wong W (2019) Who is a better teacher for children with autism? Comparison of learning outcomes between robot-based and human-based interventions in gestural production and recognition. Res Dev Disabil 86:62–75. https://doi.org/10.1016/j.ridd.2019.01.002

    Article  PubMed  Google Scholar 

  30. Ramírez-Duque A, Frizera-Neto A, Bastos T (2019) Robot-assisted autism spectrum disorder diagnostic based on artificial reasoning. J Intell Robotic Syst. 96:267–281. https://doi.org/10.1007/s10846-018-00975-y

    Article  Google Scholar 

  31. Alban A, Alhaddad A, Al-Ali A, Wing-Chee S, Connor O, Ayesh M, Qidwai U, Cabibihan J (2023) Heart rate as a predictor of challenging behaviours among children with autism from wearable sensors in social robot interactions. Robotics. 12:1–13. https://doi.org/10.3390/robotics12020055

    Article  Google Scholar 

  32. Sachar S, Kumar A (2022) Deep ensemble learning for automatic medicinal leaf identification. Int J Inf Technol 14:3089–3097. https://doi.org/10.1007/s41870-022-01055-z

    Article  PubMed  PubMed Central  Google Scholar 

  33. Patil A, Subbaraman S (2022) Performance analysis of static hand gesture recognition approaches using artificial neural network, support vector machine and two stream based transfer learning approach. Int J Inf Technol 14:3781–3792. https://doi.org/10.1007/s41870-021-00831-7

    Article  Google Scholar 

  34. Jain V, Jain A, Chauan A, Kotla S, Gautam A (2021) American sign language recognition using support vector machine and convolutional neural network. Int Inf Technol 13:1193–1200. https://doi.org/10.1007/s41870-021-00617-x

    Article  Google Scholar 

  35. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. In Proceedings of the 22nd ACM International conference on Multimedia. p 675–678

  36. García R, Irarrázavala M, López I, Rieslea S, Cabezas M, Moyanoa A, Garridoc G, Valdez D, Paulac C, Rosalic A, Cukierc S, Montiel-Navac C, Rattazzic A (2022) Encuesta para Cuidadores de Personas del Espectro Autista en Chile. Acceso a Servicios de Salud y Educación, Satisfacción, Calidad de Vida y Estigma. Rev Chil Pediatr 93:351–360. https://doi.org/10.32641/andespediatr.v93i3.3994

    Article  Google Scholar 

  37. Remington A, Hanley M, O’Brien S, Riby D, Swettenham J (2019) Implications of capacity in the classroom: simplifying tasks for autistic children may not be the answer. Res Dev Disabil 85:197–204. https://doi.org/10.1016/j.ridd.2018.12.006

    Article  PubMed  Google Scholar 

  38. Trevisan D, Hoskyn M, Birmingham E (2018) Facial expression production in autism: a meta-analysis. Autism 11:1586–1601. https://doi.org/10.1002/aur.2037

    Article  Google Scholar 

  39. Press C, Richardson D, Bird G (2010) Intact imitation of emotional facial actions in autism spectrum conditions. Neuropsychologia 48:3291–3297. https://doi.org/10.1016/j.neuropsychologia.2010.07.012

    Article  PubMed  PubMed Central  Google Scholar 

  40. Sochanski M, Snyder, K, Korneder J, Wing G (2021) Therapists’ perspectives after implementing a robot into autism therapy. In 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN). p 1216–1223.

Download references

Funding

The authors declare having received the following financial support for the research, authorship and/or publication of this article: This article was supported by the Programa Estatal de I + D + i Orientado a los Retos de la Sociedad del Ministerio de Ciencia e Innovación Español. PID2020-112611RB-I00/AEI/https://doi.org/10.13039/501100011033 and Agencia Estatal de la Investigación. Title of the project "The application of virtual reality and robotics in communication and social interaction of students with autism spectrum”. Ministerio de Ciencia e Innovación, PID2020-112611RB-I00.

Author information

Authors and Affiliations

  1. Department of Development Psychology and Teaching, University of Alicante, Carretera San Vicente del Raspeig S/N, 03690, Alicante, Spain

    G. Lorenzo & A. Lorenzo-Lledó

Authors
  1. G. Lorenzo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Lorenzo-Lledó
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to G. Lorenzo.

Ethics declarations

Conflict of interest

The authors indicate that there are no conflicts of interest.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lorenzo, G., Lorenzo-Lledó, A. The use of artificial intelligence for detecting the duration of autistic students' emotions in social interaction with the NAO robot: a case study. Int. j. inf. tecnol. 16, 625–631 (2024). https://doi.org/10.1007/s41870-023-01682-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41870-023-01682-0

Keywords

Search

Navigation