Skip to main content
SpringerLink
Log in

Using Socially Assistive Robots in Speech-Language Therapy for Children with Language Impairments

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

Socially assistive robots (SARs) have been shown to be promising therapy tools for children with primary or co-occurring language impairments (e.g., developmental language disorder and autism spectrum disorder), but only a few studies have explored the use of SARs in speech-language therapies. This work sought to address the following research goals: (1) explore the potential of using SAR for training linguistic skills of children with language impairments, targeting specific aspects of language and measuring their linguistic improvements in speech-language therapy; (2) explore children’s facial cues during SAR-supoported speech-language therapy; and (3) collect therapist perspectives on using SARs in speech-language therapy after having experienced it. Toward these goals, we conducted an 8-week between-subjects study involving 20 children with language impairments and 6 speech-language therapists who conducted the SAR-supported therapy. Children were randomly assigned to either a physical SAR or a virtual SAR condition; both provided the same language impairment therapy. We collected linguistic activity scores, video recordings, therapist questionnaires, and group interview data. The study results show that: (i) the study participants’ overall linguistic skills improved significantly in both conditions; (ii) participants who were engaged with the physical SAR (measured based on gaze direction and head position) were more likely to demonstrate linguistic skill improvements and had a significantly higher numbers of speech occurrences in the child-robot-therapist triads with the physical SAR; (iii) therapists reported skepticism about SAR efficacy in this context but believed that SAR could be beneficial for keeping children engaged, motivated, and positive during speech-language therapy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

Data and materials are available upon reasonable request to the authors. With these data, any researcher will be able to run any other type of statistical analysis.

Notes

  1. MLU is a measure of children’s linguistic productivity. It is computed by taking 100 child utterances and dividing the number of morphemes, i.e., the smallest meaningful lexical item, by the total number of utterances. A higher MLU indicates a higher degree of linguistic ability.

  2. https://github.com/pyannote/pyannote-audio.

References

  1. Abbasi NI, Spitale M, Anderson J, Ford T, Jones PB, Gunes H (2022) Can robots help in the evaluation of mental wellbeing in children? An empirical study. In: 2022 31st IEEE international conference on robot and human interactive communication (RO-MAN), pp 1459–1466. https://doi.org/10.1109/RO-MAN53752.2022.9900843

  2. Amanatiadis A, Kaburlasos VG, Dardani C, Chatzichristofis SA (2017) Interactive social robots in special education. In: 2017 IEEE 7th international conference on consumer electronics-Berlin (ICCE-Berlin). IEEE, pp 126–129

  3. Anzalone SM, Xavier J, Boucenna S, Billeci L, Narzisi A, Muratori F, Cohen D, Chetouani M (2019) Quantifying patterns of joint attention during human-robot interactions: an application for autism spectrum disorder assessment. Pattern Recognit Lett 118:42–50

    Article  Google Scholar 

  4. Arosio F, Branchini C, Barbieri L, Guasti MT (2014) Failure to produce direct object clitic pronouns as a clinical marker of SLI in school-aged Italian speaking children. Clin Linguist Phon 28(9):639–663

    Article  Google Scholar 

  5. Baltrušaitis T, Robinson P, Morency L-P (2016) Openface: an open source facial behavior analysis toolkit. In: 2016 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 1–10

  6. Bodala IP, Churamani N, Gunes H (2021) Teleoperated robot coaching for mindfulness training: a longitudinal study. In: 2021 30th IEEE international conference on robot and human interactive communication (RO-MAN). IEEE, pp 939–944

  7. Braun V, Clarke V (2012) Thematic analysis. American Psychological Association, Washington

    Book  Google Scholar 

  8. Bredin H (2017) pyannote. metrics: A toolkit for reproducible evaluation, diagnostic, and error analysis of speaker diarization systems. In: INTERSPEECH, pp 3587–3591

  9. Brooke J (1996) Sus: a “quick and dirty” usability. In: Usability evaluation in industry, p 189

  10. Cabibihan J-J, Javed H, Ang M, Aljunied SM (2013) Why robots? A survey on the roles and benefits of social robots in the therapy of children with autism. Int J Soc Robot 5(4):593–618

    Article  Google Scholar 

  11. Caldwell Marín EG, Morales CA, Solis Sanchez E, Cazorla M, Cañas Plaza JM (2021) Designing a cyber-physical robotic platform to assist speech-language pathologists. Assistive Technology (just-accepted)

  12. Cassell J, Bickmore T, Billinghurst M, Campbell L, Chang K, Vilhjálmsson H, Yan H (1999) Embodiment in conversational interfaces: Rea. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 520–527

  13. Catania F, Spitale M, Garzotto F (2022) Conversational agents in therapeutic interventions for neurodevelopmental disorders: a survey. ACM Comput Surv. ISSN 0360-0300. https://doi.org/10.1145/3564269

  14. Chaminade T, Da Fonseca D, Rosset D, Lutcher E, Cheng G, Deruelle C (2012) FMRI study of young adults with autism interacting with a humanoid robot. In: 2012 IEEE RO-MAN: The 21st IEEE international symposium on robot and human interactive communication. IEEE, pp 380–385

  15. Charron N, Lewis L, Craig M (2017) A robotic therapy case study: developing joint attention skills with a student on the autism spectrum. J Educ Technol Syst 46(1):137–148

    Article  Google Scholar 

  16. Cifuentes CA, Pinto MJ, Céspedes N, Múnera M (2020) Social robots in therapy and care. Curr Robot Rep 1:1–16

    Article  Google Scholar 

  17. Clabaugh C, Jain S, Thiagarajan B, Shi Z, Mathur L, Mahajan K, Ragusa G, Matarić MJ (2018) Attentiveness of children with diverse needs to a socially assistive robot in the home. In: 2018 International symposium on experimental robotics. University of Southern California Buenos Aires

  18. Clabaugh C, Mahajan K, Jain S, Pakkar R, Becerra D, Shi Z, Deng E, Lee R, Ragusa G, Matarić M (2019) Long-term personalization of an in-home socially assistive robot for children with autism spectrum disorders. Front Robot AI 6:110

    Article  Google Scholar 

  19. Colton MB, Ricks DJ, Goodrich MA, Dariush B, Fujimura K, Fujiki M (2009) Toward therapist-in-the-loop assistive robotics for children with autism and specific language impairment. Autism 24:25

    Google Scholar 

  20. Conti D, Di Nuovo S, Buono S, Di Nuovo A (2017) Robots in education and care of children with developmental disabilities: a study on acceptance by experienced and future professionals. Int J Soc Robot 9(1):51–62

    Article  Google Scholar 

  21. Conti D, Cattani A, Di Nuovo S, Di Nuovo A (2019) Are future psychologists willing to accept and use a humanoid robot in their practice? Italian and English students’ perspective. Front Psychol 10:2138

    Article  Google Scholar 

  22. Costa AP, Steffgen G, Lera FR, Nazarikhorram A, Ziafati P (2017) Socially assistive robots for teaching emotional abilities to children with autism spectrum disorder. In: 3rd Workshop on child–robot interaction at HRI

  23. Costescu CA, Vanderborght B, David DO (2015) Reversal learning task in children with autism spectrum disorder: a robot-based approach. J Autism Dev Disord 45(11):3715–3725

    Article  Google Scholar 

  24. Crawley MJ (2012) The R book. Wiley, Hoboken

    Book  MATH  Google Scholar 

  25. Demers L, Weiss-Lambrou R, Ska B (2002) The Quebec user evaluation of satisfaction with assistive technology (quest 2.0): an overview and recent progress. Technol Disabil 14(3):101–105

    Article  Google Scholar 

  26. Deng E, Mutlu B, Mataric MJ et al (2019) Embodiment in socially interactive robots. Found Trends Robot 7(4):251–356

    Article  Google Scholar 

  27. Dickstein-Fischer LA, Crone-Todd DE, Chapman IM, Fathima AT, Fischer GS (2018) Socially assistive robots: current status and future prospects for autism interventions. Innov Entrep Health 5:15–25

    Article  Google Scholar 

  28. Durrleman S, Delage H, Prévost P, Tuller L (2017) The comprehension of passives in autism spectrum disorder. Glossa 2(1):88

    Article  Google Scholar 

  29. Egido-García V, Estévez D, Corrales-Paredes A, Terrón-López M-J, Velasco-Quintana P-J (2020) Integration of a social robot in a pedagogical and logopedic intervention with children: a case study. Sensors 20(22):6483

    Article  Google Scholar 

  30. Estévez D, Terrón-López M-J, Velasco-Quintana PJ, Rodríguez-Jiménez R-M, Álvarez-Manzano V (2021) A case study of a robot-assisted speech therapy for children with language disorders. Sustainability 13(5):2771

    Article  Google Scholar 

  31. Eurobarometer Special (2017) Attitudes towards the impact of digitisation and automation on daily life

  32. Fachantidis N, Syriopoulou-Delli CK, Zygopoulou M (2020) The effectiveness of socially assistive robotics in children with autism spectrum disorder. Int J Dev Disabil 66(2):113–121

    Article  Google Scholar 

  33. Feil-Seifer D, Mataric MJ (2005) Defining socially assistive robotics. In: 9th International conference on rehabilitation robotics, 2005. ICORR 2005. IEEE, pp 465–468

  34. Field A (2013) Discovering statistics using IBM SPSS statistics. Sage, London

    Google Scholar 

  35. Finkelstein S, Ogan A, Vaughn C, Cassell J (2013) Alex: a virtual peer that identifies student dialect. In: Proceedings of the workshop on culturally-aware technology enhanced learning in conjuction with EC-TEL

  36. Furlong L, Morris M, Serry T, Erickson S (2018) Mobile apps for treatment of speech disorders in children: an evidence-based analysis of quality and efficacy. PLoS ONE 13(8):e0201513

    Article  Google Scholar 

  37. Georgiou N, Spanoudis G (2021) Developmental language disorder and autism: commonalities and differences on language. Brain Sci 11(5):589

    Article  Google Scholar 

  38. Guasti MT (2017) Language acquisition: The growth of grammar. MIT Press, Cambridge

    Google Scholar 

  39. Guasti MT, Palma S, Genovese E, Stagi P, Saladini G, Arosio F (2016) The production of direct object clitics in pre-school-and primary school-aged children with specific language impairments. Clin Linguist Phon 30(9):663–678

    Article  Google Scholar 

  40. Heerink M, Kröse B, Evers V, Wielinga B (2010) Assessing acceptance of assistive social agent technology by older adults: the Almere model. Int J Soc Robot 2(4):361–375

    Article  Google Scholar 

  41. Hoffman HJ, Black LI, Vahratian A (2015) Communication disorders and use of intervention services among children aged 3–17 years: United states, 2012. NCHS Data Brief 205:7

    Google Scholar 

  42. Isbell R, Sobol J, Lindauer L, Lowrance A (2004) The effects of storytelling and story reading on the oral language complexity and story comprehension of young children. Early Child Educ J 32(3):157–163

    Article  Google Scholar 

  43. Jain S, Thiagarajan B, Shi Z, Clabaugh C, Matarić MJ (2020) Modeling engagement in long-term, in-home socially assistive robot interventions for children with autism spectrum disorders. Sci Robot 5(39):eaaz3791

    Article  Google Scholar 

  44. Kidd CD, Breazeal C (2008) Robots at home: understanding long-term human–robot interaction. In: 2008 IEEE/RSJ international conference on intelligent robots and systems. IEEE, pp 3230–3235

  45. Krueger RA (2014) Focus groups: a practical guide for applied research. Sage Publications, London

    Google Scholar 

  46. Lee H, Hyun E (2015) The intelligent robot contents for children with speech-language disorder. J Educ Technol Soc 18(3):100–113

    Google Scholar 

  47. Leonard LB, Wong AM-Y, Deevy P, Stokes SF, Fletcher P (2006) The production of passives by children with specific language impairment: acquiring English or Cantonese. Appl Psycholinguist 27(2):267–299

    Article  Google Scholar 

  48. Looije R, Neerincx MA, Peters JK, Henkemans OAB (2016) Integrating robot support functions into varied activities at returning hospital visits. Int J Soc Robot 8(4):483–497

    Article  Google Scholar 

  49. Loucas T, Charman T, Pickles A, Simonoff E, Chandler S, Meldrum D, Baird G (2008) Autistic symptomatology and language ability in autism spectrum disorder and specific language impairment. J Child Psychol Psychiatry 49(11):1184–1192

    Article  Google Scholar 

  50. Martinez-Martin E, Escalona F, Cazorla M (2020) Socially assistive robots for older adults and people with autism: an overview. Electronics 9(2):367

    Article  Google Scholar 

  51. Matarić Maja J, Brian Scassellati (2016) Socially assistive robotics. In: Siciliano B, Khatib O, Kroger T (eds) Springer handbook of robotics. Springer, Berlin, pp 1973–1994

    Chapter  Google Scholar 

  52. Nasir J, Bruno B, Chetouani M, Dillenbourg P (2021) What if social robots look for productive engagement? Int J Soc Robot 14:1–17

    Google Scholar 

  53. Nederhof AJ (1985) Methods of coping with social desirability bias: a review. Eur J Soc Psychol 15(3):263–280

    Article  Google Scholar 

  54. Pacelli C, Kinkini T, Spitale M, Beccaluva E, Garzotto F et al (2022) “How would you communicate with a robot?”: People with neourodevelopmental disorder’s perspective. In: 2022 17th ACM/IEEE international Cconference on human–robot interaction (HRI). IEEE, pp 968–972

  55. Prévost P, Tuller L, Zebib R, Barthez MA, Malvy J, Bonnet-Brilhault F (2018) Pragmatic versus structural difficulties in the production of pronominal clitics in French-speaking children with autism spectrum disorder. Autism Dev Lang Impair. https://doi.org/10.1177/2396941518799643

    Article  Google Scholar 

  56. Qtrobot: Humanoid social robot for research and teaching. URL http://luxai.com/qtrobot-for-research/

  57. Riches NG, Loucas T, Baird G, Charman T, Simonoff E (2010) Sentence repetition in adolescents with specific language impairments and autism: an investigation of complex syntax. Int J Lang Commun Disord 45(1):47–60

    Article  Google Scholar 

  58. Robles-Bykbaev V, Guamán-Heredia M, Robles-Bykbaev Y, Lojano-Redrován J, Pesántez-Avilés F, Quisi-Peralta D, López-Nores M, Pazos-Arias J (2017) Onto-speltra: a robotic assistant based on ontologies and agglomerative clustering to support speech-language therapy for children with disabilities. In: Colombian conference on computing. Springer, pp 343–357

  59. Robles-Bykbaev V, Ochoa-Guaraca M, Carpio-Moreta M, Pulla-Sánchez D, Serpa-Andrade L, López-Nores M, García-Duque J (2016) Robotic assistant for support in speech therapy for children with cerebral palsy. In: 2016 IEEE international autumn meeting on power, electronics and computing (ROPEC). IEEE, pp 1–6

  60. Robles-Bykbaev VE, Lopez-Nores M, Pazos-Arias JJ, Garcia-Duque J (2015) Ramses: a robotic assistant and a mobile support environment for speech and language therapy. In: Fifth international conference on the innovative computing technology (INTECH 2015). IEEE, pp 1–4

  61. Rogers EM, Singhal A, Quinlan MM (2014) Diffusion of innovations. Routledge, New York

    Google Scholar 

  62. Scassellati B, Admoni H, Matarić M (2012) Robots for use in autism research. Annu Rev Biomed Eng 14:275–294

    Article  Google Scholar 

  63. Scassellati B, Boccanfuso L, Huang C-M, Mademtzi M, Qin M, Salomons N, Ventola P, Shic F (2018) Improving social skills in children with ASD using a long-term, in-home social robot. Sci Robot 3(21):eaat7544

    Article  Google Scholar 

  64. Shi Z, Groechel TR, Jain S, Chima K, Rudovic O, Matarić MJ (2021) Toward personalized affect-aware socially assistive robot tutors in long-term interventions for children with autism. arXiv preprint arXiv:2101.10580

  65. Shimaya J, Yoshikawa Y, Matsumoto Y, Kumazaki H, Ishiguro H, Mimura M, Miyao M (2016) Advantages of indirect conversation via a desktop humanoid robot: Case study on daily life guidance for adolescents with autism spectrum disorders. In: 2016 25th IEEE international symposium on robot and human interactive communication (RO-MAN). IEEE, pp 831–836

  66. Short ES, Matarić MJ (2015) Understanding interaction dynamics in socially assistive robotics with children with ASD. In: International meeting for autism research (IMFAR), Salt Lake City, Utah

  67. Short ES, Deng EC, Feil-Seifer DJ, Mataric MJ (2017) Understanding agency in interactions between children with autism and socially assistive robots

  68. Sidner CL, Kidd CD, Lee C, Lesh N (2004) Where to look: a study of human–robot engagement. In: Proceedings of the 9th international conference on Intelligent user interfaces, pp 78–84

  69. Silleresi S, Tuller L, Delage H, Durrelaman S, Bonnet-Brilhault F, Malvy J, Prévosti P (2018) Sentence repetition and language impairment in French-speaking children with ASD. In: On the acquisition of the syntax of romance, pp 235–258

  70. Silvera-Tawil D, Roberts-Yates C (2018) Socially-assistive robots to enhance learning for secondary students with intellectual disabilities and autism. In: 2018 27th IEEE international symposium on robot and human interactive communication (RO-MAN). IEEE, pp 838–843

  71. Simut RE, Vanderfaeillie J, Peca A, Van de Perre G, Vanderborght B (2016) Children with autism spectrum disorders make a fruit salad with probo, the social robot: an interaction study. J Autism Dev Disord 46(1):113–126

    Article  Google Scholar 

  72. Siqueira H, Sutherland A, Barros P, Kerzel M, Magg S, Wermter S (2018) Disambiguating affective stimulus associations for robot perception and dialogue. In: 2018 IEEE-RAS 18th international conference on humanoid robots (humanoids). IEEE, pp 1–9

  73. So W-C, Cheng C-H, Lam W-Y, Huang Y, Ng K-C, Tung H-C, Wong W (2020) A robot-based play-drama intervention may improve the joint attention and functional play behaviors of Chinese-speaking preschoolers with autism spectrum disorder: a pilot study. J Autism Dev Disord 50(2):467–481

    Article  Google Scholar 

  74. Spitale M, Silleresi S, Cosentino G, Panzeri F, Garzotto F (2020) Whom would you like to talk with? Exploring conversational agents for children’s linguistic assessment. In: Proceedings of the interaction design and children conference, pp 262–272

  75. Spitale M, Birmingham C, Swan RM, Matarić MJ (2021) Composing harmoni: an open-source tool for human and robot modular open interaction. In: 2021 IEEE international conference on robotics and automation (ICRA). IEEE, pp 3322–3329

  76. Spitale M, Silleresi S, Leonardi G, Arosio F, Giustolisi B, Guasti MT, Garzotto F (2021) Design patterns of technology-based therapeutic activities for children with language impairments: a psycholinguistic-driven approach. In: Extended abstracts of the 2021 CHI conference on human factors in computing systems, pp 1–7

  77. Spitale M, Okamoto S, Gupta M, Xi H, Matarić MJ (2022) Socially assistive robots as storytellers that elicit empathy. ACM Trans Hum Robot Interact 11:1–29

    Article  Google Scholar 

  78. Taheri A, Meghdari A, Alemi M, Pouretemad H (2019) Teaching music to children with autism: a social robotics challenge. Sci Iran 26(Special Issue on: Socio-Cognitive Engineering):40–58

  79. Tapus A, Maja M, Scassellatti B (2007) The grand challenges in socially assistive robotics

  80. Williams D, Botting N, Boucher J (2008) Language in autism and specific language impairment: Where are the links? Psychol Bull 134(6):944

    Article  Google Scholar 

  81. Wittke K, Mastergeorge AM, Ozonoff S, Rogers SJ, Naigles LR (2017) Grammatical language impairment in autism spectrum disorder: exploring language phenotypes beyond standardized testing. Front Psychol 8:532

    Article  Google Scholar 

  82. World Health Organization et al (2018) International classification of diseases, 11th revision (ICD-11). Number 2018. Retrieved from http://www.who.int/classifications/icd/en

  83. Zampini L, Zanchi P, Suttora C, Spinelli M, Fasolo M, Salerni N (2017) Assessing sequential reasoning skills in typically developing children. BPA Appl Psychol Bull (Boll Psicol Appl) 65(279):44–50

    Google Scholar 

  84. Zhao R, Sinha T, Black AW, Cassell J (2016) Socially-aware virtual agents: automatically assessing dyadic rapport from temporal patterns of behavior. In: International conference on intelligent virtual agents. Springer, pp 218–233

Download references

Acknowledgements

This research was supported in part by EIT Digital and IBM Italy (supporting Micol Spitale), in part by the Politecnico di Milano (supporting Silvia Silleresi, and Franca Garzotto), and in part by the University of Southern California (supporting Maja Matarić). The authors thank the speech-language therapists involved in the study for their help with the recruitment process, empirical study design, and for running the study with a great enthusiasm. The entire research team thanks the study participants.

Author information

Authors and Affiliations

  1. Department of Electronics, Information, and Bioengineering, Politecnico di Milano, Via Golgi 39, Milan, 20133, Italy

    Micol Spitale, Silvia Silleresi & Franca Garzotto

  2. Department of Psychology, University of Milano Bicocca, Piazza Dell’Ateneo Nuovo 1, 20126, Milan, Italy

    Silvia Silleresi

  3. Computer Science Department, University of Southern California, 3710 McClintock Ave, Los Angeles, California, 90089, USA

    Maja J. Matarić

Authors
  1. Micol Spitale
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Silvia Silleresi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Franca Garzotto
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Maja J. Matarić
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Micol Spitale.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Spitale, M., Silleresi, S., Garzotto, F. et al. Using Socially Assistive Robots in Speech-Language Therapy for Children with Language Impairments. Int J of Soc Robotics 15, 1525–1542 (2023). https://doi.org/10.1007/s12369-023-01028-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-023-01028-7

Keywords

Search

Navigation