Skip to main content

Advertisement

SpringerLink
Log in

How are software technologies being evaluated with autistic users? A systematic mapping

  • Review Paper
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

Evaluating software technologies with users is a way to ensure the software purpose is accomplished before the system becomes available to the final users. There is a need for researches proposing and evaluating software technologies for autistic users. However, there is a difficulty to involve autistic users in interaction tasks, due to their tendency to isolation and inability to communicate their needs. We aim to explore how are software technologies being evaluated with autistic users. In this research, we carried out a systematic mapping of the literature to identify and analyze scientific publications that report software technologies evaluation with Autism Spectrum Disorder (ASD) users. The data extraction enables us to identify evaluation methods, data collection procedures, professionals involved, test environments, and ASD aspects evaluated. We noted that user tests were conducted in clinics or classrooms with health professionals, teachers, or caregivers intermediating the autistic user interaction with software technologies to promote better engagement. Observation, questionnaires, and interviews are the most common data collection methods adopted. However, we did not identify specific instruments for evaluating ASD aspects related to software technologies. With our results, we aim to encourage the proposal of evaluation protocols for diverse kinds of software technologies such as serious games or educational software. We also expect to support researchers or professionals in selecting and adopting suitable evaluation procedures to assess software technologies with autistic users. In this way, we can obtain more evidence regarding the impact of technologies in ASD behavioral and social skills development.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. http://dl.acm.org

  2. https://engineeringvillage.com/home

  3. https://www.ieee.org

  4. http://www.scopus.com

References

  1. Aziz NSA, Ahmad WFW, binti Zulkifli NJ (2015) User experience on numerical application between children with Down Syndrome and autism. International HCI and UX Conference in Indonesia. https://doi.org/https://doi.org/10.1145/2742032.2742036

  2. Aburukba R, Aloul F, Mahmoud A, Kamili K, Ajmal S (2017) AutiAid: A learning mobile application for autistic children. International Conference on e-Health Networking, Applications and Services. https://doi.org/https://doi.org/10.1109/HealthCom.2017.8210788

  3. American Psychiatric Association: Diagnostic and statistical manual of mental disorders: diagnostic and statistical manual of mental disorders, 5th edn. American Psychiatric Association, Arlington, VA (2013)

    Book  Google Scholar 

  4. An, S., Feng, X., Dai, Y., Bo, H., Wang, X., Li, M., Woo, J.Z., Liang, X., Guo, C., Liu, C.X., Wei, L.: Development and evaluation of a speech-generating AAC mobile app for minimally verbal children with autism spectrum disorder in Mainland China. Molecular Autism. 8, 1–52 (2017). https://doi.org/10.1186/s13229-017-0165-5

    Article  Google Scholar 

  5. Azahari INNA, Ahmad WFW, Jamaludin Z, Hashim AS (2016) The design of mobile social application for children with autism. International Conference on Computer and Information Sciences. https://doi.org/https://doi.org/10.1109/ICCOINS.2016.7783274

  6. Banire B, Al Thani D, Qaraqe M, Mansoor B (2017) A systematic review: Attention assessment of virtual reality-based intervention for learning in children with autism spectrum disorder. IEEE International Conference on Control System, Computing and Engineering. https://doi.org/https://doi.org/10.1109/ICCSCE.2017.8284387

  7. Bradley, R., Newbutt, N.: Autism and virtual reality head-mounted displays: a state of the art systematic review. J Enabling Tech 12, 101–113 (2018). https://doi.org/10.1108/JET-01-2018-0004

    Article  Google Scholar 

  8. Cabielles-Hernández, D., Pérez-Pérez, J.R., Paule-Ruiz, M., Fernández-Fernández, S.: Specialized intervention using tablet devices for communication deficits in children with autism spectrum disorders. IEEE Trans. Learn. Technol. 10, 182–193 (2016). https://doi.org/10.1109/TLT.2016.2559482

    Article  Google Scholar 

  9. Carlsson, E., Miniscalco, C., Gillberg, C., Johnels, J.Å.: Assessing false-belief understanding in children with autism using a computer application: a pilot study. J. Psycholinguist. Res. 47, 1085–1099 (2018). https://doi.org/10.1007/s10936-018-9579-2

    Article  Google Scholar 

  10. Carmien S. Fischer G. (2005) Tools for living and tools for learning. In: Proceedings of the HCI International Conference (HCII), pp. 1–10.

  11. Cho, K.W., Lin, F., Song, C., Xu, X., Hartley-McAndrew, M., Doody, K.R., Xu, W.: Gaze-Wasserstein: a quantitative screening approach to autism spectrum disorders. IEEE Wireless Health (2016). https://doi.org/10.1109/WH.2016.7764551

    Article  Google Scholar 

  12. Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Measur. 20, 37–46 (1960). https://doi.org/10.1177/001316446002000104

    Article  Google Scholar 

  13. Constain G, Collazos C, Moreira F (2018) Use of HCI for the development of emotional skills in the treatment of Autism Spectrum Disorder: A systematic review. Iberian Conference on Information Systems and Technologies. https://doi.org/https://doi.org/10.23919/CISTI.2018.8399209

  14. El-Seoud MSA, Karkar A, Al Ja'am JM, Karam OH (2014) A pictorial mobile-based communication application for non-verbal people with autism. International Conference on Interactive Collaborative Learning https://doi.org/https://doi.org/10.1109/ICL.2014.7017828

  15. Frutos-Pascual M, Zapirain BG, Buldian KC (2014) Adaptive cognitive rehabilitation interventions based on serious games for children with ADHD using biofeedback techniques: assessment and evaluation. International Conference on Pervasive Computing Technologies for Healthcare. https://doi.org/https://doi.org/10.4108/icst.pervasivehealth.2014.255249

  16. Granich, J., Dass, A., Busacca, M., Moore, D., Anderson, A., Venkatesh, S., Duong, T., Vellanki, P., Richdale, A., Trembath, D., Cairns, D., Marshall, W., Rodwell, T., Rayner, M., Whitehouse, A.J.O.: Randomised controlled trial of an iPad based early intervention for autism: TOBY playpad study protocol. BMC pediatrics 16, 1–12 (2016). https://doi.org/10.1186/s12887-016-0704-9

    Article  Google Scholar 

  17. Hernandes, E., Zamboni, A., Fabbri, S., Thommazo, A.D.: Using GQM and TAM to evaluate StArt-a tool that supports Systematic Review. CLEI Electron. J. 15, 1–12 (2012). https://doi.org/10.19153/cleiej.15.1.2

    Article  Google Scholar 

  18. Kanne, S.M., Carpenter, L.A., Warren, Z.: Screening in toddlers and preschoolers at risk for autism spectrum disorder: evaluating a novel mobile-health screening tool. Autism Research 11, 1038–1049 (2018). https://doi.org/10.1002/aur.1959

    Article  Google Scholar 

  19. Keay-Bright, W., Howarth, I.: Is simplicity the key to engagement for children on the autism spectrum? Pers. Ubiquit. Comput. 16, 129–141 (2012). https://doi.org/10.1007/s00779-011-0381-5

    Article  Google Scholar 

  20. Khowaja, K., Salim, S.S., Asemi, A., Ghulamani, S., Shah, A.: A systematic review of modalities in computer-based interventions (CBIs) for language comprehension and decoding skills of children with autism spectrum disorder (ASD). Univ. Access Inf. Soc. (2019). https://doi.org/10.1007/s10209-019-00646-1

    Article  Google Scholar 

  21. Kilicaslan Y, Ucar O, Ucar E, Guner ES (2008) Visualization of Turkish for autistic and mentally retarded children. International Conference on Pervasive Computing Technologies for Healthcare. https://doi.org/https://doi.org/10.1109/PCTHEALTH.2008.4571054

  22. Kitchenham B, Charters S (2007) Guidelines for performing systematic literature reviews in software engineering. Vol 2.3 EBSE Technical Report, EBSE-2007–01, Software Engineering Group, School of Computer Science and Mathematics, Keele University, Keele, UK.

  23. Kołakowska A, Landowska A, Karpienko K (2017) Gyroscope-based game revealing progress of children with autism. International Conference on Machine Learning and Soft Computing. https://doi.org/https://doi.org/10.1145/3036290.3036324

  24. Li B, Atyabi A, Kim M, Barney E, Ahn AY, Luo Y, Aubertine M, Corrigan S, St. John T, Wang Q, Mademtzi M, Best M, Shic F (2018) Social influences on executive functioning in Autism: design of a mobile gaming platform. Conference on Human Factors in Computing Systems. https://doi.org/https://doi.org/10.1145/3173574.3174017

  25. Centers for Disease Control and Prevention, U.S. Department of Health & Human Services, 23 September 2019. [Online]. Available: http://www.cdc.gov/ncbddd/autism/data.html.

  26. Mintz, J., Branch, C., March, C., Lerman, S.: Key factors mediating the use of a mobile technology tool designed to develop social and life skills in children with Autistic spectrum disorders. Comput. Educ. 58, 53–62 (2012). https://doi.org/10.1016/j.compedu.2011.07.013

    Article  Google Scholar 

  27. Mintz, J.: Additional key factors mediating the use of a mobile technology tool designed to develop social and life skills in children with Autism spectrum disorders: evaluation of the 2nd HANDS prototype. Comput. Educ. 63, 17–27 (2013). https://doi.org/10.1016/j.compedu.2012.11.006

    Article  Google Scholar 

  28. Nubia RM, Fabian GR, Wilson RA, Wilmer PB (2015) Development of a mobile application in augmented reality to improve the communication field of autistic children at a Neurorehabilitar Clinic. Workshop on Engineering Applications-International Congress on Engineering. https://doi.org/https://doi.org/10.1109/WEA.2015.7370154

  29. Shminan AS, Adzani RA, Sharif S, Lee NK (2017) AutiPECS: Mobile based learning of picture exchange communication intervention for caregivers of autistic children. International Conference on Computer and Drone Applications. https://doi.org/10.1109/ICONDA.2017.8270398

  30. Silva, M.D., Soares, A.C.B., Moura, I.C.: Application of computational tools for the development of teaching of children with autism: a systematic mapping of literature. Braz. J. Comput. Educ. 27, 351–368 (2019). https://doi.org/10.5753/RBIE.2019.27.03.351

    Article  Google Scholar 

  31. Sousa TAS, Ferreira VD, Marques ABS (2019) How do software technologies impact the daily of people with autism in Brazil: a survey. Brazilian Symposium on Information Systems. https://doi.org/10.1145/3330204.3330274

  32. Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Experimentation in software engineering. Springer Sci Bus Media (2012). https://doi.org/10.1007/978-3-642-29044-2

    Article  MATH  Google Scholar 

  33. Ying, K.T., Sah, S.B.M., Abdullah, M.H.L.: Personalised avatar on social stories and digital storytelling: fostering positive behavioural skills for children with autism spectrum disorder. Int. Conf. User Sci. Eng. (2016). https://doi.org/10.1109/IUSER.2016.7857970

    Article  Google Scholar 

Download references

Acknowledgements

The authors acknowledge the financial support granted for this research by Fundação Cearense de Apoio ao Desenvolvimento Científico e Tecnológico (FUNCAP) process BP4-00172-00228.01.00/20.

Author information

Authors and Affiliations

  1. Federal University of Ceara, Russas, CE, Brazil

    Anna Beatriz Marques & Leandro da Silva Monte

Authors
  1. Anna Beatriz Marques
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Leandro da Silva Monte
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anna Beatriz Marques.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Marques, A.B., da Silva Monte, L. How are software technologies being evaluated with autistic users? A systematic mapping. Univ Access Inf Soc 21, 587–597 (2022). https://doi.org/10.1007/s10209-021-00794-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-021-00794-3

Keywords

Search

Navigation