Skip to main content
SpringerLink
Log in

Accessibility Inspections of Mobile Applications by Professionals with Different Expertise Levels: An Empirical Study Comparing with User Evaluations

  • Conference paper
  • First Online:
Human-Computer Interaction – INTERACT 2023 (INTERACT 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14142))

Included in the following conference series:

  • 725 Accesses

Abstract

Providing accessibility in mobile applications is essential for the appropriate use by people with disabilities. Different evaluation methods yield different results, and professionals and researchers must be aware of the types of results obtained by user evaluations and inspections performed by professionals with different expertise levels. This study aimed to compare the results from manual inspection of mobile apps performed by two groups of professionals with different expertise levels and to compare them with user evaluations conducted by users with visual disabilities. The Saraiva and Receita Federal applications usability evaluations carried out with nine visually impaired users encountered 189 problems divided into 39 violations of accessibility guidelines. Then, the applications were inspected by two groups of professionals: 17 specialists in different areas of software development (full-stack developers, testers, and front-end developers) and ten specialists in Human-Computer Interaction (HCI) with previous experience in accessibility. The results indicated a difference between accessibility assessment methods. In accessibility inspections, there was a difference between software development specialists (DEV group) and HCI specialists (HCI group). The results indicated a difference in the number of violations the DEV group encounters compared to the HCI group. Inspections by HCI experts and testing with users with disabilities encountered a greater diversity of problem types. HCI professionals also showed a broader repertoire of accessibility inspection approaches. The results allow for an initial understanding of the extent to which evaluations by developers without HCI background can cover compared to inspections by HCI specialists and users with disabilities.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://m.apkpure.com/br/.

  2. 2.

    https://meet.google.com/.

  3. 3.

    https://www.teamviewer.com/.

References

  1. Abascal, J., Arrue, M., Valencia, X.: Tools for web accessibility evaluation. In: Web Accessibility: A Foundation for Research, pp. 479–503 (2019)

    Google Scholar 

  2. Associação Brasileira de Normas Técnicas (ABNT): ABNT NBR 17060 - Acessibilidade em aplicativos de dispositivos móveis - Requisitos (Accessibility in Mobile Apps - requirements) (2022). shorturl.at/lvwJ2

    Google Scholar 

  3. BBC: HTML accessibility standards v2.0 (2014). http://www.bbc.co.uk/guidelines/futuremedia/accessibility/html/. Accessed 15 Feb 2021

  4. Brajnik, G.: A comparative test of web accessibility evaluation methods. In: Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 113–120 (2008)

    Google Scholar 

  5. Brajnik, G., Vigo, M., Yesilada, Y., Harper, S.: Group vs individual web accessibility evaluations: effects with novice evaluators. Interact. Comput. 28(6), 843–861 (2016)

    Article  Google Scholar 

  6. Brajnik, G., Yesilada, Y., Harper, S.: Testability and validity of WCAG 2.0: the expertise effect. In: Proceedings of the 12th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 43–50 (2010)

    Google Scholar 

  7. Brajnik, G., Yesilada, Y., Harper, S.: The expertise effect on web accessibility evaluation methods. Hum.-Comput. Interact. 26(3), 246–283 (2011)

    Google Scholar 

  8. British Broadcasting Corporation - BBC: Mobile accessibility standards and guidelines v1.0 (2014). http://shorturl.at/wCPX3. Accessed 15 Feb 2021

  9. Carvalho, M.C.N., Dias, F.S., Reis, A.G.S., Freire, A.P.: Accessibility and usability problems encountered on websites and applications in mobile devices by blind and normal-vision users. In: Proceedings of the 33rd Annual ACM Symposium on Applied Computing, SAC 2018, pp. 2022–2029. Association for Computing Machinery, New York (2018)

    Google Scholar 

  10. Dias, F.S.: Análise de causas técnicasde problemas de acessibilidade encontrados em aplicativos nativos para dispositivos móveis. UFLA, Lavras (2018)

    Google Scholar 

  11. Freire, A.P.: Disabled people and the Web: User-based measurement of accessibility. Ph.D. thesis, University of York (2012)

    Google Scholar 

  12. Governo Brasileiro: Lei brasileira de inclusão da pessoa com deficiência (2015). https://www.planalto.gov.br/ccivil_03/_ato2015-2018/2015/lei/l13146.htm. Accessed 31 Jan 2023

  13. ISO - International Organization for Standardization: ISO 9241-11. usability: Definitions and concepts (2018). https://www.iso.org/obp/ui/iso:std:iso:9241:-11:ed-2:v1:en. Accessed 31 Jan 2023

  14. Jaeger, P.T.: Assessing section 508 compliance on federal e-government web sites: a multi-method, user-centered evaluation of accessibility for persons with disabilities. Gov. Inf. Q. 23(2), 169–190 (2006)

    Article  Google Scholar 

  15. Mankoff, J., Fait, H., Tran, T.: Is your web page accessible? A comparative study of methods for assessing web page accessibility for the blind. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 41–50 (2005)

    Google Scholar 

  16. Power, C., Freire, A., Petrie, H., Swallow, D.: Guidelines are only half of the story: accessibility problems encountered by blind users on the web. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 433–442 (2012)

    Google Scholar 

  17. Power, C., Petrie, H.: Working with participants. In: Web Accessibility: A Foundation for Research, pp. 153–168 (2019)

    Google Scholar 

  18. Rømen, D., Svanæs, D.: Validating WCAG versions 1.0 and 2.0 through usability testing with disabled users. Univers. Access Inf. Soc. 11(4), 375–385 (2012)

    Google Scholar 

  19. Statista: Market share of mobile operating systems in Brazil from January 2019 to August 2022 (2022). https://www.statista.com/statistics/262167/market-share-held-by-mobile-operating-systems-in-brazil/. Accessed 20 May 2023

  20. Statista: Mobile OS share in North America 2018–2023 (2022). https://www.statista.com/statistics/1045192/share-of-mobile-operating-systems-in-north-america-by-month/. Accessed 20 May 2023

  21. Vollenwyder, B., Petralito, S., Iten, G.H., Brühlmann, F., Opwis, K., Mekler, E.D.: How compliance with web accessibility standards shapes the experiences of users with and without disabilities. Int. J. Hum.-Comput. Stud. 170, 102956 (2023)

    Google Scholar 

  22. W3C: Web content accessibility guidelines 1.0, web accessibility initiative (1999). https://www.w3.org/TR/WCAG10/. Accessed 31 Jan 2023

  23. W3C: W3c - world wide web consortium. web content accessibility guidelines (WCAG) 2.0 (2008). https://www.w3.org/TR/WCAG20/. Accessed 31 Jan 2023

  24. W3C: W3c. web content accessibility guidelines (WCAG) 2.1 (2018). https://www.w3.org/TR/WCAG21/. Accessed 31 Jan 2023

  25. W3C: W3c. web content accessibility guidelines (WCAG) 2.2 (2020). https://www.w3.org/TR/WCAG22/. Accessed 31 Jan 2023

  26. W3C: W3c. web content accessibility guidelines (WCAG) 3.0 (2021). https://www.w3.org/TR/wcag-3.0/. Accessed 31 Jan 2023

Download references

Acknowledgements

We would like to thank all the participants who took part in this study for their valuable contribution. We also thank FAPEMIG, CNPq and FAPESP.

Author information

Authors and Affiliations

  1. Universidade Federal de Lavras, Lavras, MG, 37200-900, Brazil

    Delvani Antônio Mateus, Maurício Ronny de Almeida Souza & André Pimenta Freire

  2. Universidade Federal do Estado do Rio de Janeiro, Rio de Janeiro, RJ, 22290-250, Brazil

    Simone Bacellar Leal Ferreira

Authors
  1. Delvani Antônio Mateus
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simone Bacellar Leal Ferreira
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maurício Ronny de Almeida Souza
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. André Pimenta Freire
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to André Pimenta Freire .

Editor information

Editors and Affiliations

  1. University of West London, London, UK

    José Abdelnour Nocera

  2. Reykjavik University, Reykjavik, Iceland

    Marta Kristín Lárusdóttir

  3. University of York, York, UK

    Helen Petrie

  4. University of Bari Aldo Moro, Bari, Italy

    Antonio Piccinno

  5. Université Côte d’Azur, Sophia Antipolis Cedex, France

    Marco Winckler

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mateus, D.A., Ferreira, S.B.L., de Almeida Souza, M.R., Freire, A.P. (2023). Accessibility Inspections of Mobile Applications by Professionals with Different Expertise Levels: An Empirical Study Comparing with User Evaluations. In: Abdelnour Nocera, J., Kristín Lárusdóttir, M., Petrie, H., Piccinno, A., Winckler, M. (eds) Human-Computer Interaction – INTERACT 2023. INTERACT 2023. Lecture Notes in Computer Science, vol 14142. Springer, Cham. https://doi.org/10.1007/978-3-031-42280-5_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-42280-5_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-42279-9

  • Online ISBN: 978-3-031-42280-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation