Using Online Artificial Vision Services to Assist the Blind - an Assessment of Microsoft Cognitive Services and Google Cloud Vision

  • Arsénio Reis
  • Dennis Paulino
  • Vitor Filipe
  • João Barroso
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 746)


The visually impaired must face several well-known difficulties on their daily life. The use of technology in assistive systems can greatly improve their lives by helping with navigation and orientation, for which several approaches and technologies have been proposed. Lately, it has been introduced powerful online image processing services, based on machine learning and deep learning, promising truly cognitive assessment capacities. Google and Microsoft are two of these main players. In this work we built a device to be used by the blind in order to test the usage of the Google and Microsoft services to assist the blind. The online services were tested by researchers in a laboratory environment and by blind users on a large meeting room, familiar to them. This work reports on our findings regarding the online services effectiveness, the user interface and system latency.


Cognitive services Visually impaired  Human-computer interaction Assistive systems 



This work was supported by Project “NanoSTIMA: Macro-to-Nano Human Sensing: Towards Integrated Multimodal Health Monitoring and Analytics/NORTE-01-0145-FEDER-000016” financed by the North Portugal Regional Operational Programme (NORTE 2020), under the PORTUGAL 2020 Partnership Agreement, and through the European Regional Development Fund (ERDF).


  1. 1.
    Gretchen, A.S., Richard, A.W., Seth, R.F., Holly, P., Jost, B.J., Jill, K., Janet, L., Kovin, N., Konrad, P., Serge, R., Hugh, T., Rupert, R.A.: Global prevalence of vision impairment and blindness: magnitude and temporal trends, 1990–2010. Ophthalmology 120(12), 2377–2384 (2013). ISSN 0161-6420CrossRefGoogle Scholar
  2. 2.
    Ramrattan, R.S., Wolfs, R.C., Panda-Jonas, S., et al.: Prevalence and causes of visual field loss in the elderly and associations with impairment in daily functioning: the Rotterdam study. Arch. Ophthalmol. 119, 1788–1794 (2001)CrossRefGoogle Scholar
  3. 3.
    Desrosiers, J., Wanet-Defalque, M.C., Témisjian, K., Gresset, J., Dubois, M.F., Renaud, J., Overbury, O.: Participation in daily activities and social roles of older adults with visual impairment. Disabil. Rehabil. 31(15), 1227–1234 (2009). Scholar
  4. 4.
    Ran, L., Helal, S., Moore, S.: Drishti: an integrated indoor/outdoor blind navigation system and service. In: 2004 Proceedings of the Second IEEE Annual Conference on Pervasive Computing and Communications, PerCom 2004, pp. 23–30, March 2004Google Scholar
  5. 5.
    Makino, H., Ishii, I., Nakashizuka, M.: Development of navigation system for the blind using GPS and mobile phone combination. In: 1996 Proceedings of the 18th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Bridging Disciplines for Biomedicine, vol. 2, pp. 506–507 (1996)Google Scholar
  6. 6.
    Brock, M., Kristensson, P.O.: Supporting blind navigation using depth sensing and sonification. In: Proceedings of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication, pp. 255–258. ACM, September 2013Google Scholar
  7. 7.
    Chumkamon, S., Tuvaphanthaphiphat, P., Keeratiwintakorn, P.: A blind navigation system using r_d for indoor environments. In: 2008 5th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology, ECTI-CON, vol. 2, pp. 765–768 (2008)Google Scholar
  8. 8.
    Santos, C., Santos, V., Tavares, A., Varajão, J.: Project management success in health–the need of additional research in public health projects. Procedia Technol. 16, 1080–1085 (2014)CrossRefGoogle Scholar
  9. 9.
    Reis, A., Lains, J., Paredes, H., Filipe, V., Abrantes, C., Ferreira, F., Mendes, R., Amorim, P., Barroso, J.: Developing a system for post-stroke rehabilitation: an Exergames approach. In: Antona, M., Stephanidis, C. (eds.) Universal Access in Human-Computer Interaction. Users and Context Diversity, pp. 403–413. Springer, Cham (2016). ISBN 978-3-319-40237-6CrossRefGoogle Scholar
  10. 10.
    Paulino, D., Reis, A., Barroso, J., Paredes, H.: Mobile devices to monitor physical activity and health data. In: 12th Iberian Conference on Information Systems and Technologies (CISTI), June 2017 (2017).
  11. 11.
    Abreu, J., Rebelo, S., Paredes, H., Barroso, J., Martins, P., Reis, A., Filipe, V.: Assessment of Microsoft Kinect in the monitoring and rehabilitation of stroke patients. In: Rocha, Á., Correia, A.M., Adeli, H., Reis, L.P., Costanzo, S. (eds.) Recent Advances in Information Systems and Technologies, vol. 2, pp. 167–174. Springer, Cham (2017). ISBN 978-3-319-56537-8Google Scholar
  12. 12.
    Yánez, D.V., Marcillo, D., Fernandes, H., Barroso, J., Pereira, A.: Blind Guide: anytime, anywhere. In: Proceedings of the 7th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion, pp. 346–352. ACM, December 2016Google Scholar
  13. 13.
    Vera, D., Marcillo, D., Pereira, A.: Blind Guide: anytime, anywhere solution for guiding blind people. In: World Conference on Information Systems and Technologies, pp. 353–363. Springer, Cham, April 2017Google Scholar
  14. 14.
    Rocha, T., Fernandes, H., Reis, A., Paredes, H., Barroso, J.: Assistive platforms for the visual impaired: bridging the gap with the general public. In: Recent Advances in Information Systems and Technologies, pp. 602–608 (2017). ISBN 978-3-319-56537-8Google Scholar
  15. 15.
    Reis, A., Martins, P., Borges, J., Sousa, A., Rocha, T., Barroso, J.: Supporting accessibility in higher education information systems: a 2016 update. In: International Conference on Universal Access in Human-Computer Interaction, pp. 227–237. Springer, Cham, July 2017Google Scholar
  16. 16.
    Reis, A., Barroso, J., Gonçalves, R.: Supporting accessibility in higher education information systems. In: International Conference on Universal Access in Human-Computer Interaction, pp. 250–255. Springer, Heidelberg, July 2013CrossRefGoogle Scholar
  17. 17.
    Paulino, D., Amaral, D., Amaral, M., Reis, A., Barroso, J., Rocha, T.: Professor Piano: a music application for people with intellectual disabilities. In: Proceedings of the 7th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion, Vila Real, Portugal (2016).
  18. 18.
    Gonçalves C., Rocha T., Reis A., Barroso J.: AppVox: an application to assist people with speech impairments in their speech therapy sessions. In: Rocha Á., Correia A., Adeli H., Reis L., Costanzo S. (eds) Recent Advances in Information Systems and Technologies, WorldCIST 2017. Advances in Intelligent Systems and Computing, vol. 570. Springer, Cham (2017)Google Scholar
  19. 19.
    Someren, M.V., Barnard, Y.F., Sandberg, J.A.: The Think Aloud Method: A Practical Approach to Modelling Cognitive Processes. Academic Press, London (1994)Google Scholar
  20. 20.
    Willis, G.B.: Cognitive Interviewing: A Tool for Improving Questionnaire Design. Sage Publications, Thousand Oaks (2004)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Arsénio Reis
    • 1
  • Dennis Paulino
    • 1
  • Vitor Filipe
    • 1
  • João Barroso
    • 1
  1. 1.INESC TEC and University of Trás-os-Montes e Alto DouroVila RealPortugal

Personalised recommendations