Eyesight Sharing in Blind Grocery Shopping: Remote P2P Caregiving through Cloud Computing
Product recognition continues to be a major access barrier for visually impaired (VI) and blind individuals in modern supermarkets. R&D approaches to this problem in the assistive technology (AT) literature vary from automated vision-based solutions to crowdsourcing applications where VI clients send image identification requests to web services. The former struggle with run-time failures and scalability while the latter must cope with concerns about trust, privacy, and quality of service. In this paper, we investigate a mobile cloud computing framework for remote caregiving that may help VI and blind clients with product recognition in supermarkets. This framework emphasizes remote teleassistance and assumes that clients work with dedicated caregivers (helpers). Clients tap on their smartphones’ touchscreens to send images of products they examine to the cloud where the SURF algorithm matches incoming image against its image database. Images along with the names of the top 5 matches are sent to remote sighted helpers via push notification services. A helper confirms the product’s name, if it is in the top 5 matches, or speaks or types the product’s name, if it is not. Basic quality of service is ensured through human eyesight sharing even when image matching does not work well. We implemented this framework in a module called EyeShare on two Android 2.3.3/2.3.6 smartphones. EyeShare was tested in three experiments with one blindfolded subject: one lab study and two experiments in Fresh Market, a supermarket in Logan, Utah. The results of our experiments show that the proposed framework may be used as a product identification solution in supermarkets.
KeywordsCloud Computing Speech Recognition Cloud Server Image Match Mobile Cloud Computing
Unable to display preview. Download preview PDF.
- 2.Peake, P., Leonard, J.: The Use of Heart-Rate as an Index of Stress in Blind Pedestrians. Ergonomics (1971)Google Scholar
- 3.Kutiyanawala, A., Kulyukin, V., Nicholson, J.: Teleassistance in Accessible Shopping for the Blind. In: Proceedings of the 2011 International Conference on Internet Computing, July 18-21, pp. 190–193. ICOMP Press, Las Vegas (2011)Google Scholar
- 4.Kulyukin, V., Kutiyanawala, A.: Accessible Shopping Systems for Blind and Visually Impaired Individuals: Design Requirements and the State of the Art. The Open Rehabilitation Journal 2, 158–168 (2010), 10.2174/1874943701003010158, ISSN: 1874-9437Google Scholar
- 6.Sam, S., Tsai, S., Chen, D., Chandrasekhar, V., Takacs, G., Ngai-Man, C.:Google Scholar
- 8.Girod, B., Chandrasekhar, V., Chen, D.M., Ngai-Man, C., Grzeszczuk, R., Reznik, Y., Takacs, G., Tsai, S.S., Vedantham, R.: Mobile Visual Search. IEEE Signal Processing Magazine 28(4), 61–76 (2011), doi:10.1109/MSP.2011.940881Google Scholar
- 9.von Reischach, F., Michahelles, F., Guinard, D., Adelmann, R., Fleisch, E., Schmidt, A.: An Evaluation of Product Identification Techniques for Mobile Phones. In: Gross, T., Gulliksen, J., Kotzé, P., Oestreicher, L., Palanque, P., Prates, R.O., Winckler, M. (eds.) INTERACT 2009. LNCS, vol. 5726, pp. 804–816. Springer, Heidelberg (2009)CrossRefGoogle Scholar