Eyesight Sharing in Blind Grocery Shopping: Remote P2P Caregiving through Cloud Computing

  • Vladimir Kulyukin
  • Tanwir Zaman
  • Abhishek Andhavarapu
  • Aliasgar Kutiyanawala
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7383)


Product recognition continues to be a major access barrier for visually impaired (VI) and blind individuals in modern supermarkets. R&D approaches to this problem in the assistive technology (AT) literature vary from automated vision-based solutions to crowdsourcing applications where VI clients send image identification requests to web services. The former struggle with run-time failures and scalability while the latter must cope with concerns about trust, privacy, and quality of service. In this paper, we investigate a mobile cloud computing framework for remote caregiving that may help VI and blind clients with product recognition in supermarkets. This framework emphasizes remote teleassistance and assumes that clients work with dedicated caregivers (helpers). Clients tap on their smartphones’ touchscreens to send images of products they examine to the cloud where the SURF algorithm matches incoming image against its image database. Images along with the names of the top 5 matches are sent to remote sighted helpers via push notification services. A helper confirms the product’s name, if it is in the top 5 matches, or speaks or types the product’s name, if it is not. Basic quality of service is ensured through human eyesight sharing even when image matching does not work well. We implemented this framework in a module called EyeShare on two Android 2.3.3/2.3.6 smartphones. EyeShare was tested in three experiments with one blindfolded subject: one lab study and two experiments in Fresh Market, a supermarket in Logan, Utah. The results of our experiments show that the proposed framework may be used as a product identification solution in supermarkets.


Cloud Computing Speech Recognition Cloud Server Image Match Mobile Cloud Computing 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bujacz, M., Baranski, P., Moranski, M., Strumillo, P., Materka, A.: Remote Guidance for the Blind - A Proposed Teleassistance System and Navigation Trials. In: Proceedings of the Conference on Human System Interactions, pp. 888–892. IEEE, Krakow (2008)CrossRefGoogle Scholar
  2. 2.
    Peake, P., Leonard, J.: The Use of Heart-Rate as an Index of Stress in Blind Pedestrians. Ergonomics (1971)Google Scholar
  3. 3.
    Kutiyanawala, A., Kulyukin, V., Nicholson, J.: Teleassistance in Accessible Shopping for the Blind. In: Proceedings of the 2011 International Conference on Internet Computing, July 18-21, pp. 190–193. ICOMP Press, Las Vegas (2011)Google Scholar
  4. 4.
    Kulyukin, V., Kutiyanawala, A.: Accessible Shopping Systems for Blind and Visually Impaired Individuals: Design Requirements and the State of the Art. The Open Rehabilitation Journal 2, 158–168 (2010), 10.2174/1874943701003010158, ISSN: 1874-9437Google Scholar
  5. 5.
    Bay, H., Tuytelaars, T., Van Gool, L.: SURF: Speeded Up Robust Features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  6. 6.
    Sam, S., Tsai, S., Chen, D., Chandrasekhar, V., Takacs, G., Ngai-Man, C.:Google Scholar
  7. 7.
    Vedantham, R., Grzeszczuk, R., Girod, B.: Mobile Product Recognition. In: Proceedings of the International Conference on Multimedia (MM 2010), pp. 1587–1590. ACM, New York (1873),, doi:10.1145/1873951.1874293Google Scholar
  8. 8.
    Girod, B., Chandrasekhar, V., Chen, D.M., Ngai-Man, C., Grzeszczuk, R., Reznik, Y., Takacs, G., Tsai, S.S., Vedantham, R.: Mobile Visual Search. IEEE Signal Processing Magazine 28(4), 61–76 (2011), doi:10.1109/MSP.2011.940881Google Scholar
  9. 9.
    von Reischach, F., Michahelles, F., Guinard, D., Adelmann, R., Fleisch, E., Schmidt, A.: An Evaluation of Product Identification Techniques for Mobile Phones. In: Gross, T., Gulliksen, J., Kotzé, P., Oestreicher, L., Palanque, P., Prates, R.O., Winckler, M. (eds.) INTERACT 2009. LNCS, vol. 5726, pp. 804–816. Springer, Heidelberg (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Vladimir Kulyukin
    • 1
  • Tanwir Zaman
    • 1
  • Abhishek Andhavarapu
    • 1
  • Aliasgar Kutiyanawala
    • 1
  1. 1.Department of Computer ScienceUtah State UniversityLoganUSA

Personalised recommendations