A Wearable Face Recognition System Built into a Smartwatch and the Blind and Low Vision Users

  • Laurindo de Sousa Britto NetoEmail author
  • Vanessa Regina Margareth Lima Maike
  • Fernando Luiz Koch
  • Maria Cecília Calani Baranauskas
  • Anderson de Rezende Rocha
  • Siome Klein Goldenstein
Conference paper
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 241)


Assistive technologies need to be affordable, ergonomic and easy to use. In this work we argue that smartwatches could be assistive devices for the visually impaired, if they have the potential to run complex applications. Hence, in this paper we propose a face recognition system to show that it’s technically possible to develop a real-time computer vision system in a wearable device with limited hardware, since such systems generally require powerful hardware. A case study is presented using the first generation Samsung Galaxy Gear smartwatch. The system runs only on the watch’s hardware and consists in a face detection and recognition software that emits an audio feedback so that visually impaired users know who is around them. The case study includes an evaluation of the proposal with users. Results are shown and discussed validating the technological aspects of the proposal and pointing out room for improving the aspects of interaction.


Human-Computer interaction Assistive technology Computer vision Accessibility Wearable device Wearable system 



The authors wish to express their gratitude to all the volunteers who participated in the experiments in this study, and also for Samsung Research that loaned the hardware equipment. Laurindo de Sousa Britto Neto receives a Ph.D. fellowship from CNPq (grant #141254/2014-9). Vanessa Regina Margareth Lima Maike receives a Ph.D. fellowship from CAPES (grant #01-P-04554/2013). Maria Cecília Calani Baranauskas, Anderson de Rezende Rocha and Siome Klein Goldenstein receive a Productivity Research Fellowship from CNPq (grants #308618/2014-9, #304352/2012-8 and #308882/2013-0, respectively). This work is part of a project that was approved by Unicamp Institutional Review Board CAAE 31818014.0.0000.5404. This paper is an extended version of work published in [3].


  1. 1.
    Ahonen, T., Hadid, A., Pietikainen, M.: Face description with local binary patterns: application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 2037–2041 (2006)CrossRefGoogle Scholar
  2. 2.
    Astler, D., Chau, H., Hsu, K., Hua, A., Kannan, A., Lei, L., Nathanson, M., Paryavi, E., Rosen, M., Unno, H., Wang, C., Zaidi, K., Zhang, X., Tang, C.: Increased accessibility to nonverbal communication through facial and expression recognition technologies for blind/visually impaired subjects. In: 13th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 259–260 (2011)Google Scholar
  3. 3.
    Britto Neto, L.S., Maike, V.R.M.L., Koch, F.L., Baranauskas, M.C.C., Rocha, A.R., Goldenstein, S.K.: A wearable face recognition system built into smartwatch and the visually impaired user. In: 17th International Conference on Enterprise Information Systems, vol. 3, pp. 5–12. Scitepress (2015)Google Scholar
  4. 4.
    Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 711–720 (1997)CrossRefGoogle Scholar
  5. 5.
    Chen, X., Flynn, P., Bowyer, K.: PCA-based face recognition in infrared imagery: baseline and comparative studies. In: IEEE International Workshop on Analysis and Modeling of Faces and Gestures, pp. 127–134 (2003)Google Scholar
  6. 6.
    Cover, T., Hart, P.: Nearest neighbor pattern classification. IEEE Trans. Inf. Theory 13(1), 21–27 (1967)CrossRefzbMATHGoogle Scholar
  7. 7.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 886–893 (2005)Google Scholar
  8. 8.
    Freevoxtouch. the only smart watch in the world for the visually impaired.
  9. 9.
    Fusco, G., Noceti, N., Odone, F.: Combining retrieval and classification for real-time face recognition. In: 10th IEEE International Conference on Advanced Video and Signal Based Surveillance, pp. 276–281 (2012)Google Scholar
  10. 10.
    Gordon, G.: Face recognition based on depth maps and surface curvature. In: SPIE1570, Geometric methods in Computer Vision, pp. 234–247 (1991)Google Scholar
  11. 11.
    Hadid, A., Li, S.Z., Pietikäinen, M.: Learning Personal Specific Facial Dynamics for Face Recognition from Videos. In: Zhou, S., Zhao, W., Tang, X., Gong, S. (eds.) AMFG 2007. LNCS, vol. 4778, pp. 1–15. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  12. 12.
    Jain, A. K., Chandrasekaran, B.: Dimensionality and sample size considerations in pattern recognition practice. In Handbook of Statistics, pp. 835–855 (1982)Google Scholar
  13. 13.
    Kistler, D., Wightman, F.: A model of head-related transfer functions based on principal components analysis and minimum-phase reconstruction. J. Acoust. Soc. Am. 91(3), 1637–1647 (1992)CrossRefGoogle Scholar
  14. 14.
    Kramer, K., Hedin, D., Rolkosky, D.: Smartphone based face recognition tool for the blind. In: 32nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 4538–4541 (2010)Google Scholar
  15. 15.
    Krishna, S., Little, G., Black, J., Panchanathan, S.: A Wearable face recognition system for individuals with visual impairments. In: 7th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 106–113 (2005)Google Scholar
  16. 16.
    Li, B., Mian, A., L., W., and Krishna, A.: Using kinect for face recognition under varying poses, expressions, illumination and disguise. In: IEEE Workshop on Applications of Computer Vision, pp. 186–192 (2013)Google Scholar
  17. 17.
    Luxand, Inc.: Detect and Recognize Faces with Luxand FaceSDK. Accessed 27 August 2013
  18. 18.
    Manduchi, R., Coughlan, J.: Computer vision without sight. Commun. ACM 55(1), 96–104 (2012)CrossRefGoogle Scholar
  19. 19.
    Martínez, A.M.: Recognizing imprecisely localized, partially occluded, and expression variant faces from a single sample per class. IEEE Trans. Pattern Anal. Mach. Intell. 24(6), 748–762 (2002)CrossRefGoogle Scholar
  20. 20.
    NEUROtechnology.: VeriLook SDK: Face Identification for Stand-Alone or Web Applications. Accessed on 15 April 2014
  21. 21.
    Porzi, L., Messelodi, S., Modena, C. M., Ricci, E.: A smart watch-based gesture recognition system for assisting people with visual impairments. In: 3rd ACM International Workshop on Interactive Multimedia on Mobile, pp. 19–24 (2013)Google Scholar
  22. 22.
    Pun, T., Roth, P., Bologna, G., Moustakas, K., Tzovaras, D.: Image and video processing for visually handicapped people. EURASIP J. Image Video Process. 025214(5), 4:1–4:12 (2007)Google Scholar
  23. 23.
    Schwartz, W., Guo, H., Choi, J., Davis, L.: Face Identification Using Large Feature Sets. IEEE Trans. Image Process. 21(4), 2245–2255 (2012)CrossRefMathSciNetGoogle Scholar
  24. 24.
    Tan, X., Chen, S., Zhou, Z., Zhang, F.: Face recognition from a single image per person: a survey. J. Pattern Recogn. 39(9), 1725–1745 (2006). Elsevier, New YorkCrossRefzbMATHGoogle Scholar
  25. 25.
    Tanveer, M., Anam, A., Rahman, A., Ghosh, S., Yeasin, M.: FEPS: A sensory substitution system for the blind to perceive facial expressions. In: 14th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 207–208 (2012)Google Scholar
  26. 26.
    Tistarelli, M., Grosso, E.: Human face analysis: from identity to emotion and intention recognition. In: Kumar, A., Zhang, D. (eds.) ICEB 2010. LNCS, vol. 6005, pp. 76–88. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  27. 27.
    Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cogn. Neurosci. 3(1), 71–86 (1991)CrossRefGoogle Scholar
  28. 28.
    Viola, P., Jones, M.: Robust real-time face detection. Int. J. Comput. Vision 57(2), 137–154 (2004)CrossRefGoogle Scholar
  29. 29.
    Watanabe, H., Terada, T., Tsukamoto, M.: A sound-based lifelog system using ultrasound. In: 5th Augmented Human International Conference, pp. 59:1–59:2 (2014)Google Scholar
  30. 30.
    Wilder, J., Phillips, P.J., Jiang, C., Wiener, S.: Comparison of visible and infra-red imagery for face recognition. In: 2nd International Conference on Automatic Face and Gesture Recognition, pp. 182–187 (1996)Google Scholar
  31. 31.
    Wiscott, L., Fellous, J., Malsburg, C.: Face recognition by elastic bunch graph matching. IEEE Trans. Pattern Anal. Mach. Intell. 19(1), 775–779 (1997)CrossRefGoogle Scholar
  32. 32.
    World Health Organization: Visual impairment and blindness: Fact sheet N. 282 (2013).
  33. 33.
    Yan, H., Lu, J., Zhou, X., Shang, Y.: Multi-feature multi-manifold learning for single-sample face recognition. NeuroComputing 143, 134–143 (2014). ElsevierCrossRefGoogle Scholar
  34. 34.
    Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–928 (2007)CrossRefGoogle Scholar
  35. 35.
    Zhao, W., Chellappa, R., Phillips, P.J., Rosenfeld, A.: Face recognition: a literature survey. ACM Comput. Surv. 35(4), 399–458 (2003)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Laurindo de Sousa Britto Neto
    • 1
    • 2
    Email author
  • Vanessa Regina Margareth Lima Maike
    • 2
  • Fernando Luiz Koch
    • 3
  • Maria Cecília Calani Baranauskas
    • 2
  • Anderson de Rezende Rocha
    • 2
  • Siome Klein Goldenstein
    • 2
  1. 1.Department of ComputingFederal University of Piauí (UFPI)TeresinaBrazil
  2. 2.Institute of ComputingUniversity of Campinas (UNICAMP)CampinasBrazil
  3. 3.Samsung Research InstituteCampinasBrazil

Personalised recommendations