Face Recognition by 3D Registration for the Visually Impaired Using a RGB-D Sensor

  • Wei LiEmail author
  • Xudong Li
  • Martin Goldberg
  • Zhigang Zhu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8927)


To help visually impaired people recognize people in their daily life, a 3D face feature registration approach is proposed with a RGB-D sensor. Compared to 2D face recognition methods, 3D data based approaches are more robust to the influence of face orientations and illumination changes. Different from most 3D data based methods, we employ a one-step ICP registration approach that is much less time consuming. The error tolerance of the 3D registration approach is analyzed with various error levels in 3D measurements. The method is tested with a Kinect sensor, by analyzing both the angular and distance errors to recognition performance. A number of other potential benefits in using 3D face data are also discussed, such as RGB image rectification, multiple-view face integration, and facial expression modeling, all useful for social interactions of visually impaired people with others.


Face recognition Assistive computer vision 3D registration RGB-D sensor 


  1. 1.
    Asthana, A., Marks, T.K., Jones, M.J., Tieu, K.H., Rohith, M.: Fully automatic pose-invariant face recognition via 3D pose normalization. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 937–944. IEEE (2011)Google Scholar
  2. 2.
    Berretti, S., Del Bimbo, A., Pala, P.: 3D face recognition using isogeodesic stripes. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(12), 2162–2177 (2010)CrossRefGoogle Scholar
  3. 3.
    Besl, P.J., McKay, N.D.: Method for registration of 3-D shapes. In: Robotics-DL Tentative, pp. 586–606. International Society for Optics and Photonics (1992)Google Scholar
  4. 4.
    Cao, X., Wei, Y., Wen, F., Sun, J.: Face alignment by explicit shape regression. International Journal of Computer Vision 107(2), 177–190 (2014)CrossRefMathSciNetGoogle Scholar
  5. 5.
    Huang, D., Zhang, G., Ardabilian, M., Wang, Y., Chen, L.: 3D face recognition using distinctiveness enhanced facial representations and local feature hybrid matching. In: 2010 Fourth IEEE International Conference on Biometrics: Theory Applications and Systems (BTAS), pp. 1–7. IEEE (2010)Google Scholar
  6. 6.
    Jafri, R., Arabnia, H.R.: A survey of face recognition techniques. JIPS 5(2), 41–68 (2009)Google Scholar
  7. 7.
    Li, B.Y., Mian, A.S., Liu, W., Krishna, A.: Using kinect for face recognition under varying poses, expressions, illumination and disguise. In: 2013 IEEE Workshop on Applications of Computer Vision (WACV), pp. 186–192. IEEE (2013)Google Scholar
  8. 8.
    Liu, C.: Gabor-based kernel pca with fractional power polynomial models for face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(5), 572–581 (2004)CrossRefGoogle Scholar
  9. 9.
    Pang, Y., Yuan, Y., Li, X.: Gabor-based region covariance matrices for face recognition. IEEE Transactions on Circuits and Systems for Video Technology 18(7), 989–993 (2008)CrossRefGoogle Scholar
  10. 10.
    Passalis, G., Perakis, P., Theoharis, T., Kakadiaris, I.A.: Using facial symmetry to handle pose variations in real-world 3D face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(10), 1938–1951 (2011)CrossRefGoogle Scholar
  11. 11.
    Phillips, P.J., Flynn, P.J., Scruggs, T., Bowyer, K.W., Chang, J., Hoffman, K., Marques, J., Min, J., Worek, W.: Overview of the face recognition grand challenge. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 947–954. IEEE (2005)Google Scholar
  12. 12.
    Queirolo, C.C., Silva, L., Bellon, O.R., Pamplona Segundo, M.: 3D face recognition using simulated annealing and the surface interpenetration measure. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(2), 206–219 (2010)Google Scholar
  13. 13.
    Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Deepface: Closing the gap to human level performance in face verification. In: IEEE CVPR (2014)Google Scholar
  14. 14.
    Torr, P.H., Zisserman, A.: Mlesac: A new robust estimator with application to estimating image geometry. Computer Vision and Image Understanding 78(1), 138–156 (2000)CrossRefGoogle Scholar
  15. 15.
    Wang, Y., Liu, J., Tang, X.: Robust 3D face recognition by local shape difference boosting. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(10), 1858–1870 (2010)CrossRefGoogle Scholar
  16. 16.
    Yang, H., Wang, Y.: A lbp-based face recognition method with hamming distance constraint. In: Fourth International Conference on Image and Graphics, ICIG 2007, pp. 645–649. IEEE (2007)Google Scholar
  17. 17.
    Zhang, Z.: Microsoft kinect sensor and its effect. IEEE MultiMedia 19(2), 4–10 (2012)CrossRefGoogle Scholar
  18. 18.
    Zhu, X., Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2879–2886. IEEE (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Wei Li
    • 1
    Email author
  • Xudong Li
    • 2
  • Martin Goldberg
    • 3
  • Zhigang Zhu
    • 1
    • 3
  1. 1.The City College of New YorkNew YorkUSA
  2. 2.Beihang UniversityBeijingChina
  3. 3.The CUNY Graduate CenterNew YorkUSA

Personalised recommendations