Gappy PCA Classification for Occlusion Tolerant 3D Face Detection

  • Alessandro Colombo
  • Claudio Cusano
  • Raimondo Schettini


This paper presents an innovative approach for the detection of faces in three dimensional scenes. The method is tolerant against partial occlusions produced by the presence of any kind of object. The detection algorithm uses invariant properties of the surfaces to segment salient facial features, namely the eyes and the nose. At least two facial features must be clearly visible in order to perform face detection. Candidate faces are then registered using an ICP (Iterative Correspondent Point) based approach aimed to avoid those samples which belong to the occluding objects. The final face versus non-face discrimination is computed by a Gappy PCA (GPCA) classifier which is able to classify candidate faces using only those regions of the surface which are considered to be non-occluded. The algorithm has been tested using the UND database obtaining 100% of correct detection and only one false alarm. The database has been then processed with an artificial occlusions generator producing realistic acquisitions that emulate unconstrained scenarios. A rate of 89.8% of correct detections shows that 3D data is particularly suited for handling occluding objects. The results have been also verified on a small test set containing real world occlusions obtaining 90.4% of correctly detected faces. The proposed approach can be used to improve the robustness of all those systems requiring a face detection stage in non-controlled scenarios.


Three dimensional face detection Three dimensional face recognition Face occlusions Curvatures Gappy principal component analysis Global registration 


  1. 1.
    Belhumeur, P.N., Hespanha, J.P., Kriegman, D.J.: Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19, 711–720 (1997) CrossRefGoogle Scholar
  2. 2.
    Qing, L., Shan, S., Chen, X.: Face relighting for face recognition under generic illumination. In: IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (ICASSP), vol. 5, pp. V-733-6 (2004) Google Scholar
  3. 3.
    Bronstein, A.M., Bronstein, M.M., Kimmel, R.: Expression-invariant 3D face recognition. In: Proc. Audio and Video-Based Person Authentication. LCNS, vol. 2688, pp. 62–70. Springer, Berlin (2003) CrossRefGoogle Scholar
  4. 4.
    Lu, X., Jain, A.K.: Deformation analysis for 3d face matching. In: Application of Computer Vision. WACV/MOTIONS, vol. 1, pp. 99–104 (2005) Google Scholar
  5. 5.
    Y.-Y., Lin, T.-L., Liu, C.-S., Fuh: Fast object detection with occlusions. In: The 8th European Conference on Computer Vision (ECCV), Prague, May 2004 Google Scholar
  6. 6.
    Hotta, K.: A robust face detector under partial occlusion. In: Proceedings of ICIP, pp. 597–600 (2004) Google Scholar
  7. 7.
    Park, J.S., Oh, Y.H., Ahn, S.C., Lee, S.W.: Glasses removal from facial image using recursive error compensation. IEEE Trans. Pattern Anal. Mach. Intell. 27(5), 805–811 (2005) CrossRefGoogle Scholar
  8. 8.
    Martinez, A.M.: Recognition of partially occluded and/or imprecisely localized faces using a probabilistic approach. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 712–717 (2000) Google Scholar
  9. 9.
    Martinez, A.M.: Recognizing imprecisely localized, partially occluded and expression variant faces from a single sample per class. IEEE Trans. Pattern Anal. Mach. Intell. 24(6), 748–763 (2002) CrossRefGoogle Scholar
  10. 10.
    Kim, J., Choi, J., Yi, J., Turk, M.: Effective representation using ICA for face recognition robust to local distortion and partial occlusion. IEEE Trans. Pattern Anal. Mach. Intell. 27(12), 1977–1981 (2005) Google Scholar
  11. 11.
    Tarrés, F., Rama, A.: A novel method for face recognition under partial occlusion or facial expression variations. In: Proc. 47th Int’l Symp. ELMAR, pp. 163–166 (2005) Google Scholar
  12. 12.
    Colombo, A., Cusano, C., Schettini, R.: 3D face detection using curvature analysis. Pattern Recogn. 39(3), 444–455 (2006) zbMATHCrossRefGoogle Scholar
  13. 13.
    Everson, R., Sirovich, L.: Karhunen-Loève procedure for gappy data. J. Optical Soc. Am. A 12(8), 657–1664 (1995) CrossRefGoogle Scholar
  14. 14.
    Colombo, A., Cusano, C., Schettini, R.: A 3D face recognition system using curvature-based detection and holistic multimodal classification. In: Proc. 4th Int’l Symp. on Image and Signal Processing and Analysis, pp. 179–184 (2005) Google Scholar
  15. 15.
    Lu, X., Jain, A.K., Colbry, D.: Matching 2.5D face scans to 3D models. IEEE Trans. Pattern Anal. Mach. Intell. 28(1), 31–43 (2006) CrossRefGoogle Scholar
  16. 16.
    Besl, P.J., McKay, N.D.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14, 239–256 (1992) CrossRefGoogle Scholar
  17. 17.
    Chang, K., Bowyer, K.W., Flynn, P.J.: Face recognition using 2D and 3D facial data. In: ACM Workshop on Multimodal User Application, pp. 25–32 (2003) Google Scholar
  18. 18.
    Pulli, K.: Multiview registration for large data sets. In: Proc. 3DIM (1999) Google Scholar
  19. 19.
    Rusinkiewicz, S., Levoy, M.: Efficient variants of the ICP algorithm. In: Third International Conference on 3D Digital Imaging and Modeling (3DIM) (2001) Google Scholar
  20. 20.
    De Smet, M., Fransens, R., Van Gool, L.: A generalized EM approach for 3D model based face recognition under occlusions, In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 1423–1430 (2006) Google Scholar
  21. 21.
    Watt, A.H.: 3D Computer Graphics. Addison Wesley, Reading (1999) Google Scholar
  22. 22.
    Horn, B.K.P.: Closed-form solution of absolute orientation using quaternions. J. Opt. Soc. Am. A 4(4), 629–642 (1987) CrossRefMathSciNetGoogle Scholar
  23. 23.
    Wang, Y., Chua, C.-S.: Face recognition from 2D and 3D images using 3D Gabor filters. Image Vis. Comput. 23(11), 1018–1028 (2005) CrossRefGoogle Scholar
  24. 24.
    Faltemier, T.C., Bowyer, K.W., Flynn, P.J.: Using multi-instance enrollment representation to improve 3D face recognition. In: Int. Conference on Biometrics, Theory, Applications, and Systems, BTAS, pp. 1–6 (2007) Google Scholar
  25. 25.
    Gross, R., Matthews, I., Baker, S.: Active appearance models with occlusion. Image Vis. Comput. 24(6), 593–604 (2006) CrossRefGoogle Scholar
  26. 26.
    Huang, P.S., Zhang, S.: Fast three-step phase shifting algorithm. Appl. Opt. 45, 5086–5091 (2006) CrossRefGoogle Scholar
  27. 27.
    Yin, L., Wei, X., Sun, Y., Wang, J., Rosato, M.J.: A 3D facial expression database for facial behavior research. In: FGR’06: Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition, pp. 211–216 (2006) Google Scholar
  28. 28.
    Alyuz, N., Gokberk, B., Akarun, L.: A 3D face recognition system for expression and occlusion invariance. In: 2nd IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS 2008), pp. 1–7 (2008) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Alessandro Colombo
    • 1
  • Claudio Cusano
    • 1
  • Raimondo Schettini
    • 1
  1. 1.DISCo (Dipartimento di Informatica, Sistemistica e Comunicazione)Università degli Studi di Milano–BicoccaMilanoItaly

Personalised recommendations