Face Detection Coupling Texture, Color and Depth Data



In this chapter, we propose an ensemble of face detectors for maximizing the number of true positives found by the system. Unfortunately, combining different face detectors increases both the number of true positives and false positives. To overcome this difficulty, several methods for reducing false positives are tested and proposed. The different filtering steps are based on the characteristics of the depth map related to the subwindows of the whole image that contain the candidate faces. The most simple and easiest criteria to use, for instance, is to filter the candidate face region by considering its size in metric units.

The experimental section demonstrates that the proposed set of filtering steps greatly reduces the number of false positives without decreasing the detection rate. The proposed approach has been validated on a dataset of 549 images (each including both 2D and depth data) representing 614 upright frontal faces. The images were acquired both outdoors and indoors, with both first and second generation Kinect sensors. This was done in order to simulate a real application scenario. Moreover, for further validation and comparison with the state-of-the-art, our ensemble of face detectors is tested on the widely used BioID dataset where it obtains 100 % detection rate with an acceptable number of false positives.

A MATLAB version of the filtering steps and the dataset used in this paper will be freely available from


Face detection Depth map Ensemble 


  1. 1.
    L. Nanni, A. Lumini, F. Dominio, P. Zanuttigh, Effective and precise face detection based on color and depth data. Appl. Comput. Inform. 10(1), 1–13 (2014)CrossRefGoogle Scholar
  2. 2.
    C. Zhang, Z. Zhang, A survey of recent advances in face detection. Microsoft Research Technical Report, MSR-TR-2010-66, June 2010Google Scholar
  3. 3.
    Z. Zeng, M. Pantic, G.I. Roisman, T.S. Huang, A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Trans. Pattern Anal. Mach. Intell. 31(1), 39–58 (2009)CrossRefGoogle Scholar
  4. 4.
    H.L. Jin, Q.S. Liu, H.Q. Lu, Face detection using one-class based support vectors, in Proceedings 6th IEEE International Conference Automatic Face Gesture Recognition, Hoboken, NJ 07030 USA, (2004), pp. 457–462Google Scholar
  5. 5.
    P. Viola, M.J. Jones, Rapid object detection using a boosted cascade of simple features, Proceedings of the 2001 IEEE Computer Society conference on Computer Vision and Pattern Recognition, 1, 511--518 (2001)Google Scholar
  6. 6.
    C. Küblbeck, A. Ernst, Face detection and track in video sequences using the modified census transformation. Image Vis. Comput. 24(6), 564–572 (2006)CrossRefGoogle Scholar
  7. 7.
    C. Huang, H. Ai, Y. Li, S. Lao, High-performance rotation invariant multiview face detection. IEEE Trans. Pattern Anal. Mach. Intell. 29(4), 671–686 (2007)CrossRefGoogle Scholar
  8. 8.
    J. Wu, S. Charles Brubaker, M.D. Mullin, J.M. Rehg, Fast asymmetric learning for cascade face detection. IEEE Trans. Pattern Anal. Mach. Intell. 30, 369–382 (2008)CrossRefGoogle Scholar
  9. 9.
    M. Anisetti, Fast and robust face detection, in Multimedia Techniques for Device and Ambient Intelligence, Chapter 3 (Springer, US, 2009). ISBN: 978-0-387-88776-0Google Scholar
  10. 10.
    J. Li, Y. Zhang, Learning surf cascade for fast and accurate object detection, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, 2013) Hoboken, NJ 07030 USAGoogle Scholar
  11. 11.
    M. Mathias, et al. Face detection without bells and whistles, in Computer Vision–ECCV 2014 (Springer International Publishing, 2014), Zurich, Switzerland, pp. 720–735Google Scholar
  12. 12.
    F. Tsalakanidou, D. Tzovaras, M.G. Strintzis, Use of depth and colour eigenfaces for face recognition. Pattern Recogn. Lett. 24(910), 1427–1435 (2003)CrossRefzbMATHGoogle Scholar
  13. 13.
    R.I. Hg, P. Jasek, C. Rofidal, K. Nasrollahi, T.B. Moeslund, G. Tranchet, An RGB-D database using Microsoft’s Kinect for windows for face detection, in 2012 Eighth International Conference on Signal Image Technology and Internet Based Systems (SITIS) (IEEE, 2012), Hoboken, NJ 07030 USA, pp. 42–46Google Scholar
  14. 14.
    M.-Y. Shieh, T.-M. Hsieh, Fast facial detection by depth map analysis. Math. Problems Eng. (2013), Article ID 694321, pp.~10, doi:  10.1155/2013/694321 Google Scholar
  15. 15.
    J. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, A. Kipman, A. Blake, Real-time human pose recognition in parts from single depth images. CVPR 2, 3 (2011)Google Scholar
  16. 16.
    R. Mattheij, E. Postma, Y. Van den Hurk, P. Spronck, Depth-based detection using Haarlike features, in Proceedings of the BNAIC 2012 Conference (Maastricht University, The Netherlands, 2012), pp. 162–169Google Scholar
  17. 17.
    M. Anisetti, V. Bellandi, E. Damiani, L. Arnone, B. Rat, A3FD: Accurate 3D face detection, in Signal Processing for Image Enhancement and Multimedia Processing (Springer, US, 2008), pp. 155–165Google Scholar
  18. 18.
    F. Jiang, M. Fischer, H.K. Ekenel, B.E. Shi, Combining texture and stereo disparity cues for real-time face detection. Signal Process. Image Commun. 28(9), 1100–1113 (2013)CrossRefGoogle Scholar
  19. 19.
    G. Goswami, S. Bharadwaj, M. Vatsa, R. Singh, On RGB-D face recognition using Kinect, in 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS) (IEEE, 2013), Hoboken, NJ 07030 USA, pp. 1–6Google Scholar
  20. 20.
    Y. Taigman, M. Yang, M.A., Ranzato, L. Wolf, Deepface: Closing the gap to human-level performance in face verification, in 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, 2014), Hoboken, NJ 07030 USA, pp. 1701–1708Google Scholar
  21. 21.
    O. Jesorsky, K. Kirchberg, R. Frischholz, in Face Detection Using the Hausdorff Distance, eds. by J. Bigun, F. Smeraldi. Audio and Video based Person Authentication—AVBPA, (Springer, 2001), Berlin, Germany, pp. 90–95Google Scholar
  22. 22.
    N. Markuš, M. Frljak, I.S. Pandžić, J. Ahlberg, R. Forchheimer, Fast localization of facial landmark points, in Proceedings of the Croatian Computer Vision Workshop, Zagreb, Croatia, (2014)Google Scholar
  23. 23.
    M. Nilsson, J. Nordberg, I. Claesson, Face detection using local SMQT features and split up SNOW classifier. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP), 2, Hoboken, NJ 07030 USA, 589–592 (2007)Google Scholar
  24. 24.
    A. Asthana, S. Zafeiriou, S. Cheng, M. Pantic, Robust discriminative response map fitting with constrained local models, in 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, 2013), Hoboken, NJ 07030 USA, pp. 3444–3451Google Scholar
  25. 25.
    D. Herrera, J. Kannala, J. Heikkilä, Joint depth and color camera calibration with distortion correction. IEEE Trans. Pattern Anal. Mach. Intell. 34, 2058–782 (2012)CrossRefGoogle Scholar
  26. 26.
    D. Comaniciu, P. Meer, Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)CrossRefGoogle Scholar
  27. 27.
    C. Dal Mutto, P. Zanuttigh, G.M. Cortelazzo, Fusion of geometry and color information for scene segmentation. IEEE J. Sel. Top. Signal Process. 6(5), 505–521 (2012)CrossRefGoogle Scholar
  28. 28.
    X. Tan, F. Song, Z.-H. Zhou, S. Chen, Enhanced pictorial structures for precise eye localization under incontrolled conditions, in IEEE Conference on Computer Vision and Pattern Recognition 2009 (CVPR 2009), Hoboken, NJ 07030 USA, 20–25 June 2009, pp. 1621, 1628Google Scholar
  29. 29.
    E. Skodras, N. Fakotakis, Precise localization of eye centers in low resolution color images. Image Vision Comput. (2015). doi: 10.1016/j.imavis.2015.01.006 Google Scholar
  30. 30.
    Z. Ren, J. Meng, J. Yuan. Depth camera based hand gesture recognition and its applications in human-computer-interaction, in Proceedings of ICICS (2011), Hoboken, NJ 07030 USA, pp. 1–5Google Scholar
  31. 31.
    F. Dominio, M. Donadeo, P. Zanuttigh, Combining multiple depth-based descriptors for hand gesture recognition. Pattern Recogn. Lett. 50, 101–111 (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.DEI, University of PadovaPadovaItaly
  2. 2.DISI, Università di BolognaCesenaItaly

Personalised recommendations