Skip to main content
SpringerLink
Log in

A comparison of face and facial feature detectors based on the Viola–Jones general object detection framework

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

The human face provides useful information during interaction; therefore, any system integrating Vision-Based Human Computer Interaction requires fast and reliable face and facial feature detection. Different approaches have focused on this ability but only open source implementations have been extensively used by researchers. A good example is the Viola–Jones object detection framework that particularly in the context of facial processing has been frequently used. The OpenCV community shares a collection of public domain classifiers for the face detection scenario. However, these classifiers have been trained in different conditions and with different data but rarely tested on the same datasets. In this paper, we try to fill that gap by analyzing the individual performance of all those public classifiers presenting their pros and cons with the aim of defining a baseline for other approaches. Solid comparisons will also help researchers to choose a specific classifier for their particular scenario. The experimental setup also describes some heuristics to increase the facial feature detection rate while reducing the face false detection rate.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Pentland A.: Looking at people: sensing for ubiquitous and wearable computing. IEEE Trans. Pattern Anal. Mach. Intell., 22(1), 107–119 (2000)

    Article  Google Scholar 

  2. Turk M.: Computer vision in the interface. Commun. Assoc. Comput. Mach. 47(1), 61–67 (2004)

    Google Scholar 

  3. Li, S.Z., Zhu, L., Zhang, Z., Blake, A., Zhang, H., Shum, H.: Statistical learning of multi-view face detection. In: European Conference Computer Vision, pp. 67–81 (2002)

  4. Schneiderman, H., Kanade, T.: A statistical method for 3d object detection applied to faces and cars. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1746–1759 (2000)

  5. Viola P., Jones M.J.: Robust real-time face detection. Int. J. Comp. Vis. 57(2), 151–173 (2004)

    Article  Google Scholar 

  6. Lienhart, R., Maydt, J.: An extended set of Haar-like features for rapid object detection. In: IEEE ICIP 2002, vol. 1, pp. 900–903 (2002)

  7. Castrillón, M., Déniz, O., Antón, L., Lorenzo, J.: Face and facial feature detection evaluation. In: Proceedings of the International Joint Conference on Computer Vision and Computer Graphics Theory and Applications, VISIGRAPP, pp. 167–172 (2008)

  8. Lienhart, R., Kuranov, A., Pisarevsky, V.: Empirical analysis of detection cascades of boosted classifiers for rapid object detection. In: DAGM’03, 25th Pattern Recognition Symposium, Madgeburg, Germany, pp. 297–304 (2003)

  9. Schmidt, A., Kasinski, A.: The performance of the haar cascade classifiers applied to the face and eyes detection. In: Springer (ed.) Proceedings of the 5th International Conference on Computer Recognition Systems CORES2007. Advances in Soft Computing, vol. 45. Wroclaw, Poland, pp. 816–823 (2007)

  10. Lam K.-M., Yan H.: Locating and extracting the eye in human face images. Pattern Recogn. 29(5), 771–779 (1996)

    Article  MathSciNet  Google Scholar 

  11. Chow G., Li X.: Towards a system for automatic facial feature detection. Pattern Recogn. 26(12), 1739–1755 (1993)

    Article  Google Scholar 

  12. Yacoob Y., Davis L.S.: Labeling of human face components from range data. CVGIP: Image Underst. 60(2), 168–178 (1994)

    Article  Google Scholar 

  13. Morimoto, C.H., Flickner, M.: Real time multiple face detection using active illumination. In: 4th IEEE International Conference on Automatic Face and Gesture Recognition, p. 8 (2000)

  14. Smeraldi, F., Carmona, O., Bigün, J.: Saccadic search with Gabor features applied to eye detection and real-time head tracking. Image Vis. Comput. 18

  15. Hsu R.-L., Abdel-Mottsleb M.: Face detection in color images. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 686–706 (2002)

    Google Scholar 

  16. Yuille A.L., Hallinan P.W., Cohen D.S.: Feature extraction from faces using deformable templates. Int. J. Comp. Vis. 8(2), 99–111 (1992)

    Article  Google Scholar 

  17. Huang, W., Sun, Q., Lam, C.-P., Wu, J.-K.: A robust approach to face and eyes detection from image with cluttered background. In: International Conference Pattern Recognition, vol. 1, pp. 110–114 (1998)

  18. Ding, L., Martinez, A.: Precise detailed detection of faces and facial features. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–7 (2008)

  19. Bala, L.-P., Talmi, K., Liu, J.: Automatic detection and tracking of faces and facial features in video. In: Picture Coding Symposium, pp. 251–256 (1997)

  20. Sobottka K., Pitas I.: A novel method for automatic face segmentation, facial feature extraction and tracking. Signal Process. Image Commun. 12(3), 263–281 (1998)

    Article  Google Scholar 

  21. Feyrer, S., Zell, A.: Detection, tracking and pursuit of humans with autonomous mobile robot. In: Proceedings of International Conference on Intelligent Robots and Systems, Kyongju, Korea, pp. 864–869 (1999)

  22. Nikolaidis A., Pitas I.: Facial feature extraction and determination of pose. Pattern Recogn. 33, 1783–1791 (2000)

    Article  Google Scholar 

  23. Gorodnichy D.O., Roth G.: Nouse ’use your nose as a mouse’ perceptual vision technology for hands-free games and interfaces. Image Vis. Comput. 22(12), 931–942 (2004)

    Article  Google Scholar 

  24. Horprasert, T., Yacoob, Y., Davis, L.: Computing 3-d head orientation from a monocular image sequence. In: Proceedings International Conference Automatic Face and Gesture Recognition, Killington, Vermont, USA, pp. 242–247 (1996)

  25. Rabie, A., Lang, C., Hanheide, M., Castrillón, M., Sagerer, G.: Automatic initialization for facial analysis in interactive robotics. In: 6th International Conference on Computer Vision Systems, Vision for Cognitive Systems, pp. 517–526 (2008)

  26. Castrillón Santana, M., Déniz Suárez, O., Hernández Tejera, M., Guerra Artal, C.: ENCARA2: Real-time detection of multiple faces at different resolutions in video streams. J. Vis. Commun. Image Represent. pp. 130–140 (2007)

  27. Lin, K., Huang, J., Chen, J., Zhou, C.: Real-time eye detection in video streams. In: Fourth International Conference on Natural Computation, pp. 193–197 (2008)

  28. Ren, X., Song, J., Ying, H., Zhu, Y., Qiu, X.: Robust nose detection and tracking using gentleboost and improved Lucas–Kanade optical flow algorithms. Lecture Notes in Computer Science: Advanced Intelligent Computing Theories and Applications. With Aspects of Theoretical and Methodological Issues, vol. 4681, pp. 1240–1246 (2007)

  29. Liang, L., Liu, X., Pi, X., Zhao, Y., Nefian, A.V.: Speaker independent audio-visual continuous speech recognition. In: International Conference on Multimedia and Expo, pp. 25–28 (2002)

  30. Hjelmas E., Low B.K.: Face detection: A survey. Comp. Vis. Image Underst. 83(3), 236–274 (2001). doi:10.1006/cviu.2001.0921

    Article  MATH  Google Scholar 

  31. Yang M.-H., Kriegman D., Ahuja N.: Detecting faces in images: A survey. Trans. Pattern Anal. Mach. Intell. 24(1), 34–58 (2002). doi:10.1109/34.982883

    Article  Google Scholar 

  32. Intel, Intel Open Source Computer Vision Library, v1.1ore, http://sourceforge.net/projects/opencvlibrary/ (October 2008)

  33. Hewitt, R.: Seeing with opencv. a computer-vision library, Servo, pp. 62–65 (2007)

  34. Lienhart, R., Liang, L., Kuranov, A.: A detector tree of boosted classifiers for real-time object detection and tracking. In: IEEE ICME2003, pp. 277–280 (2003)

  35. Wilson P.I., Fernandez J.: Facial feature detection using haar classifiers. J. Comput. Sci. Coll. 21, 127–133 (2006)

    Google Scholar 

  36. Bradley, D.: Profile face detection. http://www.davidbradley.info/publications/bradley-iurac-03.swf, last accessed 5/11/2007 (2003)

  37. Reimondo. A.: Haar cascades repository, http://alereimondo.no-ip.org/OpenCV/34 (2007)

  38. Kruppa, H., Castrillón Santana, M., Schiele, B.: Fast and robust face finding via local context. In: Joint IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance (VS-PETS), pp. 157–164 (2003)

  39. Yu, S.: Tree-based 20 × 20 eye detectors, http://yushiqi.cn/research/eyedetection (2009)

  40. Wimmer, M.: Eyefinder, http://www9.cs.tum.edu/people/wimmerm/se/project.eyefinder/, last accesed 5/11/2007 (2004)

  41. Urtho, Eye detector, http://face.urtho.net/, last accesed 5/9/2007 (2006)

  42. Shan, T.: Security and surveillance, http://www.itee.uq.edu.au/~sas/people.htm (2008)

  43. Beumer, G., Tao, Q., Bazen, A., Veldhuis, R.: A landmark paper in face recognition. In: 7th International Conference on Automatic Face and Gesture Recognition, vol. 78, 2006. FGR 2006, IEEE Computer Society Press, Southampton (2006)

  44. Hameed, S.: Eye cascade, http://umich.edu/~shameem (October 2008)

  45. Bediz, Y., Akar, G.B.: View point tracking for 3d display systems. In: 3th European Signal Processing Conference, EUSIPCO-2005, (2005)

  46. Sung K.-K., Poggio T.: Example-based learning for view-based human face detection. IEEE Trans. Pattern Anal. Mach. Intell. 20(1), 39–51 (1998)

    Article  Google Scholar 

  47. Rowley H.A., Baluja S., Kanade T.: Neural network-based face detection. IEEE Trans. Pattern Anal. Machine Intell. 20(1), 23–38 (1998)

    Article  Google Scholar 

  48. Carnegie Mellon University, CMU/VACS image database: Frontal face images, http://vasc.ri.cmu.edu/idb/html/face/frontal_images/index.html, last accesed 5/11/2007 (1999)

  49. Belhumeur P., Hespanha J., Kriegman D.: Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. IEEE Trans PAMI 19(7), 711–720 (1997)

    Google Scholar 

  50. Huang G.B., Ramesh M., Berg T., Learned-Miller E.: Labeled faces in the wild: A database for studying face recognition in unconstrained environments, Tech. Rep. 07-49. University of Massachusetts, Amherst (2007)

    Google Scholar 

  51. Jesorsky, O., Kirchberg, K.J., Frischholz, R. W.: Robust face detection using the Hausdorff distance, Lecture Notes in Computer Science. Procs. of the Third International Conference on Audio- and Video-Based Person Authentication 2091, 90–95 (2001)

Download references

Author information

Author notes

  1. Oscar Déniz

    Present address: E.T.S. Ingenieros Industriales, Universidad de Castilla-La Mancha Campus Universitario, Avda. Camilo José Cela, s/n, 13071, Ciudad Real, Spain

Authors and Affiliations

  1. Universidad de Las Palmas de Gran Canaria, SIANI, Edificio Central del Parque Científico-Tecnológico, 35017, Las Palmas, Spain

    Modesto Castrillón, Oscar Déniz, Daniel Hernández & Javier Lorenzo

Authors
  1. Modesto Castrillón
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Oscar Déniz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Hernández
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Javier Lorenzo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Modesto Castrillón.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Castrillón, M., Déniz, O., Hernández, D. et al. A comparison of face and facial feature detectors based on the Viola–Jones general object detection framework. Machine Vision and Applications 22, 481–494 (2011). https://doi.org/10.1007/s00138-010-0250-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-010-0250-7

Keywords

Search

Navigation