Machine Vision and Applications

, Volume 22, Issue 3, pp 481–494 | Cite as

A comparison of face and facial feature detectors based on the Viola–Jones general object detection framework

  • Modesto CastrillónEmail author
  • Oscar Déniz
  • Daniel Hernández
  • Javier Lorenzo
Original Paper


The human face provides useful information during interaction; therefore, any system integrating Vision-Based Human Computer Interaction requires fast and reliable face and facial feature detection. Different approaches have focused on this ability but only open source implementations have been extensively used by researchers. A good example is the Viola–Jones object detection framework that particularly in the context of facial processing has been frequently used. The OpenCV community shares a collection of public domain classifiers for the face detection scenario. However, these classifiers have been trained in different conditions and with different data but rarely tested on the same datasets. In this paper, we try to fill that gap by analyzing the individual performance of all those public classifiers presenting their pros and cons with the aim of defining a baseline for other approaches. Solid comparisons will also help researchers to choose a specific classifier for their particular scenario. The experimental setup also describes some heuristics to increase the facial feature detection rate while reducing the face false detection rate.


Face and facial feature detection Haar wavelets Human computer interaction Face datasets OpenCV 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Pentland A.: Looking at people: sensing for ubiquitous and wearable computing. IEEE Trans. Pattern Anal. Mach. Intell., 22(1), 107–119 (2000)CrossRefGoogle Scholar
  2. 2.
    Turk M.: Computer vision in the interface. Commun. Assoc. Comput. Mach. 47(1), 61–67 (2004)Google Scholar
  3. 3.
    Li, S.Z., Zhu, L., Zhang, Z., Blake, A., Zhang, H., Shum, H.: Statistical learning of multi-view face detection. In: European Conference Computer Vision, pp. 67–81 (2002)Google Scholar
  4. 4.
    Schneiderman, H., Kanade, T.: A statistical method for 3d object detection applied to faces and cars. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1746–1759 (2000)Google Scholar
  5. 5.
    Viola P., Jones M.J.: Robust real-time face detection. Int. J. Comp. Vis. 57(2), 151–173 (2004)CrossRefGoogle Scholar
  6. 6.
    Lienhart, R., Maydt, J.: An extended set of Haar-like features for rapid object detection. In: IEEE ICIP 2002, vol. 1, pp. 900–903 (2002)Google Scholar
  7. 7.
    Castrillón, M., Déniz, O., Antón, L., Lorenzo, J.: Face and facial feature detection evaluation. In: Proceedings of the International Joint Conference on Computer Vision and Computer Graphics Theory and Applications, VISIGRAPP, pp. 167–172 (2008)Google Scholar
  8. 8.
    Lienhart, R., Kuranov, A., Pisarevsky, V.: Empirical analysis of detection cascades of boosted classifiers for rapid object detection. In: DAGM’03, 25th Pattern Recognition Symposium, Madgeburg, Germany, pp. 297–304 (2003)Google Scholar
  9. 9.
    Schmidt, A., Kasinski, A.: The performance of the haar cascade classifiers applied to the face and eyes detection. In: Springer (ed.) Proceedings of the 5th International Conference on Computer Recognition Systems CORES2007. Advances in Soft Computing, vol. 45. Wroclaw, Poland, pp. 816–823 (2007)Google Scholar
  10. 10.
    Lam K.-M., Yan H.: Locating and extracting the eye in human face images. Pattern Recogn. 29(5), 771–779 (1996)CrossRefMathSciNetGoogle Scholar
  11. 11.
    Chow G., Li X.: Towards a system for automatic facial feature detection. Pattern Recogn. 26(12), 1739–1755 (1993)CrossRefGoogle Scholar
  12. 12.
    Yacoob Y., Davis L.S.: Labeling of human face components from range data. CVGIP: Image Underst. 60(2), 168–178 (1994)CrossRefGoogle Scholar
  13. 13.
    Morimoto, C.H., Flickner, M.: Real time multiple face detection using active illumination. In: 4th IEEE International Conference on Automatic Face and Gesture Recognition, p. 8 (2000)Google Scholar
  14. 14.
    Smeraldi, F., Carmona, O., Bigün, J.: Saccadic search with Gabor features applied to eye detection and real-time head tracking. Image Vis. Comput. 18Google Scholar
  15. 15.
    Hsu R.-L., Abdel-Mottsleb M.: Face detection in color images. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 686–706 (2002)Google Scholar
  16. 16.
    Yuille A.L., Hallinan P.W., Cohen D.S.: Feature extraction from faces using deformable templates. Int. J. Comp. Vis. 8(2), 99–111 (1992)CrossRefGoogle Scholar
  17. 17.
    Huang, W., Sun, Q., Lam, C.-P., Wu, J.-K.: A robust approach to face and eyes detection from image with cluttered background. In: International Conference Pattern Recognition, vol. 1, pp. 110–114 (1998)Google Scholar
  18. 18.
    Ding, L., Martinez, A.: Precise detailed detection of faces and facial features. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–7 (2008)Google Scholar
  19. 19.
    Bala, L.-P., Talmi, K., Liu, J.: Automatic detection and tracking of faces and facial features in video. In: Picture Coding Symposium, pp. 251–256 (1997)Google Scholar
  20. 20.
    Sobottka K., Pitas I.: A novel method for automatic face segmentation, facial feature extraction and tracking. Signal Process. Image Commun. 12(3), 263–281 (1998)CrossRefGoogle Scholar
  21. 21.
    Feyrer, S., Zell, A.: Detection, tracking and pursuit of humans with autonomous mobile robot. In: Proceedings of International Conference on Intelligent Robots and Systems, Kyongju, Korea, pp. 864–869 (1999)Google Scholar
  22. 22.
    Nikolaidis A., Pitas I.: Facial feature extraction and determination of pose. Pattern Recogn. 33, 1783–1791 (2000)CrossRefGoogle Scholar
  23. 23.
    Gorodnichy D.O., Roth G.: Nouse ’use your nose as a mouse’ perceptual vision technology for hands-free games and interfaces. Image Vis. Comput. 22(12), 931–942 (2004)CrossRefGoogle Scholar
  24. 24.
    Horprasert, T., Yacoob, Y., Davis, L.: Computing 3-d head orientation from a monocular image sequence. In: Proceedings International Conference Automatic Face and Gesture Recognition, Killington, Vermont, USA, pp. 242–247 (1996)Google Scholar
  25. 25.
    Rabie, A., Lang, C., Hanheide, M., Castrillón, M., Sagerer, G.: Automatic initialization for facial analysis in interactive robotics. In: 6th International Conference on Computer Vision Systems, Vision for Cognitive Systems, pp. 517–526 (2008)Google Scholar
  26. 26.
    Castrillón Santana, M., Déniz Suárez, O., Hernández Tejera, M., Guerra Artal, C.: ENCARA2: Real-time detection of multiple faces at different resolutions in video streams. J. Vis. Commun. Image Represent. pp. 130–140 (2007)Google Scholar
  27. 27.
    Lin, K., Huang, J., Chen, J., Zhou, C.: Real-time eye detection in video streams. In: Fourth International Conference on Natural Computation, pp. 193–197 (2008)Google Scholar
  28. 28.
    Ren, X., Song, J., Ying, H., Zhu, Y., Qiu, X.: Robust nose detection and tracking using gentleboost and improved Lucas–Kanade optical flow algorithms. Lecture Notes in Computer Science: Advanced Intelligent Computing Theories and Applications. With Aspects of Theoretical and Methodological Issues, vol. 4681, pp. 1240–1246 (2007)Google Scholar
  29. 29.
    Liang, L., Liu, X., Pi, X., Zhao, Y., Nefian, A.V.: Speaker independent audio-visual continuous speech recognition. In: International Conference on Multimedia and Expo, pp. 25–28 (2002)Google Scholar
  30. 30.
    Hjelmas E., Low B.K.: Face detection: A survey. Comp. Vis. Image Underst. 83(3), 236–274 (2001). doi: 10.1006/cviu.2001.0921 CrossRefzbMATHGoogle Scholar
  31. 31.
    Yang M.-H., Kriegman D., Ahuja N.: Detecting faces in images: A survey. Trans. Pattern Anal. Mach. Intell. 24(1), 34–58 (2002). doi: 10.1109/34.982883 CrossRefGoogle Scholar
  32. 32.
    Intel, Intel Open Source Computer Vision Library, v1.1ore, (October 2008)
  33. 33.
    Hewitt, R.: Seeing with opencv. a computer-vision library, Servo, pp. 62–65 (2007)Google Scholar
  34. 34.
    Lienhart, R., Liang, L., Kuranov, A.: A detector tree of boosted classifiers for real-time object detection and tracking. In: IEEE ICME2003, pp. 277–280 (2003)Google Scholar
  35. 35.
    Wilson P.I., Fernandez J.: Facial feature detection using haar classifiers. J. Comput. Sci. Coll. 21, 127–133 (2006)Google Scholar
  36. 36.
    Bradley, D.: Profile face detection., last accessed 5/11/2007 (2003)
  37. 37.
    Reimondo. A.: Haar cascades repository, (2007)
  38. 38.
    Kruppa, H., Castrillón Santana, M., Schiele, B.: Fast and robust face finding via local context. In: Joint IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance (VS-PETS), pp. 157–164 (2003)Google Scholar
  39. 39.
    Yu, S.: Tree-based 20 × 20 eye detectors, (2009)
  40. 40.
    Wimmer, M.: Eyefinder,, last accesed 5/11/2007 (2004)
  41. 41.
    Urtho, Eye detector,, last accesed 5/9/2007 (2006)
  42. 42.
    Shan, T.: Security and surveillance, (2008)
  43. 43.
    Beumer, G., Tao, Q., Bazen, A., Veldhuis, R.: A landmark paper in face recognition. In: 7th International Conference on Automatic Face and Gesture Recognition, vol. 78, 2006. FGR 2006, IEEE Computer Society Press, Southampton (2006)Google Scholar
  44. 44.
    Hameed, S.: Eye cascade, (October 2008)
  45. 45.
    Bediz, Y., Akar, G.B.: View point tracking for 3d display systems. In: 3th European Signal Processing Conference, EUSIPCO-2005, (2005)Google Scholar
  46. 46.
    Sung K.-K., Poggio T.: Example-based learning for view-based human face detection. IEEE Trans. Pattern Anal. Mach. Intell. 20(1), 39–51 (1998)CrossRefGoogle Scholar
  47. 47.
    Rowley H.A., Baluja S., Kanade T.: Neural network-based face detection. IEEE Trans. Pattern Anal. Machine Intell. 20(1), 23–38 (1998)CrossRefGoogle Scholar
  48. 48.
    Carnegie Mellon University, CMU/VACS image database: Frontal face images,, last accesed 5/11/2007 (1999)
  49. 49.
    Belhumeur P., Hespanha J., Kriegman D.: Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. IEEE Trans PAMI 19(7), 711–720 (1997)Google Scholar
  50. 50.
    Huang G.B., Ramesh M., Berg T., Learned-Miller E.: Labeled faces in the wild: A database for studying face recognition in unconstrained environments, Tech. Rep. 07-49. University of Massachusetts, Amherst (2007)Google Scholar
  51. 51.
    Jesorsky, O., Kirchberg, K.J., Frischholz, R. W.: Robust face detection using the Hausdorff distance, Lecture Notes in Computer Science. Procs. of the Third International Conference on Audio- and Video-Based Person Authentication 2091, 90–95 (2001)Google Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  • Modesto Castrillón
    • 1
    Email author
  • Oscar Déniz
    • 1
    • 2
  • Daniel Hernández
    • 1
  • Javier Lorenzo
    • 1
  1. 1.Universidad de Las Palmas de Gran Canaria, SIANI, Edificio Central del Parque Científico-TecnológicoLas PalmasSpain
  2. 2.E.T.S. Ingenieros IndustrialesUniversidad de Castilla-La Mancha Campus UniversitarioCiudad RealSpain

Personalised recommendations