3D Facial Landmark Localization Using Combinatorial Search and Shape Regression

  • Federico M. Sukno
  • John L. Waddington
  • Paul F. Whelan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7583)


This paper presents a method for the automatic detection of facial landmarks. The algorithm receives a set of 3D candidate points for each landmark (e.g. from a feature detector) and performs combinatorial search constrained by a deformable shape model. A key assumption of our approach is that for some landmarks there might not be an accurate candidate in the input set. This is tackled by detecting partial subsets of landmarks and inferring those that are missing so that the probability of the deformable model is maximized. The ability of the model to work with incomplete information makes it possible to limit the number of candidates that need to be retained, substantially reducing the number of possible combinations to be tested with respect to the alternative of trying to always detect the complete set of landmarks. We demonstrate the accuracy of the proposed method in a set of 144 facial scans acquired by means of a hand-held laser scanner in the context of clinical craniofacial dysmorphology research. Using spin images to describe the geometry and targeting 11 facial landmarks, we obtain an average error below 3 mm, which compares favorably with other state of the art approaches based on geometric descriptors.


Shape Model Deformable Model Gabor Wavelet Spin Image Statistical Shape Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Gupta, S., Markey, M., Bovik, A.: Antopometric 3D face recognition. Int. J. Comput. Vis. 90, 331–349 (2010)CrossRefGoogle Scholar
  2. 2.
    Zhang, G., Wang, Y.: Robust 3D face recognition based on resolution invariant features. Pattern Recogn. Lett. 32, 1009–1019 (2011)CrossRefGoogle Scholar
  3. 3.
    Hennessy, R., Kinsella, A., Waddington, J.: 3D laser surface scanning and geometric morphometric analysis of craniofacial shape as an index of cerebro-craniofacial morphogenesis: initial application to sexual dimorphism. Biol. Psychiatry. 51, 507–514 (2002)CrossRefGoogle Scholar
  4. 4.
    Segundo, M., Silva, L., Bellon, O.P., et al.: Automatic face segmentation and facial landmark detection in range images. IEEE Trans. Syst., Man, Cybern. B 40, 1319–1330 (2010)CrossRefGoogle Scholar
  5. 5.
    Szeptycki, P., Ardabilian, M., Chen, L.: A coarse-to-fine curvature analysis-based rotation invariant 3D face landmarking. In: BTAS, pp. 1–6 (2009)Google Scholar
  6. 6.
    Faltemier, T., Bowyer, K., Flynn, P.: Rotated profile signatures for robust 3D feature detection. In: FG, pp. 1–7 (2008)Google Scholar
  7. 7.
    Yu, T., Moon, Y.: A novel genetic algorithm for 3D facial landmark localization. In: BTAS, pp. 1–6 (2008)Google Scholar
  8. 8.
    D’ Hose, J., Colineau, J.: Bichon, et al.: Precise localization of landmarks on 3D faces using Gabor wavelets. In: BTAS, pp. 1–6 (2007)Google Scholar
  9. 9.
    Romero-Huertas, M., Pears, N.: Landmark localisation in 3D face data. In: Int. Conf. on Adv. Video and Signal Based Surveillance, pp. 73–78 (2009)Google Scholar
  10. 10.
    Zhao, X., Szeptycki, P.: Dellandrea, et al.: Precise 2.5D facial landmarking via an analysis by synthesis approach. In: Workshop on Appl. of Comput. Vis., pp. 1–7 (2009)Google Scholar
  11. 11.
    Nair, P., Cavallaro, A.: 3-D face detection, landmark localization and registration using a point distribution model. IEEE Trans. Multimedia 11, 611–623 (2009)CrossRefGoogle Scholar
  12. 12.
    Creusot, C., Pears, N., Austin, J.: 3D face landmark labelling. In: ACM Workshop on 3D Object Retrieval, pp. 23–32 (2010)Google Scholar
  13. 13.
    Amberg, B., Vetter, T.: Optimal landmark detection using shape models and branch and bound. In: ICCV, pp. 455–462 (2011)Google Scholar
  14. 14.
    Passalis, G., Perakis, N., Theoharis, T., et al.: Using facial symmetry to handle pose variations in real-world 3D face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1938–1951 (2011)CrossRefGoogle Scholar
  15. 15.
    Jiang, H., Yu, S., Martin, D.: Linear scale and rotation invariant matching. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1339–1355 (2011)CrossRefGoogle Scholar
  16. 16.
    Fischler, M., Bolles, R.: Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Comm. of the ACM 24, 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Johnson, A., Hebert, M.: Using spin images for efficient object recognition in cluttered 3D scenes. IEEE Trans. Pattern Anal. Mach. Intell. 21, 433–449 (1999)CrossRefGoogle Scholar
  18. 18.
    Cootes, T., Taylor, C.: Statistical models of appearance for computer vision. Technical report, Wolfson Image Analysis Unit, University of Manchester, UK (2001)Google Scholar
  19. 19.
    Blanc, R., Syrkina, E., Székely, G.: Estimating the Confidence of Statistical Model Based Shape Prediction. In: Prince, J.L., Pham, D.L., Myers, K.J. (eds.) IPMI 2009. LNCS, vol. 5636, pp. 602–613. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  20. 20.
    de Bruijne, M., Lund, M., Tanko, et al.: Quantitative vertebral morphometry using neighbor-conditional shape models. Med. Image Anal. 11, 503–512 (2007)CrossRefGoogle Scholar
  21. 21.
    Kudo, M., Sklansky, J.: Comparison of algorithms that select features for pattern classifiers. Pattern Recogn. 33, 25–41 (2000)CrossRefGoogle Scholar
  22. 22.
    Rousseeuw, P.: Least median of squares regression. J. Am. Stat. Assoc. 79, 871–880 (1984)MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Federico M. Sukno
    • 1
    • 2
  • John L. Waddington
    • 2
  • Paul F. Whelan
    • 1
  1. 1.Centre for Image Processing & AnalysisDublin City UniversityDublin 9Ireland
  2. 2.Molecular & Cellular TherapeuticsRoyal College of Surgeons in IrelandDublin 2Ireland

Personalised recommendations