Deformable Face Alignment via Local Measurements and Global Constraints

  • Jason M. Saragih
Part of the Lecture Notes in Computational Vision and Biomechanics book series (LNCVB, volume 7)


This chapter will review a particular approach to deformable face alignment coined constrained local models (CLM). The approach leverages the excellent generalisation properties of local appearance representations of parts and the strong global constraints imposed by the geometrical relationships between part locations. We begin by posing CLM in the general context of deformable face alignment, highlighting its similarities and differences with other approaches and motivating its benefits. An overview of the approach is then presented, explicating its various components and touching briefly on the interrelated issues of optimisation, feature representation and geometry regularisation. The following three sections discuss each of these three components in detail. The chapter concludes with a general discussion and directions of future work.


Facial Feature Facial Shape Active Appearance Model Active Shape Model Local Appearance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Amberg B, Vetter T (2011) Optimal landmark detection using shape models and branch and bound. In: Proceedings ICCV’11: international conference on computer visionGoogle Scholar
  2. 2.
    Blanz V, Vetter T (1999) A morphable model for the synthesis of 3D-faces. In: SIGGRAPH’99Google Scholar
  3. 3.
    Carreira-Perpinan M (2007) Gaussian mean-shift is an EM algorithm. PAMI 29(5):767–776CrossRefGoogle Scholar
  4. 4.
    Carreira-Perpinan M, Williams C (2003) On the number of modes of a gaussian mixture. Lect Notes Comput Sci 2695:625–640CrossRefGoogle Scholar
  5. 5.
    Cootes T, Taylor C (1992) Active shape models–‘smart snakes’. In: British achine vision conference (BMVC’92), pp 266–275Google Scholar
  6. 6.
    Cootes TF, Cooper D, Taylor CJ, Graham J (1995) Active shape models–their training and application. Comput Vis Image Underst 61:38–59CrossRefGoogle Scholar
  7. 7.
    Cootes T, Edwards G, Taylor C (1998) Active appearance models. In: ECCV’98, pp 484–498, 1998Google Scholar
  8. 8.
    Cootes TF, Edwards GJ, Taylor CJ (1998) A comparative evaluation of active appearance model algorithms. In: BMVC’98: proceedings of the 9th British machine vision conference, vol 2, pp 680–689Google Scholar
  9. 9.
    Cortes C (1995) Vapnik V (1995) Support-vector networks. Mach Learn 20:273–297zbMATHGoogle Scholar
  10. 10.
    Cristinacce D, Cootes T (2004) Feature detection and tracking with constrained local models. In: EMCV’04, pp 929–938Google Scholar
  11. 11.
    Cristinacce D, Cootes T (2007) Boosted active shape models. In: BMVC’07: proceedings of the 18th British machine vision conference, vol 2, pp 880–889Google Scholar
  12. 12.
    Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: ICCV, vol 2, pp 886–893Google Scholar
  13. 13.
    di Mauro EC, Cootes TF, Page GJ, Jackson CB (1996) Check!: a generic and specific industrial inspection tool. VISP’96: proceedings of the conference on vitalizing city centres through integrated spatial, planning 143:241–249Google Scholar
  14. 14.
    Donner R, Reiter M, Langs G, Peloschek P, Bischof H (2006) Fast active appearance model search using canonical correlation analysis. Trans Pattern Anal Mach Intell (TPAMI) 28:1690–1694CrossRefGoogle Scholar
  15. 15.
    Fashing M, Tomasi C (2005) Mean shift as a bound optimization. PAMI 27(3):471–474CrossRefGoogle Scholar
  16. 16.
    Felzenszwalb PR, Girshick DM, Ramanan D (2009) Object detection with discriminatively trained part-based models. IEEE Pattern Anal Mach IntellGoogle Scholar
  17. 17.
    Ginneken BV, Stegmann MB, Loog M (2006) Segmentation of anatomical structures in chest radiographs using supervised methods: a comparative study on a public database. Med Image Anal 10:19–40CrossRefGoogle Scholar
  18. 18.
    Gross R, Matthews I, Baker S (2005) Generic vs. person specific active appearance models. IVC 23:1080–1093CrossRefGoogle Scholar
  19. 19.
    Gross R, Sweeney L, la Torre Frade FD, Baker S (2006) Model-based face de-identification. In: Proceedings of the 2006 conference on computer vision and pattern recognition, workshop, pp 161–168Google Scholar
  20. 20.
    Gross R, Matthews I, Baker S, Kanade T (2007) The CMU multiple pose, illumination and expression (MultiPIE) database. Technical report, Robotics Institute, Carnegie Mellon UniversityGoogle Scholar
  21. 21.
    Gu L, Kanade T (2008) A generative shape regularization model for robust face alignment. In: ECCV’08Google Scholar
  22. 22.
    Hansen DW, Hansen JP, Niels M, Stegmann MB (2002) Eye typing using Markov and active appearance models. In: WACV’02: proceedings of the 6th IEEE workshop on applications of computer vision, p 132Google Scholar
  23. 23.
    Horn BKP, Schunck BG (1981) Determining optical flow. Artif Intell 17:185–203CrossRefGoogle Scholar
  24. 24.
    Huang G, Ramesh M, Berg T, Learned-Miller E (2007) Labeled faces in the wild: a atabase for studying face recognition in unconstrained environments. Techical report 07–49, University of Massachusetts, AmherstGoogle Scholar
  25. 25.
    la Torre Frade FD, Romea AC, Cohn J, Kanade T (2007) Filtered component analysis to increase robustness to local minima in appearance models. In: CVPR’07: proceedings of the IEEE computer society conference on computer vision and pattern recognitionGoogle Scholar
  26. 26.
    Liu X (2007) Generic face alignment using boosted appearance model. In: CVPR’07: proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1–8Google Scholar
  27. 27.
    Lucas B, Kanade T (1981) An iterative image registration technique with an application to stereo vision. In: Proceedings of imaging understanding workshopGoogle Scholar
  28. 28.
    Matthews I, Baker S (2004) Active appearance models revisited. IJCV’04 60:135–164Google Scholar
  29. 29.
    Mercer J (1909) Functions of positive and negative type and their connection with the theory of integral equations. Philos Trans R Soc A 209:415–446zbMATHCrossRefGoogle Scholar
  30. 30.
    Mittrapiyanuruk P, DeSouza GN, Kak AC (2005) Accurate 3D tracking of rigid objects with occlusion using active appearance models. In: WACV-MOTION’05: proceedings of the IEEE workshop on motion and video, computing vol 2, pp 90–95Google Scholar
  31. 31.
    Moghaddam B, Pentland A (1997) Probabilistic visual learning for object representation. PAMI 19(7):696–710CrossRefGoogle Scholar
  32. 32.
    Nguyen M, De la Torre Frade F (2008) Local minima free parameterized appearance models. In: CVPR’08Google Scholar
  33. 33.
    Ong EJ, Lan Y, Theobald B, Harvey R, Bowden R (2009) Robust facial feature tracking using selected multi-resolution linear predictors. In: computer vision, 2009 IEEE 12th international conference on, pp 1483–1490Google Scholar
  34. 34.
    Peyras J, Bartoli A, Mercier H, Dalle P (2007) Segmented AAMs improve person-independent face fitting. In: BMVC’07: Proceedings of the 18th British machine vision conferenceGoogle Scholar
  35. 35.
    Platt JC (1999) Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. advances in large margin classifiers. MIT Press, CambridgeGoogle Scholar
  36. 36.
    Roberts M, Cootes T, Adams J (2007) Vertebral morphometry: semi-automatic determination of detailed vertebral shape from DXA images using active appearance models. Invest Radiol 41:849–859CrossRefGoogle Scholar
  37. 37.
    Romdhani S, Gong S, Psarrou A (1999) A multi-view nonlinear active shape model using kernel PCA. In: BMVC’99: proceedings of the 10th British machine vision conference, pp 438–492Google Scholar
  38. 38.
    Saragih J (2011) Principal regression analysis. In: CVPR’11: proceedings of the IEEE computer society conference on computer vision and pattern recognitionGoogle Scholar
  39. 39.
    Saragih J, Goecke R (2006) Iterative error bound minimisation for AAM alignment. In: ICPR’06, vol 2, pp 1192–1195Google Scholar
  40. 40.
    Saragih J, Goecke R (2007) A nonlinear discriminative approach to AAM fitting. In: ICCV’07Google Scholar
  41. 41.
    Saragih J, Lucey S, Cohn J (2009) Subspace constrained mean-shift. In: ICCV’09Google Scholar
  42. 42.
    Saragih J, Lucey S, Cohn J (2010) Deformable face fitting using subspace constrained mean shifts. IJCVGoogle Scholar
  43. 43.
    Saragih JM, Lucey S, Cohn JF (2011) Real-time avatar animation from a single image. In: Automatic face gesture recognition and workshops (FG 2011), 2011 IEEE international conference on, pp 117–124Google Scholar
  44. 44.
    Silverman B (1986) Density estimation for statistics and data analysis. Chapman and Hall/CRC, LondonzbMATHGoogle Scholar
  45. 45.
    Theobald BS, Kruse GC, Bangham JA (2003) Towards a low bandwidth talking head using appearance models. J Image Vision Comput 21:1077–1205CrossRefGoogle Scholar
  46. 46.
    Tian Y, Narasimhan SG (2011) Globally optimal estimation of nonrigid image distortion. IJCVGoogle Scholar
  47. 47.
    Wang S, Wang Y, Li B (2006) Face decorating system based on improved active shape models. In: ACE ’06: proceedings of the ACM SIGCHI international conference on advances in computer entertainment technology, p 65Google Scholar
  48. 48.
    Wang Y, Lucey S, Cohn J (2008) Enforcing convexity for improved alignment with constrained local models. In: CVPR’08Google Scholar
  49. 49.
    Wei X, Yin L, Zhu Z, Ji Q (2004) Avatar-mediated face tracking and lip reading for human computer interaction. In: MULTIMEDIA’04: proceedings of the 12th annual international conference on multimedia, pp 500–503Google Scholar
  50. 50.
    Zhang J, Zhou S, McMillan L, Comaniciu D (2007) Joint real-time object detection and pose estimation using probabilistic Boosting Network. In: CVPR’07: proceedings of the IEEE computer society conference on computer vision and, pattern recognition, pp 1–8Google Scholar
  51. 51.
    Zhou X, Comaniciu D, Gupta A (2005) An information fusion framework for robust shape tracking. PAMI 27(1):115–129CrossRefGoogle Scholar
  52. 52.
    Zhou SK, Georgescu B, Zhou, XS, Comaniciu D (2005) Image based regression using boosting method. In: ICCV’05: proceedings of the 10th international conference on computer vision, vol 1, pp 541–548Google Scholar
  53. 53.
    Zhou S, Guo F, Park JH, Carneiro G, Jackson J, Brendel M, Simopoulos C, Otsuki J, Comaniciu D (2007) A probabilistic, hierarchical, and discriminant (PHD) framework for rapid and accurate detection of deformable anatomic structure. In: ICCV’07: proceedings of the 11th international conference on computer visionGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  1. 1.1 Technology CrtCSIROPullenvaleAustralia

Personalised recommendations