Multimedia Tools and Applications

, Volume 76, Issue 6, pp 8677–8694 | Cite as

Face alignment under occlusion based on local and global feature regression

  • Songrui Guo
  • Guanghua Tan
  • Huawei Pan
  • Lin Chen
  • Chunming Gao


Shape alignment or estimation under occlusion is one of the most challenging tasks in computer vision field. Most previous works treat occlusion as noises or part models, which usually lead to low accuracy or inefficiencies. This paper proposes an efficient and accurate regression-based algorithm for face alignment. In this framework, local and global regressions are iteratively used to train a series of random forests in a cascaded manner. In training and testing process, each step consists of two layers. In the first layer, a set of highly discriminative local features are extracted from local regions according to locality principle. The regression forests are trained for each facial landmark independently using those local features. Then the leaf node of the regression tree is encoded by histogram statistic method and the final shape is estimated by a linear regression matrix. In the second layer, our proposed global features are generated. Then we use those features to train a random fern to keep the global shape constraints. Experiments show that our method has a high speed, but same or slightly lower accuracy than state of the art methods under occlusion condition. In order to gain a higher accuracy we use multi-random shape for initialization, which may slightly reduce the calculation efficiency as a trade-off.


Occlusion Local regression Global regression Shape-indexed feature Face alignment 


  1. 1.
    Belhumeur PN, Jacobs DW, Kriegman DJ, Kumar N (2013) Localizing parts of faces using a consensus of exemplars. IEEE Int Conf Comput Vis 35(12):545–552Google Scholar
  2. 2.
    Breiman L (2001) Random forests. Mach Learn 45:5–32CrossRefMATHGoogle Scholar
  3. 3.
    Burgos-Atrizzu XP, Perona P, Dollar P (2013) Robust face landmark estimation under occlusion. IEEE Int Conf Comput Vis:1–6Google Scholar
  4. 4.
    Cao X, Wei Y, Wen F, Sun J (2012) Face alignment by explicit shape regression. Int J Comput Vis 107(2):2887–2894MathSciNetGoogle Scholar
  5. 5.
    Cao C, Weng Y, Lin S, Zhou K (2013) 3d shape regression for real-time facial animation. ACM Trans Graph 32(4):96–96CrossRefMATHGoogle Scholar
  6. 6.
    Cootes TF, Edwards GJ, Taylor CJ (1998) Active appearance models. Springer Berlin Heidelberg 1407(6):484–498Google Scholar
  7. 7.
    Cootes T, Taylor C (1995) Active shape models. Br Mach Vis Conf (BMVC):266–275Google Scholar
  8. 8.
    Cristinacce D, Cootes T (2007) Boosted regression active shape models. Br Mach Vis Conf (BMVC). doi:10.5244/C.21.79 MATHGoogle Scholar
  9. 9.
    Cui Y, Zhang J, Guo D, Jin Z (2015) Robust facial landmark localization using classified random ferns and pose-based initialization. Signal Process 110:46–53CrossRefGoogle Scholar
  10. 10.
    Dantone M, Gall J, Fanelli G, Van Gool L (2012) Real-time facial feature detection using conditional regression forests. IEEE Conf Comput Vis Pattern Recog 157(10):2578–2585Google Scholar
  11. 11.
    Ding L, Martinez A (2010) Features vs. context: an approach for precise and detailed det. and delineation of faces and facial features. IEEE Trans Pattern Anal Mach Intell (PAMI) 32(11):2022–2038CrossRefGoogle Scholar
  12. 12.
    Dollar P, Welinder P, Perona P (2010) Cascaded pose regression. IEEE Conf Comput Vis Pattern Recog 238(6):1078–1085Google Scholar
  13. 13.
    Ghiasi G, Fowlkes CC (2014) Occlusion coherence: localizing occluded faces with a hierarchical deformable part model. IEEE Conf Comput Vis Pattern Recog:1899–1906Google Scholar
  14. 14.
    Gonzalez RC, Woods RE (2007) Digital image processing [M], 3rd edn. [S.1.], Prentice HallGoogle Scholar
  15. 15.
    Kass M, Witkin A, Terzopoulos D (1988) Snakes: active contour models. Int J Comput Vis (IJCV) 1(4):321–331CrossRefMATHGoogle Scholar
  16. 16.
    Kazemi V, Sullivan J (2014) One millisecond face alignment with an ensemble of regression trees. IEEE Conf Comput Vis Pattern Recog:1867–1874Google Scholar
  17. 17.
    Le V, Brandt J, Lin Z, Bourdev L, Huang TS (2012) Interactive facial feature localization. Springer Berlin Heidelberg 7574(1):679–692Google Scholar
  18. 18.
    Matthews I, Baker S (2004) Active appearance models revisited. Int J Comput Vis (IJCV) 60(2):135–164CrossRefGoogle Scholar
  19. 19.
    Milborrow S, Nicolls F (2008) Locating facial features with an extended active shape model. Eur Conf Comput Vis: Part IV 5305:504–513Google Scholar
  20. 20.
    Ren S, Cao X, Wei Y, Sun J (2014) Face Alignment at 3000 FPS via regressing local binary features. IEEE Conf Comput Vis Pattern Recog:1685–1692Google Scholar
  21. 21.
    Sauer CP, Cootes T (2011) Accurate regression procedures for active appearance models. Br Mach Vis Conf (BMVC) 1(6):681–685Google Scholar
  22. 22.
    Tzimiropoulos G, Pantic M (2013) Optimization problem for fast AAM fitting in the wild. IEEE Int Conf Comput Vis 120(10):593–600Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Songrui Guo
    • 1
  • Guanghua Tan
    • 1
  • Huawei Pan
    • 1
  • Lin Chen
    • 1
  • Chunming Gao
    • 1
  1. 1.College of Information Science and EngineeringHunan UniversityChangshaChina

Personalised recommendations