Skip to main content
SpringerLink
Log in

Facial Landmark Detection: A Literature Survey

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

The locations of the fiducial facial landmark points around facial components and facial contour capture the rigid and non-rigid facial deformations due to head movements and facial expressions. They are hence important for various facial analysis tasks. Many facial landmark detection algorithms have been developed to automatically detect those key points over the years, and in this paper, we perform an extensive review of them. We classify the facial landmark detection algorithms into three major categories: holistic methods, Constrained Local Model (CLM) methods, and the regression-based methods. They differ in the ways to utilize the facial appearance and shape information. The holistic methods explicitly build models to represent the global facial appearance and shape information. The CLMs explicitly leverage the global shape model but build the local appearance models. The regression based methods implicitly capture facial shape and appearance information. For algorithms within each category, we discuss their underlying theories as well as their differences. We also compare their performances on both controlled and in the wild benchmark datasets, under varying facial expressions, head poses, and occlusion. Based on the evaluations, we point out their respective strengths and weaknesses. There is also a separate section to review the latest deep learning based algorithms. The survey also includes a listing of the benchmark databases and existing software. Finally, we identify future research directions, including combining methods in different categories to leverage their respective strengths to solve landmark detection “in-the-wild”.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. In this paper, we refer Active Appearance Model to the model, independent of the fitting algorithms.

  2. For Ranjan et al. (2016), we list the landmark prediction model instead of the multi-task prediction model for fair comparison.

  3. Ibug 300-W database contains public available training images and private testing images. The training images include the annotations of public available databases and several newly collected images. Here, we name the newly collected images as Ibug 300-W database.

References

  • Ahlberg, J. (2002). An active model for facial feature tracking. EURASIP Journal on Advances in Signal Processing, 2002(6), 569,028.

    Article  Google Scholar 

  • Alabort-I-Medina, J., & Zafeiriou, S. (2014). Bayesian active appearance models. In IEEE conference on computer vision and pattern recognition.

  • Asthana, A., Zafeiriou, S., Cheng, S., & Pantic, M. (2013). Robust discriminative response map fitting with constrained local models. In IEEE conference on computer vision and pattern recognition, CVPR ’13, pp. 3444–3451.

  • Asthana, A., Zafeiriou, S., Cheng, S., & Pantic, M. (2014). Incremental face alignment in the wild. In IEEE conference on computer vision and pattern recognition, pp. 1859–1866.

  • Baker, S., Gross, R., & Matthews, I. (2002). Lucas-kanade 20 years on: A unifying framework: Part 3. International Journal of Computer Vision, 56, 221–255.

    Article  MATH  Google Scholar 

  • Baltrusaitis, T., Robinson, P., & Morency, L. P. (2014). Continuous conditional neural fields for structured regression. In European conference on computer vision (pp. 593–608). Springer.

  • Baltrušaitis, T., Robinson, P., & Morency, L. P. (2012). 3D constrained local model for rigid and non-rigid facial tracking. In IEEE conference on computer vision and pattern recognition.

  • Belhumeur, P., Jacobs, D., Kriegman, D., & Kumar, N. (2013). Localizing parts of faces using a consensus of exemplars. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(12), 2930–2940.

    Article  Google Scholar 

  • Belhumeur, P. N., Jacobs, D. W., Kriegman, D. J., & Kumar, N. (2011). Localizing parts of faces using a consensus of exemplars. In IEEE conference on computer vision and pattern recognition.

  • BioID. https://www.bioid.com/About/BioID-Face-Database. Accessed 30 August 2015.

  • Bourel, F., Chibelushi, C., & Low, A. (2000). Robust facial feature tracking. In British Machine Vision Conference, pp. 24.1–24.10.

  • Burgos-Artizzu, X. P., Perona, P., & Dollar, P. (2013). Robust face landmark estimation under occlusion. In IEEE international conference on computer vision, pp. 1513–1520.

  • Cao, X., Wei, Y., Wen, F., & Sun, J. (2014). Face alignment by explicit shape regression. International Journal of Computer Vision, 107, 177–190.

    Article  MathSciNet  Google Scholar 

  • Chen, D., Ren, S., Wei, Y., Cao, X., & Sun, J. (2014). Joint cascade face detection and alignment. In D. Fleet, T. Pajdla, B. Schiele, & T. Tuytelaars (Eds.), European Conference on Computer Vision, Lecture Notes in Computer Science (Vol. 8694, pp. 109–122). Berlin: Springer.

    Google Scholar 

  • Chrysos, G. G., Antonakos, E., Snape, P., Asthana, A., & Zafeiriou, S. (2017). A comprehensive performance evaluation of deformable face tracking "in-the-wild". International Journal of Computer Vision, 126, 198–232.

    Article  MathSciNet  Google Scholar 

  • Cootes, T., Walker, K., & Taylor, C. (2000). View-based active appearance models. In IEEE international conference on automatic face and gesture recognition, pp. 227–232.

  • Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001). Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 681–685.

    Article  Google Scholar 

  • Cootes, T. F., Ionita, M. C., Lindner, C., & Sauer, P. (2012). Robust and accurate shape model fitting using random forest regression voting. In European Conference on Computer Vision—Volume Part VII, pp. 278–291.

  • Cootes, T. F., Taylor, C. J., Cooper, D. H., & Graham, J. (1995). Active shape models their training and application. Computer Vision and Image Understanding, 61(1), 38–59.

    Article  Google Scholar 

  • Cosar, S., & Cetin, M. (2011). A graphical model based solution to the facial feature point tracking problem. Image and Vision Computing, 29(5), 335–350.

    Article  Google Scholar 

  • Cristinacce, D., & Cootes, T. (2007). Boosted regression active shape models. In British Machine Vision Conference, pp. 880–889.

  • Cristinacce, D., & Cootes, T. F. (2004). A comparison of shape constrained facial feature detectors. In International conference on automatic face and gesture recognition, pp. 375–380.

  • Cristinacce, D., & Cootes, T. F. (2006). Feature detection and tracking with constrained local models. In British Machine Vision Conference.

  • Dantone, M., Gall, J., Fanelli, G., & Gool, L. V. (2012). Real-time facial feature detection using conditional regression forests. In IEEE conference on computer vision and pattern recognition.

  • Donner, R., Reiter, M., Langs, G., Peloschek, P., & Bischof, H. (2006). Fast active appearance model search using canonical correlation analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(10), 1690–1694.

    Article  Google Scholar 

  • Edwards, G. J., Taylor, C. J., & Cootes, T. F. (1998). Interpreting face images using active appearance models. In IEEE international conference on face and gesture recognition (pp. 300–305). IEEE Computer Society.

  • Fan, H., & Zhou, E. (2016). Approaching human level facial landmark localization by deep learning. Image and Vision Computing, 47(C), 27–35.

    Article  Google Scholar 

  • Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010). Object detection with discriminatively trained part-based models. IEEE Transactions on Pattern Analysis and Machine Intellgence, 32(9), 1627–1645.

    Article  Google Scholar 

  • Feng, Z. H., Huber, P., Kittler, J., Christmas, W., & Wu, X. J. (2015). Random cascaded-regression copse for robust facial landmark detection. IEEE Signal Processing Letters, 22(1), 76–80.

    Article  Google Scholar 

  • Georghiades, A., Belhumeur, P., & Kriegman, D. (2001). From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 643–660.

    Article  Google Scholar 

  • Ghiasi, G., & Fowlkes, C. (2014). Occlusion coherence: Localizing occluded faces with a hierarchical deformable part model. In IEEE conference on computer vision and pattern recognition, pp. 1899–1906.

  • Girshick, R. (2015). Fast r-cnn. In The IEEE international conference on computer vision (ICCV).

  • Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. In The IEEE conference on computer vision and pattern recognition (CVPR).

  • Gou, C., Wu, Y., Wang, F. Y., & Ji, Q. (2016). Shape augmented regression for 3D face alignment, pp. 604–615. Cham.

  • Gower, J. C. (1975). Generalized procrustes analysis. Psychometrika, 40(1), 33–51.

    Article  MathSciNet  MATH  Google Scholar 

  • Gross, R., Matthews, I., & Baker, S. (2004). Appearance-based face recognition and light-fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(4), 449–465.

    Article  Google Scholar 

  • Gross, R., Matthews, I., & Baker, S. (2005). Generic vs. person specific active appearance models. Image Vision and Computing, 23(12), 1080–1093.

    Article  Google Scholar 

  • Gross, R., Matthews, I., Cohn, J., Kanade, T., & Baker, S. (2010). Multi-pie. Image Vision and Computing, 28(5), 807–813.

    Article  Google Scholar 

  • Gu, L., & Kanade, T. (2008). A generative shape regularization model for robust face alignment. In European Conference on Computer Vision: Part I (pp. 413–426). Berlin, Heidelberg: Springer.

  • Hansen, D. W., & Ji, Q. (2010). In the eye of the beholder: A survey of models for eyes and gaze. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(3), 478–500.

    Article  Google Scholar 

  • Heisele, B., Serre, T., & Poggio, T. (2007). A component-based framework for face detection and identification. International Journal of Computer Vision, 74(2), 167–181.

    Article  Google Scholar 

  • Hou, X., Li, S., Zhang, H., & Cheng, Q. (2001). Direct appearance models. In IEEE conference on computer vision and pattern recognition, Vol. 1.

  • Hsu, G. S., Chang, K. H., & Huang, S. C. (2015). Regressive tree structured model for facial landmark localization. In IEEE International conference on computer vision, pp. 3855–3861.

  • Hu, C., Feris, R., & Turk, M. (2003). Real-time view-based face alignment using active wavelet networks. In IEEE international workshop on analysis and modeling of faces and gestures, pp. 215–221.

  • Jeni, L. A., Cohn, J. F., & Kanade, T. (2015). Dense 3D face alignment from 2D videos in real-time. In 2015 11th IEEE international conference and workshops on automatic face and gesture recognition (FG). articles/Jeni15FG_ZFace.pdf.

  • Jiao, F., Li, S., Shum, H., & Schuurmans, D. (2003). Face alignment using statistical models and wavelet features. In IEEE conference on computer vision and pattern recognition.

  • Jones, M., & Poggio, T. (1998). Multidimensional morphable models: A framework for representing and matching object classes. International Journal of Computer Vision, 29(2), 107–131.

    Article  Google Scholar 

  • Jourabloo, A., & Liu, X. (2015). Pose-invariant 3D face alignment. In 2015 IEEE international conference on computer vision (ICCV), pp. 3694–3702.

  • Jourabloo, A., & Liu, X. (2016). Large-pose face alignment via CNN-based dense 3D model fitting. In IEEE conference on computer vision and pattern recognition. Las Vegas, NV.

  • Kanade, T., Cohn, J. F., & Tian, Y. Comprehensive database for facial expression analysis. In IEEE international conference on automatic face and gesture recognition, pp. 46–53.

  • Kazemi, V., & Sullivan, J. (2014). One millisecond face alignment with an ensemble of regression trees. In IEEE conference on computer vision and pattern recognition (CVPR), pp. 1867–1874.

  • Koestinger, M., Wohlhart, P., Roth, P. M., & Bischof, H. (2011). Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization. In First IEEE international workshop on benchmarking facial image analysis technologies.

  • Landmark annotation for AR database. http://personalpages.manchester.ac.uk/staff/timothy.f.cootes/data/tarfd_markup/tarfd_markup.html.

  • Le, V., Brandt, J., Lin, Z., Bourdev, L., & Huang, T. S. (2012). Interactive facial feature localization. In European Conference on Computer Vision—Volume Part III, pp. 679–692.

  • Levi, G., & Hassncer, T. (2015). Age and gender classification using convolutional neural networks. In 2015 IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp. 34–42.

  • Li, Y., Wang, S., Zhao, Y., & Ji, Q. (2013). Simultaneous facial feature tracking and facial expression recognition. IEEE Transactions on Image Processing, 22(7), 2559–2573.

    Article  Google Scholar 

  • Liang S Wu J, Liang, S., Wu, J., Weinberg, S. M., & Shapiro, L. G. (2013). Improved detection of landmarks on 3D human face data. In Annual international conference of the IEEE Engineering in Medicine and Biology Society.

  • Lopes, A. T., de Aguiar, E., Souza, A. F. D., & Oliveira-Santos, T. (2017). Facial expression recognition with convolutional neural networks: Coping with few data and the training sample order. Pattern Recognition, 61, 610–628.

    Article  Google Scholar 

  • Lucey, P., Cohn, J., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010). The extended Cohn-Kanade dataset (CK+): A complete dataset for action unit and emotion-specified expression. In IEEE conference on computer vision and pattern recognition workshops, pp. 94–101.

  • Martínez, A., & Benavente, R. (1998). The AR face database.

  • Martinez, B., Valstar, M. F., Binefa, X., & Pantic, M. (2013). Local evidence aggregation for regression-based facial point detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(5), 1149–1163.

    Article  Google Scholar 

  • Mathias, M., Benenson, R., Pedersoli, M., & Van Gool, L. (2014). Face detection without bells and whistles. In European Conference on Computer Vision.

  • Matthews, I., & Baker, S. (2004). Active appearance models revisited. International Journal of Computer Vision, 60(2), 135–164.

    Article  Google Scholar 

  • Messer, K., Matas, J., Kittler, J., & Jonsson, K. (1999). XM2VTSDB: The extended M2VTS database. In International conference on audio and video-based biometric person authentication, pp. 72–77.

  • Milborrow, S., & Nicolls, F. (2008). Locating facial features with an extended active shape model. In European Conference on Computer Vision: Part IV (pp. 504–513). Berlin, Heidelberg: Springer.

  • Murphy-Chutorian, E., & Trivedi, M. (2009). Head pose estimation in computer vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(4), 607–626.

    Article  Google Scholar 

  • Nickels, K., & Hutchinson, S. (2002). Estimating uncertainty in SSD-based feature tracking. Image and Vision Computing, 20, 47–58.

    Article  Google Scholar 

  • Pantic, M., & Rothkrantz, L. J. M. (2000). Automatic analysis of facial expressions: The state of the art. IEEE Transanctions on Pattern Analysis and Machine Intellgence, 22(12), 1424–1445.

    Article  Google Scholar 

  • Papazov, C., Marks, T., & Jones, M. (2015). Real-time head pose and facial landmark estimation from depth images using triangular surface patch features. In IEEE conference on computer vision and pattern recognition (CVPR) (pp. 4722–4730). IEEE.

  • Patacchiola, M., & Cangelosi, A. (2017). Head pose estimation in the wild using convolutional neural networks and adaptive gradient methods. Pattern Recognition, 71, 132–143.

    Article  Google Scholar 

  • Patrick Sauer, T. C., & Taylor, C. (2011). Accurate regression procedures for active appearance models. In British Machine Vision Conference.

  • Perakis, P., Passalis, G., Theoharis, T., & Kakadiaris, I. A. (2013). 3D facial landmark detection under large yaw and expression variations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(7), 1552–1564.

    Article  Google Scholar 

  • Phillips, P. J., Flynn, P. J., Scruggs, T., Bowyer, K. W., Chang, J., Hoffman, K., et al. (2005). Overview of the face recognition grand challenge. In IEEE conference on computer vision and pattern recognition, CVPR ’05 (pp. 947–954). Washington, DC: IEEE Computer Society.

  • Phillips, P. J., Moon, H., Rauss, P., & Rizvi, S. A. (1997). The FERET evaluation methodology for face-recognition algorithms. In IEEE conference on computer vision and pattern recognition, CVPR ’97 (pp. 137–143). Washington, DC: IEEE Computer Society.

  • Ranjan, R., Patel, V. M., & Chellappa, R. (2016). Hyperface: A deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. CoRR arXiv:1603.01249.

  • Ren, S., Cao, X., Wei, Y., & Sun, J. (2014). Face alignment at 3000 FPS via regressing local binary features. In IEEE conference on computer vision and pattern recognition (CVPR), pp. 1685–1692.

  • Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. In NIPS.

  • Sagonas, C., Antonakos, E., Tzimiropoulos, G., Zafeiriou, S., & Pantic, M. (2016). 300 faces in-the-wild challenge: Database and results. Image and Vision Computing, 47, 3–18. 300-W, the First Automatic Facial Landmark Detection in-the-Wild Challenge.

  • Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., & Pantic, M. (2013). 300 faces in-the-wild challenge: The first facial landmark localization challenge. In IEEE international conference on computer vision, 300 Faces in-the-Wild Challenge (300-W). Sydney, Australia.

  • Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., & Pantic, M. (2013a). A semi-automatic methodology for facial landmark annotation. In 2013 IEEE conference on computer vision and pattern recognition workshops, pp. 896–903.

  • Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., & Pantic, M. (2013b). A semi-automatic methodology for facial landmark annotation. In IEEE conference on computer vision and pattern recognition workshop. Portland Oregon, USA.

  • Saragih, J., & Gocke, R. (2009). Learning AAM fitting through simulation. Pattern Recognition, 42(11), 2628–2636.

    Article  MATH  Google Scholar 

  • Saragih, J., & Goecke, R. (2007). A nonlinear discriminative approach to AAM fitting. In International conference on computer vision, pp. 1–8.

  • Saragih, J. M., Lucey, S., & Cohn, J. F. (2011). Deformable model fitting by regularized landmark mean-shift. International Journal of Computer Vision, 91(2), 200–215.

    Article  MathSciNet  MATH  Google Scholar 

  • Schroff, F., Kalenichenko, D., & Philbin, J. (2015). Facenet: A unified embedding for face recognition and clustering.

  • Shen, J., Zafeiriou, S., Chrysos, G. G., Kossaifi, J., Tzimiropoulos, G., & Pantic, M. (2015). The first facial landmark tracking in-the-wild challenge: Benchmark and results. In The IEEE international conference on computer vision (ICCV) workshops.

  • Shen, X., Lin, Z., Brandt, J., & Wu, Y. (2013). Detecting and aligning faces by image retrieval. In IEEE conference on computer vision and pattern recognition.

  • Smith, B., Brandt, J., Lin, Z., & Zhang, L. (2014). Nonparametric context modeling of local appearance for pose- and expression-robust facial landmark localization. In IEEE conference on computer vision and pattern recognition, pp. 1741–1748.

  • Smith, B. M., & Zhang, L. (2014). Collaborative facial landmark localization for transferring annotations across datasets (pp. 78–93). Cham: Springer.

  • Sun, Y., Liang, D., Wang, X., & Tang, X. (2015). Deepid3: Face recognition with very deep neural networks. CoRR arXiv:1502.00873.

  • Sun, Y., Wang, X., & Tang, X. (2013). Deep convolutional network cascade for facial point detection. In IEEE conference on computer vision and pattern recognition, pp. 3476–3483.

  • Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2014). Deepface: Closing the gap to human-level performance in face verification.

  • Tong, Y., Liu, X., Wheeler, F. W., & Tu, P. H. (2012). Semi-supervised facial landmark annotation. Computer Vision and Image Understanding, 116(8), 922–935.

    Article  Google Scholar 

  • Tong, Y., Wang, Y., Zhu, Z., & Ji, Q. (2007). Robust facial feature tracking under varying face pose and facial expression. Pattern Recognition, 40(11), 3195–3208.

    Article  MATH  Google Scholar 

  • Tresadern, P., Sauer, P., & Cootes, T. (2010). Additive update predictors in active appearance models. In British Machine Vision Conference (pp. 91.1–91.12). BMVA Press.

  • Trigeorgis, G., Snape, P., Nicolaou, M. A., Antonakos, E., & Zafeiriou, S. (2016). Mnemonic descent method: A recurrent process applied for end-to-end face alignment. In IEEE conference on computer vision and pattern recognition (CVPR), pp. 4177–4187. Las Vegas, NV, USA.

  • Tulyakov, S., & Sebe, N. (2015). Regressing a 3D face shape from a single image. In IEEE international conference on computer vision, pp. 3748–3755.

  • Tzimiropoulos, G., i medina, J. A., Zafeiriou, S., Pantic, M. (2012). Generic active appearance models revisited. In Asian Conference on Computer Vision, pp. 650–663. Daejeon, Korea.

  • Tzimiropoulos, G., & Pantic, M. Optimization problems for fast aam fitting in-the-wild. In IEEE international conference on computer vision, pp. 593–600.

  • Tzimiropoulos, G., & Pantic, M. (2014). Gauss-Newton deformable part models for face alignment in-the-wild. In IEEE conference on computer vision and pattern recognition, pp. 1851–1858.

  • Uřičář, M., Franc, V., & Hlaváč, V. (2012). Detector of facial landmarks learned by the structured output SVM. In International conference on computer vision theory and applications (Vol. 1, pp. 547–556). Portugal.

  • Valstar, M., Martinez, B., Binefa, V., & Pantic, M. (2010). Facial point detection using boosted regression and graph models. In IEEE conference on computer vision and pattern recognition, pp. 13–18.

  • Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In IEEE conference on computer vision and pattern recognition, Vol. 1, pp. I-511–I-518.

  • Williams, O., Blake, A., & Cipolla, R. (2005). Sparse Bayesian learning for efficient visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8), 1292–1304.

    Article  Google Scholar 

  • Wu, Y., & Ji, Q. (2015). Discriminative deep face shape model for facial point detection. International Journal of Computer Vision, 113(1), 37–53.

    Article  MathSciNet  Google Scholar 

  • Wu, Y., & Ji, Q. (2015). Robust facial landmark detection under significant head poses and occlusion. In International conference on computer vision.

  • Wu, Y., & Ji, Q. (2016). Constrained joint cascade regression framework for simultaneous facial action unit recognition and facial landmark detection. In IEEE conference on computer vision and pattern recognition.

  • Wu, Y., Wang, Z., & Ji, Q. (2013). Facial feature tracking under varying facial expressions and face poses based on restricted Boltzmann machines. In IEEE conference on computer vision and pattern recognition, pp. 3452–3459.

  • Wu, Y., Wang, Z., & Ji, Q. (2014). A hierarchical probabilistic model for facial feature detection. In IEEE conference on computer vision and pattern recognition, pp. 1781–1788.

  • Xiong, X., & De la Torre Frade, F. (2013). Supervised descent method and its applications to face alignment. In IEEE international conference on computer vision and pattern recognition.

  • Xiong, X., & la Torre, F. D. (2015). Global supervised descent method. In IEEE conference on computer vision and pattern recognition, pp. 2664–2673.

  • Yan, S., Hou, X., Li, S. Z., Zhang, H., & Cheng, Q. (2003). Face alignment using view-based direct appearance models. Special issue on facial image processing, analysis and synthesis. International Journal of Imaging Systems and Technology, 13, 106–112.

    Article  Google Scholar 

  • Yang, H., & Patras, I. (2013). Privileged information-based conditional regression forest for facial feature detection. In IEEE international conference and workshops on automatic face and gesture recognition, pp. 1–6.

  • Yin, L., Chen, X., Sun, Y., Worm, T., & Reale, M. (2008). A high-resolution 3D dynamic facial expression database. FG 2,3,5.

  • Yu, X., Huang, J., Zhang, S., Yan, W., & Metaxas, D. (2013). Pose free facial landmark fitting via optimized part mixtures and cascaded deformable shape model. In IEEE international conference on computer vision.

  • Yu, X., Lin, Z., Brandt, J., & Metaxas, D. N. (2014). Consensus of regression for occlusion-robust facial feature localization. In D. Fleet, T. Pajdla, B. Schiele, & T. Tuytelaars (Eds.), European Conference on Computer Vision, Lecture Notes in Computer Science (Vol. 8692, pp. 105–118). Berlin: Springer.

    Google Scholar 

  • Zhang, C., & Zhang, Z. (2010). A survey of recent advances in face detection. Tech. Rep. MSR-TR-2010-66.

  • Zhang, J., Shan, S., Kan, M., & Chen, X. (2014). Coarse-to-fine auto-encoder networks (CFAN) for real-time face alignment. In European Conference on Computer Vision, Part II, pp. 1–16.

  • Zhang, Z., Luo, P., Loy, C., & Tang, X. (2014). Facial landmark detection by deep multi-task learning. In European Conference on Computer Vision, Part II, pp. 94–108.

  • Zhang, Z., Luo, P., Loy, C. C., & Tang, X. (2016). Learning deep representation for face alignment with auxiliary attributes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(5), 918–930.

    Article  Google Scholar 

  • Zhao, X., Kim, T. K., & Luo, W. (2014). Unified face analysis by iterative multi-output random forests. In IEEE conference on computer vision and pattern recognition, pp. 1765–1772.

  • Zhou, E., Fan, H., Cao, Z., Jiang, Y., & Yin, Q. (2013). Extensive facial landmark localization with coarse-to-fine convolutional network cascade. In IEEE international conference on computer vision workshops, pp. 386–391.

  • Zhu, S., Li, C., Change Loy, C., & Tang, X. (2015). Face alignment by coarse-to-fine shape searching. In IEEE conference on computer vision and pattern recognition.

  • Zhu, X., Lei, Z., Liu, X., Shi, H., Li, S. (2016). Face alignment across large poses: A 3D solution. In IEEE conference on computer vision and pattern recognition. Las Vegas, NV.

  • Zhu, X., & Ramanan, D. (2012). Face detection, pose estimation, and landmark localization in the wild. In IEEE conference on computer vision and pattern recognition, pp. 2879–2886.

Download references

Author information

Authors and Affiliations

  1. Department of Electrical, Computer, and Systems Engineering, Rensselaer Polytechnic Institute, 110 8th Street, Troy, NY, 12180-3590, USA

    Yue Wu & Qiang Ji

Authors
  1. Yue Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qiang Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qiang Ji.

Additional information

Communicated by T.E. Boult.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, Y., Ji, Q. Facial Landmark Detection: A Literature Survey. Int J Comput Vis 127, 115–142 (2019). https://doi.org/10.1007/s11263-018-1097-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-018-1097-z

Keywords

Search

Navigation