Abstract
Facial landmark localization plays a critical role in facial recognition and analysis. In this chapter, we first propose a novel cascaded backbone-branches fully convolutional neural network (BB-FCN) for rapidly and accurately localizing facial landmarks in unconstrained and cluttered settings. The proposed BB-FCN generates facial landmark response maps directly from raw images without any preprocessing. It follows a coarse-to-fine cascaded pipeline, which consists of a backbone network for roughly detecting the locations of all facial landmarks and one branch network for each type of detected landmark to further refine their locations (©[2019] IEEE. Reprinted, with permission, from [1].). At the end of this chapter, we also introduce the progress in face hallucination, a fundamental problem in the face analysis field that refers to generating a high-resolution facial image from a low-resolution input image (©[2019] IEEE. Reprinted, with permission, from [2].).
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
L. Liu, G. Li, Y. Xie, Y. Yu, Q. Wang, L. Lin, Facial landmark machines: a backbone-branches architecture with progressive representation learning. IEEE Trans. Multimedia. https://doi.org/10.1109/TMM.2019.2902096
Q. Cao, L. Lin, Y. Shi, X. Liang, G. Li, Attention-aware face hallucination via deep reinforcement learning, in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, pp. 1656–1664 (2017). https://doi.org/10.1109/CVPR.2017.180
P. Luo, X. Wang, X. Tang, A deep sum-product architecture for robust facial attributes analysis, in ICCV, pp. 2864–2871 (2013)
C. Lu, X. Tang, Surpassing human-level face verification performance on lfw with gaussianface, in AAAI (2015)
L. Liu, C. Xiong, H. Zhang, Z. Niu, M. Wang, S. Yan, Deep aging face verification with large gaps. TMM 18(1), 64–75 (2016)
Z. Zhu, P. Luo, X. Wang, X. Tang, Deep learning identity-preserving face space, in ICCV, pp. 113–120 (2013)
C. Ding, D. Tao, Robust face recognition via multimodal deep face representation. TMM 17(11), 2049–2058 (2015)
Y. Li, L. Liu, L. Lin, Q. Wang, Face recognition by coarse-to-fine landmark regression with application to atm surveillance, in CCCV (Springer, 2017), pp. 62–73
P. Viola, M. Jones, Rapid object detection using a boosted cascade of simple features, in CVPR, vol. 1. IEEE, pp. I–511 (2001)
X. Zhu, D. Ramanan, Face detection, pose estimation, and landmark localization in the wild, in CVPR (IEEE, 2012), pp. 2879–2886
Z. Yan, H. Zhang, R. Piramuthu, V. Jagadeesh, D. DeCoste, W. Di, Y. Yu, Hd-cnn: hierarchical deep convolutional neural networks for large scale visual recognition, in ICCV, pp. 2740–2748 (2015)
M. Köstinger, P. Wohlhart, P.M. Roth, H. Bischof, Annotated facial landmarks in the wild: a large-scale, real-world database for facial landmark localization, in ICCV Workshops (IEEE, 2011), pp. 2144–2151
Z. Zhang, P. Luo, C.C. Loy, X. Tang, Facial landmark detection by deep multi-task learning, in ECCV (Springer, 2014), pp. 94–108
X. Burgos-Artizzu, P. Perona, P. Dollár, Robust face landmark estimation under occlusion, in ICCV, pp. 1513–1520 (2013)
X. Cao, Y. Wei, F. Wen, J. Sun, Face alignment by explicit shape regression. IJCV 107(2), 177–190 (2014)
X. Yu, J. Huang, S. Zhang, W. Yan, D. Metaxas, Pose-free facial landmark fitting via optimized part mixtures and cascaded deformable shape model, in ICCV, pp. 1944–1951 (2013)
X. Xiong, F. Torre, Supervised descent method and its applications to face alignment, in CVPR, pp. 532–539 (2013)
K. Zhang, Z. Zhang, Z. Li, Y. Qiao, Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process. Lett. 23(10), 1499–1503 (2016)
S. Xiao, J. Feng, J. Xing, H. Lai, S. Yan, A. Kassim, Robust facial landmark detection via recurrent attentive-refinement networks, in ECCV (Springer, 2016), pp. 57–72
Z. Liu, P. Luo, X. Wang, X. Tang, Deep learning face attributes in the wild, in ICCV, pp. 3730–3738 (2015)
Z. Zhang, P. Luo, C.C. Loy, X. Tang, Learning deep representation for face alignment with auxiliary attributes. IEEE Trans. Pattern Anal. Mach. Intell. 38(5), 918–930 (2016)
E. Zhou, Z. Cao, Q. Yin, Naive-deep face recognition: touching the limit of lfw benchmark or not? arXiv preprint arXiv:1501.04690 (2015)
E. Zhou, H. Fan, Z. Cao, Y. Jiang, Q. Yin, Learning face hallucination in the wild, in AAAI, pp. 3871–3877 (2015)
S. Zhu, S. Liu, C.C. Loy, X. Tang, Deep cascaded bi-network for face hallucination. arXiv preprint arXiv:1607.05046 (2016)
C. Liu, H.-Y. Shum, W.T. Freeman, Face hallucination: theory and practice. Int. J. Comput. Vis. 75(1), 115–134 (2007)
C. Dong, C.C. Loy, K. He, X. Tang, Learning a deep convolutional network for image super-resolution, in ECCV, pp. 184–199 (2014)
J. Najemnik, W.S. Geisler, Optimal eye movement strategies in visual search. Nature 434(7031), 387–391 (2005)
Y. Sun, D. Liang, X. Wang, X. Tang, Deepid3: face recognition with very deep neural networks. arXiv preprint arXiv:1502.00873 (2015)
J.C. Caicedo, S. Lazebnik, Active object localization with deep reinforcement learning, in ICCV, pp. 2488–2496 (2015)
K. Gregor, I. Danihelka, A. Graves, D.J. Rezende, D. Wierstra, DRAW: a recurrent neural network for image generation, in ICLR, pp. 1462–1471 (2015)
D. Silver, A. Huang, C.J. Maddison, A. Guez, L. Sifre et al., Mastering the game of go with deep neural networks and tree search. Nature 529, 484–503 (2016)
J. Kim, J.K. Lee, K.M. Lee, Accurate image super-resolution using very deep convolutional networks (2016)
O. Tuzel, Y. Taguchi, J.R. Hershey, Global-local face upsampling network. arXiv preprint arXiv:1603.07235 (2016)
S. Gu, W. Zuo, Q. Xie, D. Meng, X. Feng, L. Zhang, Convolutional sparse coding for image super-resolution, in ICCV, pp. 1823–1831 (2015)
R.J. Williams, Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8(3), 229–256 (1992)
V. Mnih, N. Heess, A. Graves, K. kavukcuoglu, Recurrent models of visual attention, in NIPS, pp. 2204–2212 (2014)
O. Jesorsky, K.J. Kirchberg, R. Frischholz, Robust face detection using the hausdorff distance, in AVBPA, pp. 90–95 (2001)
G.B. Huang, M. Ramesh, T. Berg, E. Learned-Miller, Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, Amherst, October 2007
G.B. Huang, V. Jain, E. Learned-Miller, Unsupervised joint alignment of complex images, in ICCV (2007)
C.-Y. Yang, S. Liu, M.-H. Yang, Structured face hallucination, in CVPR, pp. 1099–1106 (2013)
X. Ma, J. Zhang, C. Qi, Hallucinating face by position-patch. Pattern Recogn. 43(6), 2224–2236 (2010)
D.P. Kingma, J. Ba. Adam: amethod for stochastic optimization, in ICLR (2015)
T. Chen, L. Lin, L. Liu, X. Luo, X. Li, Disc: deep image saliency computing via progressive representation learning. TNNLS 27(6), 1135–1149 (2016)
L. Liu, H. Wang, G. Li, W. Ouyang, L. Lin, Crowd counting using deep recurrent spatial-aware network, in IJCAI (2018)
L. Liu, R. Zhang, J. Peng, G. Li, B. Du, L. Lin, Attentive crowd flow machines, in ACM MM (ACM, 2018), pp. 1553–1561
Y. Sun, X. Wang, X. Tang, Deep convolutional network cascade for facial point detection, in CVPR, pp. 3476–3483 (2013)
R. Weng, J. Lu, Y.-P. Tan, J. Zhou, Learning cascaded deep auto-encoder networks for face alignment. TMM 18(10), 2066–2078 (2016)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Lin, L., Zhang, D., Luo, P., Zuo, W. (2020). Face Localization and Enhancement. In: Human Centric Visual Analysis with Deep Learning. Springer, Singapore. https://doi.org/10.1007/978-981-13-2387-4_3
Download citation
DOI: https://doi.org/10.1007/978-981-13-2387-4_3
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-2386-7
Online ISBN: 978-981-13-2387-4
eBook Packages: Computer ScienceComputer Science (R0)