Skip to main content
SpringerLink
Log in

Learning from Longitudinal Face Demonstration—Where Tractable Deep Modeling Meets Inverse Reinforcement Learning

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

This paper presents a novel subject-dependent deep aging path (SDAP), which inherits the merits of both generative probabilistic modeling and inverse reinforcement learning to model the facial structures and the longitudinal face aging process of a given subject. The proposed SDAP is optimized using tractable log-likelihood objective functions with convolutional neural networks (CNNs) based deep feature extraction. Instead of applying a fixed aging development path for all input faces and subjects, SDAP is able to provide the most appropriate aging development path for individual subject that optimizes the reward aging formulation. Unlike previous methods that can take only one image as the input, SDAP further allows multiple images as inputs, i.e. all information of a subject at either the same or different ages, to produce the optimal aging path for the given subject. Finally, SDAP allows efficiently synthesizing in-the-wild aging faces. The proposed model is experimented in both tasks of face aging synthesis and cross-age face verification. The experimental results consistently show SDAP achieves the state-of-the-art performance on numerous face aging databases, i.e. FG-NET, MORPH, aging faces in the wild (AGFW), and cross-age celebrity dataset (CACD). Furthermore, we also evaluate the performance of SDAP on large-scale Megaface challenge to demonstrate the advantages of the proposed solution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. The results of other methods are provided in MegaFace website.

References

  • Agustsson. E., Timofte, R., & Van Gool, L. (2017). Anchored regression networks applied to age estimation and super resolution. In IEEE International Conference on Computer Vision (ICCV), (pp. 1652–1661).

  • Angulu, R., & Tapamo, J. R. (2018). Adewumi AO (2018) Age estimation via face images: a survey. EURASIP Journal on Image and Video Processing, 1, 42.

    Article  Google Scholar 

  • Antipov, G., Baccouche, M., & Dugelay, JL. (2017). Face aging with conditional generative adversarial networks. arXiv preprint arXiv:1702.01983

  • Attia, A., & Dayan, S. (2018). Global overview of imitation learning. arXiv preprint arXiv:1801.06503

  • Burt, D. M., & Perrett, D. I. (1995). Perception of age in adult caucasian male faces: Computer graphic manipulation of shape and colour information. In Proceedings of the Royal Society of London B: Biological Sciences, 259(1355), (pp. 137–143).

  • Chang, KY., Chen, CS., & Hung, YP. (2011). Ordinal hyperplanes ranker with cost sensitivities for age estimation. In CVPR, IEEE, (pp. 585–592).

  • Chen, BC., Chen, CS., & Hsu, WH. (2014). Cross-age reference coding for age-invariant face recognition and retrieval. In ECCV, (pp. 768–783).

  • Chen, C., Yang, W., Wang, Y., Ricanek, K., & Luu, K. (2011). Facial feature fusion and model selection for age estimation. In IEEE Conference on Automatic Face and Gesture Recognition(FG), (pp. 1–7).

  • Chen, S., Zhang, C., Dong, M., Le, J., & Rao, M. (2017). Using ranking-cnn for age estimation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

  • Choi, S. E., Lee, Y. J., Lee, S. J., Park, K. R., & Kim, J. (2011). Age estimation using a hierarchical classifier based on global and local facial features. Pattern Recognition, 44(6), 1262–1281.

    Article  MATH  Google Scholar 

  • Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001). Active appearance models. IEEE Transactions on Pattern Analysis & Machine Intelligence, 6, 681–685.

    Article  Google Scholar 

  • Duan, Y., Chen, X., Houthooft, R., Schulman, J., & Abbeel, P. (2016). Benchmarking deep reinforcement learning for continuous control. In International Conference on Machine Learning, (pp. 1329–1338).

  • Duong, CN., Luu, K., Quach, KG., & Bui, TD. (2015). Beyond principal components: Deep Boltzmann machines for face modeling. In CVPR, IEEE, (pp. 4786–4794).

  • Duong, CN., Luu, K., Quach, KG., Bui, TD. (2016). Longitudinal face modeling via temporal deep restricted boltzmann machines. In CVPR, (pp. 5772–5780).

  • Duong, CN., Quach, KG., Luu, K., Le, N., & Savvides, M. (2017). Temporal non-volume preserving approach to facial age-progression and age-invariant face recognition. In The IEEE International Conference on Computer Vision (ICCV), (pp. 3755–3763).

  • Duong, C. N., Quach, K. G., Luu, K., Le, H. B., & Ricanek, K. (2011). Fine tuning age estimation with global and local facial features. In IEEE International Conference on Acoustics (pp. 2032–2035). Speech and Signal Processing: Prague, Czech Republic.

  • FGNET Aging Database (2004). http://www.fgnet.rsunit.com.

  • Finn, C., Levine, S., & Abbeel, P. (2016). Guided cost learning: Deep inverse optimal control via policy optimization. In International Conference on Machine Learning, (pp. 49–58).

  • Fu, Y., Guo, G., & Huang, T. S. (2010). Age synthesis and estimation via faces: A survey. IEEE transactions on pattern analysis and machine intelligence, 32(11), 1955–1976.

    Article  Google Scholar 

  • Geng, X., Smith-Miles, K., & Zhou, ZH. (2008). Facial age estimation by nonlinear aging pattern subspace. In Proceedings of the 16th ACM international conference on Multimedia, ACM, (pp. 721–724).

  • Geng, X., Zhou, Z. H., & Smith-Miles, K. (2007). Automatic age estimation based on facial aging patterns. PAMI, 29(12), 2234–2240.

    Article  Google Scholar 

  • Guo, G., & Wang, X. (2012). A study on human age estimation under facial expression changes. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, IEEE, (pp. 2547–2553).

  • Guo, G., & Zhang, C. (2014). A study on cross-population age estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (pp. 4257–4263).

  • Guo, G., Mu, G., Fu, Y., & Huang, TS. (2009). Human age estimation using bio-inspired features. In Computer Vision and Pattern Recognition, 2009. CVPR, IEEE Conference on 2009, (pp. 112–119).

  • Guo, G., Fu, Y., Dyer, C. R., & Huang, T. S. (2008). Image-based human age estimation by manifold learning and locally adjusted robust regression. IEEE Transactions on Image Processing, 17(7), 1178–1188.

    Article  MathSciNet  Google Scholar 

  • Han, H., Jain, A. K., Wang, F., Shan, S., & Chen, X. (2018). Heterogeneous face attribute estimation: A deep multi-task learning approach. IEEE transactions on pattern analysis and machine intelligence, 40(11), 2597–2609.

    Article  Google Scholar 

  • Huo, Z., Yang, X., Xing, C., Zhou, Y., Hou, P., Lv, J., & Geng, X. (2016). Deep age distribution learning for apparent age estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, (pp. 17–24).

  • Kemelmacher Shlizerman, I., Suwajanakorn, S., & Seitz, SM. (2014). Illumination-aware age progression. In CVPR, IEEE, (pp. 3334–3341).

  • Kemelmacher-Shlizerman, I., Seitz, SM., Miller, D., Brossard, E. (2016). The megaface benchmark: 1 million faces for recognition at scale. In CVPR, (pp. 4873–4882).

  • Kwon, Y. H., & da Vitoria, Lobo N. (1999). Age classification from facial images. CVIU, 74(1), 1–21.

    Google Scholar 

  • Lanitis, A., Draganova, C., & Christodoulou, C. (2004). Comparing different classifiers for automatic age estimation. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 34(1), 621–628.

    Article  Google Scholar 

  • Lanitis, A., Taylor, C. J., & Cootes, T. F. (2002). Toward automatic simulation of aging effects on face images. PAMI, 24(4), 442–455.

    Article  Google Scholar 

  • Le, T. H. N., Seshadri, K., Luu, K., & Savvides, M. (2015). Facial aging and asymmetry decomposition based approaches to identification of twins. Pattern Recognition, 48(12), 3843–3856.

    Article  Google Scholar 

  • Levine, S., & Abbeel, P. (2014). Learning neural network policies with guided policy search under unknown dynamics. In Advances in Neural Information Processing Systems, (pp. 1071–1079).

  • Li, K., Xing, J., Su, C., Hu, W., Zhang, Y., & Maybank, S. (2018). Deep cost-sensitive and order-preserving feature learning for cross-population age estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (pp. 399–408).

  • Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., & Song, L. (2017). Sphereface: Deep hypersphere embedding for face recognition. arXiv preprint arXiv:1704.08063.

  • Lou, Z., Alnajar, F., Alvarez, J. M., Hu, N., & Gevers, T. (2018). Expression-invariant age estimation using structured learning. IEEE transactions on pattern analysis and machine intelligence, 40(2), 365–375.

    Article  Google Scholar 

  • Luu, K. (2010). Computer approaches for face aging problems. In The 23th Canadian Conference on Artificial Intelligence (CAI), Ottawa, Canada.

  • Luu, K., Ricanek Jr, K., Bui, TD., & Suen, CY. (2009a). Age estimation using active appearance models and support vector machine regression. In BTAS’09, IEEE, (pp. 1–5).

  • Luu, K., Suen, C., Bui, T., & Ricanek, JK. (2009b). Automatic child-face age-progression based on heritability factors of familial faces. In BIdS, IEEE, (pp. 1–6).

  • Niu, Z., Zhou, M., Wang, L., Gao, X., & Hua, G. (2016). Ordinal regression with multiple output cnn for age estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition, (pp. 4920–4928).

  • Pan, H., Han, H., Shan, S., & Chen, X. (2018). Mean-variance loss for deep age estimation from a face. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (pp. 5285–5294).

  • Patterson, E., Ricanek, K., Albert, M., & Boone, E. (2006). Automatic representation of adult aging in facial images. In Proceedings IASTED International Conference Visualization, Imaging, and Image Processing, (pp. 171–176).

  • Ricanek Jr, K., & Tesafaye, T. (2006). Morph: A longitudinal image database of normal adult age-progression. In FGR 2006, IEEE, (pp. 341–345).

  • Rothe, R., Timofte, R., & Gool, L. V. (2016). Deep expectation of real and apparent age from a single image without facial landmarks. International Journal of Computer Vision, 126(2–4), 144–157. https://doi.org/10.1007/s11263-016-0940-3.

    MathSciNet  Google Scholar 

  • Rowland, D., Perrett, D., et al. (1995). Manipulating facial appearance through shape and color. Computer Graphics and Applications, IEEE, 15(5), 70–76.

    Article  Google Scholar 

  • Shen, W., Guo, Y., Wang, Y., Zhao, K., Wang, B., & Yuille, AL. (2018). Deep regression forests for age estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (pp. 2304–2313).

  • Shu, X., Tang, J., Lai, H., Liu, L., & Yan, S. (2015). Personalized age progression with aging dictionary. In ICCV, (pp. 3970–3978).

  • Suo, J., Chen, X., Shan, S., Gao, W., & Dai, Q. (2012). A concatenational graph evolution aging model. PAMI, 34(11), 2083–2096.

    Article  Google Scholar 

  • Suo, J., Zhu, S. C., Shan, S., & Chen, X. (2010). A compositional and dynamic model for face aging. PAMI, 32(3), 385–401.

    Article  Google Scholar 

  • Taylor, GW., & Hinton, GE. (2009). Factored conditional restricted boltzmann machines for modeling motion style. In Proceedings of the 26th Annual International Conference on Machine Learning, ACM, (pp. 1025–1032).

  • Tsai, M. H., Liao, Y. K., & Lin, I. C. (2014). Human face aging with guided prediction and detail synthesis. Multimedia Tools and Applications, 72(1), 801–824.

    Article  Google Scholar 

  • Wang, W., Cui, Z., Yan, Y., Feng, J., Yan, S., Shu, X., & Sebe, N. (2016). Recurrent face aging. In CVPR, (pp. 2378–2386).

  • Wang, F., Han, H., Shan, S., & Chen, X. (2017). Deep multi-task learning for joint prediction of heterogeneous face attributes. In Automatic Face & Gesture Recognition (FG 2017), 2017 12th IEEE International Conference on, IEEE, (pp. 173–179).

  • Wang, Z., X Tang, WL., & Gao, S. (2018). Face aging with identity-preserved conditional generative adversarial networks. In CVPR.

  • Xu, J., Luu, K., Savvides, M., Bui, TD., & Suen, CY. (2011). Investigating age invariant face recognition based on periocular biometrics. In International Joint Conference on Biometrics (IJCB), IEEE.

  • Xu, F., Luu, K., & Savvides, M. (2015). Spartans: Single-sample periocular-based alignment-robust recognition technique applied to non-frontal scenarios. Trans on Image Processing (TIP), 24, 4780–4795.

    Article  MathSciNet  MATH  Google Scholar 

  • Yan, S., Liu, M., & Huang, TS. (2008). Extracting age information from local spatially flexible patches. In Acoustics, Speech and Signal Processing, 2008. ICASSP 2008. IEEE International Conference on, IEEE, (pp. 737–740).

  • Yang, X., Gao, BB., Xing, C., Huo, ZW., Wei, XS., Zhou, Y., Wu, J., & Geng, X. (2015). Deep label distribution learning for apparent age estimation. In Proceedings of the IEEE International Conference on Computer Vision Workshops, (pp. 102–108).

  • Yang, H., Huang, D., Wang, Y., Jain, AK. (2018). Learning face age progression: A pyramid architecture of gans. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

  • Yang, H., Huang, D., Wang, Y., Wang, H., & Tang, Y. (2016). Face aging effect simulation using hidden factor analysis joint sparse representation. TIP, 25(6), 2493–2507.

    MathSciNet  MATH  Google Scholar 

  • Yan, S., Wang, H., Fu, Y., Yan, J., Tang, X., & Huang, T. S. (2009). Synchronized submanifold embedding for person-independent pose estimation and beyond. IEEE Transactions on Image Processing, 18(1), 202–210.

    Article  MathSciNet  MATH  Google Scholar 

  • Yi, D., Lei, Z., Li, SZ. (2014). Age estimation by multi-scale convolutional network. In Asian conference on computer vision, Springer, (pp. 144–158).

  • Zhang, Y., & Yeung, DY. (2010). Multi-task warped Gaussian process for personalized age estimation. In CVPR, IEEE, (pp. 2622–2629).

  • Zhang, Z., Song, Y., & Qi, H. (2017). Age progression/regression by conditional adversarial autoencoder. In CVPR.

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Software Engineering, Concordia University, Montréal, Québec, Canada

    Chi Nhan Duong, Kha Gia Quach & Tien D. Bui

  2. Computer Science and Computer Engineering Department, University of Arkansas, Fayetteville, AR, USA

    Khoa Luu

  3. CyLab Biometrics Center and the Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, PA, USA

    T. Hoang Ngan Le & Marios Savvides

Authors
  1. Chi Nhan Duong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kha Gia Quach
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Khoa Luu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. T. Hoang Ngan Le
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marios Savvides
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tien D. Bui
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chi Nhan Duong.

Additional information

Communicated by Dr. Rama Chellappa, Dr. Xiaoming Liu, Dr. Tae-Kyun Kim, Dr. Fernando De la Torre and Dr. Chen Change Loy.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 5255 KB)

Supplementary material 2 (mp4 119181 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Duong, C.N., Quach, K.G., Luu, K. et al. Learning from Longitudinal Face Demonstration—Where Tractable Deep Modeling Meets Inverse Reinforcement Learning. Int J Comput Vis 127, 957–971 (2019). https://doi.org/10.1007/s11263-019-01165-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-019-01165-5

Keywords

Search

Navigation