Skip to main content
SpringerLink
Log in

3D face dense reconstruction based on sparse points using probabilistic principal component analysis

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Reconstructing 3D face from sparse points is an ill-posed problem. While there already exits available solutions addressing this problem, to our knowledge, we propose a better-performed approach which can robustly reconstruct fine 3D face shape. Our method includes two modules: face model establishment based on probabilistic principal component analysis (PPCA) trained in an unsupervised manner to learn transformation between landmarks and point cloud in their low-dimensional representation, and 3D face reconstruction based on learned relation between them to reconstruct fine face shape. Overall, our method considers the probability of face shape and learns more useful information of 3D face shape. We compare our method with 3 typical and state-of-the-art methods on 2 datasets and the effectiveness of our method is demonstrated generally. Further experiments on datasets with noise of different intensities show the stability of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Aji OP, Purnama IKE, Yuniarno EM (2019) Facial model deformation based on landmarks using laplacian. In: 2019 International conference on computer engineering, network, and intelligent multimedia (CENIM), IEEE, pp 1–6

  2. Blanz V, Vetter T (1999) A morphable model for the synthesis of 3d faces. In: Proceedings of the 26th annual conference on Computer graphics and interactive techniques, pp 187–194

  3. Cao C, Weng Y, Zhou S, Tong Y, Zhou K (2013) Facewarehouse: A 3d facial expression database for visual computing. IEEE Transactions on Visualization and Computer Graphics 20(3):413–425

    Google Scholar 

  4. Chen Y, Wu F, Wang Z, Song Y, Ling Y, Bao L (2020) Self-supervised learning of detailed 3d face reconstruction. IEEE Transactions on Image Processing 29:8696–8705

    Article  Google Scholar 

  5. Deng Q, Zhou M, Shui W, Wu Z, Ji Y, Bai R (2011) A novel skull registration based on global and local deformations for craniofacial reconstruction. Forensic Science International 208(1–3):95–102

    Article  Google Scholar 

  6. Duan F, Huang D, Tian Y, Lu K, Wu Z, Zhou M (2015) 3d face reconstruction from skull by regression modeling in shape parameter spaces. Neurocomputing 151:674–682

    Article  Google Scholar 

  7. Feng Y, Wu F, Shao X, Wang Y, Zhou X (2018) Joint 3d face reconstruction and dense alignment with position map regression network. In: Proceedings of the european conference on computer vision (ECCV), pp 534–551

  8. Ferrari C, Berretti S, Pala P, Del Bimbo A (2018) Learning 3dmm deformation coefficients for rendering realistic expression images. In: International conference on smart multimedia, Springer, pp 320–333

  9. Geraci M, Farcomeni A (2016) Probabilistic principal component analysis to identify profiles of physical activity behaviours in the presence of non-ignorable missing data. Journal of the Royal Statistical Society: Series C (Applied Statistics) 65(1):51–75

    MathSciNet  Google Scholar 

  10. Gruszczynski M, Klos A, Bogusz J (2019) A filtering of incomplete gnss position time series with probabilistic principal component analysis. In: Geodynamics and earth tides observations from global to micro scale, Springer, pp 247–273

  11. Gu M, Shen W (2020) Generalized probabilistic principal component analysis of correlated data. Journal of Machine Learning Research 21(13):1–41

    MathSciNet  MATH  Google Scholar 

  12. Guo X, Li S, Yu J, Zhang J, Ma J, Ma L, Liu W, Ling H (2019) Pfld: a practical facial landmark detector. arXiv:190210859

  13. Hu X, Wang Y, Zhu F, Pan C (2016) Learning-based fully 3d face reconstruction from a single image. In: 2016 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE, pp 1651–1655

  14. Hubert M, Rousseeuw PJ, Vanden Branden K (2005) Robpca: a new approach to robust principal component analysis. Technometrics 47(1):64–79

    Article  MathSciNet  Google Scholar 

  15. Jhanani R, Harshitha S, Kalaichelvi T, Subedha V (2020) Mobile application for human facial recognition to identify criminals and missing people using tensor flow pp 16–20

  16. King DE (2009) Dlib-ml: A machine learning toolkit. Journal of Machine Learning Research 10:1755–1758

    Google Scholar 

  17. Knothe R, Romdhani S, Vetter T (2006) Combining pca and lfa for surface reconstruction from a sparse set of control points. In 7th international conference on automatic face and gesture recognition (FGR06), pp 637–644, https://doi.org/10.1109/FGR.2006.31

  18. Liu F, Zeng D, Zhao Q, Liu X (2016) Joint face alignment and 3d face reconstruction. In: European conference on computer vision, Springer, pp 545–560

  19. Liu P, Han X, Lyu M, King I, Xu J (2020) Learning 3d face reconstruction with a pose guidance network. In: Proceedings of the asian conference on computer vision

  20. Mredhula L, Dorairangaswamy M (2016) An effective filtering technique for image denoising using probabilistic principal component analysis (ppca). Journal of Medical Imaging and Health Informatics 6(1):194–203

    Article  Google Scholar 

  21. Paysan P, Knothe R, Amberg B, Romdhani S, Vetter T (2009a) A 3d face model for pose and illumination invariant face recognition. In: 2009 sixth IEEE international conference on advanced video and signal based surveillance, IEEE, pp 296–301

  22. Paysan P, Knothe R, Amberg B, Romdhani S, Vetter T (2009b) A 3d face model for pose and illumination invariant face recognition. In: 2009 Sixth IEEE international conference on advanced video and signal based surveillance, IEEE, pp 296–301

  23. Schölkopf B, Smola A, Müller KR (1997) Kernel principal component analysis. In: International conference on artificial neural networks, Springer, pp 583–588

  24. Shah SMS, Batool S, Khan I, Ashraf MU, Abbas SH, Hussain SA (2017) Feature extraction through parallel probabilistic principal component analysis for heart disease diagnosis. Physica A: Statistical Mechanics and its Applications 482:796–807

    Article  Google Scholar 

  25. Tekalp AM, Ostermann J (2000) Face and 2-d mesh animation in mpeg-4. Signal Processing: Image Communication 15(4–5):387–421

    Google Scholar 

  26. Thomas D (2020) Real-time simultaneous 3D head modeling and facial motion capture with an RGB-D camera. arXiv:2004.10557

  27. Tipping ME, Bishop CM (1999) Probabilistic principal component analysis. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 61(3):611–622

    Article  MathSciNet  Google Scholar 

  28. Tran L, Liu X (2018) Nonlinear 3d face morphable model. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)

  29. Tran L, Liu F, Liu X (2019) Towards high-fidelity nonlinear 3d face morphable model. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR)

  30. Tuan Tran A, Hassner T, Masi I, Medioni G (2017) Regressing robust and discriminative 3d morphable models with a very deep neural network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5163–5172

  31. Vaddi R, Manoharan P (2018) Probabilistic pca based hyper spectral image classification for remote sensing applications. In: International conference on intelligent systems design and applications, Springer, pp 863–869

  32. Wold S, Esbensen K, Geladi P (1987) Principal component analysis. Chemometrics and Intelligent Laboratory Systems 2(1–3):37–52

    Article  Google Scholar 

  33. Wu CJ (1983) On the convergence properties of the em algorithm. Ann Stat :95–103

  34. Xiao Q, Han L, Liu P (2014) 3d face reconstruction via feature point depth estimation and shape deformation. In: 2014 22nd international conference on pattern recognition, IEEE, pp 2257–2262

  35. Yi H, Li C, Cao Q, Shen X, Li S, Wang G, Tai YW (2019) Mmface: A multi-metric regression network for unconstrained face reconstruction. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 7663–7672

  36. Zhang K, Zhang Z, Li Z, Qiao Y (2016) Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters 23(10):1499–1503

    Article  Google Scholar 

  37. Zhong Y, Pei Y, Li P, Guo Y, Ma G, Liu M, Bai W, Wu W, Zha H (2020) Face denoising and 3d reconstruction from a single depth image. In: 2020 15th IEEE international conference on automatic face and gesture recognition (FG 2020), pp 117–124, https://doi.org/10.1109/FG47880.2020.00005

  38. Zhu X, Lei Z, Liu X, Shi H, Li SZ (2016) Face alignment across large poses: A 3d solution. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR)

  39. Zou H, Hastie T, Tibshirani R (2006) Sparse principal component analysis. Journal of Computational and Graphical Statistics 15(2):265–286

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work is partially supported by National Natural Science Foundation of China (No.61972041, No.62072045), the National Key Cooperation between the BRICS of China (No.2017YFE0100500), National Key R&D Program of China (No.2017YFB1002604, No.2017YFB1402105).

Author information

Authors and Affiliations

  1. School of Artificial Intelligence, Beijing Normal University, Beijing, China

    Xiaoxiao Xie, Xingce Wang & Zhongke Wu

  2. Engineering Research Center of Virtual Reality and Applications, Ministry of Education, Beijing Key Laboratory of Digital Preservation and Virtual Reality for Cultural Heritage, Beijing Normal University, 100875, Beijing, China

    Xiaoxiao Xie, Xingce Wang & Zhongke Wu

Authors
  1. Xiaoxiao Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xingce Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhongke Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhongke Wu.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xie, X., Wang, X. & Wu, Z. 3D face dense reconstruction based on sparse points using probabilistic principal component analysis. Multimed Tools Appl 81, 2937–2957 (2022). https://doi.org/10.1007/s11042-021-11707-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11707-0

Keywords

Search

Navigation