Skip to main content
Book cover

Joint Conference of the Asian Council on Ergonomics and Design and the Southeast Asian Network of Ergonomics Societies

ACEDSEANES 2020: Convergence of Ergonomics and Design pp 357–366Cite as

A Brief Review of 3D Face Reconstruction Methods for Face-Related Product Design

  • 342 Accesses

Part of the Advances in Intelligent Systems and Computing book series (AISC,volume 1298)

Abstract

3D face reconstruction is highly important in the ergonomics study of 3D face, especially in terms of designing face-related products. With the development of machine vision and deep learning, it becomes feasible to reconstruct the 3D face from a single image, which can make it practical to obtain a large scale data of 3D face shape instead of using the 3D scanning technology. The 3D face reconstruction methods, to recover the 3D facial geometry under unconstrained situations from 2D images, are roughly classified into two categories, namely (1) 3D Morphable Model (3DMM) fitting based method and (2) End-to-end deep convolutional neural network (CNN) based method. The 3DMM as a general face representation is introduced emphatically and two kinds of 3DMM fitting based methods are introduced when improving the 3DMM modeling mechanism. Four representative CNN based methods are compared when regressing from pixels of face image to the 3D face coordinates in different gird-like data structures. Finally, six common face datasets largely used in the training and testing are listed.

Keywords

  • Face-related product design
  • 3D face reconstruction
  • 3D morphable model

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-63335-6_37
  • Chapter length: 10 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   219.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-63335-6
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   279.99
Price excludes VAT (USA)
Hardcover Book
USD   279.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.

References

  1. Chu C-H, Huang S-H, Yang C-K, Tseng C-Y (2015) Design customization of respiratory mask based on 3D face anthropometric data. Int J Prec Eng Manuf 16(3):487–494. https://doi.org/10.1007/s12541-015-0066-5

    CrossRef  Google Scholar 

  2. Lacko D, Vleugels J, Fransen E, Huysmans T, De Bruyne G, Van Hulle MM, Sijbers J, Verwulgen S (2017) Ergonomic design of an EEG headset using 3D anthropometry. Appl Ergon 58:128–136

    CrossRef  Google Scholar 

  3. Skals S, Ellena T, Subic A, Mustafa H, Pang TY (2016) Improving fit of bicycle helmet liners using 3D anthropometric data. Int J Ind Ergon 55:86–95

    CrossRef  Google Scholar 

  4. Long J, Helland, M, Anshel J (2011) A vision for strengthening partnerships between optometry and ergonomics. In: HFESA 47th Annual Conference

    Google Scholar 

  5. Blanz V, Vetter T (1999) A morphable model for the synthesis of 3D faces. In: Siggraph

    Google Scholar 

  6. Blanz V, Vetter T (2003) Face recognition based on fitting a 3d morphable model. IEEE Trans Pattern Anal Mach Intell 25(9):1063–1074

    CrossRef  Google Scholar 

  7. Egger B, Smith WA, Tewari A, Wuhrer S, Zollhoefer M, Beeler T, Bernard F, Bolkart T, Kortylewski A, Romdhani S (2019) 3D Morphable Face Models–Past, Present and Future. arXiv preprint arXiv:1909.01815

  8. Jolliffe I (2011) Principal component analysis. Springer

    Google Scholar 

  9. Cao C, Weng Y, Zhou S, Tong Y, Zhou K (2013) Facewarehouse: A 3d facial expression database for visual computing. IEEE Trans Visual Comput Graph 20(3):413–425

    Google Scholar 

  10. Paysan P, Knothe R, Amberg B, Romdhani S, Vetter T (2010) A 3D face model for pose and illumination invariant face recognition. In: 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance. IEEE

    Google Scholar 

  11. Cristinacce D, Cootes T (2008) Automatic feature localisation with constrained local models. Pattern Recogn 41(10):3054–3067

    CrossRef  Google Scholar 

  12. Cootes TF, Edwards GJ, Taylor CJ (2001) Active appearance models. IEEE Trans Pattern Anal Mach Intell 6:681–685

    CrossRef  Google Scholar 

  13. Zhu X, Liu X, Lei Z, Li SZ (2017) Face alignment in full pose range: A 3d total solution. IEEE Trans Pattern Anal Mach Intell 41(1):78–92

    CrossRef  Google Scholar 

  14. Dollár P, Welinder P, Perona P (2010) Cascaded pose regression. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE

    Google Scholar 

  15. Sun Y, Wang X, Tang X (2013) Deep convolutional network cascade for facial point detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    Google Scholar 

  16. Zhang Z, Luo P, Loy CC, Tang X (2014) Facial landmark detection by deep multi-task learning. In: European Conference on Computer Vision. Springer

    Google Scholar 

  17. Jourabloo A, Liu X (2016) Large-pose face alignment via CNN-based dense 3D model fitting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    Google Scholar 

  18. Feng Y, Wu F, Shao X, Wang Y, Zhou X (2018) Joint 3d face reconstruction and dense alignment with position map regression network. In: Proceedings of the European Conference on Computer Vision (ECCV)

    Google Scholar 

  19. Jackson  AS, Bulat A, Argyriou V,  Tzimiropoulos G (2017) Large pose 3D face reconstruction from a single image via direct volumetric CNN regression. In: Proceedings of the IEEE International Conference on Computer Vision

    Google Scholar 

  20. Zeng X, Peng X, Qiao Y (2019) DF2Net: A dense-fine-finer network for detailed 3D face reconstruction. In: Proceedings of the IEEE International Conference on Computer Vision

    Google Scholar 

  21. Wei H, Liang S, Wei Y (2019) 3D Dense Face Alignment via Graph Convolution Networks. arXiv preprint arXiv:1904.05562

  22. Liu Z, Luo P, Wang X, Tang X (2015) Deep learning face attributes in the wild. In: Proceedings of the IEEE International Conference on Computer Vision)

    Google Scholar 

  23. Koestinger M, Wohlhart P, Roth PM (2011) Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV workshops). IEEE

    Google Scholar 

  24. Burgos-Artizzu XP, Perona P, Dollár P (2013) Robust face landmark estimation under occlusion. In: Proceedings of the IEEE International Conference on Computer Vision

    Google Scholar 

  25. Belhumeur PN, Jacobs DW, Kriegman DJ, Kumar N (2013) Localizing parts of faces using a consensus of exemplars. IEEE Trans Anal Mach Intell 35(12):2930–2940

    CrossRef  Google Scholar 

  26. Jeni LA, Tulyakov S, Yin L, Sebe N, Cohn JF (2016) The first 3d face alignment in the wild (3dfaw) challenge. In: European Conference on Computer Vision. Springer

    Google Scholar 

  27. Sagonas C, Tzimiropoulos G, Zafeiriou S, Pantic M (2013) 300 faces in-the-wild challenge: The first facial landmark localization challenge. In: Proceedings of the IEEE International Conference on Computer Vision Workshops

    Google Scholar 

  28. Zhu X, Ramanan D (2012) Face detection, pose estimation, and landmark localization in the wild. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE

    Google Scholar 

  29. Le V, Brandt J, Lin, Z, Bourdev L, Huang TS (2012) Interactive facial feature localization. In: European Conference on Computer Vision. Springer

    Google Scholar 

  30. Messer K, Matas J, Kittler J, Luettin J, Maitre G (1999) XM2VTSDB: The extended M2VTS database. In: Second International Conference on Audio and Video-Based Biometric Person Authentication

    Google Scholar 

  31. Shen J, Zafeiriou S, Chrysos GG, Kossaifi J, Tzimiropoulos G, Pantic M (2015) The first facial landmark tracking in-the-wild challenge: Benchmark and results. In: Proceedings of the IEEE International Conference On Computer Vision Workshops

    Google Scholar 

  32. Wu W, Qian C, Yang S, Wang Q, Cai Y, Zhou Q (2018) Look at boundary: A boundary-aware face alignment algorithm. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    Google Scholar 

  33. Yin L, Sun XCY, Worm T, Reale M (2008) A high-resolution 3D dynamic facial expression database. In: IEEE International Conference on Automatic Face and Gesture Recognition, Amsterdam, The Netherlands

    Google Scholar 

  34. Bagdanov AD, Del Bimbo A, Masi I (2011) The florence 2D/3D hybrid face dataset. In: Proceedings of the 2011 Joint ACM Workshop on Human Gesture and Behavior Understanding. ACM

    Google Scholar 

  35. Zhang X, Yin L, Cohn JF, Canavan S, Reale M, Horowitz A, Liu P, Girard JM (2014) Bp4d-spontaneous: a high-resolution spontaneous 3d dynamic facial expression database. Image Vis Comput 32(10):692–706

    CrossRef  Google Scholar 

  36. Bulat A, Tzimiropoulos G (2017) How far are we from solving the 2D & 3D face alignment problem? (and a dataset of 230,000 3d facial landmarks). In: Proceedings of the IEEE International Conference on Computer Vision

    Google Scholar 

  37. Jain V, Learned-Miller E (2010) Fddb: A benchmark for face detection in unconstrained settings. UMass Amherst Technical Report

    Google Scholar 

Download references

Acknowledgement

This work was supported by The Hong Kong Research Grants Council (RGC PolyU. 15603419).

Author information

Authors and Affiliations

  1. School of Design, The Hong Kong Polytechnic University, Hong Kong SAR, China

    Jie Zhang & Yan Luximon

  2. School of Computer and Communication Engineering, University of Science and Technology, Beijing, China

    Kangneng Zhou

Authors
  1. Jie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kangneng Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Luximon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yan Luximon .

Editor information

Editors and Affiliations

  1. Industrial and Systems Engineering, John Gokongwei Jr. College of Engineering, De La Salle University, Manila, Philippines

    Prof. Alma Maria Jennifer Gutierrez

  2. Department of Industrial Engineering and Decision Analytics, Hong Kong University of Science and Technology, Hong Kong, Hong Kong

    Prof. Ravindra S. Goonetilleke

  3. Mapúa University, Manila, Philippines

    Prof. Rex Aurellius C. Robielos

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Zhang, J., Zhou, K., Luximon, Y. (2021). A Brief Review of 3D Face Reconstruction Methods for Face-Related Product Design. In: Gutierrez, A.M.J., Goonetilleke, R.S., Robielos, R.A.C. (eds) Convergence of Ergonomics and Design. ACEDSEANES 2020. Advances in Intelligent Systems and Computing, vol 1298. Springer, Cham. https://doi.org/10.1007/978-3-030-63335-6_37

Download citation