Abstract
Facial analysis from photography supports the early identification of genetic syndromes, but clinically-acquired uncalibrated images suffer from image pose and illumination variability. Although 3D photography overcomes some of the challenges of 2D images, 3D scanners are not typically available. We present an optimization method for 3D face reconstruction from uncalibrated 2D photographs of the face using a novel statistical shape model of the infant face. First, our method creates an initial estimation of the camera pose for each 2D photograph using the average shape of the statistical model and a set of 2D facial landmarks. Second, it calculates the camera pose and the parameters of the statistical model by minimizing the distance between the projection of the estimated 3D face in the image plane of each camera and the observed 2D face geometry. Using the reconstructed 3D faces, we automatically extract a set of 3D geometric and appearance descriptors and we use them to train a classifier to identify facial dysmorphology associated with genetic syndromes. We evaluated our face reconstruction method on 3D photographs of 54 subjects (age range 0–3 years), and we obtained a point-to-surface error of 2.01 \( \pm \) 0.54%, which was a significant improvement over 2.98 \( \pm \) 0.64% using state-of-the-art methods (p < 0.001). Our classifier detected genetic syndromes from the reconstructed 3D faces from the 2D photographs with 100% sensitivity and 92.11% specificity.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Sivakumar, S., Larkins, S.: Accuracy of clinical diagnosis in down’s syndrome. Arch. Dis. Child. 89(7), 691–693 (2004)
Kruszka, P., et al.: 22q11.2 deletion syndrome in diverse populations. Am. J. Med. Genet. Part A 173(4), 879–888 (2017)
Zhao, Q., et al.: Digital facial dysmorphology for genetic screening: hierarchical constrained local model using ICA. Med. Image Anal. 18(5), 699–710 (2014)
Cerrolaza, J.J., et al.: Identification of dysmorphic syndromes using landmark-specific local texture descriptors. In: Proceedings of IEEE International Symposium on Biomedical Imaging (ISBI), pp. 1080–1083 (2016)
Gurovich, Y., et al.: Identifying facial phenotypes of genetic disorders using deep learning. Nat. Med. 25(1), 60–64 (2019)
Paternoster, L., et al.: Genome-wide association study of three-dimensional facial morphology identifies a variant in PAX3 associated with nasion position. Am. J. Hum. Genet. 90(3), 478–485 (2012)
Weinberg, S.M., et al.: The 3D facial norms database: part 1. a web-based craniofacial anthropometric and image repository for the clinical and research community. Cleft Palate Craniofac. J. 53(6), e185–e197 (2016)
Meulstee, J.W., et al.: A new method for three-dimensional evaluation of the cranial shape and the automatic identification of craniosynostosis using 3D stereophotogrammetry. Int. J. Oral Maxillofac. Surg. 46(7), 819–826 (2017)
Ph, D., et al.: Cranial growth in infants─a longitudinal three-dimensional analysis of the first months of life. J. Craniomaxillofac. Surg. 46(6), 987–993 (2018)
Hassner, T.: Viewing real-world faces in 3D. In: Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3607–3614 (2013)
Roth, J., et al.: Adaptive 3D face reconstruction from unconstrained photo collections. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4197–4206 (2016)
Bas, A., et al.: Fitting a 3D morphable model to edges: a comparison between hard and soft correspondences. In: Asian Conference on Computer Vision (ACCV) Workshops, pp. 377–391 (2016)
Zhu, X., et al.: Face alignment across large poses: a 3D solution. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 146–155 (2016)
Tuan Tran, A., et al.: Regressing robust and discriminative 3D morphable models with a very deep neural network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)
Tu, L., et al.: Analysis of 3D facial dysmorphology in genetic syndromes from unconstrained 2D photographs. In: Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention. (MICCAI), 2018, pp. 347–355 (2018)
Lévy, B., et al.: Least squares conformal maps for automatic texture atlas generation. ACM Trans. Graph. 21(3), 362–371 (2002)
Coleman, T.F., Li, Y.: An interior trust region approach for nonlinear minimization subject to bounds. SIAM J. Optim. 6(2), 418–445 (1996)
Salti, S., et al.: SHOT: unique signatures of histograms for surface and texture description. Comput. Vis. Image Underst. 125, 251–264 (2014)
Kim, H.-Y.: Statistical notes for clinical researchers: assessing normal distribution (2) using skewness and kurtosis. Restor. Dent. Endod. 38(1), 52–54 (2013)
Guyon, I., et al.: Gene selection for cancer classification using support vector machines. Mach. Learn. 46(1), 389–422 (2002)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Tu, L. et al. (2019). Three-Dimensional Face Reconstruction from Uncalibrated Photographs: Application to Early Detection of Genetic Syndromes. In: Greenspan, H., et al. Uncertainty for Safe Utilization of Machine Learning in Medical Imaging and Clinical Image-Based Procedures. CLIP UNSURE 2019 2019. Lecture Notes in Computer Science(), vol 11840. Springer, Cham. https://doi.org/10.1007/978-3-030-32689-0_19
Download citation
DOI: https://doi.org/10.1007/978-3-030-32689-0_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32688-3
Online ISBN: 978-3-030-32689-0
eBook Packages: Computer ScienceComputer Science (R0)
