Abstract
Skull prediction from the head is a challenging issue toward a cost-effective therapeutic solution for facial disorders. This issue was initially studied in our previous work using full head-to-skull relationship learning. However, the head-skull thickness topology is locally shaped, especially in the face region. Thus, the objective of the present study was to enhance our head-to-skull prediction problem by using local topological features for training and predicting. Head and skull feature points were sampled on 329 head and skull models from computed tomography (CT) images. These feature points were classified into the back and facial topologies. Head-to-skull relations were trained using the partial least square regression (PLSR) models separately in the two topologies. A hyperparameter tuning process was also conducted for selecting optimal parameters for each training model. Thus, a new skull could be generated so that its shape was statistically fitted with the target head. Mean errors of the predicted skulls using the topology-based learning method were better than those using the non-topology-based learning method. After tenfold cross-validation, the mean error was enhanced 36.96% for the skull shapes and 14.17% for the skull models. Mean error in the facial skull region was especially improved with 4.98%. The mean errors were also improved 11.71% and 25.74% in the muscle attachment regions and the back skull regions respectively. Moreover, using the enhanced learning strategy, the errors (mean ± SD) for the best and worst prediction cases are from 1.1994 ± 1.1225 mm (median: 0.9036, coefficient of multiple determination (R2): 0.997274) to 3.6972 ± 2.4118 mm (median: 3.9089, R2: 0.999614) and from 2.0172 ± 2.0454 mm (median: 1.2999, R2: 0.995959) to 4.0227 ± 2.6098 mm (median: 3.9998, R2: 0.998577) for the predicted skull shapes and the predicted skull models respectively. This present study showed that more detailed information on the head-skull shape leads to a better accuracy level for the skull prediction from the head. In particular, local topological features on the back and face regions of interest should be considered toward a better learning strategy for the head-to-skull prediction problem. In perspective, this enhanced learning strategy was used to update our developed clinical decision support system for facial disorders. Furthermore, a new class of learning methods, called geometric deep learning will be studied.
Graphical abstract
Similar content being viewed by others
References
Y. Lee, D. Terzopoulos, K. Waters, Constructing physics-based facial models of individuals, Proc. Gr Aphics Interface’93 Conf. (1993) 1–8. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.29.2662.
K. Kähler, J. Haber, H.-P. Seidel, Geometry-based muscle modeling for facial animation, in: Graph. Interface, 2001: pp. 37–46.
Claes P, Vandermeulen D, De Greef S, Willems G, Clement JG, Suetens P (2010) Computerized craniofacial reconstruction: conceptual framework and review. Forensic Sci Int 201:138–145. https://doi.org/10.1016/j.forsciint.2010.03.008
Wei M, Liu Y, Dong H, El Saddik A (2017) Human head stiffness rendering. IEEE Trans Instrum Meas 66:2083–2096. https://doi.org/10.1109/TIM.2017.2676258
Ping HY, Abdullah LN, Sulaiman PS, Halin AA (2013) Computer facial animation: a review. Int J Comput Theory Eng 5:658–662. https://doi.org/10.7763/ijcte.2013.v5.770
A.X. Fan, S. Dakpé, T.T. Dao, P. Pouletaut, M. Rachik, M.C. Ho Ba Tho, MRI-based finite element modeling of facial mimics: a case study on the paired zygomaticus major muscles, Comput. Methods Biomech. Biomed. Engin. 20 (2017) 919–928. https://doi.org/10.1080/10255842.2017.1305363.
T.T. Dao, A.X. Fan, S. Dakpé, P. Pouletaut, M. Rachik, M.C. Ho Ba Tho, Image-based skeletal muscle coordination: case study on a subject specific facial mimic simulation, J. Mech. Med. Biol. 18 (2018) 1–15. https://doi.org/10.1142/S0219519418500203.
Wu T, Hung APL, Hunter P, Mithraratne K (2013) Modelling facial expressions: a framework for simulating nonlinear soft tissue deformations using embedded 3D muscles. Finite Elem Anal Des 76:63–70. https://doi.org/10.1016/j.finel.2013.08.002
Cordea MD, Petriu EM (2006) A 3-D anthropometric-muscle-based active appearance model. IEEE Trans Instrum Meas 55:91–98. https://doi.org/10.1109/TIM.2005.860861
Cordea MD, Petriu EM, Petriu DC (2008) Three-dimensional head tracking and facial expression recovery using an anthropometric muscle-based active appearance model. IEEE Trans Instrum Meas 57:1578–1588. https://doi.org/10.1109/TIM.2008.923784
King SA, Parent RE (2005) Creating speech-synchronized animation. IEEE Trans Vis Comput Graph 11:341–352. https://doi.org/10.1109/TVCG.2005.43
Delingette H (1998) Toward realistic soft-tissue modeling in medical simulation. Proc IEEE 86:512–523. https://doi.org/10.1109/5.662876
Hutto JR, Vattoth S (2015) A practical review of the muscles of facial mimicry with special emphasis on the superficial musculoaponeurotic system, AJR. Am J Roentgenol 204:W19–W26. https://doi.org/10.2214/AJR.14.12857
Wang SF, Lai SH (2011) Reconstructing 3D face model with associated expression deformation from a single face image via constructing a low-dimensional expression deformation manifold. IEEE Trans Pattern Anal Mach Intell 33:2115–2121. https://doi.org/10.1109/TPAMI.2011.88
Marcos S, Gómez-García-Bermejo J, Zalama E (2010) A realistic, virtual head for human-computer interaction. Interact Comput 22:176–192. https://doi.org/10.1016/j.intcom.2009.12.002
Matsuoka A, Yoshioka F, Ozawa S, Takebe J (2019) Development of three-dimensional facial expression models using morphing methods for fabricating facial prostheses. J Prosthodont Res 63:66–72. https://doi.org/10.1016/j.jpor.2018.08.003
Turban L, Girard D, Kose N, Dugelay JL (2015) From Kinect video to realistic and animatable MPEG-4 face model: a complete framework, 2015 IEEE Int. Conf Multimed Expo Work ICMEW 2015:1–6. https://doi.org/10.1109/ICMEW.2015.7169783
T.-N. Nguyen, S. Dakpe, M.-C. Ho Ba Tho, T.-T. Dao, Kinect-driven patient-specific head, skull, and muscle network modelling for facial palsy patients, Comput. Methods Programs Biomed. (2020) 105846. https://doi.org/10.1016/j.cmpb.2020.105846.
P. Claes, D. Vandermeulen, R. Suetens, G. Willems, S. De Greef, Volumetric deformable face models for cranio-facial reconstruction, ISPA 2005. Proc. 4th Int. Symp. Image Signal Process. Anal. 2005. (2008) 353–358. https://doi.org/10.1109/ispa.2005.195437.
R. Liang, Y. Lin, L. Jiang, J. Bao, X. Huang, Craniofacial model reconstruction from skull data based on feature points, Proc. - 2009 11th IEEE Int. Conf. Comput. Des. Comput. Graph. CAD/Graphics 2009. (2009) 602–605. https://doi.org/10.1109/CADCG.2009.5246828.
Y.F. Zhang, M.Q. Zhou, G.H. Geng, J. Feng, Face appearance reconstruction based on a regional statistical craniofacial model (RCSM), Proc. - Int. Conf. Pattern Recognit. (2010) 1670–1673. https://doi.org/10.1109/ICPR.2010.413.
W. Shui, M. Zhou, Q. Deng, Z. Wu, F. Duan, 3D craniofacial reconstruction using reference skull-face database, Int. Conf. Image Vis. Comput. New Zeal. (2010) 1–7. https://doi.org/10.1109/IVCNZ.2010.6148864.
Yang Y, Zhou M, Lu K, Duan F, Li Y, Tian Y, Wu Z (2014) Skull identification via correlation measure between skull and face shape. IEEE Trans Inf Forensics Secur 9:1322–1332. https://doi.org/10.1109/tifs.2014.2332981
Duan F, Huang D, Tian Y, Lu K, Wu Z, Zhou M (2015) 3D face reconstruction from skull by regression modeling in shape parameter spaces. Neurocomputing 151:674–682. https://doi.org/10.1016/j.neucom.2014.04.089
Bai F, Zhengxin C, Qiao X, Qingqiong D, Duan F, Tian Y, Face reconstruction from skull based on Least Squares Canonical Dependency Analysis, (2016) IEEE Int. Conf. Syst. Man, Cybern. SMC 2016 - Conf. Proc 2017:3612–3617. https://doi.org/10.1109/SMC.2016.7844794
D. Madsen, M. Lüthi, A. Schneider, T. Vetter, Probabilistic joint face-skull modelling for facial reconstruction, in: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2018: pp. 5295–5303.
K. Pearson, LIII. On lines and planes of closest fit to systems of points in space, London, Edinburgh, Dublin Philos. Mag. J. Sci. 2 (1901) 559–572.
H. Hotelling, Analysis of a complex of statistical variables into principal components., J. Educ. Psychol. 24 (1933) 417.
Soummer R, Pueyo L, Larkin J (2012) Detection and characterization of exoplanets and disks using projections on Karhunen-Loève eigenimages. Astrophys J Lett 755:L28
Wold S, Geladi P, Esbensen K, Öhman J (1987) Multi-way principal components-and PLS-analysis. J Chemom 1:41–56
Garthwaite PH (1994) An interpretation of partial least squares. J Am Stat Assoc 89:122–127
Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401:788–791
Paatero P, Tapper U (1994) Positive matrix factorization: a non-negative factor model with optimal utilization of error estimates of data values. Environmetrics 5:111–126
W. Shui, M. Zhou, S. Maddock, Y. Ji, Q. Deng, K. Li, Y. Fan, Y. Li, X. Wu, A computerized craniofacial reconstruction method for an unidentified skull based on statistical shape models, Multimed. Tools Appl. (2020) 25589–25611. https://doi.org/10.1007/s11042-020-09189-7.
Jia B, Zhao J, Xin S, Duan F, Pan Z, Wu Z, Li J, Zhou M (2021) Craniofacial reconstruction based on heat flow geodesic grid regression (HF-GGR) model. Comput Graph 97:258–267. https://doi.org/10.1016/j.cag.2021.04.029
Nguyen T-N, Tran V-D, Nguyen H-Q, Dao T-T (2020) A statistical shape modeling approach for predicting subject-specific human skull from head surface. Med Biol Eng Comput In Press. https://doi.org/10.1007/s11517-020-02219-4
Floater MS (2003) Mean value coordinates. Comput Aided Geom Des 20:19–27. https://doi.org/10.1016/S0167-8396(03)00002-5
Srivastava A, Joshi SH, Mio W, Liu X (2005) Statistical shape analysis: clustering, learning, and testing. IEEE Trans Pattern Anal Mach Intell 27:590–602. https://doi.org/10.1109/TPAMI.2005.86
Carlsson G (2009). Topology and data. https://doi.org/10.1090/S0273-0979-09-01249-X
Heimann T, Meinzer H-P (2009) Statistical shape models for 3D medical image segmentation: a review. Med Image Anal 13:543–563. https://doi.org/10.1016/j.media.2009.05.004
Mutsvangwa T, Burdin V, Schwartz C, Roux C (2015) An automated statistical shape model developmental pipeline: application to the human scapula and humerus. IEEE Trans Biomed Eng 62:1098–1107. https://doi.org/10.1109/TBME.2014.2368362
Zhang J, Malcolm D, Hislop-Jambrich J, Thomas CDL, Nielsen PMF (2014) An anatomical region-based statistical shape model of the human femur. Comput Methods Biomech Biomed Eng Imaging Vis 2:176–185. https://doi.org/10.1080/21681163.2013.878668
Clark K, Vendt B, Smith K, Freymann J, Kirby J, Koppel P, Moore S, Phillips S, Maffitt D, Pringle M, Tarbox L, Prior F (2013) The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository. J Digit Imaging 26:1045–1057. https://doi.org/10.1007/s10278-013-9622-7
Vallières M, Kay-Rivest E, Perrin LJ, Liem X, Furstoss C, Aerts HJWL, Khaouam N, Nguyen-Tan PF, Wang C-S, Sultanem K, Seuntjens J, El Naqa I (2017) Radiomics strategies for risk assessment of tumour failure in head-and-neck cancer. Sci Rep 7:10117. https://doi.org/10.1038/s41598-017-10371-5
Grossberg AJ, Mohamed ASR, El Halawani H, Bennett WC, Smith KE, Nolan TS, Williams B, Chamchod S, Heukelom J, Kantor ME, Browne T, Hutcheson KA, Gunn GB, Garden AS, Morrison WH, Frank SJ, Rosenthal DI, Freymann JB, Fuller CD (2018) Data descriptor: imaging and clinical data archive for head and neck squamous cell carcinoma patients treated with radiotherapy. Sci Data 5:1–10. https://doi.org/10.1038/sdata.2018.173
S. Pieper, M. Halle, R. Kikinis, 3D Slicer, (2005) 632–635. https://doi.org/10.1109/isbi.2004.1398617.
P. Cignoni, M. Callieri, M. Corsini, M. Dellepiane, F. Ganovelli, G. Ranzuglia, Meshlab: an open-source mesh processing tool., in: Eurographics Ital. Chapter Conf., 2008: pp. 129–136. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.649.4449.
S. Marden, J. Guivant, Improving the performance of ICP for real-time applications using an approximate nearest neighbour search, Australas. Conf. Robot. Autom. ACRA. (2012) 3–5.
P.J. Besl, N.D. McKay, A method for registration of 3-D shapes, in: P.S. Schenker (Ed.), Sens. Fusion IV Control Paradig. Data Struct., 1992: pp. 586–606. https://doi.org/10.1117/12.57955.
T.-N. Nguyen, S. Dakpé, M.-C. Ho Ba Tho, T.-T. Dao, Real-time computer vision system for tracking simultaneously subject-specific rigid head and non-rigid facial mimic movements using a contactless sensor and system of systems approach, Comput. Methods Programs Biomed. 191 (2020) 105410. https://doi.org/10.1016/j.cmpb.2020.105410.
T. Ju, S. Schaefer, J. Warren, Mean value coordinates for closed triangular meshes, in: ACM SIGGRAPH 2005 Pap. - SIGGRAPH ’05, ACM Press, New York, New York, USA, 2005: p. 561. https://doi.org/10.1145/1186822.1073229.
P. Geladi, Bruce R. Kowalski, Partial Least-Squares Regression: A tutorial, J. Optoelectron. Adv. Mater. 10 (1986) 1–17. https://doi.org/10.1016/0003-2670(86)80028-9.
Lindgren F, Geladi P, Wold S (1993) The kernel algorithm for PLS. J Chemom 7:45–59. https://doi.org/10.1002/cem.1180070104
Dayal BS, Macgregor JF (1997) Improved PLS algorithms. J Chemom 11:73–85. https://doi.org/10.1002/(SICI)1099-128X(199701)11:1%3C73::AID-CEM435%3E3.0.CO;2-%23
Corsini M, Cignoni P, Scopigno R (2012) Efficient and Flexible sampling with blue noise properties of triangular meshes. IEEE Trans Vis Comput Graph 18:914–924. https://doi.org/10.1109/TVCG.2012.34
Bernardini F, Mittleman J, Rushmeier H, Silva C, Taubin G (1999) The ball-pivoting algorithm for surface reconstruction. IEEE Trans Vis Comput Graph 5:349–359. https://doi.org/10.1109/2945.817351
N. Aspert, D. Santa-Cruz, T. Ebrahimi, MESH: measuring errors between surfaces using the Hausdorff distance, in: Proceedings. IEEE Int. Conf. Multimed. Expo, IEEE, 1978: pp. 705–708. https://doi.org/10.1109/ICME.2002.1035879.
D.J.J. Farnell, S. Richmond, J. Galloway, A.I. Zhurov, P. Pirttiniemi, T. Heikkinen, V. Harila, H. Matthews, P. Claes, An exploration of adolescent facial shape changes with age via multilevel partial least squares regression, Comput. Methods Programs Biomed. 200 (2021). https://doi.org/10.1016/j.cmpb.2021.105935.
A. Jacobson, D. Panozzo, libigl, in: SIGGRAPH Asia 2017 Courses, ACM, New York, NY, USA, 2017: pp. 1–172. https://doi.org/10.1145/3134472.3134497.
Schroeder WJ, Avila LS, Hoffman W (2000) Visualizing with VTK: a tutorial. IEEE Comput Graph Appl 20:20–27. https://doi.org/10.1109/38.865875
Smith KE, Bhatia G, Vannier MW (1995) Assessment of mass properties of human head using various three-dimensional imaging modalities. Med Biol Eng Comput 33:278–284. https://doi.org/10.1007/BF02510500
Rittey C (2007) The facial nerve. Pediatr ENT 83:479–484. https://doi.org/10.1007/978-3-540-33039-4_47
Constantinides M, Galli SKD, Miller PJ (2001) Complications of static facial suspensions with expanded polytetrafluoroethylene (ePTFE). Laryngoscope 111:2114–2121. https://doi.org/10.1097/00005537-200112000-00006
M. Wernick Robinson, J. Baiungo, M. Hohman, T. Hadlock, Facial rehabilitation, Oper. Tech. Otolaryngol. - Head Neck Surg. 23 (2012) 288–296. https://doi.org/10.1016/j.otot.2012.10.002.
Khalifian S, Brazio PS, Mohan R, Shaffer C, Brandacher G, Barth RN, Rodriguez ED (2014) Facial transplantation: the first 9 years. Lancet 384:2153–2163. https://doi.org/10.1016/S0140-6736(13)62632-X
Samsudin WSW, Sundaraj K (2014) Clinical and non-clinical initial assessment of facial nerve paralysis: a qualitative review, Biocybern. Biomed Eng 34:71–78. https://doi.org/10.1016/j.bbe.2014.02.005
S. De Greef, P. Claes, D. Vandermeulen, W. Mollemans, P. Suetens, G. Willems, Large-scale in-vivo Caucasian facial soft tissue thickness database for craniofacial reconstruction, Forensic Sci. Int. 159 (2006). https://doi.org/10.1016/j.forsciint.2006.02.034.
Bronstein MM, Bruna J, LeCun Y, Szlam A, Vandergheynst P (2017) Geometric deep learning: going beyond euclidean data. IEEE Signal Process Mag 34:18–42
J. Zhao, X. Qi, C. Wen, N. Lei, X. Gu, Automatic and robust skull registration based on discrete uniformization, Proc. IEEE Int. Conf. Comput. Vis. 2019-Octob (2019) 431–440. https://doi.org/10.1109/ICCV.2019.00052.
Acknowledgements
The authors would like to thank the Métropole Européenne de Lille (MEL) and the I-SITE ULNE (Université Lille Nord Europe) for funding.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix 1
Mathematical formulations of distances, thicknesses, and sampling rays of the back and facial topologies
in which \({x}_{ij}^{BS}\)(or \({x}_{ij}^{FS}\)), \({y}_{ij}^{BS}\)(or \({y}_{ij}^{FS}\)), and \({z}_{ij}^{BS}\)(or \({z}_{ij}^{BS}\)) are \(x\), \(y\), and \(z\) coordinates of the \({j\mathrm{th}}\) back (or facial) skull feature point on the \({i\mathrm{th}}\) head-skull pair. \({x}_{ij}^{BH}\)(or \({x}_{ij}^{FH}\)), \({y}_{ij}^{BH}\)(or \({y}_{ij}^{FH}\)), and \({z}_{ij}^{BH}\)(or \({z}_{ij}^{FH}\)) are \(x\), \(y\), and \(z\) coordinates of the \({j\mathrm{th}}\) back (or facial) head feature point on the \({i\mathrm{th}}\) head-skull pair. \({x}_{j}^{BSV}\)(or \({x}_{j}^{FSV}\)), \({y}_{j}^{BSV}\)(or \({y}_{j}^{FSV}\)), and \({z}_{j}^{BSV}\)(or \({z}_{j}^{FSV}\)) are \(x\), \(y\), and \(z\) coordinates of the \({j\mathrm{th}}\) back (or facial) sampling vertices. \({d}_{ij}^{B}\) (or \({d}_{ij}^{F}\)) is the Euclidean distance between the \({j\mathrm{th}}\) back (or facial) head feature point of the \({i\mathrm{th}}\) head-skull pair and the centroid of the first skull convex (\({{\varvec{v}}}^{SC}\)). \({t}_{ij}^{B}\) (or \({t}_{ij}^{F}\)) is the Euclidean distance from the \({j\mathrm{th}}\) back (facial) head feature point to the \({j\mathrm{th}}\) back (facial) skull feature point of the \({i\mathrm{th}}\) head-skull pair. \(M\) is the number of back sampling rays. \(N\) is the number of facial sampling rays. \({N}_{\mathrm{subjects}}\) is the number of head-skull pairs (\({N}_{\mathrm{subjects}} = 329\)) in the whole dataset.
Appendix 2
Mathematical formulations of predictor and response variables of the topology-based head-to-skull predicting method
in which \({N}_{\mathrm{training}}\) is the number of head-skull pairs in the training dataset.
Rights and permissions
About this article
Cite this article
Nguyen, TN., Tran, VD., Nguyen, HQ. et al. Enhanced head-skull shape learning using statistical modeling and topological features. Med Biol Eng Comput 60, 559–581 (2022). https://doi.org/10.1007/s11517-021-02483-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11517-021-02483-y