Abstract
Multilevel principal components analysis (mPCA) has previously been shown to provide a simple and straightforward method of forming point distribution models that can be used in (active) shape models. Here we extend the mPCA approach to model image texture as well as shape. As a test case, we consider a set of (2D frontal) facial images from a group of 80 Finnish subjects (34 male; 46 female) with two different facial expressions (smiling and neutral) per subject. Shape (in terms of landmark points) and image texture are considered separately in this initial analysis. Three-level models are constructed that contain levels for biological sex, “within-subject” variation (i.e., facial expression), and “between-subject” variation (i.e., all other sources of variation). By considering eigenvalues, we find that the order of importance as sources of variation for facial shape is: facial expression (47.5%), between-subject variations (45.1%), and then biological sex (7.4%). By contrast, the order for image texture is: between-subject variations (55.5%), facial expression (37.1%), and then biological sex (7.4%). The major modes for the facial expression level of the mPCA models clearly reflect changes in increased mouth size and increased prominence of cheeks during smiling for both shape and texture. Even subtle effects such as changes to eyes and nose shape during smile are seen clearly. The major mode for the biological sex level of the mPCA models similarly relates clearly to changes between male and female. Model fits yield “scores” for each principal component that show strong clustering for both shape and texture by biological sex and facial expression at appropriate levels of the model. We conclude that mPCA correctly decomposes sources of variation due to biological sex and facial expression (etc.) and that it provides a reliable method of forming models of both shape and image texture.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
(1) Glabella (g); (2) Nasion (n); (3) Endocanthion left (enl); (4) Endocanthion right (enr); (5) Exocanthion left (exl); (6) Exocanthion right (exr); (7) Palpebrale superius left (psl); (8) Palpebrale superius right (psr); (9) Palpebrale inferius left (pil); (10) Palpebrale inferius right (pir); (11) Pronasale (prn); (12) Subnasale (sn); (13) Alare left (all); (14) Alare right (alr); (15) Labiale superius (ls); (16) Crista philtri left (cphl); (17) Crista philtri right (cphr); (18) Labiale inferius (li); (19) Cheilion left (chl); (20) Cheilion right (chr); (21) Pogonion (pg).
References
Gelman, A., Hill, J.: Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge University Press, Cambridge (2007)
Lecron, F., Boisvert, J., Benjelloun, M., Labelle, H., Mahmoudi, S.: Multilevel statistical shape models: a new framework for modeling hierarchical structures. In: 9th IEEE International Symposium on Biomedical Imaging (ISBI), pp. 1284–1287 (2012)
Farnell, D.J.J., Popat, H., Richmond, S.: Multilevel principal component analysis (mPCA) in shape analysis: a feasibility study in medical and dental imaging. Comput. Methods Programs Biomed. 129, 149–159 (2016)
Farnell, D.J.J., Galloway, J., Zhurov, A., Richmond, S., Perttiniemi, P., Katic, V.: Initial results of multilevel principal components analysis of facial shape. In: Valdés Hernández, M., González-Castro, V. (eds.) MIUA 2017. CCIS, vol. 723, pp. 674–685. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-60964-5_59. ISBN 978-3-319-60964-5
Farnell, D.J.J., Galloway, J., Zhurov, A., Richmond, S., Perttiniemi, P., Katic, V.: An initial exploration of ethnicity, sex, and subject variation on facial shape (2018, in preparation)
Cootes, T.F., Hill, A., Taylor, C.J., Haslam, J.: Use of active shape models for locating structure in medical images. Image Vis. Comput. 12, 355–365 (1994)
Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J.: Active shape models - their training and application. Comput. Vis. Image Underst. 61, 38–59 (1995)
Hill, A., Cootes, T.F., Taylor, C.J.: Active shape models and the shape approximation problem. Image Vis. Comput. 14, 601–607 (1996)
Cootes, T.F., Taylor, C.J.: A mixture model for representing shape variation. Image Vis. Comput. 17, 567–573 (1999)
Allen, P.D., et al.: Detecting reduced bone mineral density from dental radiographs using statistical shape models. IEEE Trans. Inf Technol. Biomed. 11, 601–610 (2007)
Edwards, G.J., Lanitis, A., Taylor, C.J., Cootes, T.: Statistical models of face images: improving specificity. In: British Machine Vision Conference 1996, Edinburgh, UK (1996)
Taylor, C.J., Cootes, T.F., Lanitis, A., Edwards, G., Smyth, P., Kotcheff, A.C.W.: Model-based interpretation of complex and variable images. Philos. Trans. R. Soc. Lond. Ser. B-Biol. Sci. 352, 1267–1274 (1997)
Edwards, G.J., Cootes, T.F., Taylor, C.J.: Face recognition using active appearance models. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, pp. 581–595. Springer, Heidelberg (1998). https://doi.org/10.1007/BFb0054766
Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23, 681–685 (2001)
Cootes, T.F., Taylor, C.J.: Anatomical statistical models and their role in feature extraction. Br. J. Radiol. 77, S133–S139 (2004)
Matthews, I., Baker, S.: Active appearance models revisited. Int. J. Comput. Vis. 60, 135–164 (2004)
Di, C.-Z., Crainiceanu, C.M., Caffo, B.S., Punjabi, N.M.: Multilevel functional principal component analysis. Ann. Appl. Stat. 3, 458–488 (2009)
Velemínská, J., et al.: Surface facial modelling and allometry in relation to sexual dimorphism. HOMO – J. Comp. Hum. Biol. 63, 81–93 (2012)
Toma, A.M., Zhurov, A., Playle, R., Richmond, S.: A three-dimensional look for facial differences between males and females in a British-Caucasian sample aged 15 ½ years old. Orthod. Craniofac. Res. 11, 180–185 (2008)
Etemad, K., Chellappa, R.: Discriminant analysis for recognition of human face images. J. Opt. Soc. Am. A 14, 1724–1733 (1997)
Kim, T.-K., Kittler, J.: Locally linear discriminant analysis for multi-modally distributed classes for face recognition with a single model image. IEEE Trans. Pattern Anal. Mach. Intell. 27, 318–327 (2005)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Farnell, D.J.J., Galloway, J., Zhurov, A., Richmond, S., Pirttiniemi, P., Lähdesmäki, R. (2018). What’s in a Smile? Initial Results of Multilevel Principal Components Analysis of Facial Shape and Image Texture. In: Nixon, M., Mahmoodi, S., Zwiggelaar, R. (eds) Medical Image Understanding and Analysis. MIUA 2018. Communications in Computer and Information Science, vol 894. Springer, Cham. https://doi.org/10.1007/978-3-319-95921-4_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-95921-4_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-95920-7
Online ISBN: 978-3-319-95921-4
eBook Packages: Computer ScienceComputer Science (R0)