Copula Eigenfaces with Attributes: Semiparametric Principal Component Analysis for a Combined Color, Shape and Attribute Model

  • Bernhard EggerEmail author
  • Dinu Kaufmann
  • Sandro Schönborn
  • Volker Roth
  • Thomas Vetter
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 693)


Principal component analysis is a ubiquitous method in parametric appearance modeling for describing dependency and variance in datasets. The method requires the observed data to be Gaussian-distributed. We show that this requirement is not fulfilled in the context of analysis and synthesis of facial appearance. The model mismatch leads to unnatural artifacts which are severe to human perception. As a remedy, we use a semiparametric Gaussian copula model, where dependency and variance are modeled separately. This model enables us to use arbitrary Gaussian and non-Gaussian marginal distributions. Moreover, facial color, shape and continuous or categorical attributes can be analyzed in an unified way. Accounting for the joint dependency between all modalities leads to a more specific face model. In practice, the proposed model can enhance performance of principal component analysis in existing pipelines: The steps for analysis and synthesis can be implemented as convenient pre- and post-processing steps.


Copula Component Analysis Gaussian copula Principal component analysis Parametric Appearance Models 3D Morphable Model Face modeling Face synthesis Attributes 



This work was partially supported by the Swiss National Science Foundation, project 200021_146178: Copula Distributions in Machine Learning. We would like to thank Clemens Blumer, Antonia Bertschinger and Anna Engler for their valuable inputs and proofreading.


  1. 1.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, pp. 484–498. Springer, Heidelberg (1998). doi: 10.1007/BFb0054760 CrossRefGoogle Scholar
  2. 2.
    Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques. ACM Press/Addison-Wesley Publishing Co., pp. 187–194 (1999)Google Scholar
  3. 3.
    Jolliffe, I.: Principal Component Analysis. Wiley, Hoboken (2002)zbMATHGoogle Scholar
  4. 4.
    Mohammed, U., Prince, S.J., Kautz, J.: Visio-lization: generating novel facial images. ACM Trans. Graph. (TOG) 28, 57 (2009)CrossRefGoogle Scholar
  5. 5.
    Walker, M., Vetter, T.: Portraits made to measure: manipulating social judgments about individuals with a statistical face model. J. Vis. 9, 12 (2009)CrossRefGoogle Scholar
  6. 6.
    Schönborn, S., Forster, A., Egger, B., Vetter, T.: A monte carlo strategy to integrate detection and model-based face analysis. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 101–110. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-40602-7_11 CrossRefGoogle Scholar
  7. 7.
    Rasmussen, C.E.: The infinite gaussian mixture model. NIPS 12, 554–560 (1999)Google Scholar
  8. 8.
    Han, F., Liu, H.: Semiparametric principal component analysis. In: Advances in Neural Information Processing Systems, pp. 171–179 (2012)Google Scholar
  9. 9.
    Sirovich, L., Kirby, M.: Low-dimensional procedure for the characterization of human faces. JOSA A 4, 519–524 (1987)CrossRefGoogle Scholar
  10. 10.
    Turk, M., Pentland, A.P., et al.: Face recognition using eigenfaces. In: Proceedings: 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 1991, pp. 586–591. IEEE (1991)Google Scholar
  11. 11.
    Nelsen, R.B.: An Introduction to Copulas, vol. 139. Springer Science & Business Media, Heidelberg (2013)zbMATHGoogle Scholar
  12. 12.
    Joe, H.: Multivariate Models and Multivariate Dependence Concepts. CRC Press, Boca Raton (1997)CrossRefzbMATHGoogle Scholar
  13. 13.
    Genest, C., Ghoudi, K., Rivest, L.P.: A semiparametric estimation procedure of dependence parameters in multivariate families of distributions. Biometrika 82, 543–552 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Tsukahara, H.: Semiparametric estimation in copula models. Can. J. Stat. 33, 357–375 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Hoff, P.D.: Extending the rank likelihood for semiparametric copula estimation. Ann. Appl. Stat. 1(1), 265–283 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Paysan, P., Knothe, R., Amberg, B., Romdhani, S., Vetter, T.: A 3D face model for pose and illumination invariant face recognition. In: Sixth IEEE International Conference On Advanced Video and Signal Based Surveillance AVSS 2009, pp. 96–301. IEEE (2009)Google Scholar
  17. 17.
    Massey Jr., F.J.: The Kolmogorov-Smirnov test for goodness of fit. J. Am. Stat. Assoc. 46, 68–78 (1951)CrossRefzbMATHGoogle Scholar
  18. 18.
    Lothar Sachs, J.H.: Angewandte Statistik, 7th edn. Springer, Heidelberg (2006)zbMATHGoogle Scholar
  19. 19.
    Amberg, B., Romdhani, S., Vetter, T.: Optimal step Nonrigid ICP algorithms for surface registration. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2007, pp. 1–8. IEEE (2007)Google Scholar
  20. 20.
    Styner, M.A., Rajamani, K.T., Nolte, L.-P., Zsemlye, G., Székely, G., Taylor, C.J., Davies, R.H.: Evaluation of 3D correspondence methods for model building. In: Taylor, C., Noble, J.A. (eds.) IPMI 2003. LNCS, vol. 2732, pp. 63–75. Springer, Heidelberg (2003). doi: 10.1007/978-3-540-45087-0_6 CrossRefGoogle Scholar
  21. 21.
    Schumacher, M., Blanz, V.: Exploration of the correlations of attributes and features in faces. In: 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp. 1–8. IEEE (2015)Google Scholar
  22. 22.
    Blanc, R., Seiler, C., Székely, G., Nolte, L.P., Reyes, M.: Statistical model based shape prediction from a combination of direct observations and various surrogates: application to orthopaedic research. Med. Image Anal. 16, 1156–1166 (2012)CrossRefGoogle Scholar
  23. 23.
    Edwards, G.J., Lanitis, A., Taylor, C.J., Cootes, T.F.: Statistical models of face images improving specificity. Image Vis. Comput. 16, 203–211 (1998)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Bernhard Egger
    • 1
    Email author
  • Dinu Kaufmann
    • 1
  • Sandro Schönborn
    • 1
  • Volker Roth
    • 1
  • Thomas Vetter
    • 1
  1. 1.Department of Mathematics and Computer ScienceUniversity of BaselBaselSwitzerland

Personalised recommendations