Predicting Facial Beauty without Landmarks

  • Douglas Gray
  • Kai Yu
  • Wei Xu
  • Yihong Gong
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6316)


A fundamental task in artificial intelligence and computer vision is to build machines that can behave like a human in recognizing a broad range of visual concepts. This paper aims to investigate and develop intelligent systems for learning the concept of female facial beauty and producing human-like predictors. Artists and social scientists have long been fascinated by the notion of facial beauty, but study by computer scientists has only begun in the last few years. Our work is notably different from and goes beyond previous works in several aspects: 1) we focus on fully-automatic learning approaches that do not require costly manual annotation of landmark facial features but simply take the raw pixels as inputs; 2) our study is based on a collection of data that is an order of magnitude larger than that of any previous study; 3) we imposed no restrictions in terms of pose, lighting, background, expression, age, and ethnicity on the face images used for training and testing. These factors significantly increased the difficulty of the learning task. We show that a biologically-inspired model with multiple layers of trainable feature extractors can produce results that are much more human-like than the previously used eigenface approach. Finally, we develop a novel visualization method to interpret the learned model and revealed the existence of several beautiful features that go beyond the current averageness and symmetry hypotheses.


Face Image Multiscale Model Facial Attractiveness Absolute Score Luminance Channel 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cross, J., Cross, J.: Age, Sex, Race, and the Perception of Facial Beauty. Developmental Psychology 5, 433–439 (1971)CrossRefGoogle Scholar
  2. 2.
    Aarabi, P., Hughes, D., Mohajer, K., Emami, M.: The automatic measurement of facial beauty. In: IEEE International Conference on Systems, Man, and Cybernetics, vol. 4 (2001)Google Scholar
  3. 3.
    Alley, T., Cunningham, M.: Averaged faces are attractive, but very attractive faces are not average. Psychological Science 2, 123–125 (1991)CrossRefGoogle Scholar
  4. 4.
    Grammer, K., Thornhill, R.: Human (Homo sapiens) facial attractiveness and sexual selection: the role of symmetry and averageness. J. Comp. Psychol. 108, 233–242 (1994)CrossRefGoogle Scholar
  5. 5.
    Zhou, Y., Gu, L., Zhang, H.: Bayesian tangent shape model: estimating shape and pose parameters via Bayesian inference. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1 (2003)Google Scholar
  6. 6.
    Eisenthal, Y., Dror, G., Ruppin, E.: Facial Attractiveness: Beauty and the Machine (2005)Google Scholar
  7. 7.
    Kagian, A., Dror, G., Leyvand, T., Cohen-Or, D., Ruppin, E.: A Humanlike Predictor of Facial Attractiveness. In: Advances in Neural Information Processing Systems, pp. 649–656 (2005)Google Scholar
  8. 8.
    Guo, D., Sim, T.: Digital face makeup by example. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2009)Google Scholar
  9. 9.
    Hubel, D., Wiesel, T.: Receptive fields and functional architecture of monkey striate cortex. The Journal of Physiology 195, 215–243 (1968)Google Scholar
  10. 10.
    Likert, R.: Technique for the measurement of attitudes. Arch. Psychol. 22, 55 (1932)Google Scholar
  11. 11.
    Oliva, A., Torralba, A.: Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope. International Journal of Computer Vision 42, 145–175 (2001)zbMATHCrossRefGoogle Scholar
  12. 12.
    Chopra, S., Hadsell, R., LeCun, Y.: Learning a similarity metric discriminatively, with application to face verification. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1 (2005)Google Scholar
  13. 13.
    Fukushima, K.: Neocognitron: A hierarchical neural network capable of visual pattern recognition. Neural Networks 1, 119–130 (1988)CrossRefGoogle Scholar
  14. 14.
    Hubel, D., Wiesel, T.: Receptive fields, binocular interaction and functional architecture in the cats visual cortex. Journal of Physiology 160, 106–154 (1962)Google Scholar
  15. 15.
    Huang, G., Jain, V., Amherst, M., Learned-Miller, E.: Unsupervised Joint Alignment of Complex Images. In: IEEE International Conference on Computer Vision (2007)Google Scholar
  16. 16.
    Gunes, H., Piccardi, M., Jan, T.: Comparative beauty classification for pre-surgery planning. In: IEEE International Conference on Systems, Man and Cybernetics, vol. 3 (2004)Google Scholar
  17. 17.
    Joy, K., Primeaux, D.: A Comparison of Two Contributive Analysis Methods Applied to an ANN Modeling Facial Attractiveness. In: International Conference on Software Engineering Research, Management and Applications, pp. 82–86 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Douglas Gray
    • 1
  • Kai Yu
    • 2
  • Wei Xu
    • 2
  • Yihong Gong
    • 1
  1. 1.Akiira Media Systems 
  2. 2.NEC Labs America 

Personalised recommendations