Advertisement

Physiognomy: Personality traits prediction by learning

  • Ting Zhang
  • Ri-Zhen Qin
  • Qiu-Lei Dong
  • Wei Gao
  • Hua-Rong Xu
  • Zhan-Yi Hu
Research Article

Abstract

Evaluating individuals’ personality traits and intelligence from their faces plays a crucial role in interpersonal relationship and important social events such as elections and court sentences. To assess the possible correlations between personality traits (also measured intelligence) and face images, we first construct a dataset consisting of face photographs, personality measurements, and intelligence measurements. Then, we build an end-to-end convolutional neural network for prediction of personality traits and intelligence to investigate whether self-reported personality traits and intelligence can be predicted reliably from a face image. To our knowledge, it is the first work where deep learning is applied to this problem. Experimental results show the following three points: 1) “Rule-consciousness” and “Tension” can be reliably predicted from face images. 2) It is difficult, if not impossible, to predict intelligence from face images, a finding in accord with previous studies. 3) Convolutional neural network (CNN) features outperform traditional handcrafted features in predicting traits.

Keywords

Personality traits physiognomy face image deep learning convolutional neural network (CNN) 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgement

We are grateful to all the students of Xiamen Institute of Technology in China who participated in this study.

References

  1. [1]
    D. McNeill. The Face: A Natural History, New York, USA: Back Bay Books, 2000.Google Scholar
  2. [2]
    A. Todorov, C. P. Said, A. D. Engell, N. N. Oosterhof. Understanding evaluation of faces on social dimensions. Trends in Cognitive Sciences, vol. 12, no. 12, pp. 455–460, 2008.CrossRefGoogle Scholar
  3. [3]
    C. C. Ballew II, A. Todorov. Predicting political elections from rapid and unreflective face judgments. Proceedings of the National Academy of Sciences of the United States of America, vol. 104, no. 46, pp. 17948–17953, 2007.CrossRefGoogle Scholar
  4. [4]
    A. C. Little, R. P. Burriss, B. C. Jones, S. C. Roberts. Facial appearance affects voting decisions. Evolution and Human Behavior, vol. 28, no. 1, pp. 18–27, 2007.CrossRefGoogle Scholar
  5. [5]
    I. V. Blair, C. M. Judd, K. M. Chapleau. The influence of afrocentric facial features in criminal sentencing. Psychological Science, vol. 15, no. 10, pp. 674–679, 2004.CrossRefGoogle Scholar
  6. [6]
    D. R. Carney, C. R. Colvin, J. A. Hall. A thin slice perspective on the accuracy of first impressions. Journal of Research in Personality, vol. 41, no. 5, pp. 1054–1072, 2007.CrossRefGoogle Scholar
  7. [7]
    R. S. S. Kramer, J. E. King, R. Ward. Identifying personality from the static, nonexpressive face in humans and chimpanzees: Evidence of a shared system for signaling personality. Evolution and Human Behavior, vol. 32, no. 3, pp. 179–185, 2011.CrossRefGoogle Scholar
  8. [8]
    Q. M. Rojas, D. Masip, A. Todorov, J. Vitriä. Automatic point-based facial trait judgments evaluation. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, San Francisco, USA, pp. 2715–2720, 2010.Google Scholar
  9. [9]
    Q. M. Rojas, D. Masip, A. Todorov, J. Vitria. Automatic prediction of facial trait judgments: Appearance vs. structural models. PLoS One, vol. 6, no. 8, Article number e23323, 2011.Google Scholar
  10. [10]
    K. Wolffhechel, J. Fagertun, U. P. Jacobsen, W. Majewski, A. S. Hemmingsen, C. L. Larsen, S. K. Lorentzen, H. Jarmer. Interpretation of appearance: The effect of facial features on first impressions and personality. PLoS One, vol. 9, no. 9, Article number e107721, 2014.CrossRefGoogle Scholar
  11. [11]
    K. Kleisner, V. Chvátalová, J. Flegr. Perceived intelligence is associated with measured intelligence in men but not women. PLoS One, vol. 9, no. 3, Article number e81237, 2014.CrossRefGoogle Scholar
  12. [12]
    J. Yosinski, J. Clune, Y. Bengio, H. Lipson. How transferable are features in deep neural networks? In Proceedings of the 27th International Conference on Neural Information Processing Systems, NIPS, Montréal, Canada, pp. 3320–3328, 2014.Google Scholar
  13. [13]
    M. S. Long, Y. Cao, J. M. Wang, M. I. Jordan. Learning transferable features with deep adaptation networks. In Proceedings of the 32nd International Conference on Machine Learning, JMLR, Lille, France, 2015.Google Scholar
  14. [14]
    O. M. Parkhi, A. Vedaldi, A. Zisserman. Deep face recognition. In Proceedings of British Machine Vision Conference, Swansea, UK, vol. 41, pp. 1–12, 2015.Google Scholar
  15. [15]
    G. B. Huang, M. Ramesh, T. Berg, E. Learned-Miller. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, USA, 2007.Google Scholar
  16. [16]
    Y. Taigman, M. Yang, M. A. Ranzato, L. Wolf. Deepface: Closing the gap to human-level performance in face verification. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Columbus, USA, pp. 1701–1708, 2014.Google Scholar
  17. [17]
    Y. Sun, X. G. Wang, X. O. Tang. Deep learning face representation from predicting 10,000 classes. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Columbus, USA, pp. 1891–1898, 2014.Google Scholar
  18. [18]
    Z. Y. Zhu, P. Luo, X. G. Wang, X. O. Tang. Deep learning identity-preserving face space. In Proceedings of IEEE International Conference on Computer Vision, IEEE, Sydney, Australia, pp. 113–120, 2013.Google Scholar
  19. [19]
    Y. Sun, Y. H. Chen, X. G. Wang, X. O. Tang. Deep learning face representation by joint identification-verification. In Proceedings of the 27th International Conference on Neural Information Processing Systems, NIPS, Montréal, Canada, pp. 1988–1996, 2014.Google Scholar
  20. [20]
    Y. Taigman, M. Yang, M. A. Ranzato, L. Wolf. Webscale training for face identification. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Boston, USA, pp. 2746–2754, 2015.Google Scholar
  21. [21]
    T. Zhang, Q. L. Dong, Z. Y. Hu. Pursuing face identity from view-specific representation to view-invariant representation. In Proceedings of IEEE International Conference on Image Processing, IEEE, Phoenix, USA, pp. 3244–3248, 2016.Google Scholar
  22. [22]
    B. Zhao, J. S. Feng, X. Wu, S. C. Yan. A survey on deep learning-based fine-grained object classification and semantic segmentation. International Journal of Automation and Computing, vol. 14, no. 2, pp. 1–17, 2017.CrossRefGoogle Scholar
  23. [23]
    N. Kumar, A. C. Berg, P. N. Belhumeur, S. K. Nayar. Attribute and simile classifiers for face verification. In Proceedings of IEEE International Conference on Computer Vision, IEEE, Kyoto, Japan, pp. 365–372, 2009.Google Scholar
  24. [24]
    Y. Taigman, L. Wolf, T. Hassner. Multiple one-shots for utilizing class label information. In Proceedings of British Machine Vision Conference, London, UK, vol. 2, pp. 1–12, 2009.Google Scholar
  25. [25]
    M. Guillaumin, J. Verbeek, C. Schmid. Is that you? Metric learning approaches for face identification. In Proceedings of IEEE International Conference on Computer Vision, IEEE, Kyoto, Japan, pp. 498–505, 2009.Google Scholar
  26. [26]
    Q. Yin, X. O. Tang, J. Sun. An associate-predict model for face recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Colorado Springs, USA, pp. 497–504, 2011.Google Scholar
  27. [27]
    C. Huang, S. H. Zhu, K. Yu. Large scale strongly supervised ensemble metric learning, with applications to face verification and retrieval. arXiv:1212.6094, 2012.Google Scholar
  28. [28]
    D. Chen, X. D. Cao, L. W. Wang, F. Wen, J. Sun. Bayesian face revisited: A joint formulation. In Proceedings of the 12th European Conference on Computer Vision, Florence, Italy, pp. 566–579, 2012.Google Scholar
  29. [29]
    T. Berg, P. N. Belhumeur. Tom-vs-pete classifiers and identity-preserving alignment for face verification. In Proceedings of British Machine Vision Conference, Guildford, UK, vol. 129, pp. 1–11, 2012.Google Scholar
  30. [30]
    D. Chen, X. D. Cao, F. Wen, J. Sun. Blessing of dimensionality: High-dimensional feature and its efficient compression for face verification. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Portland, USA, pp. 3025–3032, 2013.Google Scholar
  31. [31]
    X. D. Cao, D. Wipf, F. Wen, G. Q. Duan, J. Sun. A practical transfer learning algorithm for face verification. In Proceedings of IEEE International Conference on Computer Vision, IEEE, Barcelona, Spain, pp. 3208–3215, 2013.Google Scholar
  32. [32]
    F. K. Zaman, A. A. Shafie, Y. M. Mustafah. Robust face recognition against expressions and partial occlusions. International Journal of Automation and Computing, vol. 13, no. 4, pp. 319–337, 2016.CrossRefGoogle Scholar
  33. [33]
    N. N. Oosterhof, A. Todorov. The functional basis of face evaluation. Proceedings of the National Academy of Sciences of the United States of America, vol. 105, no. 32, pp. 11087–11092, 2008.CrossRefGoogle Scholar
  34. [34]
    S. Karson. A Guide to the Clinical Use of the 16 pf, Savoy, USA: Institute for Personality and Ability Testing, 1976.Google Scholar
  35. [35]
    R. M. Kaplan, D. P. Saccuzzo. Psychological Testing: Principles, Applications, and Issues, Boston, USA: Wadsworth Publishing, 2012.Google Scholar
  36. [36]
    R. Z. Qin, T. Zhang. Shape initialization without ground truth for face alignment. In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE, Shanghai, China, pp. 1278–1282, 2016.Google Scholar
  37. [37]
    Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, T. Darrell. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of ACM International Conference on Multimedia, ACM, Orlando, USA, 2014.Google Scholar
  38. [38]
    N. Dalal, B. Triggs. Histograms of oriented gradients for human detection. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, San Diego, USA, pp. 886–893, 2005.Google Scholar
  39. [39]
    T. Ahonen, A. Hadid, M. Pietikainen. Face description with local binary patterns: Application to face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 12, pp. 2037–2041, 2006.CrossRefzbMATHGoogle Scholar
  40. [40]
    C. J. Liu, H. Wechsler. Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition. IEEE Transactions on Image Processing, vol. 11, no. 4, pp. 467–476, 2002.CrossRefGoogle Scholar
  41. [41]
    D. G. Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, vol. 60, no. 2, pp. 91–110, 2004.CrossRefGoogle Scholar
  42. [42]
    A. Oliva, A. Torralba. Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision, vol. 42, no. 3, pp. 145–175, 2001.CrossRefzbMATHGoogle Scholar

Copyright information

© Institute of Automation, Chinese Academy of Sciences and Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  1. 1.National Laboratory of Pattern Recognition, Institute of AutomationChinese Academy of SciencesBeijingChina
  2. 2.Center for Excellence in Brain Science and Intelligence TechnologyChinese Academy of SciencesBeijingChina
  3. 3.University of Chinese Academy of SciencesBeijingChina
  4. 4.Department of Computer Science & TechnologyXiamen Institute of TechnologyXiamenChina

Personalised recommendations