Skip to main content
SpringerLink
Log in

Understanding personality of portrait by social embedding visual features

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Image understanding achieves unprecedented performance in content recognition and emotion rating recently. However, previous research mainly focused on visual features. In this paper, inspired by human cognitive activities, we discuss how to measure portrait distance in respective of high level semantic: personality, by employing features from both visual contents and behavior contents. Firstly, a new image distance metric, named Social and Visual Portrait Distance, is designed by jointly considering visual features and human social media behavior features. In portrait images, visual features are defined globally and locally utilizing theoretical and empirical concepts from psychology theory. While in social media, behavior features are designed with considerations of demographic factors, identity claims and behavioral residue. And the new distance is estimated referred to feature reliability. Secondly, we modify the proposed distance calculation formula in order to apply it on only visual features but preserving both visual and social relations by a new Social Embedding Portrait Distance Learning method. In this manner, we could measure the social embedding visual distance of common portrait images in absence of social media information, such as web portrait, or daily photos. Comprehensive experiments are employed to investigate the effectiveness of the new portrait distance and the metric learning method in representing personality distance compared with several baselines. Moreover, the learned distance matrix reveals a reasonable explanation of social preferred visual features with contribute partitions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Back MD et al (2010) Facebook profiles reflect actual personality, not self-idealization. Psychol Sci 21(3):372–374

    Article  Google Scholar 

  2. Bhaskar H, Mihaylova L, Maskell S (2013) Articulated human body parts detection based on cluster background subtraction and foreground matching. Neurocomputing 100:58–73

    Article  Google Scholar 

  3. Big Five personality traits (2017) Available https://en.wikipedia.org/wiki/Big_Five_personality_traits. Accessed 29 May 2017

  4. Buffardi LE, Campbell WK (2008) Narcissism and social networking web sites. Personal Soc Psychol Bull 34(10):1303–1314

    Article  Google Scholar 

  5. Cristani M, Vinciarelli A, Segalin C, Perina A (2013) Unveiling the multimedia unconscious: Implicit cognitive processes and multimedia content analysis. In: Proceedings of the 21st ACM international conference on Multimedia, pp 213–222

  6. Deng J, Berg AC, Fei-fei L (2011) Hierarchical semantic indexing for large scale image retrieval. In: IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs

  7. Gosling SD, Ko SJ, Mannarelli T, Morris ME (2002) A room with a cue: personality judgments based on offices and bedrooms. J Pers Soc Psychol 82(3):379

    Article  Google Scholar 

  8. Gosling SD, Ko SJ, Mannarelli T, Morris ME (Mar. 2002) A room with a cue: personality judgments based on offices and bedrooms. J Pers Soc Psychol 82(3):379–398

    Article  Google Scholar 

  9. Gosling SD, Gaddis S, Vazire S et al (2007) Personality impressions based on Facebook profiles. Int Conf Weblogs Soc Media 7:1–4

    Google Scholar 

  10. Ivcevic Z, Ambady N (2012) Personality impressions from identity claims on Facebook. Psychol Pop Media Cult 1(1):38–45

    Article  Google Scholar 

  11. Jia J, Wu S, Wang X, Hu P, Cai L, Tang J (2012) Can we understand van gogh’s mood?: learning to infer affects from images in social networks. In: Proceedings of the 20th ACM international conference on Multimedia, pp 857–860

  12. Kendall MG (1938) A new measure of rank correlation. Biometrika. [Online]. Available https://eurekamag.com/research/029/704/029704155.php. Accessed 29 May 2017

  13. Kiapour MH, Yamaguchi K, Berg AC, Berg TL (2014) Hipster wars: discovering elements of fashion styles. In: European Conference on Computer Vision, Zurich, pp 472–488

  14. Le Q, Mikolov T (2014) Distributed Representations of Sentences and Documents. In: PMLR, pp 1188–1196

  15. Li F, Ding Y, Zhou M, Hao K, Chen L (2016) An affection-based dynamic leader selection model for formation control in multirobot systems. IEEE Trans Syst Man Cybern Syst PP(99):1–12

    Article  Google Scholar 

  16. Liu S, Cui P, Zhu W, Yang S, Tian Q (2014) Social Embedding Image Distance Learning. In: Proceedings of the 22Nd ACM International Conference on Multimedia, New York, pp 617–626

  17. Liu L, Preotiuc-Pietro D, Samani ZR, Moghaddam ME, Ungar L (2016) Analyzing Personality through Social Media Profile Picture Choice. In: Tenth International AAAI Conference on Web and Social Media

  18. Machajdik J, Hanbury A (2010) Affective image classification using features inspired by psychology and art theory. In: Proceedings of the 18th ACM international conference on Multimedia, pp 83–92

  19. Naumann LP, Vazire S, Rentfrow PJ, Gosling SD (2009) Personality judgments based on physical appearance. Personal Soc Psychol Bull 35(12):1661–1671

    Article  Google Scholar 

  20. Nie J, Cui P, Yan Y, Huang L, Li Z, Wei Z (2014) How your portrait impresses people? Inferring personality impressions from portrait contents. In: MM 2014 - Proceedings of the 2014 ACM Conference on Multimedia, pp 905–908

  21. Nie J, Huang L, Li Z, Wei C, Hong B, Zhu W (2016) Thinking like psychologist: Learning to predict personality by using features from portrait and social media. In: 2016 4th International Conference on Cloud Computing and Intelligence Systems (CCIS), pp 21–26

  22. Picard RW, Picard R (1997) Affective computing, vol 252. MIT press, Cambridge

    Google Scholar 

  23. Steele F, Evans DC, Green RK (2009) Is Your Profile Picture Worth 1000 Words? Photo Characteristics Associated with Personality Impression Agreement. In: AAAI International Conference on Weblogs & Social Media, San Jose

  24. Todorov A, Said C, Verosky S (2011) Personality impressions from facial appearance. In: Calder A, Haxby JV, Johnson M, Rhodes G (eds) Handbook of face perception. Oxford University Press, New York, pp 631–652

  25. Wang HL, Cheong L-F (2006) Affective understanding in film. IEEE Trans Circuits Syst Video Technol 16(6):689–704

    Article  Google Scholar 

  26. Wang W, He Q (2008) A survey on emotional semantic image retrieval. In: IEEE International Conference on Image Processing, pp 117–120

  27. Weinberger KQ, Saul LK (2009) Distance metric learning for large margin nearest neighbor classification. J Mach Learn Res 10(Feb):207–244

    MATH  Google Scholar 

  28. Weisbuch M, Ivcevic Z, Ambady N (2009) On being liked on the web and in the real world: consistency in first impressions across personal webpages and spontaneous behavior. J Exp Soc Psychol 45(3):573–576

    Article  Google Scholar 

  29. Yan Y, Nie J, Huang L, Li Z, Cao Q, Wei Z (2015) Is your first impression reliable? trustworthy analysis using facial traits in portraits. In: International Conference on Multimedia Modeling, Sydney, pp 148–158

  30. Yan Y, Nie J, Huang L, Li Z, Cao Q, Wei Z (2016) Exploring relationship between face and trustworthy impression using mid-level facial features. In: International Conference on Multimedia Modeling, Miami, pp 540–549

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (No.61402428, No.61702471, No. 61672475); Aoshan Innovation Project in Science and Technology of Qingdao National Laboratory for Marine Science and Technology (No. 2016ASKJ07); Qingdao Science and Technology Development Plan (No. 16-5-1-13-jch).

Author information

Authors and Affiliations

  1. College of Information Science and Engineering, Ocean University of China, Qingdao, China

    Jie Nie, Zhiqiang Wei, Zhen Li, Yan Yan & Lei Huang

Authors
  1. Jie Nie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhiqiang Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhen Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yan Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lei Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lei Huang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nie, J., Wei, Z., Li, Z. et al. Understanding personality of portrait by social embedding visual features. Multimed Tools Appl 78, 727–746 (2019). https://doi.org/10.1007/s11042-017-5577-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5577-x

Keywords

Search

Navigation