Detecting personality and emotion traits in crowds from video sequences

  • Rodolfo Migon FavarettoEmail author
  • Paulo Knob
  • Soraia Raupp Musse
  • Felipe Vilanova
  • Ângelo Brandelli Costa
Special Issue Paper


This paper presents a methodology to detect personality and basic emotion characteristics of crowds in video sequences. Firstly, individuals are detected and tracked, and then groups are recognized and characterized. Such information is then mapped to OCEAN dimensions, used to find out personality and emotion in videos, based on OCC emotion models. Although it is a clear challenge to validate our results with real life experiments, we evaluate our method with the available literature information regarding OCEAN values of different Countries and also emergent Personal distance among people. Hence, such analysis refer to cultural differences of each country too. Our results indicate that this model generates coherent information when compared to data provided in available literature, as shown in qualitative and quantitative results.


Computer vision Crowd features Big Five model Cultural dimensions Crowd emotion 

Supplementary material

138_2018_979_MOESM1_ESM.mp4 (138.7 mb)
Supplementary material 1 (mp4 142068 KB)


  1. 1.
    Baig, M.W., Baig, M.S., Bastani, V., Barakova, E.I., Marcenaro, L., Regazzoni, C.S., Rauterberg, M.: Perception of emotions from crowd dynamics. In: 2015 IEEE International Conference on Digital Signal Processing (DSP), pp. 703–707. IEEE (2015)Google Scholar
  2. 2.
    Barry, B., Stewart, G.L.: Composition, process, and performance in self-managed groups: the role of personality. J. Appl. Psychol. 82, 62 (1997)CrossRefGoogle Scholar
  3. 3.
    Bins, J., Dihl, L.L., Jung, C.R.: Target tracking using multiple patches and weighted vector median filters. MIV 45(3), 293–307 (2013). MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Bon, G.: The Crowd: A Study of the Popular Mind. Cosimo Classics Personal Development. Cosimo Classics, New York (2006)Google Scholar
  5. 5.
    Cai, Z., Yu, Z.L., Liu, H., Zhang, K.: Counting people in crowded scenes by video analyzing. In: 9th IEEE ICIEA, pp. 1841–1845 (2014).
  6. 6.
    Cattell, R.B.: The primary personality factors in women compared with those in men. Br. J. Stat. Psychol. 1(2), 114–130 (1948). CrossRefGoogle Scholar
  7. 7.
    Cattell, R.B.: Personality: A Systematic, Theoretical, and Factual Study, 1st edn. McGraw-Hill, New York (1950)CrossRefGoogle Scholar
  8. 8.
    Chan, A.B., Vasconcelos, N.: Bayesian poisson regression for crowd counting. In: 12th IEEE ICCV, pp. 545–551 (2009)Google Scholar
  9. 9.
    Chandran, A., Poh, L.A., Vadakkepat, P.: Identifying social groups in pedestrian crowd videos. In: ICAPR, pp. 1–6 (2015).
  10. 10.
    Chattaraj, U., Seyfried, A., Chakroborty, P.: Comparison of pedestrian fundamental diagram across cultures. ACS 12(03), 393–405 (2009). CrossRefGoogle Scholar
  11. 11.
    Chattaraj, U., Seyfried, A., Chakroborty, P.: Comparison of pedestrian fundamental diagram across cultures. ACS (2009). CrossRefGoogle Scholar
  12. 12.
    Clore, G.L., Ortony, A.: Psychological construction in the OCC model of emotion. Emot. Rev. 5(4), 335–343 (2013)CrossRefGoogle Scholar
  13. 13.
    Costa, Jr. P.T., McCrae, R.R.: Revised NEO Personality Inventory (NEO-PI-R) and the NEO Five-Factor Inventory (NEO-FFI) Professional Manual. Psychological Assessment Resources, Odessa (1992)Google Scholar
  14. 14.
    Costa, Jr. P.T., McCrae, R.R.: NEO PI-R: inventrio de personalidade NEO revisado. Sao Paulo (2007)Google Scholar
  15. 15.
    Davis, K.L., Panksepp, J.: The brain’s emotional foundations of human personality and the affective neuroscience personality scales. Neurosci. Biobehav. Rev. 35(9), 1946–1958 (2011)CrossRefGoogle Scholar
  16. 16.
    Davis, K.L., Panksepp, J., Normansell, L.: The affective neuroscience personality scales: normative data and implications. Neuropsychoanalysis 5(1), 57–69 (2003)CrossRefGoogle Scholar
  17. 17.
    Digman, J.M.: Personality structure: emergence of the five-factor model. Annu. Rev. Psychol. 41(1), 417–440 (1990). CrossRefGoogle Scholar
  18. 18.
    Durupinar, F., Allbeck, J., Pelechano, N., Badler, N.: Creating crowd variation with the ocean personality model. In: Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems—vol. 3, pp. 1217–1220. IFAAMAS, Richland (2008).
  19. 19.
    Ekman, P., Friesen, W.V.: Constants across cultures in the face and emotion. J. Pers. Soc. Psychol. 17(2), 124 (1971)CrossRefGoogle Scholar
  20. 20.
    Favaretto, R.M., Dihl, L., Musse, S.R.: Detecting crowd features in video sequences. In: Proceedings of Conference on Graphics, Patterns and Images (SIBGRAPI). IEEE Computer Society Conference Publishing Services (2016).
  21. 21.
    Favaretto, R.M., Dihl, L., Musse, S.R., Vilanova, F., Costa, A.B.: Using big five personality model to detect cultural aspects in crowds. In: 2017 30th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 223–229 (2017).
  22. 22.
    Feng, L., Bhanu, B.: Understanding dynamic social grouping behaviors of pedestrians. IEEE STSP 9(2), 317–329 (2015)Google Scholar
  23. 23.
    Fiske, D.W.: Consistency of The Factorial Structures in Personality Ratings from Different Sources. University of MICHIGAN (1948).
  24. 24.
    Gorbova, J., Lsi, I., Litvin, A., Anbarjafari, G.: Automated screening of job candidate based on multimodal video processing. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1679–1685 (2017).
  25. 25.
    Guy, S.J., Kim, S., Lin, M.C., Manocha, D.: Simulating heterogeneous crowd behaviors using personality trait theory. In: Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA ’11, pp. 43–52. ACM, New York (2011).
  26. 26.
    Hall, E.: The Hidden Dimension. A Doubleday Anchor Book. Anchor Books (1990).
  27. 27.
    Hall, C.S., Lindzey, G., Campbell, J.B.: Theories of Personality, 4th edn. Wiley, New Jersey (1998)Google Scholar
  28. 28.
    Jo, H., Chug, K., Sethi, R.J.: A review of physics-based methods for group and crowd analysis in computer vision. J. Postdr. Res. 1(1), 4–7 (2013)Google Scholar
  29. 29.
    Kaup, D., Clarke, T., Malone, L., Oleson, R.: Society for computer simulation. Simul. Ser. 38(4), 365–370 (2006)Google Scholar
  30. 30.
    Lord, W.: Neo Pi-R A Guide to Interpretation and Feedback in a Work Context, 1st edn. Hogrefe Ltd, Oxford (2007)Google Scholar
  31. 31.
    McCrae, R., Terracciano, A.: Universal features of personality traits from the observer’s perspective : data from 50 cultures. J. Pers. Soc. Psychol. 88(3), 547–561 (2005)CrossRefGoogle Scholar
  32. 32.
    McCrae, R.R.: NEO-PI-R Data from 36 Cultures, pp. 105–125. Springer US, Boston (2002). CrossRefGoogle Scholar
  33. 33.
    Montag, C., Panksepp, J.: Primary emotional systems and personality: an evolutionary perspective. Front. Psychol. 8, 464 (2017)CrossRefGoogle Scholar
  34. 34.
    Ortony, A., Clore, G.L., Collins, A.: The Cognitive Structure of Emotions. Cambridge University Press, Cambridge (1990)Google Scholar
  35. 35.
    Rabiee, H., Haddadnia, J., Mousavi, H., Nabi, M., Murino, V., Sebe, N.: Emotion-based crowd representation for abnormality detection. arXiv preprint arXiv:1607.07646 (2016)
  36. 36.
    Revelle, W., Scherer, K.R.: Personality and emotion. In: Sander D, Scherer K (eds) Oxford Companion to Emotion and the Affective Sciences. Oxford University Press, New York, pp. 304–306 (2009)Google Scholar
  37. 37.
    Saifi, L., Boubetra, A., Nouioua, F.: An approach for emotions and behavior modeling in a crowd in the presence of rare events. Adapt. Behav. 24(6), 428–445 (2016)CrossRefGoogle Scholar
  38. 38.
    Sethi, R.J.: Towards defining groups and crowds in video using the atomic group actions dataset. In: 2015 IEEE International Conference on Image Processing, ICIP 2015, Quebec City, QC, Canada, September 27-30, 2015, pp. 2925–2929 (2015).
  39. 39.
    Shao, J., Loy, C.C., Wang, X.: Scene-independent group profiling in crowd. In: IEEE CVPR, pp. 2227–2234 (2014).
  40. 40.
    Solera, F., Calderara, S., Cucchiara, R.: Structured learning for detection of social groups in crowd. In: 10th IEEE AVSS (2013)Google Scholar
  41. 41.
    Solmaz, B., Moore, B.E., Shah, M.: Identifying behaviors in crowd scenes using stability analysis for dynamical systems. IEEE PAMI 34(10), 2064–2070 (2012). CrossRefGoogle Scholar
  42. 42.
    Sorokowska, A., Sorokowski, P.: P.H.: preferred interpersonal distances: a global comparison. J. Cross Cult. Psychol. 48(4), 577–592 (2017). CrossRefGoogle Scholar
  43. 43.
    Vilanova, F., Beria, F., et al.: A.C.: deindividuation: from le bon to the social identity model of deindividuation effects. CP 4(1) (2017).
  44. 44.
    Zhang, Y., Qin, L., Ji, R., Zhao, S., Huang, Q., Luo, J.: Exploring coherent motion patterns via structured trajectory learning for crowd mood modeling. IEEE Trans. Circuits Syst. Video Technol. 27(3), 635–648 (2017)CrossRefGoogle Scholar
  45. 45.
    Zhou, B., Tang, X., Zhang, H., Wang, X.: Measuring crowd collectiveness. IEEE PAMI 36(8), 1586–1599 (2014). CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.VHLab, Graduate Program in Computer SciencePUCRSPorto AlegreBrazil
  2. 2.Graduate Program in PsychologyPUCRSPorto AlegreBrazil

Personalised recommendations