Abstract
Recently, the automatic prediction of personality traits has received a lot of attention. Specifically, personality trait prediction from multimodal data has emerged as a hot topic within the field of affective computing. In this paper, we review significant machine learning models which have been employed for personality detection, with an emphasis on deep learning-based methods. This review paper provides an overview of the most popular approaches to automated personality detection, various computational datasets, its industrial applications, and state-of-the-art machine learning models for personality detection with specific focus on multimodal approaches. Personality detection is a very broad and diverse topic: this survey only focuses on computational approaches and leaves out psychological studies on personality detection.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
The Great Hack, a recent documentary about the Cambridge Analytica data scandal
References
Agarwal B (2014) Personality detection from text: a review. Int J Comput Syst 1:1–4
Al Moubayed N, Vazquez-Alvarez Y, McKay A, Vinciarelli A (2014) Face-based automatic personality perception. In: Proceedings of the 22nd ACM international conference on multimedia. ACM, pp 1153–1156
An G, Levitan R (2018) Lexical and acoustic deep learning model for personality recognition. In: Proceedings of interspeech, pp 1761–1765. https://doi.org/10.21437/Interspeech.2018-2263
Baltrusaitis T, Robinson P, Morency LP (2016) Openface: an open source facial behavior analysis toolkit. In: IEEE winter conference on applications of computer vision (WACV). IEEE, pp 1–10
Biel JI, Teijeiro-Mosquera L, Gatica-Perez D (2012) Facetube: predicting personality from facial expressions of emotion in online conversational video. In: Proceedings of the 14th ACM international conference on multimodal interaction. ACM, pp 53–56
Blei DM, Ng AY, Jordan MI (2003) Latent dirichlet allocation. J Mach Lear Res 3:993–1022
Briggs-Myers I (1993) Introduction to type: a guide to understanding your results on the Myers-Briggs type indicator (revised by lk kirby & kd myers), Palo alto
Bruce V, Young A (1986) Understanding face recognition. Br J Psychol 77(3):305–327
Busso C, Bulut M, Lee CC, Kazemzadeh A, Mower E, Kim S, Chang JN, Lee S, Narayanan SS (2008) IEMOCAP: interactive emotional dyadic motion capture database. Lang Resour Eval 42(4):335
Cambria E, Poria S, Hazarika D, Kwok K (2018) SenticNet 5: discovering conceptual primitives for sentiment analysis by means of context embeddings. In: AAAI, pp 1795–1802
Cambria E, Poria S, Gelbukh A, Thelwall M (2017) Sentiment analysis is a big suitcase. IEEE Int Syst 32(6):74–80
Caprara GV, Schwartz S, Capanna C, Vecchione M, Barbaranelli C (2006) Personality and politics: values, traits, and political choice. Polit Psychol 27(1):1–28
Carletta J (2007) Unleashing the killer corpus: experiences in creating the multi-everything AMI meeting corpus. Lang Resour Eval 41(2):181–190
Carletta J, Ashby S, Bourban S, Flynn M, Guillemot M, Hain T, Kadlec J, Karaiskos V, Kraaij W, Kronenthal M et al (2005) The ami meeting corpus: a pre-announcement. In: International workshop on machine learning for multimodal interaction. Springer, pp 28–39
Cattell HE, Mead AD (2008) The sixteen personality factor questionnaire (16pf). The SAGE handbook of personality theory and assessment 2:135–178
Celli F (2012) Unsupervised personality recognition for social network sites. In: Proceedings of sixth international conference on digital society
Celli F, Poesio M (2014) Pr2: a language independent unsupervised tool for personality recognition from text. arXiv preprint arXiv:1402.2796
Celli F, Di Lascio FML, Magnani M, Pacelli B, Rossi L (2010) Social network data and practices: the case of friendfeed. In: International conference on social computing, behavioral modeling, and prediction. Springer, pp 346–353
Chen B, Escalera S, Guyon I, Ponce-López V, Shah N, Simón MO (2016) Overcoming calibration problems in pattern labeling with pairwise ratings: application to personality traits. In: European conference on computer vision. Springer, pp 419–432
Chittaranjan G, Blom J, Gatica-Perez D (2011) Who’s who with big-five: analyzing and classifying personality traits with smartphones. In: 15th Annual international symposium on wearable computers (ISWC). IEEE, pp 29–36
Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555
Cristani M, Vinciarelli A, Segalin C, Perina A (2013) Unveiling the multimedia unconscious: implicit cognitive processes and multimedia content analysis. In: Proceedings of the 21st ACM international conference on multimedia. ACM, pp 213–222
CVPR (2017) Chalearn dataset looking at people challenge. http://chalearnlap.cvc.uab.es/dataset/24/description/. Accessed 09 Oct 2019
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE computer society conference on computer vision and pattern recognition, CVPR 2005, vol 1. IEEE, pp 886–893
Davis SB, Mermelstein P (1990) Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. In: Readings in speech recognition. Elsevier, pp 65–74
Digman JM (1990) Personality structure: emergence of the five-factor model. Ann Rev Psychol 41(1):417–440
Eddine Bekhouche S, Dornaika F, Ouafi A, Taleb-Ahmed A (2017) Personality traits and job candidate screening via analyzing facial videos. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 10–13
Escalante HJ, Guyon I, Escalera S, Jacques J, Madadi M, Baró X, Ayache S, Viegas E, Güçlütürk Y, Güçlü U et al (2017) Design of an explainable machine learning challenge for video interviews. In: International joint conference on neural networks (IJCNN). IEEE, pp 3688–3695
Escalante HJ, Kaya H, Salah AA, Escalera S, Gucluturk Y, Guclu U, Baró X, Guyon I, Junior JJ, Madadi M et al (2018) Explaining first impressions: modeling, recognizing, and explaining apparent personality from videos. arXiv preprint arXiv:1802.00745
Escalera S, Baró X, Guyon I, Escalante HJ (2018) Guest editorial: apparent personality analysis. IEEE Trans Affect Comput 9(3):299–302
Eyben F, Wollmer M, Schuller B (2010) Opensmile: the munich versatile and fast open-source audio feature extractor. In: Proceedings of the 18th ACM international conference on multimedia. ACM, pp 1459–1462
Eysenck HJ (2012) A model for personality. Springer, Berlin
Freund Y, Schapire RE et al (1996) Experiments with a new boosting algorithm. ICML, Bari, Italy 96:148–156
Furnham A (1990) Can people accurately estimate their own personality test scores? Eur J Personal 4(4):319–327
Furnham A (1996) The big five versus the big four: the relationship between the Myers-Briggs type indicator (MBTI) and NEO-PI five factor model of personality. Personal Individ Differ 21(2):303–307
Golbeck J (2016) Predicting personality from social media text. AIS Trans Replication Res 2(1):2
Gonzalez-Gallardo CE, Montes A, Sierra G, Nunez-Juarez JA, Salinas-Lopez AJ, Ek J (2015) Tweets classification using corpus dependent tags, character and pos n-grams. In: CLEF (working notes)
Gorbova J, Lusi I, Litvin A, Anbarjafari G (2017) Automated screening of job candidate based on multimodal video processing. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 29–35
Gucluturk Y, Guclu U, Perez M, Balderas HJE, Baro X, Guyon I, Andujar C, Junior J, Madadi M, Escalera S et al (2017) Visualizing apparent personality analysis with deep residual networks. In: International conference on computer vision-ICCV 2017
Gurpinar F, Kaya H, Salah AA (2016) Combining deep facial and ambient features for first impression estimation. In: European conference on computer vision. Springer, pp 372–385
Gurpinar F, Kaya H, Salah AA (2016) Multimodal fusion of audio, scene, and face features for first impression estimation. In: 23rd International conference on pattern recognition (ICPR). IEEE, pp 43–48
Haghighat M, Abdel-Mottaleb M, Alhalabi W (2016) Discriminant correlation analysis: real-time feature level fusion for multimodal biometric recognition. IEEE Trans Inf Forensics Secur 11(9):1984–1996
Hernandez RK, Scott I (2017) Predicting Myers-Briggs type indicator with text
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Hirsch HG, Pearce D (2000) The aurora experimental framework for the performance evaluation of speech recognition systems under noisy conditions. In: ASR2000-Automatic speech recognition: challenges for the new Millenium ISCA tutorial and research workshop (ITRW)
Hoerl AE, Kennard RW (1970) Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12(1):55–67
Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B (Cybern) 42(2):513–529
Ilmini K, Fernando TL (2016) Persons’ personality traits recognition using machine learning algorithms and image processing techniques
Ireland ME, Pennebaker JW (2010) Language style matching in writing: Synchrony in essays, correspondence, and poetry. J Personal Soc Psychol 99(3):549
Jaynes ET (1982) On the rationale of maximum-entropy methods. Proc IEEE 70(9):939–952
Ji S, Xu W, Yang M, Yu K (2013) 3d convolutional neural networks for human action recognition. IEEE Trans Pattern Anal Mach Intell 35(1):221–231
Judge TA, Higgins CA, Thoresen CJ, Barrick MR (1999) The big five personality traits, general mental ability, and career success across the life span. Pers Psychol 52(3):621–652
Junior J, Jacques C, Güçlütürk Y, Pérez M, Güçlü U, Andujar C, Baró X, Escalante HJ, Guyon I, van Gerven MA et al (2018) First impressions: a survey on computer vision-based apparent personality trait analysis. arXiv preprint arXiv:1804.08046
Kalghatgi MP, Ramannavar M, Sidnal NS (2015) A neural network approach to personality prediction based on the big-five model. Int J Innov Res Adv Eng (IJIRAE) 2(8):56–63
Kamenskaya E, Kukharev G (2008) Recognition of psychological characteristics from face. Metody Inform Stosow 1(1):59–73
Kampman O, Barezi EJ, Bertero D, Fung P (2018) Investigating audio, video, and text fusion methods for end-to-end automatic personality prediction. In: Proceedings of the 56th annual meeting of the association for computational linguistics (volume 2: short papers), vol 2, pp 606–611
Kindiroglu AA, Akarun L (2017) Aran O (2017) Multi-domain and multi-task prediction of extraversion and leadership from meeting videos. EURASIP J Image Video Proc 1:77
Kohonen T (1990) The self-organizing map. Proc IEEE 78(9):1464–1480
Levitan SI, An G, Wang M, Mendels G, Hirschberg J, Levine M, Rosenberg A (2015) Cross-cultural production and detection of deception from speech. In: Proceedings of the 2015 ACM on workshop on multimodal deception detection. ACM, pp 1–8
Levitan SI, Levitan Y, An G, Levine M, Levitan R, Rosenberg A, Hirschberg J (2016) Identifying individual differences in gender, ethnicity, and personality from dialogue for deception detection. In: Proceedings of the second workshop on computational approaches to deception detection, pp 40–44
Liem CC, Langer M, Demetriou A, Hiemstra AM, Wicaksana AS, Born MP, König CJ (2018) Psychology meets machine learning: Interdisciplinary perspectives on algorithmic job candidate screening. In: Explainable and interpretable models in computer vision and machine learning. Springer, pp 197–253
Ling W, Luis T, Marujo L, Astudillo RF, Amir S, Dyer C, Black AW, Trancoso I (2015) Finding function in form: Compositional character models for open vocabulary word representation. arXiv preprint arXiv:1508.02096
Liu L, Preotiuc-Pietro D, Samani ZR, Moghaddam ME, Ungar LH (2016) Analyzing personality through social media profile picture choice. In: ICWSM, pp 211–220
Madzlan NA, Han J, Bonin F, Campbell N (2014) Automatic recognition of attitudes in video blogs—prosodic and visual feature analysis. In: Fifteenth annual conference of the international speech communication association
Majumder N, Poria S, Gelbukh A, Cambria E (2017) Deep learning-based document modeling for personality detection from text. IEEE Intell Syst 32(2):74–79
Majumder N, Poria S, Peng H, Chhaya N, Cambria E, Gelbukh A (2019) Sentiment and sarcasm classification with multitask learning. IEEE Intel Syst 34(3):38–43
Matthews G, Deary IJ, Whiteman MC (2003) Personality traits. Cambridge University Press, Cambridge
Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781
Miles J, Hempel S (2004) The Eysenck personality scales: the Eysenck personality questionnaire-revised (EPQ-R) and the Eysenck personality profiler (EPP). Comprehensive handbook of psychological assessment vol 2, pp 99–107
Mukherjee S, Kumar U (2016) Ethical issues in personality assessment. In: The Wiley handbook of personality assessment, pp 415–426
Oosterhof NN, Todorov A (2008) The functional basis of face evaluation. Proc Natl Acad Sci 105(32):11087–11092
Palaz D, Magimai-Doss M, Collobert R (2015) Analysis of cnn-based speech recognition system using raw speech as input. Technical report, Idiap
Papurt MJ (1930) A study of the woodworth psychoneurotic inventory with suggested revision. J Abnorm Soc Psychol 25(3):335
Park G, Schwartz HA, Eichstaedt JC, Kern ML, Kosinski M, Stillwell DJ, Ungar LH, Seligman ME (2015) Automatic personality assessment through social media language. J Personal Soc Psychol 108(6):934
Pennebaker JW, King LA (1999) Linguistic styles: language use as an individual difference. J Personal Soc Psychol 77(6):1296
Pennebaker JW, Francis ME, Booth RJ (2001) Linguistic inquiry and word count: LIWC 2001, vol 71. Lawrence Erlbaum Associates, Mahway
Pennington J, Socher R, Manning C (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543
Perez-Rosas V, Mihalcea R, Morency LP (2013) Utterance-level multimodal sentiment analysis. In: Proceedings of the 51st annual meeting of the association for computational linguistics (volume 1: long papers), vol 1, pp 973–982
Pervin LA, John OP (1999) Handbook of personality: theory and research. Elsevier, Amsterdam
Phillips PJ, Wechsler H, Huang J, Rauss PJ (1998) The feret database and evaluation procedure for face-recognition algorithms. Image Vis Comput 16(5):295–306
Polzehl T, Moller S, Metze F (2010) Automatically assessing personality from speech. In: IEEE fourth international conference on semantic computing (ICSC). IEEE, pp 134–140
Ponce-López V, Chen B, Oliu M, Corneanu C, Clapés A, Guyon I, Baró X, Escalante HJ, Escalera S (2016) Chalearn lap 2016: first round challenge on first impressions-dataset and results. In: European conference on computer vision. Springer, pp 400–418
Poria S, Gelbukh A, Agarwal B, Cambria E, Howard N (2013) Common sense knowledge based personality recognition from text. In: Mexican international conference on artificial intelligence. Springer, pp 484–496
Poria S, Chaturvedi I, Cambria E, Bisio F (2016) Sentic LDA: Improving on LDA with semantic similarity for aspect-based sentiment analysis. In: International joint conference on neural networks (IJCNN), pp 4465–4473
Poria S, Cambria E, Hazarika D, Majumder N, Zadeh A, Morency LP (2017) Context-dependent sentiment analysis in user-generated videos. In: Proceedings of the 55th annual meeting of the association for computational linguistics (volume 1: long papers), vol 1, pp 873–883
Rai N (2016) Bi-modal regression for apparent personality trait recognition. In: 23rd International conference on pattern recognition (ICPR). IEEE, pp 55–60
Roberto B, Poggio T (1993) Face recognition: features versus templates. IEEE Tran Pattern Anal Mach Intell 15(10):1042–1052
Rojas M, Masip D, Todorov A, Vitria J (2011) Automatic prediction of facial trait judgments: appearance vs. structural models. PLoS ONE 6(8):e23323
Saez Y, Navarro C, Mochon A, Isasi P (2014) A system for personality and happiness detection. IJIMAI 2(5):7–15
Sanchez-Cortes D, Aran O, Gatica-Perez D (2011) An audio visual corpus for emergent leader analysis. In: ICMI-MLMI, Multimodal corpora for machine learning, pp 14–18
Schapire RE, Singer Y (2000) Boostexter: a boosting-based system for text categorization. Mach Learn 39(2–3):135–168
Shaver PR, Brennan KA (1992) Attachment styles and the “big five” personality traits: their connections with each other and with romantic relationship outcomes. Personal Soc Psychol Bull 18(5):536–545
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556
Su MH, Wu CH, Zheng YT (2016) Exploiting turn-taking temporal evolution for personality trait perception in dyadic conversations. IEEE/ACM Trans Audio Speech Lang Process 24(4):733–744
Subramaniam A, Patel V, Mishra A, Balasubramanian P, Mittal A (2016) Bi-modal first impressions recognition using temporally ordered deep audio and stochastic visual features. In: European conference on computer vision. Springer, pp 337–348
Sun X, Liu B, Cao J, Luo J, Shen X (2018) Who am I? Personality detection based on deep learning for texts. In: IEEE international conference on communications (ICC). IEEE, pp 1–6
Tausczik YR, Pennebaker JW (2010) The psychological meaning of words: LIWC and computerized text analysis methods. J Lang Soc Psychol 29(1):24–54
Tseng SC (2004) Processing mandarin spoken corpora. Traitement Automatique des Langes, pp 89–108
Turk M, Pentland A (1991) Eigenfaces for recognition. J Cognit Neurosci 3(1):71–86
Valente F, Kim S, Motlicek P (2012) Annotation and recognition of personality traits in spoken conversations from the ami meetings corpus. In: Thirteenth annual conference of the international speech communication association
Ventura C, Masip D, Lapedriza A (2017) Interpreting cnn models for apparent personality trait regression. In: IEEE conference on computer vision and pattern recognition workshops (CVPRW). IEEE, pp 1705–1713
Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154
Vo NN, Liu S, He X, Xu G (2018) Multimodal mixture density boosting network for personality mining. In: Pacific-Asia conference on knowledge discovery and data mining. Springer, pp 644–655
Walker M, Vetter T (2016) Changing the personality of a face: Perceived big two and big five personality factors modeled in real photographs. J Personal Soc Psychol 110(4):609
Whissell C, Fournier M, Pelland R, Weir D, Makarec K (1986) A dictionary of affect in language: IV Reliability, validity, and applications. Percept Motor Skills 62(3):875–888
Willis J, Todorov A (2006) First impressions: making up your mind after a 100-ms exposure to a face. Psychol Sci 17(7):592–598
Yang K, Mall S, Glaser N (2017) Prediction of personality first impressions with deep bimodal LSTM
Yang HC, Huang ZR (2019) Mining personality traits from social messages for game recommender systems. Knowl Based Syst 165:157–168
Vinciarelli A, Mohammadi G (2014) A survey of personality computing. IEEE Trans Affect Comput 5(3):273–291
Yin H, Wang Y, Li Q, Xu W, Yu Y, Zhang T (2018) A network-enhanced prediction method for automobile purchase classification using deep learning
Yu J, Markov K (2017) Deep learning based personality recognition from facebook status updates. In: IEEE 8th international conference on awareness science and technology (iCAST). IEEE, pp 383–387
Zadeh A, Zellers R, Pincus E, Morency LP (2016) Mosi: multimodal corpus of sentiment intensity and subjectivity analysis in online opinion videos. arXiv preprint arXiv:1606.06259
Zhang W, Shan S, Gao W, Chen X, Zhang H (2005) Local gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition. In: Tenth IEEE international conference on computer vision, ICCV 2005, vol 1. IEEE, pp 786–791
Zhang CL, Zhang H, Wei XS, Wu J (2016) Deep bimodal regression for apparent personality analysis. In: European conference on computer vision. Springer, pp 311–324
Zhao S, Gholaminejad A, Ding G, Gao Y, Han J, Keutzer K (2019) Personalized emotion recognition by personality-aware high-order learning of physiological signals. ACM Tran Multimed Comput Commun Appl (TOMM) 15(1s):14
Zuo X, Feng B, Yao Y, Zhang T, Zhang Q, Wang M, Zuo W (2013) A weighted ML-kNN model for predicting users’ personality traits. In: Proceedings of international conference on information science and computer application (ISCA), pp 345–350
Acknowledgements
We would like to thank Prof. Bharat M Deshpande for his valuable guidance. A. Gelbukh recognizes the support of the Instituto Politecnico Nacional via the Secretaria de Investigacion y Posgrado projects SIP 20196437 and SIP 20196021.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Mehta, Y., Majumder, N., Gelbukh, A. et al. Recent trends in deep learning based personality detection. Artif Intell Rev 53, 2313–2339 (2020). https://doi.org/10.1007/s10462-019-09770-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10462-019-09770-z