Abstract
The problem solved in this article is to predict the expression of personality traits of a user, which can be obtained from the Life Style Index questionnaire, through the analysis of the graphical content published in his social media account. The proposed approach is to identify faces in photos from users’ accounts and use them to assess the expression of types of psychological defense. The proposed solution aims at testing the hypothesis that the presence of a face in the images and its position can contain the features sought. The essence of the proposed method consists in transfer training of a related model of a deep neural network for determination of emotions on pairs of faces contained in a digital image, and expressiveness of indicators of psychological defense mechanics. The accuracy of the predictions obtained with the new model when compared to the baseline is more than 2 times higher. Theoretical and practical significance lies in the fact that a new approach is formed, different from the known by the data involved in the analysis, and a neural network model is built, which allows to estimate the severity of psychological defenses of the user on the images published by him in social media, which indirectly in the future will allow to build estimates of the user protection from social engineering attacks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
We are social. DIGITAL 2021: THE RUSSIAN FEDERATION. https://datareportal.com/reports/digital-2021-russian-federation. Last accessed 21 June 2022
2022 Data Breach Investigations Report. https://www.verizon.com/business/resources/reports/2022/dbir/2022-data-breach-investigations-report-dbir.pdf. Last accessed 21 June 2022
2022 Must-Know Cyber Attack Statistics and Trends. https://www.embroker.com/blog/cyber-attack-statistics/. Last accessed 21 June 2022
Apple and Meta Gave User Data to Hackers Who Used Forged Legal Requests. https://www.bloomberg.com/news/articles/2022-03-30/apple-meta-gave-user-data-to-hackers-who-forged-legal-requests. Last accessed 21 June 2022
Azarov, A.A., Tulupyeva, T.V., Suvorova, A.V., Tulupyev, A.L., Abramov, M.V., Yusupov, R.M.: Social Engineering Attacks. The Problem of Analysis. Nauka, St. Petersburg (2016)
Khlobystova, A., Abramov, M.: Time-based model of the success of a malefactor’s multistep social engineering attack on a user. In: Kovalev, S., Tarassov, V., Snasel, V., Sukhanov, A. (eds.) Proceedings of the Fifth International Scientific Conference “Intelligent Information Technologies for Industry” (IITI’21). Lecture Notes in Networks and Systems, vol. 330, pp. 216–223. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-87178-9_22
Korepanova, A.A., Abramov, M.V., Tulupyev, A.L.: Social media user identity linkage by graphic content comparison. Sci. Tech. J. Inf. Technol., Mech. Opt. 21(6), 942–950 (2021). https://doi.org/10.17586/2226-1494-2021-21-6-942-950 (in Russian)
Tulupyeva, T.V., Abramov, M.V., Tulupyev, A.L.: Social Engineering Attacks: Social Networks and User Security Assessments. SUAI, St. Petersburg (2018) (in Russian)
Brand Analytics team. Social Media in Russia 2021. https://br-analytics.ru/blog/social-media-russia-2021. Last accessed 21 June 2022
Oliseenko, V.D., Tulupyeva, T.V.: Neural network approach in the task of multi-label classification of user posts in online social networks. In: Proceedings of the 2021 XXIV International Conference on Soft Computing and Measurements (SCM), pp. 46–48 (2021). https://doi.org/10.1109/SCM52931.2021.9507148
Plutchik, R., Kellerman, H., Conte, H.: Life style index. A structural theory of ego defence and emotions. In: Izard, E. (ed.) Emotions in Personality and Psychopatology, pp. 229–257. New York (1979)
Wasserman, L.I., Eryshev, O.F., Klubova, E.B.: Psikhologicheskaya diagnostika index zhiznii stilya: metodicheskoe posobie dlya vrachnikov i psikhologov (Psychological Diagnostics of the Index of Life Style: A Methodological Guide for Doctors and Psychologists). NIPNI im. V.M. Bekhtereva, St. Petersburg (1998)
Viola, P., Jones, M.: Robust real-time face detection. Int. J. Comput. Vision 5(57), 137–154 (2004)
Cascade Classifier. https://docs.opencv.org/3.4/db/d28/tutorial_cascade_classifier.html. Last accessed 21 June 2022
Rath, S.R.: Image Recognition Using Histogram of Oriented Gradients (HOG) Descriptor. https://debuggercafe.com/image-recognition-using-histogram-of-oriented-gradients-hog-descriptor/. Last accessed 21 June 2022
Rahmad, C., Asmara, R.A., Putra, D.R.H., Dharma, I., Darmono, H., Muhiqqin, I.: Comparison of Viola-Jones Haar cascade classifier and histogram of oriented gradients (HOG) for face detection. IOP Conf. Ser.: Mater. Sci. Eng. 732(1), 012038 (2020)
Pang, Y., Yuan, Y., Li, X., Pan, J.: Efficient hog human detection. Signal Process. 91(4), 773–781 (2011)
Xiang, J., Zhu, J.: Joint face detection and facial expression recognition with mtcnn. In: 4th International Conference on Information Science and Control Engineering (ICISCE), pp. 424–427. IEEE (2017)
Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010)
Kazakov, O.D., Mikheenko, O.V.: Transfer learning and domain adaptation based on modeling of socio-economic systems. Bus. Inform. 14(2), 7–20 (2020). https://doi.org/10.17323/2587-814X.2020.2.7.20
Thoughtworksarts. EmoPy. https://github.com/thoughtworksarts/. Last accessed 21 June 2022
Koech, K.E.: Cross-Entropy Loss Function. https://towardsdatascience.com/cross-entropy-loss-function-f38c4ec8643e. Last accessed 21 June 2022
Rynegardh, C.: Baseline. https://datascience.stackexchange.com/questions/30912/what-does-baseline-mean-in-the-context-of-machine-learning. Last accessed 21 June 2022
Acknowledgements
The research was carried out in the framework of the project on state assignment SPC RAS SPIIRAS no. FFZF-2022-0003, with the financial support of the RFBR (â„– 20-07-00839 Digital twins and soft computing in social engineering attacks modelling and associated risks assessment) and grant of the President MK-5237.2022.1.6 (Digital traces of the user and his vulnerabilities in the automated assessment of security against social engineering attacks).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Bushmelev, F., Khlobystova, A., Abramov, M., Livshits, L. (2023). Deep Machine Learning Techniques in the Problem of Estimating the Expression of Psychological Characteristics of a Social Media User. In: Dolinina, O., et al. Artificial Intelligence in Models, Methods and Applications. AIES 2022. Studies in Systems, Decision and Control, vol 457. Springer, Cham. https://doi.org/10.1007/978-3-031-22938-1_22
Download citation
DOI: https://doi.org/10.1007/978-3-031-22938-1_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-22937-4
Online ISBN: 978-3-031-22938-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)