Skip to main content
SpringerLink
Log in

DeepVisInterests : deep data analysis for topics of interest prediction

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Deep data analysis for latent information prediction has become an increasingly important area of research. In order to predict users’ interests and other latent attributes, most of the existing solutions (works, studies) have used textual data and have obtained accurate results. However, little attention has been paid to visual data that have become increasingly popular in recent years. This paper addresses the problem of discovering the attributed interests and analyzing the performance of the automatic prediction by a comparison with the self-assessed topics of interests (topics of interests provided by the user in a proposed questionnaire) based on data analysis techniques applied to users’ visual data. We analyze the content of individual images and aggregate the image-level information to predict user-level interest distribution. Thus, we employ a Convolutional Neural Network architecture pre-trained on ImageNet dataset for feature extraction. The suggested system is based on the construction of users’ interests ontology in order to learn the semantic representation for the popular Topics of interests defined by social networks (e.g., Facebook). Our experiments show that the analysis enhances the overall prediction performance. We set forth a novel evaluation database to improve our framework’s robustness and enhance its ability to generalize to new user profiles. Our proposed framework has shown promising results. Our method yields a competitive accuracy of 0.80 compared with the state of the art techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Algorithm 2
Algorithm 3
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Alam F, Alam T, Ofli F, Imran M (2021) Social media images classification models for real-time disaster response. coRR

  2. Alpaydin E (2020) Introduction to machine learning

  3. Chen C, Ren J (2017) Forum latent Dirichlet allocation for user interest discovery. Knowl-Based Syst:1–7

  4. Chung T-L(D), Johnson O, Hall-Phillips A, Kim K (2021) The effects of offline events on online connective actions: an examination of boycottNFL using social network analysis. In: Computers in human behavior

  5. D’Agostino G, D’Antonio F, De Nicola A, Tucci S (2016) Interests diffusion in social networks. Physica A: Stat Mech Appl:443–461

  6. Deng J, Dong W, Socher R, Li L-J, Li K, Li F-F (2009) Imagenet: a large-scale hierarchical image database CVPR

  7. Dinh XT, Hai VP (2020) A proposal of deep learning model for classifying user interests on social networks. In: Proceedings of the 4th international conference on machine learning and soft computing, pp 10,14

  8. Faralli S, Stilo G, Velardi P (2017) Automatic acquisition of a taxonomy of microblogs users’ interests. Web Semant: Sci Serv Agents World Wide Web:23,40

  9. Gharibshah Z, Zhu X, Hainline A, Conway M (2020) Deep learning for user interest and response prediction in online display advertising. In: Data science and engineering journal

  10. Hohenecker P, Lukasiewicz T (2017) Deep learning for ontology reasoning, arXiv:1705.10342

  11. Hong T, Choi C, Shin J (2020) CNN-based malicious user detection in social networks. Concurr Comput: Pract Experience:e4163

  12. Huang C (2019) Social network site use and Big Five personality traits: a meta-analysis. Comput Human Behav 97:280,290

    Article  Google Scholar 

  13. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2023) Caffe: convolutional architec

  14. Kaur K (2016) Development of a framework for analyzing terrorism actions via Twitter lists. In: 2016 International conference on computational techniques in information and communication technologies (ICCTICT), pp 5,10

  15. Kosinski M, Stillwell D, Graepel T (2013) Private traits and attributes are predictable from digital records of human behavior. In: Proce20Procedding of national academy of sciences of the United State of America, pp 1,6

  16. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

    Google Scholar 

  17. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25

  18. Lazzez O, Ouarda W, Alimi AM (2016) Understand me if you can! global soft biometrics recognition from social visual data. In: Proceedings of the 16th international conference on hybrid intelligent systems (HIS 2016) November 21-23, 2016. Marrakech, pp 527–538

  19. LeCun Y, Haffner P, Bottou L, Bengio Y (2000) Processing systems using graph transformer networks. In: Proceedings of computer vision and pattern recognition, object recognition with gradient-based learning

  20. Lewenberg Y, Bachrach Y, Volkova S (2015) Using emotions to predict user interest areas in online social networks. In: 2015 IEEE international conference on data science and advanced Analytics (DSAA), pp 1–10

  21. Li L-J (2007) What, where and who? classifying event by scene and object recognition. In: IEEE International conference on computer vision

  22. Li L-J (2010) What, where and who? classifying event by scene and object recognition. In: IEEE International conference on computer vision

  23. Li L-J, Li F-F (2017) What, where and who? classifying event by scene and object recognition. In: Proceedings of international conference on computer vision

  24. Li J, Wu N, Feng Z (2018) Model-based non-Gaussian interest topic distribution for user retweeting in social networks. Neurocomputing:87–98

  25. Liu Z, Luo P, Qiu S, Wang X, Tang X (2016) Deepfashion: powering robust clothes recognition and retrieval with rich annotations. In: IEEE Conference on computer vision and pattern recognition (CVPR)

  26. Lourdusamy R, John A (2020) A review on metrics for ontology evaluation. In: 2018 2nd International conference on inventive systems and control (ICISC)

  27. Lovato P, Perina A, Cheng DS, Segalin C, Sebe N, Cristani M (2013) We like it! mapping image preferences on the counting grid. In: 2013 IEEE International conference on image processing, pp 2892–2896

  28. Malviya N, Mishra N, Sahu S (2012) Developing University Ontology using protégé OWL tool: process and reasoning. Int J Sci Eng Res 2

  29. McDaniel M, Storey VC, Sugumaran V (2018) Assessing the quality of domain ontologies: metrics and an automated ranking system. Data Knowl Eng

  30. Musto C, de Gemmis M, Semeraro G, Lops P (2018) A multi-criteria recommender system exploiting aspect-based sentiment analysis of users’ reviews

  31. Ouarda W, Trichili H, Alimi AM, Solaiman B (2016) Towards a novel biometric system for smart riding club. J Inf Assur Secur 11:201–213. ISSN 1554-1010

    Google Scholar 

  32. Rich J, Haddadi H, Hospedales TM (2016) Towards bottom-up analysis of social food, coRR

  33. Segalin C, Cheng DS, Cristani M (2014) Social profiling through image understanding personality inference using convolutional neural networks. Comput Vision and Image Underst 2017.ture for Fast Feature Embedding

  34. Shen H, Wang G (2017) Can dynamic knowledge-sharing activities be mirrored from the static online social network in yahoo! answers and how to improve its quality of service?. In: IEEE transactions on systems, man, and cybernetics: systems, vol 12, pp 3363–3376

  35. Szegedy C, Liu W, Jia Y, Sermanet P, Reed SE, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2014) Going deeper with convolutions, CoRR

  36. Wu B, Jia J, Yang Y, Zhao P, Tang J (2015) Understanding the emotions behind social images: inferring with user demographics. In: 2015 IEEE international conference on multimedia and expo (ICME), pp 1,6

  37. Yang C, Wang Y (2013) Unveiling the multimedia unconscious: implicit cognitive processes and multimedia content analysis. In: Proceedings of the 21st ACM international conference on multimedia, pp 5–10

  38. Yang C, Yu X, Liu Y, Nie Y, Wang Y (2016) Collaborative filtering with weighted opinion aspects. Neurocomputing:185–196

  39. Yang L, Hsieh K, Estrin D (2017) Beyond classification: latent user interests profiling from visual contents analysis, coRR

  40. Yang C, Wang Y (2019) Online social network image classification and application based on deep learning. In: 2019 IEEE International conference on acoustics, speech and signal processing (ICASSP), pp 5,10

  41. You Q, Bhatia S, Luo J (2018) A picture tells a thousand words - about you! user interest profiling from user generated visual content, coRR

  42. Zareie A, Sheikhahmadi A, Jalili M (2019) Identification of influential users in social networks based on users’ interest. Inf Sci J 493:217,231

    MathSciNet  MATH  Google Scholar 

  43. Zarrinkalam F, Kahani M, Bagheri E (2018) Mining user interests over active topics on social networks. Inf Process Manag 54:339–357

    Article  Google Scholar 

  44. Zarrinkalam F, Kahani M, Bagheri E (2019) User interest prediction over future unobserved topics on social networks. Inf Retrieval J 22:93,128

    Article  Google Scholar 

  45. Zhou G, Mou N, Fan Y, Pi Q, Bian W, Zhou C, Zhu X, Gai K (2019) Deep interest evolution network for click-through rate prediction. In: Proceedings of the AAAI conference on artificial intelligence

  46. Zhu H, Liu D, Bayley I, Aldea A, Yang Y, Chen Y (2017) Quality model and metrics of ontology for semantic descriptions of web services. Tsinghua Sci Technol 22:254–277

    Article  Google Scholar 

Download references

Acknowledgements

We deeply acknowledge Taif University for Supporting this study through Taif University Researchers Supporting Project number (TURSP-2020/115), Taif University, Taif, Saudi Arabia. The research leading to these results has received fund-ing from the Ministry of Higher Education and Scientific Re-search of Tunisia under the grant agreement number LR11ES48.

Author information

Authors and Affiliations

  1. REGIM-Lab.: Research Groups in Intelligent Machines, University of Sfax, BP 1173, Sfax, 3038, Tunisia

    Onsa Lazzez & Adel M. Alimi

  2. Department of Computer Science, College of Computers and Information Technolgy, Taif University, P.O.Box. 11099, Taif, 21944, Saudi Arabia

    Abdulrahman M. Qahtani & Abdulmajeed Alsufyani

  3. College of Applied Computer Science, King Saud University, (Almuzahmiyah Campus), Riyadh, Saudi Arabia

    Omar Almutiry & Habib Dhahri

  4. Departimento di Information, Universita degli Studi di Milano, 20133, Milano, Italy

    Vincenzo Piuri

  5. Department of Electrical and Electronic Engineering Science, Faculty of Engineering and the Built Environment, University of Johannesburg, Johannesburg, South Africa

    Adel M. Alimi

Authors
  1. Onsa Lazzez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdulrahman M. Qahtani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Abdulmajeed Alsufyani
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Omar Almutiry
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Habib Dhahri
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Vincenzo Piuri
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Adel M. Alimi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Onsa Lazzez.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflicts of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lazzez, O., Qahtani, A.M., Alsufyani, A. et al. DeepVisInterests : deep data analysis for topics of interest prediction. Multimed Tools Appl 82, 40913–40936 (2023). https://doi.org/10.1007/s11042-023-14806-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-14806-2

Keywords

Search

Navigation