Abstract
Nowadays media overload is a pretty common scenario all around the world. The prevalence of media overload grants both individuals and governmental entities the ability to shape public opinions, highlighting the need to deploy effective fake news detection methods. In this paper, we suggest a novel model named GraMuFeN, for detecting fake news that has been posted by users on Twitter and Weibo. This model has been designed to detect fake news using both textual and image data accompanying each piece of news. We utilize Graph Convolution Neural Networks (GCN) as the text encoder and Convolutional Neural Networks (CNN) as the image encoder with the help of Supervised Contrastive Loss aiming to develop a model much lighter in terms of trainable parameters and easier to train while having a higher performance compared to previous works. Our evaluations on two different benchmarks show a promising 10% improvement in micro f1 score and a 50% reduction in terms of the model’s trainable parameters.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
No datasets were generated or analysed during the current study.
References
Boididou C, Andreadou K, Papadopoulos S, Dang Nguyen DT, Boato G, Riegler M, Kompatsiaris Y, et al (2015) Verifying multimedia use at mediaeval 2015. In: MediaEval 2015, vol 1436. CEUR-WS
Farajtabar M, Yang J, Ye X, Xu H, Trivedi R, Khalil E, Li S, Song L, Zha H (2017) Fake news mitigation via point process based intervention. In: International conference on machine learning. PMLR, pp 1097–1106
Gao X, Wang X, Chen Z, Zhou W, Hoi SC (2024) Knowledge enhanced vision and language model for multi-modal fake news detection. IEEE Trans Multim
Ghorbanpour F, Ramezani M, Fazli MA, Rabiee HR (2023) Fnr: a similarity and transformer-based approach to detect multi-modal fake news in social media. Soc Netw Anal Min 13(1):56
Gupta M, Zhao P, Han J (2012) Evaluating event credibility on twitter. In: Proceedings of the 2012 SIAM international conference on data mining. SIAM, pp 153–164
Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. Adv Neural Inform Process Syst 30
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Jadidinejad AH, Sadr H (2015) Improving weak queries using local cluster analysis as a preliminary framework. Indian J Sci Technol 8(5):495–510
Jin Z, Cao J, Guo H, Zhang Y, Luo J (2017) Multimodal fusion with recurrent neural networks for rumor detection on microblogs. In: Proceedings of the 25th ACM international conference on multimedia, pp 795–816
Kalsnes B (2018) Fake news. In: Oxford Research Encyclopedia of Communication
Kasban H, Nassar S (2020) An efficient approach for forgery detection in digital images using Hilbert–Huang transform. Appl Soft Comput 97:106728
Khattar D, Goud JS, Gupta M, Varma V (2019) Mvae: multimodal variational autoencoder for fake news detection. In: The World Wide Web conference, pp 2915–2921
Khosla P, Teterwak P, Wang C, Sarna A, Tian Y, Isola P, Maschinot A, Liu C, Krishnan D (2020) Supervised contrastive learning. Adv Neural Inf Process Syst 33:18661–18673
Kumari R, Ekbal A (2021) Amfb: attention based multimodal factorized bilinear pooling for multimodal fake news detection. Expert Syst Appl 184:115412
Kwon S, Cha M, Jung K, Chen W, Wang Y (2013) Prominent features of rumor propagation in online social media. In: 2013 IEEE 13th international conference on data mining. IEEE, pp 1103–1108
Li J, Ni S, Kao H-Y (2021) Meet the truth: Leverage objective facts and subjective views for interpretable rumor detection. arXiv:2107.10747
Meel P, Vishwakarma DK (2023) Multi-modal fusion using fine-tuned self-attention and transfer learning for veracity analysis of web information. Expert Syst Appl 229:120537
Meng R, Zhou Z, Cui Q, Lam K-Y, Kot A (2022) Traceable and authenticable image tagging for fake news detection
Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. arXiv:1301.3781
Mohades Deilami F, Sadr H, Tarkhan M (2022) Contextualized multidimensional personality recognition using combination of deep neural network and ensemble learning. Neural Process Lett 54(5):3811–3828
Ni S, Li J, Kao H-Y (2021) Mvan: multi-view attention networks for fake news detection on social media. IEEE Access 9:106907–106917
Ping Tian D et al (2013) A review on image feature extraction and representation techniques. Int J Multim Ubiquitous Eng 8(4):385–396
Qian S, Hu J, Fang Q, Xu C (2021) Knowledge-aware multi-modal adaptive graph convolutional networks for fake news detection. ACM Trans Multim Comput Commun Appl (TOMM) 17(3):1–23
Radford A, Kim JW, Hallacy C, Ramesh A, Goh G, Agarwal S, Sastry G, Askell A, Mishkin P, Clark J, Krueger G, Sutskever I (2021) Learning transferable visual models from natural language supervision
Ruchansky N, Seo S, Liu Y (2017) Csi: A hybrid deep model for fake news detection. In: Proceedings of the 2017 ACM on conference on information and knowledge management, pp 797–806
Sadr H, Soleimandarabi MN, Pedram M, Teshnelab M (2019) Unified topic-based semantic models: a study in computing the semantic relatedness of geographic terms. In: 2019 5th International Conference on Web Research (ICWR). IEEE, pp 134–140
Salama K (2021) Keras documentation: Natural Language image search with a dual encoder. https://keras.io/examples/vision/nl_image_search/. Accessed 2021
Shu K, Sliva A, Wang S, Tang J, Liu H (2017) Fake news detection on social media: a data mining perspective. ACM SIGKDD Explor Newsl 19(1):22–36
Singh B, Sharma DK (2021) Siteforge: detecting and localizing forged images on microblogging platforms using deep convolutional neural network. Comput Indust Eng 162:107733
Singh B, Sharma DK (2022) Predicting image credibility in fake news over social media using multi-modal approach. Neural Comput Appl 34(24):21503–21517
Singhal S, Shah RR, Chakraborty T, Kumaraguru P, Satoh S (2019) Spotfake: A multi-modal framework for fake news detection. In: 2019 IEEE fifth international conference on multimedia Big Data (BigMM). IEEE, pp 39–47
Song C, Ning N, Zhang Y, Wu B (2021) A multimodal fake news detection model based on crossmodal attention residual and multichannel convolutional neural networks. Inform Process Manag 58(1):102437
Steinebach M, Gotkowski K, Liu H (2019) Fake news detection by image montage recognition. In: Proceedings of the 14th international conference on availability, reliability and security, pp 1–9
Wang Y, Ma F, Jin Z, Yuan Y, Xun G, Jha K, Su L, Gao J (2018) Eann: Event adversarial neural networks for multi-modal fake news detection. In: Proceedings of the 24th Acm sigkdd international conference on knowledge discovery & data mining, pp 849–857
Wang Y, Qian S, Hu J, Fang Q, Xu C (2020) Fake news detection via knowledge-driven multimodal graph convolutional networks. In: Proceedings of the 2020 international conference on multimedia retrieval, pp 540–547
Wessel M, Thies F, Benlian A (2016) The emergence and effects of fake social information: evidence from crowdfunding. Decis Support Syst 90:75–85
Woolley SC, Howard PN (2018) Computational propaganda: political parties, politicians, and political manipulation on social media. Oxford University Press
Wu K, Yang S, Zhu KQ (2015) False rumors detection on sina weibo by propagation structures. In: 2015 IEEE 31st international conference on data engineering. IEEE, pp 651–662
Xi Y, Zhang Y, Ding S, Wan S (2020) Visual question answering model based on visual relationship detection. Signal Process: Image Commun 80:115648
Zhou X, Wu J, Zafarani R (2020) SAFE: similarity-aware multi-modal fake news detection
Author information
Authors and Affiliations
Contributions
S. AmirAli Gh. Ghahramani, contributed in conceptualizing the main idea, coding and implementation, analyzing the results and reviewed and revised the manuscript. Makan Kanaanian contributed in the data collection and data cleaning, coding and implementation, fine tuning the models and gathering results, drawing figures, writing the manuscript draft and reporting the results. Fatemeh Badiei contributed in the data collection and data cleaning, coding and implementation, fine tuning the models and gathering results, drawing figures, writing the manuscript draft and reporting the results.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Kananian, M., Badiei, F. & Gh. Ghahramani, S.A. GraMuFeN: graph-based multi-modal fake news detection in social media. Soc. Netw. Anal. Min. 14, 104 (2024). https://doi.org/10.1007/s13278-024-01267-0
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s13278-024-01267-0