Skip to main content
SpringerLink
Log in

ConvNet frameworks for multi-modal fake news detection

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

An upsurge of false information revolves around the internet. Social media and websites are flooded with unverified news posts. These posts are comprised of text, images, audio, and videos. There is a requirement for a system that detects fake content in multiple data modalities. We have seen a considerable amount of research on classification techniques for textual fake news detection, while frameworks dedicated to visual fake news detection are very few. We explored the state-of-the-art methods using deep networks such as CNNs and RNNs for multi-modal online information credibility analysis. They show rapid improvement in classification tasks without requiring pre-processing. To aid the ongoing research over fake news detection using CNN models, we build textual and visual modules to analyze their performances over multi-modal datasets. We exploit latent features present inside text and images using layers of convolutions. We see how well these convolutional neural networks perform classification when provided with only latent features and analyze what type of images are needed to be fed to perform efficient fake news detection. We propose a multi-modal Coupled ConvNet architecture that fuses both the data modules and efficiently classifies online news depending on its textual and visual content. We thence offer a comparative analysis of the results of all the models utilized over three datasets. The proposed architecture outperforms various state-of-the-art methods for fake news detection with considerably high accuracies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. https://drive.google.com/file/d/0B3e3qZpPtccsMFo5bk9Ib3VCc2c/view

  2. https://www.kaggle.com/chahatraj/ticnn-extracted-images

  3. https://www.kaggle.com/chahatraj/emergent-extracted-images

References

  1. Yang Y, Zheng L, Zhang J, Cui Q, Li Z, Yu, PS (2018) TI-CNN: Convolutional Neural Networks for fake news detection

    Google Scholar 

  2. Wu K, Yang S, Zhu KQ (2015) False rumors detection on sina weibo by propagation structures. In: 2015 IEEE 31st international conference on data engineering, pp 651–662

    Chapter  Google Scholar 

  3. Ma J, Gao W, Mitra P, Kwon S, Jansen BJ, Wong KF, Cha M (2016) Detecting rumors from microblogs with recurrent neural networks

    Google Scholar 

  4. Ma J, Gao W, Wong KF (2019) Detect rumors on twitter by promoting information campaigns with generative adversarial learning. In: The world wide web conference, pp 3049–3055

    Chapter  Google Scholar 

  5. Jin Z, Cao J, Zhang Y, Zhou J, Tian Q (2016) Novel visual and statistical image features for microblogs news verification. IEEE Transactions on Multimedia 19(3):598–608

    Article  Google Scholar 

  6. Boididou C, Andreadou K, Papadopoulos S, Dang-Nguyen DT, Boato G, Riegler M, Kompatsiaris Y (2015) Verifying multimedia use at MediaEval 2015. MediaEval 3(3):7

    Google Scholar 

  7. Boididou C, Papadopoulos S, Dang-Nguyen D-T, Boato G, Riegler M, Middleton SE, Petlund A, Kompatsiaris Y et al (2016) Verifying multimedia use at MediaEval 2016. MediaEval

  8. Jin Z, Cao J, Guo H, Zhang Y, Luo J (2017) Multi-modal fusion with recurrent neural networks for rumor detection on microblogs. In: Proceedings of the 25th ACM international conference on multimedia, pp 795–816

    Chapter  Google Scholar 

  9. Qi P, Cao J, Yang T, Guo J, Li J (2019) Exploiting multi-domain visual information for fake news detection. In: 2019 IEEE International Conference on Data Mining (ICDM), pp 518–527

    Chapter  Google Scholar 

  10. Chen T, Li X, Yin H, Zhang J (2018) Call attention to rumors: deep attention based recurrent neural networks for early rumor detection. In: Pacific-Asia conference on knowledge discovery and data mining, pp 40–52

    Google Scholar 

  11. Liu Y, Wu YFB (2018) Early detection of fake news on social media through propagation path classification with recurrent and convolutional networks. Thirty-Second AAAI Conference on Artificial Intelligence

  12. Wang Y, Ma F, Jin Z, Yuan Y, Xun G, Jha K et al (2018) Eann: event adversarial neural networks for multi-modal fake news detection. In: Proceedings of the 24th acm sigkdd international conference on knowledge discovery & data mining, pp 849–857

    Chapter  Google Scholar 

  13. Khattar D, Goud JS, Gupta M, Varma V (2019) Mvae: multi-modal variational autoencoder for fake news detection. In: The world wide web conference, pp 2915–2921

    Chapter  Google Scholar 

  14. Ajao O, Bhowmik D, Zargari S (2018) Fake news identification on twitter with hybrid cnn and rnn models. In: Proceedings of the 9th international conference on social media and society, pp 226–230

    Chapter  Google Scholar 

  15. Jindal S, Sood R, Singh R, Vatsa M, Chakraborty T (2020) NewsBag: a multi-modal benchmark dataset for fake news detection

    Google Scholar 

  16. Singhal S, Shah RR, Chakraborty T, Kumaraguru P, Satoh SI (2019) SpotFake: a multi-modal framework for fake news detection. In: 2019 IEEE fifth international conference on multimedia big data (BigMM), pp 39–47

    Chapter  Google Scholar 

  17. Marra F, Gragnaniello D, Cozzolino D, Verdoliva L (2018) Detection of Gan-generated fake images over social networks. In: 2018 IEEE conference on multimedia information processing and retrieval (MIPR), pp 384–389

    Chapter  Google Scholar 

  18. Sabir E, AbdAlmageed W, Wu Y, Natarajan P (2018) Deep multi-modal image-repurposing detection. In: Proceedings of the 26th ACM international conference on multimedia, pp 1337–1345

    Chapter  Google Scholar 

  19. Pomari T, Ruppert G, Rezende E, Rocha A, Carvalho T (2018) Image splicing detection through illumination inconsistencies and deep learning. 2018 25th IEEE International Conference on Image Processing (ICIP):3788–3792

  20. Maigrot C, Claveau V, Kijak E, Sicre R (2016) Mediaeval 2016: a multi-modal system for verifying multimedia use task

    Google Scholar 

  21. Bayar B, Stamm MC (2016) A deep learning approach to universal image manipulation detection using a new convolutional layer. In: Proceedings of the 4th ACM workshop on information hiding and multimedia security, pp 5–10

    Chapter  Google Scholar 

  22. Lago F, Phan QT, Boato G (2019) Visual and textual analysis for image trustworthiness assessment within online news. Security and Communication Networks

  23. Cui L, Wang S, Lee D (2019) SAME: sentiment-aware multi-modal embedding for detecting fake news. In: Proceedings of the 2019 IEEE/ACM international conference on advances in social networks analysis and mining, pp 41–48

    Chapter  Google Scholar 

  24. Volkova S, Ayton E, Arendt DL, Huang Z, Hutchinson B (2019) Explaining multi-modal deceptive news prediction models. In: Proceedings of the international AAAI conference on web and social media, vol 13, pp 659–662

    Google Scholar 

  25. Tariq S, Lee S, Kim H, Shin Y, Woo SS (2018) Detecting both machine and human created fake face images in the wild. In: Proceedings of the 2nd international workshop on multimedia privacy and security, pp 81–87

    Chapter  Google Scholar 

  26. Sabir E, Cheng J, Jaiswal A, AbdAlmageed W, Masi I, Natarajan P (2019) Recurrent convolutional strategies for face manipulation detection in videos. Interfaces (GUI) 3(1)

  27. Güera D, Delp EJ (2018) Deepfake video detection using recurrent neural networks. In: 2018 15th IEEE international conference on advanced video and signal based surveillance (AVSS), pp 1–6

    Google Scholar 

  28. Papadopoulou O, Zampoglou M, Papadopoulos S, Kompatsiaris Y (2017) Web video verification using contextual cues. In: Proceedings of the 2nd international workshop on multimedia forensics and security, pp 6–10

    Chapter  Google Scholar 

  29. Zhou P, Han X, Morariu VI, Davis LS (2017) Two-stream neural networks for tampered face detection. In: 2017 IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp 1831–1839

    Chapter  Google Scholar 

  30. Rahmouni N, Nozick V, Yamagishi J, Echizen I (2017) Distinguishing computer graphics from natural images using convolution neural networks. In: 2017 IEEE workshop on information forensics and security (WIFS), pp 1–6

    Google Scholar 

  31. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

    Google Scholar 

  32. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

    Google Scholar 

  33. Chollet F (2017) Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1251–1258

    Google Scholar 

  34. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D et al (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9

    Google Scholar 

  35. Ferreira W, Vlachos A (2016) Emergent: a novel dataset for stance classification. In: Proceedings of the 2016 conference of the north American chapter of the association for computational linguistics: human language technologies, pp 1163–1168

    Google Scholar 

  36. Amerini I, Ballan L, Caldelli R, Del Bimbo A, Serra G (2011) A sift-based forensic method for copy–move attack detection and transformation recovery. IEEE Transactions on Information Forensics and Security 6(3):1099–1110

    Article  Google Scholar 

  37. Conforti C, Pilehvar MT, Collier N (2018) Towards automatic fake news detection: cross-level stance detection in news articles. In: Proceedings of the first workshop on fact extraction and VERification (FEVER), pp 40–49

    Chapter  Google Scholar 

  38. Bourgonje P, Schneider JM, Rehm G (2017) From clickbait to fake news detection: an approach based on detecting the stance of headlines to articles. In: Proceedings of the 2017 EMNLP workshop: natural language processing meets journalism, pp 84–89

    Chapter  Google Scholar 

  39. Thorne J, Chen M, Myrianthous G, Pu J, Wang X, Vlachos A (2017) Fake news stance detection using stacked ensemble of classifiers. In Proceedings of the 2017 EMNLP Workshop: Natural Language Processing meets Journalism, pp 80–83

  40. Doegar A, Dutta M, Gaurav K (2019) CNN based image forgery detection using pre-trained AlexNet model. Int J Comput Intell & IoT 2(1)

  41. Uliyan DM, Jalab HA, Wahab AWA (2015) Copy move image forgery detection using hessian and center symmetric local binary pattern. In: 2015 IEEE Conference on Open Systems (ICOS), pp 7–11

    Chapter  Google Scholar 

  42. Uliyan DM, Jalab HA, Wahab AWA, Shivakumara P, Sadeghi S (2016) A novel forged blurred region detection system for image forensic applications. Expert Syst Appl 64:1–10

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Technology, Delhi Technological University, Delhi, India

    Chahat Raj & Priyanka Meel

Authors
  1. Chahat Raj
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Priyanka Meel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Priyanka Meel.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Raj, C., Meel, P. ConvNet frameworks for multi-modal fake news detection. Appl Intell 51, 8132–8148 (2021). https://doi.org/10.1007/s10489-021-02345-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02345-y

Keywords

Search

Navigation