Skip to main content
SpringerLink
Log in

Cycle mapping with adversarial event classification network for fake news detection

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In recent years, there is a increase in researchers’ interest on social evidence, particularly for fake news detection (FND). However, news posts on social media often include diverse modalities, e.g., text, image, etc., and diverse events related to politics, economics, etc., resulting in significant modality and event differences. How to jointly learn modal-invariant and event-invariant discriminative features effectively of news posts remains a challenge. This paper proposes a novel FND approach, called Cycle Mapping and Adversarial Event Classification Network (CMAECN). It consists of two parts: a multi-modal cycle feature mapping module (CMM) and an adversarial event classification module (AECM). In order to fully reduce modality difference, the CMM module is designed, which performs cross-modal generation between image and text modalities by using the generative model, conducts feature source identification between initial and generated features for each modality with the discriminative model, and reconstructs text or image features with the cross-modal fused features to avoid information loss with the reconstructor. In order to fully reduce event difference, the AECM module is designed to perform event adversarial classification between the event classification task and the event-independent classification task with a multi-task event classifier, where each dimension of the classifier output corresponds to a certain event category, and an additional dimension of the output represents the event-independent category. The network training of CMAECN is conducted by adopting an adversarial scheme. Comprehensive experiments are conducted on two public datasets, and CMAECN shows superior performance compared to the state-of-the-art multi-modal FND methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability Statement

The used benchmark multi-modal datasets Weibo and Twitter are available from the works [1] and [2], respectively.

References

  1. Jin Z, Cao J, Zhang Y, Zhou J, Tian Q (2016) Novel visual and statistical image features for microblogs news verification. IEEE Transactions on Multimedia 19(3):598–608

    Article  Google Scholar 

  2. Boididou C, Andreadou K, Papadopoulos S, Dang Nguyen DT, Boato G, Riegler M, Kompatsiaris Y (2015) Verifying multimedia use at mediaeval 2015. MediaEval 3(3):7

    Google Scholar 

  3. Zhang X, Ghorbani AA (2020) An overview of online fake news: Characterization, detection, and discussion. Information Processing & Management 57(2)102025

  4. Zhou X, Zafarani R (2020) A survey of fake news: Fundamental theories, detection methods, and opportunities. ACM Computing Surveys 53(5):1–40

    Article  Google Scholar 

  5. O’Connor C, Murphy M (2020) Going viral: doctors must tackle fake news in the covid-19 pandemic. British Med J 369(10.1136):1587

  6. Grinberg N, Joseph K, Friedland L, Swire-Thompson B, Lazer D (2019) Fake news on twitter during the 2016 us presidential election. Science 363(6425):374–378

    Article  ADS  CAS  PubMed  Google Scholar 

  7. Choudhury D, Acharjee T (2023) A novel approach to fake news detection in social networks using genetic algorithm applying machine learning classifiers. Multimedia Tools and Applications 82(6):9029–9045

    Article  Google Scholar 

  8. Palani B, Elango S, Viswanathan KV (2022) Cb-fake: A multimodal deep learning framework for automatic fake news detection using capsule neural network and bert. Multimedia Tools and Applications 81(4):5587–5620

    Article  PubMed  Google Scholar 

  9. Kaliyar RK, Goswami A, Narang P (2021) Fakebert: Fake news detection in social media with a bert-based deep learning approach. Multimedia Tools and applications 80(8):11765–11788

    Article  PubMed  PubMed Central  Google Scholar 

  10. Dun Y, Tu K, Chen C, Hou C, Yuan X (2021) Kan: Knowledge-aware attention network for fake news detection. In: AAAI Conference on artificial intelligence, 35:81–89

  11. Upadhayay B, Behzadan V (2022) Hybrid deep learning model for fake news detection in social networks (student abstract). In: AAAI Conference on artificial intelligence, 36:13067–13068

  12. Singhal S, Shah RR, Chakraborty T, Kumaraguru P, Satoh S (2019) Spotfake: A multi-modal framework for fake news detection. In: International conference on multimedia big data, pp 39–47

  13. Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805

  14. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  15. Jin Z, Cao J, Guo H, Zhang Y, Luo J (2017) Multimodal fusion with recurrent neural networks for rumor detection on microblogs. In: ACM International conference on multimedia, pp 795–816

  16. Zhou X, Wu J, Zafarani R (2020) Similarity-aware multi-modal fake news detection. In: Advances in knowledge discovery and data mining, pp 354–367

  17. Khattar D, Goud JS, Gupta M, Varma V (2019) Mvae: Multimodal variational autoencoder for fake news detection. In: The world wide web conference, pp 2915–2921

  18. Wang Y, Ma F, Jin Z, Yuan Y, Xun G, Jha K, Su L, Gao J (2018) Eann: Event adversarial neural networks for multi-modal fake news detection. In: ACM SIGKDD International conference on knowledge discovery & data mining, pp 849–857

  19. Ding Y, Guo B, Liu Y, Liang Y, Shen H, Yu Z (2022) Metadetector: Meta event knowledge transfer for fake news detection. ACM Trans Intell Syst Technol 13(6):1–25

  20. Zhang T, Wang D, Chen H, Zeng Z, Guo W, Miao C, Cui L (2020) Bdann: Bert-based domain adaptation neural network for multi-modal fake news detection. In: International joint conference on neural networks, pp 1–8

  21. Wei P, Wu F, Sun Y, Zhou H, Jing X-Y (2022) Modality and event adversarial networks for multi-modal fake news detection. IEEE Signal Processing Letters 29:1382–1386

    Article  ADS  Google Scholar 

  22. Tong S, Koller D (2001) Support vector machine active learning with applications to text classification. J Mach Learn Res 2(Nov):45–66

  23. Zhou X, Jain A, Phoha VV, Zafarani R (2020) Fake news early detection: A theory-driven model. Digital Threats: Research and Practice 1(2):1–25

    Article  Google Scholar 

  24. Sahoo SR, Gupta BB (2021) Multiple features based approach for automatic fake news detection on social networks using deep learning. Applied Soft Computing 100:106983

    Article  Google Scholar 

  25. Berahmand K, Li Y, Xu Y (2023) Dac-hpp: Deep attributed clustering with high-order proximity preserve. Neural Computing and Applications 35:24493–24511

    Article  Google Scholar 

  26. Berahmand K, Li Y, Xu Y (2023) A deep semi-supervised community detection based on point-wise mutual information. IEEE Transactions on Computational Social Systems In Press,

  27. Malik M, Prabha C, Soni P, Arya V, Alhalabi WA, Gupta BB, Albeshri AA, Almomani A (2023) Machine learning-based automatic litter detection and classification using neural networks in smart cities. International Journal on Semantic Web and Information Systems 19(1):1–20

    Article  Google Scholar 

  28. Mishra A, Hsu C-H, Arya V, Chaurasia P, Li P (2021) A hybrid approach for protection against rumours in a iot enabled smart city environment. In: International conference on cyber security, privacy and networking, pp 101–109

  29. Li P, Sun X, Yu H, Tian Y, Yao F, Xu G (2021) Entity-oriented multi-modal alignment and fusion network for fake news detection. IEEE Transactions on Multimedia 24:3455–3468

    Article  Google Scholar 

  30. Inan E (2022) Zoka: a fake news detection method using edge-weighted graph attention network with transfer models. Neural Computing and Applications 34(14):11669–11677

    Article  Google Scholar 

  31. Jain V, Kaliyar RK, Goswami A, Narang P, Sharma Y (2022) Aenet: an attention-enabled neural architecture for fake news detection using contextual features. Neural Computing and Applications 34(1):771–782

    Article  PubMed  Google Scholar 

  32. Ma J, Gao W, Mitra P, Kwon S, Jansen BJ, Wong K-F, Cha M (2016) Detecting rumors from microblogs with recurrent neural networks. In: International joint conference on artificial intelligence, pp 3818–3824

  33. Chen T, Li X, Yin H, Zhang J (2018) Call attention to rumors: Deep attention based recurrent neural networks for early rumor detection. In: Trends and applications in knowledge discovery and data mining, pp 40–52

  34. Gaurav A, Gupta BB, Hsu C, Castiglione A, Chui KT (2021) Machine learning technique for fake news detection using text-based word vector representation. Computational Data and Social Networks 13116:340–348

    Article  Google Scholar 

  35. Qian S, Hu J, Fang Q, Xu C (2021) Knowledge-aware multi-modal adaptive graph convolutional networks for fake news detection. ACM Transactions on Multimedia Computing, Communications, and Applications 17(3):1–23

    Article  Google Scholar 

  36. Udandarao V, Maiti A, Srivatsav D, Vyalla SR, Yin Y, Shah RR (2020) Cobra: Contrastive bi-modal representation algorithm. arXiv:2005.03687

  37. Jarrahi A, Safari L (2023) Evaluating the effectiveness of publishers’ features in fake news detection on social media. Multimedia Tools and Applications 82(2):2913–2939

    Article  PubMed  Google Scholar 

  38. Ma K, Tang C, Zhang W, Cui B, Ji K, Chen Z, Abraham A (2023) Dc-cnn: Dual-channel convolutional neural networks with attention-pooling for fake news detection. Applied Intelligence 53(7):8354–8369

    Article  PubMed  Google Scholar 

  39. Peng X, Xintong B (2022) An effective strategy for multi-modal fake news detection. Multimedia Tools and Applications 81(10):13799–13822

    Article  Google Scholar 

  40. Ye K, Kovashka A (2021) A case study of the shortcut effects in visual commonsense reasoning. In: AAAI Conference on artificial intelligence, pp 3181–3189

  41. Yu Z, Yu J, Cui Y, Tao D, Tian Q (2019) Deep modular co-attention networks for visual question answering. In: IEEE/CVF Conference on computer vision and pattern recognition, pp 6281–6290

  42. Chen H, Suhr A, Misra D, Snavely N, Artzi Y (2019) Touchdown: Natural language navigation and spatial reasoning in visual street environments. In: IEEE/CVF Conference on computer vision and pattern recognition, pp 12538–12547

  43. Miech A, Alayrac J-B, Laptev I, Sivic J, Zisserman A (2021) Thinking fast and slow: Efficient text-to-visual retrieval with transformers. In: IEEE/CVF Conference on computer vision and pattern recognition, pp 9826–9836

  44. Wang B, Yang Y, Xu X, Hanjalic A, Shen HT (2017) Adversarial cross-modal retrieval. In: ACM International conference on multimedia, pp 154–162

  45. Karpathy A, Joulin A, Fei-Fei LF (2014) Deep fragment embeddings for bidirectional image sentence mapping. In: Advances in neural information processing systems, vol. 27

  46. Lee K-H, Chen X, Hua G, Hu H, He X (2018) Stacked cross attention for image-text matching. In: European conference on computer vision, pp 201–216

  47. Fang H, Gupta S, Iandola F, Srivastava RK, Deng L (2015) From captions to visual concepts and back. In: IEEE Conference on computer vision and pattern recognition, pp 1473–1482

  48. Zhou Y, Ying Q, Qian Z, Li S, Zhang X (2022) Multimodal fake news detection via clip-guided learning. arXiv:2205.14304

  49. Xu X, He L, Lu H, Gao L, Ji Y (2019) Deep adversarial metric learning for cross-modal retrieval. World Wide Web 22:657–672

    Article  Google Scholar 

  50. Zhu J-Y, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE International conference on computer vision, pp 2223–2232

  51. Sandfort V, Yan K, Pickhardt PJ, Summers RM (2019) Data augmentation using generative adversarial networks (cyclegan) to improve generalizability in ct segmentation tasks. Scientific Reports 9(1):16884

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  52. Lin X, Li J, Ma Z, Li H, Li S, Xu K (2022) Learning modal-invariant and temporal-memory for video-based visible-infrared person re-identification. In: IEEE/CVF Conference on computer vision and pattern recognition, pp 20973–20982

  53. Nakamura K, Levy S, Wang WY (2019) r/fakeddit: A new multimodal benchmark dataset for fine-grained fake news detection. arXiv preprint arXiv:1911.03854

  54. Antol S, Agrawal A, Lu J, Mitchell M, Batra D (2015) Vqa: Visual question answering. In: IEEE International conference on computer vision, pp 2425–2433

  55. Vinyals O, Toshev A, Bengio S, Erhan D (2015) Show and tell: A neural image caption generator. In: IEEE Conference on computer vision and pattern recognition, pp 3156–3164

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (No. 62076139), National Key R &D Program of China (2023YFB2904000, 2023YFB2904003, 2023YFB2904004), 1311 Talent Program of Nanjing University of Posts and Telecommunications, and Postgraduate Research & Practice Innovation Program of Jiangsu Province (No. SJCX22_0289).

Author information

Authors and Affiliations

  1. College of Automation & Artificial Intelligence, Nanjing University of Posts and Telecommunications, Nanjing, 210023, People’s Republic of China

    Fei Wu, Hong Zhou, Yujian Feng & Guangwei Gao

  2. School of Computer Science, Nanjing University of Posts and Telecommunications, Nanjing, 210023, People’s Republic of China

    Yimu Ji

  3. School of Computer, Wuhan University, Wuhan, 430072, People’s Republic of China

    Xiao-Yuan Jing

Authors
  1. Fei Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hong Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yujian Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guangwei Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yimu Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xiao-Yuan Jing
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fei Wu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, F., Zhou, H., Feng, Y. et al. Cycle mapping with adversarial event classification network for fake news detection. Multimed Tools Appl (2024). https://doi.org/10.1007/s11042-024-18499-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11042-024-18499-z

Keywords

Search

Navigation