Skip to main content
SpringerLink
Log in

Fake news detection using dual BERT deep neural networks

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Fake news is a growing challenge for social networks and media. Detection of fake news always has been a problem for many years, but the evolution of social networks and increasing speed of news dissemination in recent years has been considered again. There are several approaches to solving this problem, one of which is to detect fake news based on its text style using deep neural networks. In recent years, transfer learning with transformers is one of the most used forms of deep neural networks for natural language processing. BERT is one of the most promising transformers that outperforms other models in many NLP benchmarks. In this article, we introduce MWPBert, which uses two parallel BERT networks to perform veracity detection on full-text news articles. One of the BERT networks encodes news headline, and another encodes news bodies. Since the input length of the BERT network is limited and constant and the news body is usually a long text, we cannot feed the whole text into the BERT. Therefore, using the MaxWorth algorithm, we selected the part of the news text that is more valuable for fact-checking, and fed it into the BERT network. Finally, we encode the output of the two BERT networks to an output network to classify the news. The experiment results showed that the proposed model outperformed previous models regarding accuracy and other performance measures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

The datasets analysed during the current study are available in this repository: https://github.com/KaiDMML/FakeNewsNet

Code Availability

The code are available from the corresponding author on reasonable request.

References

  1. Abouelenien M, Pérez-Rosas V, Zhao B et al. (2017) Gender-based multimodal deception detection. In: Proceedings of the ACM Symposium on Applied Computing, vol Part F1280. Association for Computing Machinery, New York, New York, USA, pp 137–144. https://doi.org/10.1145/3019612.3019644, http://dl.acm.org/citation.cfm?doid=3019612.3019644

  2. Albawi S, Mohammed TA, Al-Zawi S (2018) Understanding of a convolutional neural network. In: Proceedings of 2017 International Conference on Engineering and Technology, ICET 2017, Ieee, pp 1–6. https://doi.org/10.1109/ICEngTechnol.2017.8308186

  3. Bahdanau D, Cho KH, Bengio Y (2015) Neural machine translation by jointly learning to align and translate. In: 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings. arXiv:1409.0473

  4. Bordes A, Usunier N, Garcia-Durán A et al. (2013) Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems

  5. Bronstein MM, Bruna J, Lecun Y et al. (2017) Geometric Deep Learning: Going beyond Euclidean data. https://doi.org/10.1109/MSP.2017.2693418, arXiv:1611.08097

  6. Dagan I, Dolan B, Magnini B et al. (2009) Recognizing textual entailment: Rational, evaluation and approaches. https://doi.org/10.1017/S1351324909990209

  7. Devlin J, Chang MW, Lee K et al. (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. NAACL HLT 2019 - 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference 1(Mlm):4171–4186. arXiv:1810.04805

  8. Gravanis G, Vakali A, Diamantaras K et al (2019) Behind the cues: A benchmarking study for fake news detection. Exp Syst Appl 128:201–213. https://doi.org/10.1016/j.eswa.2019.03.036

    Article  Google Scholar 

  9. Hassan N, Arslan F, Li C et al (2017) Toward automated fact-checking: Detecting check-worthy factual claims by claimbuster. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining Part F1296:1803–1812. https://doi.org/10.1145/3097983.3098131

    Article  Google Scholar 

  10. Kaliyar RK, Goswami A, Narang P (2021) FakeBERT: Fake news detection in social media with a BERT-based deep learning approach. Multimed Tools Appl 80(8):11,765–11,788. https://doi.org/10.1007/s11042-020-10183-2

  11. Karimi H, Tang J (2019) Learning hierarchical discourse-level structure for fake news detection. NAACL HLT 2019 - 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference 1:3432–3442. https://doi.org/10.18653/v1/n19-1347, arXiv:1903.07389

  12. Kowsari K, Meimandi KJ, Heidarysafa M et al. (2019) Text classification algorithms: A survey. Inf (Switzerland) 10(4):150. https://doi.org/10.3390/info10040150, arXiv:1904.08067

  13. Kwon S, Cha M, Jung K et al. (2013) Prominent features of rumor propagation in online social media. In: Proceedings - IEEE International Conference on Data Mining, ICDM, pp 1103–1108. https://doi.org/10.1109/ICDM.2013.61

  14. Li L, Qin B, Ren W et al (2017) Document representation and feature combination for deceptive spam review detection. Neurocomput 254:33–41. https://doi.org/10.1016/j.neucom.2016.10.080

    Article  Google Scholar 

  15. Long Y, Lu Q, Xiang R et al. (2017) Fake News Detection Through Multi-Perspective Speaker Profiles. Proceedings of the Eighth International Joint Conference on Natural Language Processing Volume 2:(8):252–256. http://www.aclweb.org/anthology/I17-2043

  16. Luong MT, Pham H, Manning CD (2015) Effective approaches to attention-based neural machine translation. In: Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing, pp 1412–1421, https://doi.org/10.18653/v1/d15-1166. arXiv:1508.04025

  17. Meel P, Vishwakarma DK (2020) Fake news, rumor, information pollution in social media and web: A contemporary survey of state-of-the-arts, challenges and opportunities. https://doi.org/10.1016/j.eswa.2019.112986

  18. Mikolov T, Sutskever I, Chen K et al. (2013) Distributed representations ofwords and phrases and their compositionality. In: Advances in Neural Information Processing Systems

  19. Monti F, Frasca F, Eynard D et al. (2019) Fake News Detection on Social Media using Geometric Deep Learning. Arxiv 1902-06673v1 pp 1–15. arXiv:1902.06673

  20. Ott M, Choi Y, Cardie C et al. (2011) Finding deceptive opinion spam by any stretch of the imagination. In: ACL-HLT 2011 - Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp 309–319. arXiv:1107.4557

  21. Palani B, Elango S, Vignesh Viswanathan K (2022) CB-Fake: A multimodal deep learning framework for automatic fake news detection using capsule neural network and BERT. Multimed Tools Appl 81(4):5587–5620. https://doi.org/10.1007/s11042-021-11782-3. https://link.springer.com/article/10.1007/s11042-021-11782-3

  22. Pan JZ, Pavlova S, Li C et al. (2018) Content based fake news detection using knowledge graphs. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol 11136 LNCS. Springer, pp 669–683. https://doi.org/10.1007/978-3-030-00671-6_39, http://link.springer.com/10.1007/978-3-030-00671-6_39

  23. Panigrahi S, Nanda A, Swarnkar T (2021) A Survey on Transfer Learning. Smart Innov Syst Technol 194(10):781–789. https://doi.org/10.1007/978-981-15-5971-6_83

    Article  Google Scholar 

  24. Pennington J, Socher R, Manning CD (2014) GloVe: Global vectors for word representation. In: EMNLP 2014 - 2014 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference, pp 1532–1543. https://doi.org/10.3115/v1/d14-1162

  25. Pérez-Rosas V, Kleinberg B, Lefevre A et al. (2018) Automatic detection of fake news. https://www.aclweb.org/anthology/C18-1287/. arXiv:1708.07104

  26. Peters ME, Neumann M, Zettlemoyer L et al. (2018) Dissecting contextual word embeddings: Architecture and representation. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018, pp 1499–1509, https://doi.org/10.18653/v1/d18-1179. arXiv:1808.08949

  27. Pierri F, Ceri S (2019) False news on social media: A data-driven survey. SIGMOD Record 48(2):18–32. https://doi.org/10.1145/3377330.3377334, arXiv:1902.07539

  28. Rajpurkar P, Zhang J, Lopyrev K et al. (2016) SQuad: 100,000+ questions for machine comprehension of text. In: EMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings, pp 2383–2392. https://doi.org/10.18653/v1/d16-1264, arXiv:1606.05250

  29. Roy A, Basak K, Ekbal A et al. (2019) A Deep Ensemble Framework for Fake News Detection and Multi-Class Classification of Short Political Statements. In: Proceedings of the 16th International Conference on Natural Language Processing

  30. Sadeghi F, Bidgoly AJ, Amirkhani H (2022) Fake news detection on social media using a natural language inference approach. Multimed Tools Appl 81(23):33,801–33,821. https://doi.org/10.1007/s11042-022-12428-8, https://link.springer.com/article/10.1007/s11042-022-12428-8

  31. Shu K, Mahudeswaran D, Liu H (2019) FakeNewsTracker: a tool for fake news collection, detection, and visualization. Comput Math Org Theory 25(1):60–71. https://doi.org/10.1007/s10588-018-09280-3

    Article  Google Scholar 

  32. Shu K, Mahudeswaran D, Wang S et al. (2020) FakeNewsNet: A Data Repository with News Content, Social Context, and Spatiotemporal Information for Studying Fake News on Social Media. Big Data 8(3):171–188. https://doi.org/10.1089/big.2020.0062, arXiv:1809.01286

  33. Singhania S, Fernandez N, Rao S (2017) 3HAN: A Deep Neural Network for Fake News Detection. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 10635 LNCS:572–581. https://doi.org/10.1007/978-3-319-70096-0_59

  34. Thorne J, Vlachos A (2018) Automated fact checking: Task formulations, methods and future directions. COLING 2018 - 27th International Conference on Computational Linguistics, Proceedings pp 3346–3359. arXiv:1806.07687

  35. Thota A, Tilak P, Ahluwalia S et al. (2018) Fake news detection: a deep learning approach. SMU Data Sci Rev 1(3):10. https://scholar.smu.edu/datasciencereview/vol1/iss3/10

  36. Vaswani A, Shazeer N, Parmar N, et al (2017) Attention is all you need. In: Advances in Neural Information Processing Systems, pp 5999–6009. arXiv:1706.03762

  37. Vosoughi S, Roy D, Aral S (2018) The spread of true and false news online. Sci 359(6380):1146–1151. https://doi.org/10.1126/science.aap9559

    Article  Google Scholar 

  38. Wang B, Feng Y, cai Xiong X et al. (2022) Multi-modal transformer using two-level visual features for fake news detection. Appl Intell pp 1–15. https://doi.org/10.1007/s10489-022-04055-5, https://link.springer.com/article/10.1007/s10489-022-04055-5

  39. Wang WY (2017) "Liar, liar pants on fire": A new benchmark dataset for fake news detection. In: ACL 2017–55th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers), pp 422–426. https://doi.org/10.18653/v1/P17-2067, arXiv:1705.00648

  40. Williams A, Nangia N, Bowman SR (2018) A broad-coverage challenge corpus for sentence understanding through inference. In: NAACL HLT 2018 - 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference, pp 1112–1122. https://doi.org/10.18653/v1/n18-1101, arXiv:1704.05426

  41. Wolf T, Debut L, Sanh V et al. (2020) Transformers: State-of-the-Art Natural Language Processing. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp 38–45. https://doi.org/10.18653/v1/2020.emnlp-demos.6

  42. Wu K, Yang S, Zhu KQ (2015) False rumors detection on Sina Weibo by propagation structures. In: Proceedings - International Conference on Data Engineering, pp 651–662. https://doi.org/10.1109/ICDE.2015.7113322

  43. Zhou X, Zafarani R (2020) A Survey of Fake News: Fundamental Theories, Detection Methods, and Opportunities. ACM Comput Surv 53(5). https://doi.org/10.1145/3395046, arXiv:1812.00315

  44. Zhou X, Cao J, Jin Z et al. (2015) Real-time news certification system on sina weibo. In: WWW 2015 Companion - Proceedings of the 24th International Conference on World Wide Web, pp 983–988. https://doi.org/10.1145/2740908.2742571

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Arak University, Arak, Iran

    Mahmood Farokhian & Vahid Rafe

  2. Department of New Science and Technology, University of Tehran, Tehran, Iran

    Hadi Veisi

Authors
  1. Mahmood Farokhian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vahid Rafe
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hadi Veisi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mahmood Farokhian.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Farokhian, M., Rafe, V. & Veisi, H. Fake news detection using dual BERT deep neural networks. Multimed Tools Appl 83, 43831–43848 (2024). https://doi.org/10.1007/s11042-023-17115-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-17115-w

Keywords

Search

Navigation