Skip to main content
SpringerLink
Log in

A transformer-based architecture for fake news classification

  • Original Article
  • Published:
Social Network Analysis and Mining Aims and scope Submit manuscript

Abstract

In today’s post-truth world, the proliferation of propaganda and falsified news poses a deadly risk of misinforming the public on a variety of issues, either through traditional media or on social media. Information people acquire through these articles and posts tends to shape their world view and provides reasoning for choices they take in their day to day lives. Thus, fake news can definitely be a malicious force, having massive real-world consequences. In this paper, we focus on classifying fake news using models based on a natural language processing framework, Bidirectional Encoder Representations from Transformers, also known as BERT. We fine-tune BERT for specific domain datasets and also make use of human justification and metadata for added performance in our models. We determine that the deep-contextualizing nature of BERT is effective for this task and obtain significant improvement over binary classification, and minimal yet important improvement in six-label classification in comparison with previously explored models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Ahmed H, Traore I, Saad S (2017) Detecting opinion spams and fake news using text classification. Secur Priv. https://doi.org/10.1002/spy2.9

    Article  Google Scholar 

  • Alhindi T (2018a) Where is your evidence: improving fact-checking by justification modeling. In: Proceedings of the first workshop on fact extraction and verification (FEVER), Brussels, Belgium, pp 85–90

  • Alhindi T, Petridis S, Muresan S (2018b) Where is your evidence: improving fact-checking by justification modeling. In: Proceedings of the first workshop on fact extraction

  • Aphiwongsophon S, Chongstitvatana P (2018). Detecting fake news with machine learning method, pp 528–531. https://doi.org/10.1109/ECTICon.2018.8620051

  • Balwant MK (2019) Bidirectional LSTM Based on POS tags and CNN architecture for fake news detection. In: 2019 10th international conference on computing, communication and networking technologies (ICCCNT). https://doi.org/10.1109/ICCCNT45670.2019.8944460

  • Bojanowski P, Grave E, Joulin A, Mikolov T (2017) Enriching word vectors with subword information. Trans Assoc Comput Linguist 5:135–146

    Article  Google Scholar 

  • Bourgonje P, Schneider JM, Rehm G (2017) From clickbait to fake news detection: an approach based on detecting the stance of headlines to articles. In: Proceedings of the 2017 EMNLP workshop: natural language processing meets journalism, pp 84–89

  • Chen ZF, Cheng Y (2020) Consumer response to fake news about brands on social media: the effects of self-efficacy, media trust, and persuasion knowledge on brand trust. J Prod Brand Manag 29(2):188–198

    Article  Google Scholar 

  • Crawford M, Khoshgoftaar TM, Prusa JD, Richter AN, Al Najada H (2015) Survey of review spam detection using machine learning techniques. J Big Data 2(1):1–24

    Article  Google Scholar 

  • Devlin J, Chang MW, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

  • Eugenio T et al (2017) Some like it hoax: automated fake news detection in social networks. arXiv preprint arXiv:1704.07506

  • Feng S, Banarjee R, Choi Y (2012) Syntactic stylometry for deception detection. In: ACL’12

  • Handley L (2018) Nearly 70 percent of people are worried about fake news as a ’weapon,’ survey says. Retrieved from https://www.cnbc.com/2018/01/22/nearly-70-percent-of-peopleare-worried-about-fake-news-as-a-weapon-survey-says.html

  • Hu X, Tang J, Gao H, Liu H (2014a) Social spammer detection with sentiment information. In: ICDM’14

  • Hu X, Tang J, Liu H (2014b) Online social spammer detection. In: AAAI’14, pp 59–65

  • Kaliyar RK, Goswami A, Narang P, Sinha S (2020) FNDNet–a deep convolutional neural network for fake news detection. Cogn Syst Res 61:32–44

    Article  Google Scholar 

  • Kumar S, Asthana R, Upadhyay S, Upreti N, Akbar M (2020) Fake news detection using deep learning models: a novel approach. Trans Emerg Telecommun Technol 31(2):e3767

    Google Scholar 

  • Lichterman J (2016) Nearly half of US adults get news on Facebook, Pew Says. URL: http://www.niemanlab.org/2016/05/pew-report-44-percent-of-us-adults-get-news-onfacebook

  • Long Y et al (2017) Fake news detection through multi-perspective speaker profiles. In: Proceedings of the eighth international joint conference on natural language processing, vol 2: short papers, pp 252–256

  • Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th international conference on neural information processing systems, Lake Tahoe, NV, USA, 5–10, pp 3111–3119

  • Nandimath JN, Katkar BS, Ghadge VU, Garad AN (2017) Efficiently detecting and analyzing spam reviews using live data feed. Int Res J EngTechnol (IRJET) 4(2):1421–1424

    Google Scholar 

  • Rapoza K (2017) Can ’fake news’ impact the stock market? RealClearMarkets, Forbes

  • Resnick B (2018) False news stories travel faster and farther on Twitter than the truth, Vox.(Erişim: 09.09. 2019). https://www.vox.com/science-and-health/2018/3/8/17085928/fake-news-study-mit-science

  • Silverman C (2016) This analysis shows how viral fake election news stories outperformed real news On Facebook. BuzzFeed News, BuzzFeed News. www.buzzfeednews.com/article/craigsilverman/viral-fake-election-news-outperformed-real-news-on-facebook. Accessed 16 Nov 2016

  • Shu K, Sliva A, Wang S, Tang J, Liu H (2017) Fake news detection on social media: a data mining perspective. SIGKDD Explor Newslett 19(1):22–36

    Article  Google Scholar 

  • Shu K, Bernard H, Liu H (2018) Studying fake news via network analysis: detection and mitigation

  • Tang J, Yi C, Huan L (2014) Mining social media with social theories: a survey. ACM SIGKDD Explor Newslett 15(2):20–29

    Article  Google Scholar 

  • Vaswani A et al (2017) Attention is all you need. In: Advances in neural information processing systems, pp 6000–6010

  • Vis F (2014) 10. The rapid spread of misinformation online. World Economic Forum. Retrieved from http://reports.weforum.org/outlook-14/top-ten-trends-categorypage/10-the-rapid-spread-of-misinformation-online/

  • Vo N, Lee K (2018) The rise of guardians: fact-checking URL recommendation to combat fake news. In: The 41st international ACM SIGIR conference on research & development in information retrieval, pp 275–284

  • Volkova S, Shaffer K, Jang JY, Hodas N (2017) Separating facts from fiction: linguistic models to classify suspicious and trusted news posts on Twitter. In: ACL

  • Wakefield J (2016) Young using social media to access news. BBC News. www.bbc.com/news/uk-36528256. Accessed 15 Jun 2016

  • Wang WY (2017) “Liar, Liar Pants on Fire”: a new benchmark dataset for fake news detection. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, vol 2, Short Papers, pp 422–426

  • Wang C, Mahadevan S (2011) Heterogeneous domain adaptation using manifold alignment. In: Proceedings of the 22nd international joint conference on artificial intelligence, vol 2, pp 541–546

  • Yang, S et al (2019) Unsupervised fake news detection on social media: a generative approach. In: Proceedings of the AAAI conference on artificial intelligence, vol 33

  • Zhang S, Wang Y, Tan C (2018) Research on text classification for identifying fake news. In: IEEE 2018 international conference on security, pattern analysis, and cybernetics (SPAC). https://doi.org/10.1109/SPAC46244.2018.8965536

  • Zhang J et al (2019) FAKEDETECTOR: effective fake news detection with deep diffusive neural network. In: 2019 36th IEEE international conference. https://doi.org/10.1109/ICDE48307.2020.00180

  • Zhou JT, Tsang IW, Pan SJ, Tan M (2014) Heterogeneous domain adaptation for multiple classes. In: International conference on artificial intelligence and statistics, pp 103–1095

Download references

Author information

Authors and Affiliations

  1. Information Technology Department, NITK Surathkal, Mangalore, 575025, India

    Divyam Mehta, Aniket Dwivedi, Arunabha Patra & M. Anand Kumar

Authors
  1. Divyam Mehta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aniket Dwivedi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Arunabha Patra
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. M. Anand Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Anand Kumar.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mehta, D., Dwivedi, A., Patra, A. et al. A transformer-based architecture for fake news classification. Soc. Netw. Anal. Min. 11, 39 (2021). https://doi.org/10.1007/s13278-021-00738-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13278-021-00738-y

Keywords

Search

Navigation