Skip to main content
SpringerLink
Log in

Exploiting Transformer-Based Multitask Learning for the Detection of Media Bias in News Articles

  • Conference paper
  • First Online:
Information for a Better World: Shaping the Global Future (iConference 2022)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 13192))

Included in the following conference series:

Abstract

Media has a substantial impact on the public perception of events. A one-sided or polarizing perspective on any topic is usually described as media bias. One of the ways how bias in news articles can be introduced is by altering word choice. Biased word choices are not always obvious, nor do they exhibit high context-dependency. Hence, detecting bias is often difficult. We propose a Transformer-based deep learning architecture trained via Multi-Task Learning using six bias-related data sets to tackle the media bias detection problem. Our best-performing implementation achieves a macro \(F_{1}\) of 0.776, a performance boost of 3% compared to our baseline, outperforming existing methods. Our results indicate Multi-Task Learning as a promising alternative to improve existing baseline models in identifying slanted reporting.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The cross-domain model is not shown due to lack of space but is published at the repository mentioned in Sect. 1.

  2. 2.

    https://huggingface.co/transformers/model_doc/distilbert.html.

  3. 3.

    A detailed description of the data sets is published at the repository mentioned in Sect. 1.

  4. 4.

    IMDb, Subj, Wiki, Reddit.

  5. 5.

    https://colab.research.google.com/notebooks/intro.ipynb.

  6. 6.

    We use a subset of BABE [31], introduced in Sect. 2, to evaluate the MTL models.

  7. 7.

    Let us point out that none of the instances from the Wikipedia data set are contained in our target media bias data set.

References

  1. Baumer, E., Elovic, E., Qin, Y., Polletta, F., Gay, G.: Testing and comparing computational approaches for identifying the language of framing in political news. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1472–1482. Association for Computational Linguistics, Denver, Colorado (May–Jun 2015). https://doi.org/10.3115/v1/N15-1171. https://www.aclweb.org/anthology/N15-1171

  2. Bowman, S.R., Angeli, G., Potts, C., Manning, C.D.: A large annotated corpus for learning natural language inference. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. pp. 632–642. Association for Computational Linguistics, Lisbon, Portugal (September 2015). https://doi.org/10.18653/v1/D15-1075. https://www.aclweb.org/anthology/D15-1075

  3. Browne, M.W.: Cross-validation methods. J. Math. Psychol. 44(1), 108–132 (2000). https://doi.org/10.1006/jmps.1999.1279. https://doi.org/10.1006/jmps.1999.1279

  4. Cabot, P.H., Abadi, D., Fischer, A., Shutova, E.: Us vs. them: a dataset of populist attitudes, news bias and emotions. CoRR abs/2101.11956 (2021). https://arxiv.org/abs/2101.11956

  5. Cer, D., Diab, M., Agirre, E., Lopez-Gazpio, I., Specia, L.: Semeval-2017 task 1: semantic textual similarity multilingual and crosslingual focused evaluation. In: Proceedings of the 11th International Workshop on Semantic Evaluation, SemEval-2017 (2017). https://doi.org/10.18653/v1/s17-2001. http://dx.doi.org/10.18653/v1/S17-2001

  6. Chen, W.F., Al Khatib, K., Wachsmuth, H., Stein, B.: Analyzing political bias and unfairness in news articles at different levels of granularity. In: Proceedings of the 4th Workshop on Natural Language Processing and Computational Social Science, pp. 149–154. Association for Computational Linguistics (2020). https://doi.org/10.18653/v1/2020.nlpcss-1.16. https://www.aclweb.org/anthology/2020.nlpcss-1.16

  7. Clark, K., Luong, M.T., Le, Q.V., Manning, C.D.: ELECTRA: pre-training text encoders as discriminators rather than generators. arXiv:2003.10555 [cs] (March 2020)

  8. De Boer, P.T., Kroese, D.P., Mannor, S., Rubinstein, R.Y.: A tutorial on the cross-entropy method. Ann. Oper. Res. 134(1), 19–67 (2005)

    Article  MathSciNet  Google Scholar 

  9. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, pp. 4171–4186. Association for Computational Linguistics (June 2019). https://doi.org/10.18653/v1/N19-1423. https://www.aclweb.org/anthology/N19-1423

  10. Fan, L., et al.: In plain sight: media bias through the lens of factual reporting. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, pp. 6343–6349. Association for Computational Linguistics (November 2019). https://doi.org/10.18653/v1/D19-1664. https://www.aclweb.org/anthology/D19-1664

  11. Hube, C., Fetahu, B.: Detecting biased statements in Wikipedia. In: Companion Proceedings of the the Web Conference 2018, WWW 2018, Republic and Canton of Geneva, CHE, pp. 1779–1786. International World Wide Web Conferences Steering Committee (2018). https://doi.org/10.1145/3184558.3191640

  12. Hube, C., Fetahu, B.: Neural based statement classification for biased language. In: Proceedings of the 12th ACM International Conference on Web Search and Data Mining, WSDM 2019, New York, NY, USA, pp. 195–203. Association for Computing Machinery (2019). https://doi.org/10.1145/3289600.3291018

  13. Huo, H., Iwaihara, M.: Utilizing BERT pretrained models with various fine-tune methods for subjectivity detection. In: Wang, X., Zhang, R., Lee, Y.-K., Sun, L., Moon, Y.-S. (eds.) APWeb-WAIM 2020. LNCS, vol. 12318, pp. 270–284. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60290-1_21

    Chapter  Google Scholar 

  14. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, Conference Track Proceedings, San Diego, CA, USA, 7–9 May 2015 (2015). http://arxiv.org/abs/1412.6980

  15. Krippendorff, K.: Computing Krippendorff’s alpha-reliability. Departmental Papers (ASC); University of Pennsylvania (2011). https://repository.upenn.edu/cgi/viewcontent.cgi?article=1043&context=asc_papers

  16. Lim, S., Jatowt, A., Färber, M., Yoshikawa, M.: Annotating and analyzing biased sentences in news articles using crowdsourcing. In: Proceedings of the 12th Language Resources and Evaluation Conference, Marseille, France, pp. 1478–1484. European Language Resources Association (May 2020). https://www.aclweb.org/anthology/2020.lrec-1.184

  17. Lim, S., Jatowt, A., Yoshikawa, M.: Understanding characteristics of biased sentences in news articles. In: CIKM Workshops (2018). http://ceur-ws.org/Vol-2482/paper13.pdf

  18. Liu, X., He, P., Chen, W., Gao, J.: Multi-task deep neural networks for natural language understanding. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, pp. 4487–4496. Association for Computational Linguistics (July 2019). https://www.aclweb.org/anthology/P19-1441

  19. Maas, A.L., Daly, R.E., Pham, P.T., Huang, D., Ng, A.Y., Potts, C.: Learning word vectors for sentiment analysis. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, Oregon, USA, pp. 142–150. Association for Computational Linguistics (June 2011). https://www.aclweb.org/anthology/P11-1015

  20. Pang, B., Lee, L.: A sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts. In: Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, ACL 2004, USA, p. 271. Association for Computational Linguistics (2004). https://doi.org/10.3115/1218955.1218990. https://doi.org/10.3115/1218955.1218990

  21. Pryzant, R., Diehl Martinez, R., Dass, N., Kurohashi, S., Jurafsky, D., Yang, D.: Automatically neutralizing subjective bias in text. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 01, pp. 480–489 (April 2020). https://doi.org/10.1609/aaai.v34i01.5385. https://ojs.aaai.org/index.php/AAAI/article/view/5385

  22. Recasens, M., Danescu-Niculescu-Mizil, C., Jurafsky, D.: Linguistic models for analyzing and detecting biased language. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1650–1659 (2013). https://www.aclweb.org/anthology/P13-1162.pdf

  23. Ruder, S.: An overview of multi-task learning in deep neural networks. CoRR abs/1706.05098 (2017). http://arxiv.org/abs/1706.05098

  24. Mean squared error. In: Sammut, C., Webb, G.I. (eds.) Encyclopedia of Machine Learning. Springer, Boston, MA (2011). https://doi.org/10.1007/978-0-387-30164-8_528

  25. Sanh, V., Debut, L., Chaumond, J., Wolf, T.: Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter. CoRR abs/1910.01108 (2019). http://arxiv.org/abs/1910.01108

  26. Spinde, T.: An interdisciplinary approach for the automated detection and visualization of media bias in news articles. In: 2021 IEEE International Conference on Data Mining Workshops (ICDMW) (2021). https://media-bias-research.org/wp-content/uploads/2021/09/Spinde2021g.pdf

  27. Spinde, T., Hamborg, F., Donnay, K., Becerra, A., Gipp, B.: Enabling news consumers to view and understand biased news coverage: a study on the perception and visualization of media bias. In: Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020, JCDL 2020, Virtual Event, China, pp. 389–392. Association for Computing Machinery (2020). https://doi.org/10.1145/3383583.3398619

  28. Spinde, T., Hamborg, F., Gipp, B.: Media bias in German news articles: a combined approach. In: ECML PKDD 2020 Workshops: Workshops of the European Conference on Machine Learning and Knowledge Discovery in Databases (ECML PKDD 2020): INRA 2020, Ghent, Belgium, 14–18 September 2020, Proceedings, vol. 1323, pp. 581–590 (2020). https://doi.org/10.1007/978-3-030-65965-3_41. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7850083/

  29. Spinde, T., Kreuter, C., Gaissmaier, W., Hamborg, F., Gipp, B., Giese, H.: Do you think it’s biased? How to ask for the perception of media bias. In: Proceedings of the ACM/IEEE Joint Conference on Digital Libraries (JCDL) (September 2021)

    Google Scholar 

  30. Spinde, T., Krieger, D., Plank, M., Gipp, B.: Towards a reliable ground-truth for biased language detection. In: Proceedings of the ACM/IEEE Joint Conference on Digital Libraries (JCDL) (September 2021)

    Google Scholar 

  31. Spinde, T., Plank, M., Krieger, J.D., Ruas, T., Gipp, B., Aizawa, A.: Neural media bias detection using distant supervision with BABE - bias annotations by experts. In: Findings of the Association for Computational Linguistics, EMNLP 2021, Dominican Republic (November 2021)

    Google Scholar 

  32. Spinde, T., Rudnitckaia, L., Hamborg, F., Gipp, B.: Identification of biased terms in news articles by comparison of outlet-specific word embeddings. In: Proceedings of the iConference 2021 (March 2021)

    Google Scholar 

  33. Spinde, T., Rudnitckaia, L., Mitrović, J., Hamborg, F., Granitzer, M., Gipp, B., Donnay, K.: Automated identification of bias inducing words in news articles using linguistic and context-oriented features. Inf. Process. Manage. 58(3), 102505 (2021). https://doi.org/10.1016/j.ipm.2021.102505

  34. Spinde, T., Rudnitckaia, L., Sinha, K., Hamborg, F., Gipp, B., Donnay, K.: MBIC - a media bias annotation dataset including annotator characteristics. In: Proceedings of the iConference 2021. iSchools (2021). https://doi.org/10.5281/zenodo.4474336

  35. Spinde, T., Sinha, K., Meuschke, N., Gipp, B.: TASSY - a text annotation survey system. In: Proceedings of the ACM/IEEE Joint Conference on Digital Libraries (JCDL) (September 2021)

    Google Scholar 

  36. Sun, C., Qiu, X., Xu, Y., Huang, X.: How to fine-tune BERT for text classification? In: Sun, M., Huang, X., Ji, H., Liu, Z., Liu, Y. (eds.) CCL 2019. LNCS (LNAI), vol. 11856, pp. 194–206. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32381-3_16

    Chapter  Google Scholar 

  37. Sun, Y., et al.: ERNIE 2.0: a continual pre-training framework for language understanding. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 8968–8975 (April 2020). https://doi.org/10.1609/aaai.v34i05.6428

  38. Vaswani, A., et al.: Attention is all you need. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc. (2017). https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf

  39. Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., Bowman, S.R.: Glue: a multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461 (2018)

  40. Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., Le, Q.V.: XLNet: generalized autoregressive pretraining for language understanding. arXiv:1906.08237 [cs] (June 2019)

Download references

Author information

Authors and Affiliations

  1. University of Wuppertal, Wuppertal, Germany

    Timo Spinde, Terry Ruas & Bela Gipp

  2. University of Konstanz, Konstanz, Germany

    Jan-David Krieger

  3. Institute for Artificial Intelligence Research and Development of Serbia, University of Passau, Passau, Germany

    Jelena Mitrović

  4. University of Kassel, Kassel, Germany

    Franz Götz-Hahn

  5. NII Tokyo, Tokyo, Japan

    Akiko Aizawa

Authors
  1. Timo Spinde
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan-David Krieger
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Terry Ruas
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jelena Mitrović
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Franz Götz-Hahn
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Akiko Aizawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Bela Gipp
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Timo Spinde .

Editor information

Editors and Affiliations

  1. Humboldt-Universität zu Berlin, Berlin, Germany

    Malte Smits

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Spinde, T. et al. (2022). Exploiting Transformer-Based Multitask Learning for the Detection of Media Bias in News Articles. In: Smits, M. (eds) Information for a Better World: Shaping the Global Future. iConference 2022. Lecture Notes in Computer Science(), vol 13192. Springer, Cham. https://doi.org/10.1007/978-3-030-96957-8_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-96957-8_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-96956-1

  • Online ISBN: 978-3-030-96957-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation