Advertisement

Real Solutions for Fake News? Measuring the Effectiveness of General Warnings and Fact-Check Tags in Reducing Belief in False Stories on Social Media

  • Katherine Clayton
  • Spencer Blair
  • Jonathan A. Busam
  • Samuel Forstner
  • John Glance
  • Guy Green
  • Anna Kawata
  • Akhila Kovvuri
  • Jonathan Martin
  • Evan Morgan
  • Morgan Sandhu
  • Rachel Sang
  • Rachel Scholz-Bright
  • Austin T. Welch
  • Andrew G. Wolff
  • Amanda Zhou
  • Brendan NyhanEmail author
Original Paper

Abstract

Social media has increasingly enabled “fake news” to circulate widely, most notably during the 2016 U.S. presidential campaign. These intentionally false or misleading stories threaten the democratic goal of a well-informed electorate. This study evaluates the effectiveness of strategies that could be used by Facebook and other social media to counter false stories. Results from a pre-registered experiment indicate that false headlines are perceived as less accurate when people receive a general warning about misleading information on social media or when specific headlines are accompanied by a “Disputed” or “Rated false” tag. Though the magnitudes of these effects are relatively modest, they generally do not vary by whether headlines were congenial to respondents’ political views. In addition, we find that adding a “Rated false” tag to an article headline lowers its perceived accuracy more than adding a “Disputed” tag (Facebook’s original approach) relative to a control condition. Finally, though exposure to the “Disputed” or “Rated false” tags did not affect the perceived accuracy of unlabeled false or true headlines, exposure to a general warning decreased belief in the accuracy of true headlines, suggesting the need for further research into how to most effectively counter false news without distorting belief in true information.

Keywords

Fake news Fact check Warnings Corrections Social media Misperceptions 

Notes

Acknowledgements

We thank the Dartmouth College Office of Undergraduate Research for generous funding support. We are also grateful to Ro’ee Levy and David Rand for helpful comments.

Supplementary material

11109_2019_9533_MOESM1_ESM.pdf (16.5 mb)
Supplementary material 1 (PDF 16849 KB)

References

  1. Aklin, M., & Urpelainen, J. (2014). Perceptions of scientific dissent undermine public support for environmental policy. Environmental Science and Policy, 38, 173–177.CrossRefGoogle Scholar
  2. Allcott, H., & Gentzkow, M. (2017). Social media and fake news in the 2016 election. Journal of Economic Perspectives, 31(2), 211–236.CrossRefGoogle Scholar
  3. Atkins, L. (2017). States should require schools to teach media literacy to combat fake news. The Huffington Post, July 13, 2017. Retrieved July 8, 2018, from https://www.huffingtonpost.com/entry/states-should-require-schools-to-teach-media-literacy_us_59676573e4b07b5e1d96ed86.
  4. Berinsky, A. J., Gregory, A. H., & Gabriel, S. L. (2012). Evaluating online labor markets for experimental research: Amazoncom’s mechanical turk. Political Analysis, 20(3), 351–368.CrossRefGoogle Scholar
  5. Bode, L., & Vraga, E. K. (2015). In related news, that was wrong: The correction of misinformation through related stories functionality in social media. Journal of Communication, 65(4), 619–638.CrossRefGoogle Scholar
  6. Bolsen, T., & Druckman, J. N. (2015). Counteracting the politicization of science. Journal of Communication, 65(5), 745–769.CrossRefGoogle Scholar
  7. Boykoff, M. T., & Boykoff, J. M. (2004). Balance as bias: Global warming and the US prestige press. Global Environmetal Change, 14, 125–136.CrossRefGoogle Scholar
  8. Bullock, J. G., Green, D. P., & Shang, E. H. (2010). Yes, but what’s the mechanism? (Don’t expect an easy answer). Journal of Personality and Social Psychology, 98(4), 550–558.CrossRefGoogle Scholar
  9. Chinn, S., Lane, D. S., & Hart, P. S. (2018). In consensus we trust? Persuasive effects of scientific consensus communication. Public Understanding of Science, 27(7), 807–823.CrossRefGoogle Scholar
  10. Clayton, K., Davis, J., Hinckley, K., & Horiuchi, Y. (2018). Partisan motivated reasoning and misinformation in the media: Is news from ideologically uncongenial sources more suspicious? Unpublished manuscript. Retrieved July 8, 2018, from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3035272.
  11. Constine, J. (2017). Facebook puts link to 10 tips for spotting ‘false news’ atop feed. Tech Crunch, April 6, 2017. Retrieved July 18, from https://techcrunch.com/2017/04/06/facebook-puts-link-to-10-tips-for-spotting-false-news-atop-feed/.
  12. Coppock, A. (2016). Generalizing from survey experiments conducted on mechanical Turk: A replication approach. Unpublished manuscript, March 22, 2016. Retrieved July 8, 2018, from https://alexandercoppock.files.wordpress.com/2016/02/coppock_generalizability2.pdf.
  13. Corbett, J. B., & Durfee, J. L. (2004). Testing public (un)certainty of science media representations of global warming. Science Communication, 26(2), 129–151.CrossRefGoogle Scholar
  14. de Leeuw, E. D., Hox, J. J., & Boevé, A. (2015). Handling do-not-know answers: Exploring new approaches in online and mixed-mode surveys. Social Science Computer Review, 34(1), 116–132.CrossRefGoogle Scholar
  15. Echterhoff, G., Groll, S., & Hirst, W. (2007). Tainted truth: Overcorrection for misinformation influence on eyewitness memory. Social Cognition, 25(3), 367–409.CrossRefGoogle Scholar
  16. Ecker, U. K. H., Lewandowsky, S., Chang, E. P., & Pillai, R. (2014). The effects of subtle misinformation in news headlines. Journal of Experimental Psychology: Applied, 20(4), 323–335.Google Scholar
  17. Ecker, U. K. H., Lewandowsky, S., & Tang, D. T. W. (2010). Explicit warnings reduce but do not eliminate the continued influence of misinformation. Memory & cognition, 38(8), 1087–1100.CrossRefGoogle Scholar
  18. Flynn, D. J., Nyhan, B., & Reifler, J. (2017). The nature and origins of misperceptions: Understanding false and unsupported beliefs about politics. Political Psychology, 38(S1), 127–150.CrossRefGoogle Scholar
  19. Gabielkov, M., Ramachandran, A., Chaintreau, A., Legout, A. (2016). Social clicks: What and who gets read on twitter? In Proceedings of the 2016 ACM SIGMETRICS international conference on measurement and modeling of computer science. Ne York: ACM.Google Scholar
  20. Gottfried, J., & Shearer, E. (2017). News use across social media platforms 2016. Pew Research Center, May 26, 2016. Retrieved May 23, 2017, from http://www.journalism.org/2016/05/26/news-use-across-social-media-platforms-2016/.
  21. Horton, J. J., Rand, D. G., & Zeckhauser, R. J. (2011). The online laboratory: Conducting experiments in a real labor market. Experimental Economics, 14(3), 399–425.CrossRefGoogle Scholar
  22. Kahan, D. M. (2015). Climate-science communication and the measurement problem. Political Psychology, 36(S1), 1–43.CrossRefGoogle Scholar
  23. Kahan, D. M., Dawson, E. C., Peters, E., & Slovic, P. (2017). Motivated numeracy and enlightened self-government. Behavioural Public Policy, 1(1), 54–86.CrossRefGoogle Scholar
  24. Koehler, D. J. (2016). Can journalistic false balance distort public perception of consensus in expert opinion? Journal of Experimental Psychology: Applied, 22(1), 24–38.Google Scholar
  25. Krupnikov, Y., & Levine, A. S. (2014). Cross-sample comparisons and external validity. Journal of Experimental Political Science, 1(1), 59–80.CrossRefGoogle Scholar
  26. Kuru, O., Pasek, J., & Traugott, M. W. (2017). Motivated reasoning in the perceived credibility of public opinion polls. Public Opinion Quarterly, 81(2), 422–446.CrossRefGoogle Scholar
  27. Lazer, D. M. J., Baum, M. A., Benkler, Y., Berinsky, A. J., Greenhill, K. M., Menczer, F., et al. (2018). The science of fake news. Science, 359(6380), 1094–1096.CrossRefGoogle Scholar
  28. Lewandowsky, S., Ecker, U. K. H., Seifert, C. M., Schwarz, N., & Cook, J. (2012). Misinformation and its correction: Continued influence and successful debiasing. Psychological Science in the Public Interest, 13(3), 106–131.CrossRefGoogle Scholar
  29. Manjoo, F. (2013). You won’t finish this article. Slate Magazine, June 6, 2013. Retrieved May 23, 2017, from http://www.slate.com/articles/technology/technology/2013/06/how_people_read_online_why_you_won_t_finish_this_article.html.
  30. Mosseri, A. (2016). Addressing Hoaxes and Fake news. Facebook, December 15, 2016. Retrieved July 18, 2018, from https://newsroom.fb.com/news/2016/12/news-feed-fyi-addressing-hoaxes-and-fake-news/.
  31. Mosseri, A. (2017). A new educational tool against misinformation. Facebook, April 6, 2017. Retrieved May 23, 2017, from https://newsroom.fb.com/news/2017/04/a-new-educational-tool-against-misinformation/.
  32. Mullinix, K. J., Leeper, T. J., Druckman, J. N., & Freese, J. (2015). The generalizability of survey experiments. Journal of Experimental Political Science, 2(2), 109–138.CrossRefGoogle Scholar
  33. Mummolo, J., & Peterson, E. (2018). Demand effects in survey experiments: An empirical assessment. American Political Science Review. Retrieved January 5, 2019, from https://scholar.princeton.edu/sites/default/files/jmummolo/files/demand_effects_in_survey_experiments_an_empirical_assessment.pdf.
  34. Nyhan, B., Porter, E., Reifler, J. & Wood, T. (2017). Taking corrections literally but not seriously? The effects of information on factual beliefs and candidate favorability. Unpublished manuscript. Retrieved July 8, 2018, from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2995128.
  35. Nyhan, B., & Reifler, J. (2010). When corrections fail: The persistence of political misperceptions. Political Behavior, 32(2), 303–330.CrossRefGoogle Scholar
  36. Nyhan, B., & Reifler, J. (N.d). Do people actually learn from fact-checking? Evidence from a longitudinal study during the 2014 campaign. Unpublished manuscript. Retrieved September 20, 2017, from http://www.dartmouth.edu/~nyhan/fact-checking-effects.pdf.
  37. Oremus, W. (2017). Facebook has stopped saying fake news. Slate, August 8, 2017. Retrieved July 8, 2018, from http://www.slate.com/blogs/future_tense/2017/08/08/facebook_has_stopped_saying_fake_news_is_false_news_any_better.html.
  38. Owen, L. H. (2018). Is your fake news about immigrants or politicians? It all depends on where you live. Nieman Journalism Lab, May 25, 2018. Retrieved July 18, 2018, from http://www.niemanlab.org/2018/05/is-your-fake-news-about-immigrants-or-politicians-it-all-depends-on-where-you-live/.
  39. Pennycook, G., & Rand, D. G. (2017). The implied truth effect: Attaching warnings to a subset of fake news stories increases perceived accuracy of stories without warnings. Unpublished manuscript. Retrieved July 8, 2018, from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3035384.
  40. Pennycook, G., & Rand D. G. (2018a). Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning. Cognition. Retrieved July 8, 2018, from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3165567.
  41. Pennycook, G., & Rand, D. G. (2018b). Who falls for fake news? The roles of analytic thinking, motivated reasoning, political ideology, and bullshit receptivity. Unpublished manuscript. Retrieved July 10, 2018, from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3023545.
  42. Pennycook, G., Cannon, T. D., & Rand, D. G. (2017). Prior exposure increases perceived accuracy of fake news. Journal of Experimental Psychology: General. Retrieved May 23, 2017, from https://papers.ssrn.com/sol3/papers.cfm?abstract\_id=2958246.Google Scholar
  43. Porter, E., Wood, T. J., & Kirby, D. (2018). Sex trafficking, Russian infiltration, birth certificates, and pedophilia: A survey experiment correcting fake news. Journal of Experimental Political Science, 5(2), 159–164.CrossRefGoogle Scholar
  44. Schaedel, S. (2017). Black lives matter blocked hurricane relief? Factcheck.org, September 1, 2017. Retrieved September 26, 2017 from http://www.factcheck.org/2017/09/black-lives-matter-blocked-hurricane-relief/.
  45. Silverman, C. (2016). This analysis shows how viral fake election news stories outperformed real news on facebook. Buzzfeed, November 16, 2016. Retrieved May 22, 2017, from https://www.buzzfeed.com/craigsilverman/viral-fake-election-news-outperformed-real-news-on-facebook?utm_term=.lgQvmj974#.qqXqL1AJV.
  46. Silverman, C., & Jeremy S.-V. (2016). Most Americans who see fake news believe it, new survey says. December 6, 2016. Retrieved May 23, 2017, from https://www.buzzfeed.com/craigsilverman/fake-news-survey?utm_term=.srvopPEVR#.cqlAz0PeX.
  47. Smith, J., Jackson, G., & Raj, S. (2017). Designing against misinformation. Medium, December 20, 2017. Retrieved July 8, 2018, from https://medium.com/facebook-design/designing-against-misinformation-e5846b3aa1e2.
  48. Szpitalak, M., & Polczyk, R. (2010). Warning against warnings: Alerted subjects may perform worse. Misinformation, involvement and warning as determinants of witness testimony. Polish Psychological Bulletin, 41(3), 105–112.CrossRefGoogle Scholar
  49. Thorson, E. (2016). Belief echoes: The persistent effects of corrected misinformation. Political Communication, 33(3), 460–480.CrossRefGoogle Scholar
  50. van der Linden, S., Leiserowitz, A., Rosenthal, S., & Maibach, E. (2017). Inoculating the public against misinformation about climate change. Global Challenges, 1(2), 1600008.CrossRefGoogle Scholar
  51. Wardle, C., & Derakhshan, H. (2017). Information disorder: Toward an interdisciplinary framework for research and policy making. Council of Europe Report, September 27, 2017. Retrieved July 8, 2018, from https://rm.coe.int/information-disorder-toward-an-interdisciplinary-framework-for-researc/168076277c.
  52. Wegner, D. M., Richard Wenzlaff, R., Kerker, M., & Beattie, A. E. (1981). Incrimination through innuendo: Can Media questions become public answers? Journal of Personality and Social Psychology, 40(5), 822–832.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Katherine Clayton
    • 1
  • Spencer Blair
    • 1
  • Jonathan A. Busam
    • 1
  • Samuel Forstner
    • 1
  • John Glance
    • 1
  • Guy Green
    • 1
  • Anna Kawata
    • 1
  • Akhila Kovvuri
    • 1
  • Jonathan Martin
    • 1
  • Evan Morgan
    • 1
  • Morgan Sandhu
    • 1
  • Rachel Sang
    • 1
  • Rachel Scholz-Bright
    • 1
  • Austin T. Welch
    • 1
  • Andrew G. Wolff
    • 1
  • Amanda Zhou
    • 1
  • Brendan Nyhan
    • 2
    Email author
  1. 1.Dartmouth CollegeHanoverUSA
  2. 2.University of MichiganAnn ArborUSA

Personalised recommendations