Real Solutions for Fake News? Measuring the Effectiveness of General Warnings and Fact-Check Tags in Reducing Belief in False Stories on Social Media
Social media has increasingly enabled “fake news” to circulate widely, most notably during the 2016 U.S. presidential campaign. These intentionally false or misleading stories threaten the democratic goal of a well-informed electorate. This study evaluates the effectiveness of strategies that could be used by Facebook and other social media to counter false stories. Results from a pre-registered experiment indicate that false headlines are perceived as less accurate when people receive a general warning about misleading information on social media or when specific headlines are accompanied by a “Disputed” or “Rated false” tag. Though the magnitudes of these effects are relatively modest, they generally do not vary by whether headlines were congenial to respondents’ political views. In addition, we find that adding a “Rated false” tag to an article headline lowers its perceived accuracy more than adding a “Disputed” tag (Facebook’s original approach) relative to a control condition. Finally, though exposure to the “Disputed” or “Rated false” tags did not affect the perceived accuracy of unlabeled false or true headlines, exposure to a general warning decreased belief in the accuracy of true headlines, suggesting the need for further research into how to most effectively counter false news without distorting belief in true information.
KeywordsFake news Fact check Warnings Corrections Social media Misperceptions
We thank the Dartmouth College Office of Undergraduate Research for generous funding support. We are also grateful to Ro’ee Levy and David Rand for helpful comments.
- Atkins, L. (2017). States should require schools to teach media literacy to combat fake news. The Huffington Post, July 13, 2017. Retrieved July 8, 2018, from https://www.huffingtonpost.com/entry/states-should-require-schools-to-teach-media-literacy_us_59676573e4b07b5e1d96ed86.
- Clayton, K., Davis, J., Hinckley, K., & Horiuchi, Y. (2018). Partisan motivated reasoning and misinformation in the media: Is news from ideologically uncongenial sources more suspicious? Unpublished manuscript. Retrieved July 8, 2018, from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3035272.
- Constine, J. (2017). Facebook puts link to 10 tips for spotting ‘false news’ atop feed. Tech Crunch, April 6, 2017. Retrieved July 18, from https://techcrunch.com/2017/04/06/facebook-puts-link-to-10-tips-for-spotting-false-news-atop-feed/.
- Coppock, A. (2016). Generalizing from survey experiments conducted on mechanical Turk: A replication approach. Unpublished manuscript, March 22, 2016. Retrieved July 8, 2018, from https://alexandercoppock.files.wordpress.com/2016/02/coppock_generalizability2.pdf.
- Ecker, U. K. H., Lewandowsky, S., Chang, E. P., & Pillai, R. (2014). The effects of subtle misinformation in news headlines. Journal of Experimental Psychology: Applied, 20(4), 323–335.Google Scholar
- Gabielkov, M., Ramachandran, A., Chaintreau, A., Legout, A. (2016). Social clicks: What and who gets read on twitter? In Proceedings of the 2016 ACM SIGMETRICS international conference on measurement and modeling of computer science. Ne York: ACM.Google Scholar
- Gottfried, J., & Shearer, E. (2017). News use across social media platforms 2016. Pew Research Center, May 26, 2016. Retrieved May 23, 2017, from http://www.journalism.org/2016/05/26/news-use-across-social-media-platforms-2016/.
- Koehler, D. J. (2016). Can journalistic false balance distort public perception of consensus in expert opinion? Journal of Experimental Psychology: Applied, 22(1), 24–38.Google Scholar
- Manjoo, F. (2013). You won’t finish this article. Slate Magazine, June 6, 2013. Retrieved May 23, 2017, from http://www.slate.com/articles/technology/technology/2013/06/how_people_read_online_why_you_won_t_finish_this_article.html.
- Mosseri, A. (2016). Addressing Hoaxes and Fake news. Facebook, December 15, 2016. Retrieved July 18, 2018, from https://newsroom.fb.com/news/2016/12/news-feed-fyi-addressing-hoaxes-and-fake-news/.
- Mosseri, A. (2017). A new educational tool against misinformation. Facebook, April 6, 2017. Retrieved May 23, 2017, from https://newsroom.fb.com/news/2017/04/a-new-educational-tool-against-misinformation/.
- Mummolo, J., & Peterson, E. (2018). Demand effects in survey experiments: An empirical assessment. American Political Science Review. Retrieved January 5, 2019, from https://scholar.princeton.edu/sites/default/files/jmummolo/files/demand_effects_in_survey_experiments_an_empirical_assessment.pdf.
- Nyhan, B., Porter, E., Reifler, J. & Wood, T. (2017). Taking corrections literally but not seriously? The effects of information on factual beliefs and candidate favorability. Unpublished manuscript. Retrieved July 8, 2018, from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2995128.
- Nyhan, B., & Reifler, J. (N.d). Do people actually learn from fact-checking? Evidence from a longitudinal study during the 2014 campaign. Unpublished manuscript. Retrieved September 20, 2017, from http://www.dartmouth.edu/~nyhan/fact-checking-effects.pdf.
- Oremus, W. (2017). Facebook has stopped saying fake news. Slate, August 8, 2017. Retrieved July 8, 2018, from http://www.slate.com/blogs/future_tense/2017/08/08/facebook_has_stopped_saying_fake_news_is_false_news_any_better.html.
- Owen, L. H. (2018). Is your fake news about immigrants or politicians? It all depends on where you live. Nieman Journalism Lab, May 25, 2018. Retrieved July 18, 2018, from http://www.niemanlab.org/2018/05/is-your-fake-news-about-immigrants-or-politicians-it-all-depends-on-where-you-live/.
- Pennycook, G., & Rand, D. G. (2017). The implied truth effect: Attaching warnings to a subset of fake news stories increases perceived accuracy of stories without warnings. Unpublished manuscript. Retrieved July 8, 2018, from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3035384.
- Pennycook, G., & Rand D. G. (2018a). Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning. Cognition. Retrieved July 8, 2018, from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3165567.
- Pennycook, G., & Rand, D. G. (2018b). Who falls for fake news? The roles of analytic thinking, motivated reasoning, political ideology, and bullshit receptivity. Unpublished manuscript. Retrieved July 10, 2018, from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3023545.
- Pennycook, G., Cannon, T. D., & Rand, D. G. (2017). Prior exposure increases perceived accuracy of fake news. Journal of Experimental Psychology: General. Retrieved May 23, 2017, from https://papers.ssrn.com/sol3/papers.cfm?abstract\_id=2958246.Google Scholar
- Schaedel, S. (2017). Black lives matter blocked hurricane relief? Factcheck.org, September 1, 2017. Retrieved September 26, 2017 from http://www.factcheck.org/2017/09/black-lives-matter-blocked-hurricane-relief/.
- Silverman, C. (2016). This analysis shows how viral fake election news stories outperformed real news on facebook. Buzzfeed, November 16, 2016. Retrieved May 22, 2017, from https://www.buzzfeed.com/craigsilverman/viral-fake-election-news-outperformed-real-news-on-facebook?utm_term=.lgQvmj974#.qqXqL1AJV.
- Silverman, C., & Jeremy S.-V. (2016). Most Americans who see fake news believe it, new survey says. December 6, 2016. Retrieved May 23, 2017, from https://www.buzzfeed.com/craigsilverman/fake-news-survey?utm_term=.srvopPEVR#.cqlAz0PeX.
- Smith, J., Jackson, G., & Raj, S. (2017). Designing against misinformation. Medium, December 20, 2017. Retrieved July 8, 2018, from https://medium.com/facebook-design/designing-against-misinformation-e5846b3aa1e2.
- Wardle, C., & Derakhshan, H. (2017). Information disorder: Toward an interdisciplinary framework for research and policy making. Council of Europe Report, September 27, 2017. Retrieved July 8, 2018, from https://rm.coe.int/information-disorder-toward-an-interdisciplinary-framework-for-researc/168076277c.