Many scholars study when climate change communication increases citizen engagement. Yet, past work has largely used public opinion-based measures of engagement to evaluate alternative frames. In this paper, we argue for a new approach to evaluation, which is premised on research on the policy-making process showing that space on the political agenda and, ultimately, policy change are more likely to arise in response to changes in both public opinion and collective political action. Thus, we argue that alternative frames should be evaluated based on their consequences for both. This is especially critical given that frames can have divergent effects on attitudes and behavior. Using a combination of field and survey experiments, we apply our approach to evaluate two frames related to climate change risks. We find that they heighten people’s concern about climate change yet decrease their rate of political action to express that concern. Our results suggest caution with regard to these frames in particular and that, more generally, frames that might seem advantageous when examining public opinion may not be when political behavior is analyzed.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
There are some exceptions, such as Bain et al. (2012) and Bernauer and McGrath (2016), which examine the effect of frames on behavioral intentions but do not measure political behavior itself. Also note that our focus is on political engagement – that is, attitudes and behaviors designed to affect public policy. A separate line of work examines the effect of frames on non-political, environmentally friendly behaviors such as personal energy consumption (e.g., Bolsen 2011; Bolsen et al. 2014).
Both of our partner organizations requested that they remain anonymous in print.
With different partners, such as more conservative-leaning organizations, we would likely have chosen to rent from different lists.
Note that, unlike many previous climate change studies, the control groups in our experiments always received some climate information. The reason is because, in a field experiment setting like ours in which even our control group was receiving an email from our partner organization, it would not have made sense to have only neutral and/or non-climate information. For instance, the control groups received information about the National Climate Assessment and the scientific consensus about climate change (see the Supplementary Information).
We minimized psychological distance by focusing on impacts in the USA and using language that conveyed temporal proximity (Leiserowitz 2005).
Two theoretical points are worth noting here. First, we acknowledge that adding text can introduce a possible confound (increased email length). In another paper with a very similar experimental design Levine and Kline (2017) rule out this potential confound by showing that it is the threat to material well-being that accounts for differences across groups, and not simply lengthened text. Second, as shown in the appendix, the treatment text in all of our experiments was very personal – it encouraged all respondents to directly think about how the issue would threaten their own material well-being. Levine and Kline (2017) show that a treatment text that uses sociotropic language has a similar effect but only among people that are already experiencing health hardships in their own lives (i.e., they are the respondents most likely to personally feel threatened). We would argue that the underlying mechanism – the idea that frames highlighting threats to material well-being can make people feel poorer, which in turn reduces their willingness to spend scarce resources on politics – is the same across studies.
These outcomes were tallied one week after the emails were sent.
In fact, to put these numbers into perspective, the total number of responses to our email led to an almost 25% increase in the size of the organization’s total membership list.
Again, we omit its name in print.
Aklin M, Urpelainen J (2013) Debating clean energy: frames, counter frames, and audiences. Glob Environ Chang 23:1225–1232
Bain PG, Matthew J, Hornsey RB, Jeffries C (2012) Promoting pro-environmental action in climate change deniers. Nat Clim Chang 2:600–603
Bartels LM (2008) Unequal democracy: the political economy of the new gilded age. Princeton University Press
Baumgartner FR, Jeffrey MB, Hojnacki M, Leech BL, Kimball DC (2009) Lobbying and policy change: who wins, who loses, and why. University of Chicago Press
Berinsky A, Huber G, Lenz G (2012) Evaluating online labor markets for experimental research: Amazon. com’s mechanical turk. Polit Anal 20:351–368
Bernauer T, McGrath LF (2016) Simple reframing unlikely to boost public support for climate policy. Nature climate change forthcoming
Bolsen T (2011) A light bulb goes on: norms, rhetoric, and actions for the public good. Polit Behav 35:1–20
Bolsen T, Druckman JN, Cook FL (2014) Communication and collective actions: a survey experiment on motivating energy conservation in the U.S. J Exp Political Sci 1:24–38
Bolsen T, Leeper TJ, Shapiro MA (2014) Doing what others do: norms, science, and collective action on global warming. Amer Politics Res 42:65–89
Druckman JN, Jacobs LR (2015) Who governs? Presidents, public opinion, and manipulation. University of Chicago Press
Feinberg M, Willer R (2011) Apocalypse soon? Dire messages reduce belief in global warming by contradicting just-world beliefs. Psychol Sci 22:34–38
Gerber AS, Green DP (2012), Field experiments: design, analysis, and interpretation. WW Norton
Gilens M (2012) Affluence and influence. Princeton University Press
Han H (2014) How organizations develop activists. Oxford University Press
Hart PS, Nisbet EC (2011) Boomerang effects in science communication: how motivated reasoning and identity cues amplify opinion polarization about climate mitigation policies. Commun Res 39:701–723
Hestres LE (2014) Preaching to the choir: Internet-mediated advocacy, issue public mobilization, and climate change. Med Soc 16:323–339
Hestres LE (2015) Climate change advocacy online: theories of change, target audiences, and online strategy. Environ Politics 24:193–211
Kahan DM, Peters E, Wittlin M, Slovic P, Ouellette LL, Braman D, Mandel G (2012) The polarizing impact of science literacy and numeracy on perceived climate change risks. Nat Clim Chang 2:732–735
Karl TR (2009) Global climate change impacts in the United States. Cambridge University Press
Karpf D (2012) The moveon effect. Oxford University Press
Kollman K (1998), Outside lobbying: public opinion and interest group strategies. Princeton University Press
Krupnikov Y, Levine AS (2014) Cross-sample comparisons and external validity. J Exp Political Sci 1(01):59–80
Leiserowitz A (2005) American risk perceptions: is climate change dangerous?. Risk Anal 25:1433–1442
Levine AS (2015) American insecurity. Princeton University Press
Levine AS, Kline R (2017) When does self-interest motivate issue engagement? The case of climate change.” Available at SSRN: https://ssrn.com/abstract=2931842
Lockwood M (2011) Does the framing of climate policies make a different to public support? Evidence from UK marginal constituencies. Clim Pol 11:1097–1112
Lupia A, Levine AS, Menning JO, Sin G (2007) Were bush tax cut supporters ‘simply ignorant’? A second look at conservatives and liberals in ‘homer gets a tax cut. Perspect Polit 4:773–784
Maibach EW, Roser-Renouf C, Leiserowitz A (2009) Global Warming’s Six Americas 2009: an audience segmentation analysis. http://climatechange.gmu.edu
Maibach EW, Nisbet M, Baldwin P, Akerlof K, Diao G (2010) Reframing climate change as a public health issue: an exploratory study of public reactions. BMC Publ Health 10:299–309
McCright AM, Dunlap RE (2011) The politicization of climate change and polarization in the American public’s views of global warming, 2001-2010. Sociol Q 52:155–194
Myers TA, Nisbet MC, Maibach EW, Leiserowitz AA (2012) A public health frame arouses hopeful emotions about climate change. Clim Chang 113:1105–1112
Nisbet MC, Markowitz EM, Kotcher JE (2012) Winning the conversation: framing and moral messaging in environmental campaigns. In: Ahern L, Bortree D (eds) Talking green: exploring contemporary issues in environmental communications. Peter Lang, New York
Roser-Renouf C, Edward WM, Leiserowitz AA, Zhao X (2014) The genesis of climate change activism: from key beliefs to political action. Clim Chang 125:163–178
Scannell L, Gifford R (2013) Personally relevant climate change: the role of place attachment and local versus global message framing in engagement. Environ Behav 45:60–85
Schuldt JP, Konrath SH, Schwarz N (2011) Global warming or climate change? Whether the planet is warming depends on question wording. Pub Opin Q 75:115–124
Spence A, Pidgeon N (2010) Framing and communicating climate change: the effects of distance and outcome frame manipulations. Glob Environ Chang 20:656–667
Stern PC (2000) Toward a coherent theory of environmentally significant behavior. J Soc Issues 56:407–424
Van der Linden S, Maibach E, Leiserowitz A (2015) Improving public engagement with climate change: five best practice insights from psychological science. Perspect Psychol Sci 10(6):758–763
Van der Linden S, Leiserowitz AA, Feinberg GD, Maibach EW (2015) The scientific consensus on climate change as a gateway belief: experimental evidence. PLoS ONE 10:e0118489
Yeager DS, Larson SB, Krosnick JA, Tompson T (2011) Measuring Americans’ issue priorities: a new version of the most important problem question reveals more concern about global warming and the environment. Pub Opin Q 75:125–138
We thank Andrew Gooch, Donald Haider-Markel, Yanna Krupnikov, Brendan Nyhan, our anonymous reviewers, and seminar participants at UCSB, Syracuse University, and Dartmouth University for helpful and engaging feedback. We also thank the Atkinson Center for aSustainable Future at Cornell University for generous funding
Electronic supplementary material
Below is the link to the electronic supplementary material.
About this article
Cite this article
Levine, A.S., Kline, R. A new approach for evaluating climate change communication. Climatic Change 142, 301–309 (2017). https://doi.org/10.1007/s10584-017-1952-x
- Public Opinion
- Partner Organization
- Political Engagement
- Climate Change Risk
- Climate Change Communication