A new approach for evaluating climate change communication


Many scholars study when climate change communication increases citizen engagement. Yet, past work has largely used public opinion-based measures of engagement to evaluate alternative frames. In this paper, we argue for a new approach to evaluation, which is premised on research on the policy-making process showing that space on the political agenda and, ultimately, policy change are more likely to arise in response to changes in both public opinion and collective political action. Thus, we argue that alternative frames should be evaluated based on their consequences for both. This is especially critical given that frames can have divergent effects on attitudes and behavior. Using a combination of field and survey experiments, we apply our approach to evaluate two frames related to climate change risks. We find that they heighten people’s concern about climate change yet decrease their rate of political action to express that concern. Our results suggest caution with regard to these frames in particular and that, more generally, frames that might seem advantageous when examining public opinion may not be when political behavior is analyzed.

This is a preview of subscription content, log in to check access.

Fig. 1


  1. 1.

    There are some exceptions, such as Bain et al. (2012) and Bernauer and McGrath (2016), which examine the effect of frames on behavioral intentions but do not measure political behavior itself. Also note that our focus is on political engagement – that is, attitudes and behaviors designed to affect public policy. A separate line of work examines the effect of frames on non-political, environmentally friendly behaviors such as personal energy consumption (e.g., Bolsen 2011; Bolsen et al. 2014).

  2. 2.

    Both of our partner organizations requested that they remain anonymous in print.

  3. 3.

    With different partners, such as more conservative-leaning organizations, we would likely have chosen to rent from different lists.

  4. 4.

    Note that, unlike many previous climate change studies, the control groups in our experiments always received some climate information. The reason is because, in a field experiment setting like ours in which even our control group was receiving an email from our partner organization, it would not have made sense to have only neutral and/or non-climate information. For instance, the control groups received information about the National Climate Assessment and the scientific consensus about climate change (see the Supplementary Information).

  5. 5.

    We minimized psychological distance by focusing on impacts in the USA and using language that conveyed temporal proximity (Leiserowitz 2005).

  6. 6.

    Two theoretical points are worth noting here. First, we acknowledge that adding text can introduce a possible confound (increased email length). In another paper with a very similar experimental design Levine and Kline (2017) rule out this potential confound by showing that it is the threat to material well-being that accounts for differences across groups, and not simply lengthened text. Second, as shown in the appendix, the treatment text in all of our experiments was very personal – it encouraged all respondents to directly think about how the issue would threaten their own material well-being. Levine and Kline (2017) show that a treatment text that uses sociotropic language has a similar effect but only among people that are already experiencing health hardships in their own lives (i.e., they are the respondents most likely to personally feel threatened). We would argue that the underlying mechanism – the idea that frames highlighting threats to material well-being can make people feel poorer, which in turn reduces their willingness to spend scarce resources on politics – is the same across studies.

  7. 7.

    These outcomes were tallied one week after the emails were sent.

  8. 8.

    In fact, to put these numbers into perspective, the total number of responses to our email led to an almost 25% increase in the size of the organization’s total membership list.

  9. 9.

    Again, we omit its name in print.


  1. Aklin M, Urpelainen J (2013) Debating clean energy: frames, counter frames, and audiences. Glob Environ Chang 23:1225–1232

    Article  Google Scholar 

  2. Bain PG, Matthew J, Hornsey RB, Jeffries C (2012) Promoting pro-environmental action in climate change deniers. Nat Clim Chang 2:600–603

    Article  Google Scholar 

  3. Bartels LM (2008) Unequal democracy: the political economy of the new gilded age. Princeton University Press

  4. Baumgartner FR, Jeffrey MB, Hojnacki M, Leech BL, Kimball DC (2009) Lobbying and policy change: who wins, who loses, and why. University of Chicago Press

  5. Berinsky A, Huber G, Lenz G (2012) Evaluating online labor markets for experimental research: Amazon. com’s mechanical turk. Polit Anal 20:351–368

    Article  Google Scholar 

  6. Bernauer T, McGrath LF (2016) Simple reframing unlikely to boost public support for climate policy. Nature climate change forthcoming

  7. Bolsen T (2011) A light bulb goes on: norms, rhetoric, and actions for the public good. Polit Behav 35:1–20

    Article  Google Scholar 

  8. Bolsen T, Druckman JN, Cook FL (2014) Communication and collective actions: a survey experiment on motivating energy conservation in the U.S. J Exp Political Sci 1:24–38

    Article  Google Scholar 

  9. Bolsen T, Leeper TJ, Shapiro MA (2014) Doing what others do: norms, science, and collective action on global warming. Amer Politics Res 42:65–89

    Article  Google Scholar 

  10. Druckman JN, Jacobs LR (2015) Who governs? Presidents, public opinion, and manipulation. University of Chicago Press

  11. Feinberg M, Willer R (2011) Apocalypse soon? Dire messages reduce belief in global warming by contradicting just-world beliefs. Psychol Sci 22:34–38

    Article  Google Scholar 

  12. Gerber AS, Green DP (2012), Field experiments: design, analysis, and interpretation. WW Norton

  13. Gilens M (2012) Affluence and influence. Princeton University Press

  14. Han H (2014) How organizations develop activists. Oxford University Press

  15. Hart PS, Nisbet EC (2011) Boomerang effects in science communication: how motivated reasoning and identity cues amplify opinion polarization about climate mitigation policies. Commun Res 39:701–723

    Article  Google Scholar 

  16. Hestres LE (2014) Preaching to the choir: Internet-mediated advocacy, issue public mobilization, and climate change. Med Soc 16:323–339

    Google Scholar 

  17. Hestres LE (2015) Climate change advocacy online: theories of change, target audiences, and online strategy. Environ Politics 24:193–211

    Article  Google Scholar 

  18. Kahan DM, Peters E, Wittlin M, Slovic P, Ouellette LL, Braman D, Mandel G (2012) The polarizing impact of science literacy and numeracy on perceived climate change risks. Nat Clim Chang 2:732–735

    Article  Google Scholar 

  19. Karl TR (2009) Global climate change impacts in the United States. Cambridge University Press

  20. Karpf D (2012) The moveon effect. Oxford University Press

  21. Kollman K (1998), Outside lobbying: public opinion and interest group strategies. Princeton University Press

  22. Krupnikov Y, Levine AS (2014) Cross-sample comparisons and external validity. J Exp Political Sci 1(01):59–80

    Article  Google Scholar 

  23. Leiserowitz A (2005) American risk perceptions: is climate change dangerous?. Risk Anal 25:1433–1442

    Article  Google Scholar 

  24. Levine AS (2015) American insecurity. Princeton University Press

  25. Levine AS, Kline R (2017) When does self-interest motivate issue engagement? The case of climate change.” Available at SSRN: https://ssrn.com/abstract=2931842

  26. Lockwood M (2011) Does the framing of climate policies make a different to public support? Evidence from UK marginal constituencies. Clim Pol 11:1097–1112

    Article  Google Scholar 

  27. Lupia A, Levine AS, Menning JO, Sin G (2007) Were bush tax cut supporters ‘simply ignorant’? A second look at conservatives and liberals in ‘homer gets a tax cut. Perspect Polit 4:773–784

    Google Scholar 

  28. Maibach EW, Roser-Renouf C, Leiserowitz A (2009) Global Warming’s Six Americas 2009: an audience segmentation analysis. http://climatechange.gmu.edu

  29. Maibach EW, Nisbet M, Baldwin P, Akerlof K, Diao G (2010) Reframing climate change as a public health issue: an exploratory study of public reactions. BMC Publ Health 10:299–309

    Article  Google Scholar 

  30. McCright AM, Dunlap RE (2011) The politicization of climate change and polarization in the American public’s views of global warming, 2001-2010. Sociol Q 52:155–194

    Article  Google Scholar 

  31. Myers TA, Nisbet MC, Maibach EW, Leiserowitz AA (2012) A public health frame arouses hopeful emotions about climate change. Clim Chang 113:1105–1112

    Article  Google Scholar 

  32. Nisbet MC, Markowitz EM, Kotcher JE (2012) Winning the conversation: framing and moral messaging in environmental campaigns. In: Ahern L, Bortree D (eds) Talking green: exploring contemporary issues in environmental communications. Peter Lang, New York

    Google Scholar 

  33. Roser-Renouf C, Edward WM, Leiserowitz AA, Zhao X (2014) The genesis of climate change activism: from key beliefs to political action. Clim Chang 125:163–178

    Article  Google Scholar 

  34. Scannell L, Gifford R (2013) Personally relevant climate change: the role of place attachment and local versus global message framing in engagement. Environ Behav 45:60–85

    Article  Google Scholar 

  35. Schuldt JP, Konrath SH, Schwarz N (2011) Global warming or climate change? Whether the planet is warming depends on question wording. Pub Opin Q 75:115–124

    Article  Google Scholar 

  36. Spence A, Pidgeon N (2010) Framing and communicating climate change: the effects of distance and outcome frame manipulations. Glob Environ Chang 20:656–667

    Article  Google Scholar 

  37. Stern PC (2000) Toward a coherent theory of environmentally significant behavior. J Soc Issues 56:407–424

    Article  Google Scholar 

  38. Van der Linden S, Maibach E, Leiserowitz A (2015) Improving public engagement with climate change: five best practice insights from psychological science. Perspect Psychol Sci 10(6):758–763

  39. Van der Linden S, Leiserowitz AA, Feinberg GD, Maibach EW (2015) The scientific consensus on climate change as a gateway belief: experimental evidence. PLoS ONE 10:e0118489

  40. Yeager DS, Larson SB, Krosnick JA, Tompson T (2011) Measuring Americans’ issue priorities: a new version of the most important problem question reveals more concern about global warming and the environment. Pub Opin Q 75:125–138

    Article  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Adam Seth Levine.

Additional information

We thank Andrew Gooch, Donald Haider-Markel, Yanna Krupnikov, Brendan Nyhan, our anonymous reviewers, and seminar participants at UCSB, Syracuse University, and Dartmouth University for helpful and engaging feedback. We also thank the Atkinson Center for aSustainable Future at Cornell University for generous funding

Electronic supplementary material

Below is the link to the electronic supplementary material.

(PDF 514 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Levine, A.S., Kline, R. A new approach for evaluating climate change communication. Climatic Change 142, 301–309 (2017). https://doi.org/10.1007/s10584-017-1952-x

Download citation


  • Public Opinion
  • Partner Organization
  • Political Engagement
  • Climate Change Risk
  • Climate Change Communication