Skip to main content

An experimental examination of measurement disparities in public climate change beliefs

Climatic Change volume 154, pages 37–47 (2019)Cite this article

Abstract

The extent to which Americans—especially Republicans—believe in anthropogenic climate change (ACC) has recently been the subject of high profile academic and popular disagreement. We offer a novel framework, and experimental data, for making sense of this debate. Using a large (N = 7,019) and demographically diverse sample of US adults, we compare several widely used methods for measuring belief in ACC. We find that seemingly trivial decisions made when constructing questions can, in some cases, significantly alter the proportion of the American public who appear to believe in human-caused climate change. Critically, we find that some common measurement practices may nearly double estimates of Republicans’ acceptance of human-caused climate change. We conclude by discussing how this work can help improve the consumption of research on climate opinion.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

US$ 39.95

Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2

References

  1. Benjamin D, Por HH, Budescu D (2017) Climate change versus global warming: who is susceptible to the framing of climate change? Environ Behav 49(7):745–770

    Article  Google Scholar 

  2. Brenan M, Saad L (2018) Global warming concern steady despite some partisan shifts. Gallup. Retrieved from: https://news.gallup.com/poll/231530/global-warming-concern-steady-despite-partisan-shifts.aspx

  3. Capstick S, Whitmarsh L, Poortinga W, Pidgeon N, Upham P (2015) International trends in public perceptions of climate change over the past quarter century. Wiley Interdiscip Rev Clim Chang 6(1):35–61

    Article  Google Scholar 

  4. Coppock A, McClellan OA (2019) Validating the demographic, political, psychological, and experimental results obtained from a new source of online survey respondents. https://doi.org/10.1177/2053168018822174

  5. Funk C, Kennedy B (2016) The politics of climate. Pew Research Center. Retrieved from: http://www.pewinternet.org/2016/10/04/the-politics-of-climate/

  6. Greenhill M, Leviston Z, Leonard R, Walker I (2014) Assessing climate change beliefs: response effects of question wording and response alternatives. Public Underst Sci 23(8):947–965

    Article  Google Scholar 

  7. Groothuis PA, Whitehead JC (2002) Does don’t know mean no? Analysis of ‘don’t know’ responses in dichotomous choice contingent valuation questions. Appl Econ 34(15):1935–1940

    Article  Google Scholar 

  8. Heerwegh D (2009) Mode differences between face-to-face and web surveys: an experimental investigation of data quality and social desirability effects. International Journal of Public Opinion Research 21(1):111–121

    Article  Google Scholar 

  9. Kahan DM (2015) Climate-science communication and the measurement problem. Polit Psychol 36:1–43

    Article  Google Scholar 

  10. Kahan D (2017) The “Gateway Belief” illusion: reanalyzing the results of a scientific-consensus messaging study. JCOM: J Sci Commun 16:1b

    Article  Google Scholar 

  11. Kahan DM, Peters E, Wittlin M, Slovic P, Ouellette LL, Braman D, Mandel G (2012) The polarizing impact of science literacy and numeracy on perceived climate change risks. Nat Clim Chang 2(10):732

    Article  Google Scholar 

  12. Krosnick JA (1991) Response strategies for coping with the cognitive demands of attitude measures in surveys. Appl Cogn Psychol 5(3):213–236

    Article  Google Scholar 

  13. Krosnick JA (2018) Questionnaire design. In: The Palgrave handbook of survey research. Palgrave Macmillan, pp 439–455

  14. Krosnick JA, Berent MK (1993) Comparisons of party identification and policy preferences: The impact of survey question format. Am J Polit Sci, 941–964

  15. Krosnick JA, Malhotra N, Mittal U (2014) Public misunderstanding of political facts: how question wording affected estimates of partisan differences in birtherism. Public Opin Q 78(1):147–165

    Article  Google Scholar 

  16. Kunda Z (1990) The case for motivated reasoning. Psychol Bull 108(3):480

    Article  Google Scholar 

  17. Kwak N, Radler B (2002) A comparison between mail and web surveys: response pattern, respondent profile, and data quality. J Off Stat 18(2):257

    Google Scholar 

  18. Leiserowitz A, Maibach E, Roser-Renouf C, Rosenthal S, Cutler M, Kotcher J (2018) Politics & global warming, March 2018. Yale University and George Mason University. New Haven, CT: Yale program on climate change communication

  19. Maitland A, Tourangeau R, Sun H (2018) Separating science knowledge from religious belief two approaches for reducing the effect of identity on survey responses. Public opinion quarterly, nfx049

  20. McAdam D (2017) Social movement theory and the prospects for climate change activism in the United States. Annu Rev Polit Sci 20:189–208

    Article  Google Scholar 

  21. McCright AM, Dunlap RE (2011) The politicization of climate change and polarization in the American public’s views of global warming, 2001–2010. Sociol Q 52 (2):155–194

    Article  Google Scholar 

  22. Mutz DC (2011) Population-based survey experiments. Princeton University Press, Princeton

    Book  Google Scholar 

  23. Nadler JT, Weston R, Voyles EC (2015) Stuck in the middle: the use and interpretation of mid-points in items on questionnaires. J Gen Psychol 142(2):71–89

    Article  Google Scholar 

  24. Pasek J (2017) It’s not my consensus: motivated reasoning and the sources of scientific illiteracy. Public Understanding of Science 0963662517733681

  25. Polk E (2018) Communicating climate change where did we go wrong, how can we do better?. Handbook of communication for development and social change, pp 1–19

  26. Schuldt JP, Roh S, Schwarz N (2015) Questionnaire design effects in climate change surveys: Implications for the partisan divide. The ANNALS of the American Academy of Political and Social Science 658(1):67–85

    Article  Google Scholar 

  27. Schuldt JP, Enns PK, Cavaliere V (2017) Does the label really matter? Evidence that the US public continues to doubt “global warming” more than “climate change”. Clim Chang 143(1–2):271–280

    Article  Google Scholar 

  28. Schuman H, Presser S (1981) Questions and answers in attitude surveys: experiments on question form, wording, and context. Sage, Newbury Park

    Google Scholar 

  29. van Boven L, Sherman D (2018) Actually, republicans do believe in climate change. New York Times

  30. Van Boven L, Ehret PJ, Sherman DK (2018) Psychological barriers to bipartisan public support for climate policy. Perspect Psychol Sci 13(4):492–507

    Article  Google Scholar 

  31. van der Linden SL, Leiserowitz AA, Feinberg GD, Maibach E (2015) The scientific consensus on climate change as a gateway belief: experimental evidence. PLoS ONE, p 10

  32. Villar A, Krosnick JA (2011) Global warming vs. climate change, taxes vs. prices: does word choice matter? Clim Change 105(1):11–12

    Google Scholar 

  33. Whitmarsh L, Capstick S (2018) Perceptions of climate change. In: Psychology and climate change, pp 13–33

Download references

Author information

Affiliations

  1. Annenberg Public Policy Center, University of Pennsylvania, Philadelphia, PA, 19104, USA

    Matthew Motta, Daniel Chapman, Dominik Stecula & Kathryn Haglin

  2. Yale Law School, New Haven, CT, 06511, USA

    Matthew Motta, Daniel Chapman & Kathryn Haglin

  3. Simon Fraser University, Burnaby, BC, V5A 1S6, Canada

    Dominik Stecula

Authors
  1. Matthew Motta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel Chapman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dominik Stecula
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kathryn Haglin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matthew Motta.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(PDF 145 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Motta, M., Chapman, D., Stecula, D. et al. An experimental examination of measurement disparities in public climate change beliefs. Climatic Change 154, 37–47 (2019). https://doi.org/10.1007/s10584-019-02406-9

Download citation