A new, valid measure of climate change understanding: associations with risk perception

Abstract

The relationship between climate change understanding and other variables, including risk perception, beliefs, and worldviews, is an important consideration as we work to increase public attention to climate change. Despite significant effort to develop rigorous mechanisms for measuring affective variables, measurement of climate change understanding is often relegated to unvalidated questions or question sets. To remedy this situation, we constructed and analyzed a climate change concept inventory using a suite of validity and reliability steps, including Rasch analysis. The resultant 21-item test has a high degree of validity and reliability for measuring understanding about basic climate change processes. Inventory scores along with other variables were included in a model of climate change risk perception, providing both concurrent validity for the test and new insight into the importance of understanding, worldview, and values on risk perception. We find that environmental beliefs and cultural cognition worldview play a larger role in predicting an individual’s risk perception than knowledge. Implications for addressing climate change are considered.

This is a preview of subscription content, log in to check access.

Fig. 1

References

  1. AAAS (2012) American Association for the Advancement of Science Project 2061 Science Assessment. Weather and climate topics. http://assessment.aaas.org/pages/home Accessed 19 September 2014

  2. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (2014) AERA, APA, & NCME. Standards for Educational and Psychological Testing. AERA, Washington, DC

    Google Scholar 

  3. Andersson B, Wallin A (2000) Students’ understanding of the greenhouse effect, societal consequences of reducing CO2 emissions and why ozone layer depletion is a problem. J Res Sci Teach 37:1096–1111

    Article  Google Scholar 

  4. Aksit O, McNeal KS, Gold AU, Libarkin JC, Harris S (2018) The influence of instruction, prior knowledge, and values on climate change risk perception among undergraduates. J Res Sci Teach 55(4):550–572. https://doi.org/10.1002/tea.21430

    Article  Google Scholar 

  5. Bond T, Fox C (2007) Applying the Rasch model: fundamental measurement in the human sciences (2nd). Lawrence Erlbaum, Mahwah, New Jersey

  6. Bostrom A, Morgan MG, Fischhoff B, Read D (1994) What do people know about global climate change? 1. Mental models. Risk Anal 14:959–970

    Article  Google Scholar 

  7. Boyes E, Stanisstreet M (1992) Students’ perceptions of global warming. Int J Environ Stud 42:287–300

    Article  Google Scholar 

  8. Boyes E, Stanisstreet M (1993) The ‘greenhouse effect’: children’s perceptions of causes, consequences and cures. Int J Sci Educ 15:531–552

    Article  Google Scholar 

  9. Boyes E, Stanisstreet M (1994) The ideas of secondary school children concerning ozone layer damage. Glob Environ Chang 4:311–324

    Article  Google Scholar 

  10. Boyes E, Stanisstreet M (1997) Children’s models of understanding of two major global environmental issues (ozone layer and greenhouse effect). Res Sci Technol Educ 15:19–28

    Article  Google Scholar 

  11. Center for Research on Environmental Decisions (CRED) (2009) The psychology of climate change communication: a guide for scientists, journalists, educators, political aides, and the interested public. Columbia University, New York

    Google Scholar 

  12. Downing SM (2002) Threats to the validity of locally developed multiple-choice tests in medical education: construct-irrelevant variance and construct underrepresentation. Adv Health Sci Educ 7:235–241

    Article  Google Scholar 

  13. Embretson SE (1996) The new rules of measurement. Psychol Assess 8:341–349

    Article  Google Scholar 

  14. Embretson SE, Reise S (2000) Item response theory for psychologists. Erlbaum Publishers, Mahwah

    Google Scholar 

  15. Fisher WP Jr (1994) The Rasch debate: validity and revolution in educational measurement. In: Wilson M (ed) Objective measurement: theory into practice, vol 2. Ablex Publishing Corporation, Norwood, pp 36–72

    Google Scholar 

  16. Flora J, Saphir M, Lappe M, Roser-Renouf C, Maibach E, Leiserowitz A (2014) Evaluation of a national high school education program: the Alliance for Climate Education. Clim Chang 127:419–434

    Article  Google Scholar 

  17. Frey BB, Petersen SE, Edwards LM, Pedrotti JT, Peyton V (2005) Item-writing rules: collective wisdom. Teach Teach Educ 21:357–364

    Article  Google Scholar 

  18. Gautier C, Deutsch K, Rebich S (2006) Misconceptions about the greenhouse effect. J Geo Educ 54:386–395

    Article  Google Scholar 

  19. Haladyna TM, Downing SM (1989) A taxonomy of multiple-choice item-writing rules. Appl Meas Educ 2:37–50

    Article  Google Scholar 

  20. Haladyna TM, Downing SM, Rodriguez MC (2002) A review of multiple-choice item-writing guidelines for classroom assessment. Appl Meas Educ 15:309–333

    Article  Google Scholar 

  21. Harris SE, Gold AU (2017) Learning molecular behaviour may improve student explanatory models of the greenhouse effect. Environ Educ Res. https://doi.org/10.1080/13504622.2017.1280448

    Article  Google Scholar 

  22. Kahan DM, Braman D, Gastil J, Slovic P (2007) Culture and identity-protective cognition: explaining the white-male effect in risk perception. J Empir Leg Stud 4:465–505

    Article  Google Scholar 

  23. Kahan DM, Braman D (2008) The self-defensive cognition of self-defense. Am Crim Law Rev 45:1–65

    Google Scholar 

  24. Kahan DM, Peters E, Wittlin M, Slovic P, Ouellette LL, Braman D, Mandel G (2012) The polarizing impact of science literacy and numeracy on perceived climate change risks. Nat Clim Chang 2(10):732–735

    Article  Google Scholar 

  25. Kane MT (1992) An argument-based approach to validity. Psychol Bull 112:527

    Article  Google Scholar 

  26. Keller JM (2006) Development of a concept inventory addressing students’ beliefs and reasoning difficulties regarding the greenhouse effect (part I). Dissertation, Department of Planetary Sciences, University of Arizona. https://astronomy101.jpl.nasa.gov/files/Keller_Dissertation2006.pdf

  27. Kellstedt PM, Zahran S, Vedlitz A (2008) Personal efficacy, the information environment, and attitudes toward global warming and climate change in the United States. Risk Anal 28:113–126

    Article  Google Scholar 

  28. Lambert JL, Lindgren J, Bleicher R (2012) Assessing elementary science methods students’ understanding about global climate change. Int J Sci Educ 34:1167–1187

    Article  Google Scholar 

  29. Lee TM, Markowitz EM, Howe PD, Ko C-Y, Leiserowitz A (2015) Predictors of public climate change awareness and risk perception around the world. Nat Clim Chang 5:1014–1020

    Article  Google Scholar 

  30. Leiserowitz A, Maibach E, Roser-Renouf C (2009) Global warming’s six Americas: an audience segmentation analysis. Yale University, New Haven http://environment.yale.edu/climate-communication/files/SixAmericas2009.pdf

    Google Scholar 

  31. Libarkin JC, Anderson SW (2005) Assessment of learning in entry-level geoscience courses: results from the geoscience concept inventory. J Geo Educ 53:394–401

    Article  Google Scholar 

  32. Libarkin JC, Anderson SW (2006) The geoscience concept inventory: application of Rasch analysis to concept inventory development in higher education. In: Liu X, Boone W (eds) Applications of Rasch measurement in science education Jam Press, Maple Grove, p 45–73

  33. Libarkin JC, Thomas SR, Ording G (2015) Factor analysis of drawings: application to college student models of the greenhouse effect. Int J Sci Educ 37:2214–2236

    Article  Google Scholar 

  34. Linacre, J. M. (2016) Winsteps® Rasch measurement computer program. Winsteps.com, Beaverton

  35. Lombardi D, Sinatra GM (2012) College students’ perceptions about the plausibility of human-induced climate change. Res Sci Educ 42:201–217

    Article  Google Scholar 

  36. Lombardi D, Sinatra GM (2013) Emotions about teaching about human-induced climate change. Int J Sci Educ 35:167–191

    Article  Google Scholar 

  37. Lombardi D, Sinatra GM, Nussbaum EM (2013) Plausibility reappraisals and shifts in middle school students’ climate change conceptions. Learn Instr 27:50–62

    Article  Google Scholar 

  38. Lombardi D, Seyranian V, Sinatra GM (2014) Source effects and plausibility judgments when reading about climate change. Disc Proc 51:75–92

    Article  Google Scholar 

  39. Lord FM (1980) Applications of item response to theory to practical testing problems. Lawrence Erlbaum, Mahwah, New Jersey

  40. Lord FM, Novick MR (1968) Statistical theories of mental test scores. Addison-Welsley Publishing Company, Reading

    Google Scholar 

  41. Malka A, Krosnick JA, Langer G (2009) The association of knowledge with concern about global warming: trusted information sources shape public thinking. Risk Anal 29(5):633–647

    Article  Google Scholar 

  42. Markus KA, Borsboom D (2013) Frontiers of test validity theory: measurement, causation, and meaning. Routledge, New York

    Google Scholar 

  43. McCright AM, Dunlap RE (2011) The politicization of climate change and polarization in the American public’s views of global warming, 2001–2010. Sociol Q 52(2):155–194

    Article  Google Scholar 

  44. McNeal KS, Libarkin JC, Ledley TS, Bardar E, Haddad N, Ellins K, Dutta S (2014) The role of research in online curriculum development: The case of EarthLabs Climate Change and Earth System Modules. J Geosci Educ 62(4):560–577

    Article  Google Scholar 

  45. Morrow CA, Monsaas J, Katzenberger J, Afolabi CY (2013) Introducing a new concept inventory on climate change to support undergraduate instruction, teacher education, education research, and project evaluation. AGU Fall Meeting Abstracts, ED32A-03

  46. NGSS Lead States (2013) Next generation science standards: for states, by states. The National Academies Press, Washington, DC

    Google Scholar 

  47. Paolacci G, Chandler J, Ipeirotis PG (2010) Running experiments on amazon mechanical turk. Judgm Decis Mak 5:411–419

    Google Scholar 

  48. Pew Research Center (2015a) Americans, politics and science issues. http://www.pewinternet.org/files/2015/07/2015-07-01_science-and-politics_FINAL.pdf

  49. Pew Research Center (2015b) Global concern about climate change, broad support for limiting emissions. http://www.pewglobal.org/files/2015/11/Pew-Research-Center-Climate-Change-Report-FINAL-November-5-2015.pdf

  50. Rutherford DJ, Weber ET (2011) Ethics and environmental policy. In: Chen W-Y, Seiner J, Suzuki T, Lackner M (eds) Handbook of climate change mitigation. Springer Science Business Media, New York

    Google Scholar 

  51. Shepardson DP, Niyogi D, Choi S, Charusombat U (2011) Students’ conceptions about the greenhouse effect, global warming, and climate change. Clim Chang 104:481–507

    Article  Google Scholar 

  52. Sinatra GM, Danielson RW (2014) Adapting to a warmer climate of scientific communication. Biosci 64(4):275–276

    Article  Google Scholar 

  53. Steg L, De Groot JI, Dreijerink L, Abrahamse W, Siero F (2011) General antecedents of personal norms, policy acceptability, and intentions: the role of values, worldviews, and environmental concern. Soc Nat Resour 24(4):349–367

    Article  Google Scholar 

  54. Stevenson KT, Peterson MN, Bondell HD, Moore SE, Carrier SJ (2014) Overcoming skepticism with education: interacting influences of worldview and climate change knowledge on perceived climate change risk among adolescents. Clim Chang 126(3–4):293–304

    Article  Google Scholar 

  55. Summerville CJ, Hassol SJ (2011) Communicating the science of climate change. Phys Today 64:48–53

    Article  Google Scholar 

  56. Trenbath T-KL (2012) Undergraduate students’ conceptions of natural and anthropogenic climate change: a case study approach. Dissertation, Department of Atmospheric and Oceanic Sciences, University of Colorado Boulder. http://gradworks.umi.com/35/08/3508048.html

  57. USGCRP (U.S. Global Climate Research Program) (2009) Climate literacy: the essential principles of climate science. A guide for individuals and communities. Download: https://downloads.globalchange.gov/Literacy/climate_literacy_lowres_english.pdf

  58. Walker SL, McNeal KS (2013) Development and validation of an instrument for assessing climate change knowledge and perceptions: the Climate Stewardship Survey (CSS). IEJEE 3(1):57–73

    Google Scholar 

  59. Wright BD, Linacre JM (1994) Reasonable mean-square fit values. Rasch Meas Trans 8:370

    Google Scholar 

Download references

Acknowledgements

We are grateful to everyone who participated in this research, as well as members of the Geocognition Research Lab for a review of this manuscript.

Funding

This work was partially supported by the National Science Foundation under grant no. DUE-1504659 to Libarkin and Bowles.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Julie C. Libarkin.

Electronic supplementary material

ESM 1

(DOCX 451 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Libarkin, J.C., Gold, A.U., Harris, S.E. et al. A new, valid measure of climate change understanding: associations with risk perception. Climatic Change 150, 403–416 (2018). https://doi.org/10.1007/s10584-018-2279-y

Download citation