Climatic Change

, Volume 126, Issue 3–4, pp 485–493 | Cite as

A method to evaluate the usability of interactive climate change impact decision aids

  • Gabrielle Wong-ParodiEmail author
  • Baruch Fischhoff
  • Ben Strauss


Reducing the impacts from climate change requires people to make decisions that may prompt substantial changes in their lives. One possible way to help them is with personalized decision aids. Here we describe a method for evaluating such aids, in terms of how they affect users’ understanding of their situation, defined in terms of their (a) knowledge, (b) consistency of preferences, and (c) active mastery of the material. Our method provides a simple way to evaluate the usability of climate-change decision aids, and to address concerns that the choice of display could bias users’ attitudes. We demonstrate it with the Surging Seas Risk Finder, a decision aid focused on coastal flooding (


Down Syndrome Geographic Information System Coastal Flooding King County Land Impact 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was supported by a research grant from the Rockefeller Foundation and the Center for Climate and Energy Decision Making (SES-0949710). The authors thank Jack Wang and Dan Rizza for their support with the experiment.

Author attributions

All authors contributed to the conception and the design of the tests and the experiment used within the work presented. G.W.P. ran the experiment and analyzed the data. G.W.P. and B.F. led in the writing of the paper, with input from B.S.

Competing financial interests

The authors declare no competing financial interests.


  1. Bekker HL, Hewison J, Thornton JG (2004) Applying decision analysis to facilitate informed decision making about prenatal diagnosis for Down syndrome: a randomised controlled trial. Prenat Diagn 24(4):265–275CrossRefGoogle Scholar
  2. Bottrill, C (2007) Internet-based carbon tools for behaviour change. University of Oxford Environmental Change Institute, Oxford (available online at
  3. Braddock CH III, Edwards KA, Hasenberg NM, Laidley TL, Levinson W (1999) Informed decision making in outpatient practice. JAMA 282(24):2313–2320CrossRefGoogle Scholar
  4. Chatterton TJ, Coulter A, Musselwhite C, Lyons G, Clegg S (2009) Understanding how transport choices are affected by the environment and health: views expressed in a study on the use of carbon calculators. Public Health 123(1):e45–e49CrossRefGoogle Scholar
  5. Crump MJ, McDonnell JV, Gureckis TM (2013) Evaluating Amazon’s Mechanical Turk as a tool for experimental behavioral research. PLoS One 8(3):e57410CrossRefGoogle Scholar
  6. de Best-Waldhober M, Daamen D, Faaij A (2009) Informed and uninformed public opinions on CO2 capture and storage technologies in the Netherlands. Int J Greenh Gas Control 3(3):322–332CrossRefGoogle Scholar
  7. Dietz T, Gardner GT, Gilligan J, Stern PC, Vandenbergh MP (2009) Household actions can provide a behavioral wedge to rapidly reduce US carbon emissions. PNAS 106(44):18452–18456CrossRefGoogle Scholar
  8. Fischhoff B (2013) The sciences of science communication. PNAS 110(Supplement 3):14033–14039CrossRefGoogle Scholar
  9. Fischhoff, B, Davis, AL (2014) Communicating scientific uncertainty. PNASGoogle Scholar
  10. Fischhoff B, Brewer N, Downs JS (eds) (2011) Communicating risks and benefits: an evidence-based users’ guide. Food and Drug Administration, WashingtonGoogle Scholar
  11. Füssel HM (2007) Adaptation planning for climate change: concepts, assessment approaches, and key lessons. Sustain Sci 2(2):265–275CrossRefGoogle Scholar
  12. Kahneman D (2011) Thinking, fast and slow. Farrar Giroux & Strauss, New YorkGoogle Scholar
  13. Lichtenstein SL, Slovic P (eds) (2006) The construction of preferences. Cambridge University Press, New YorkGoogle Scholar
  14. Lusardi, A (2008) Financial Literacy: An Essential Tool for Informed Consumer Choice? (No. w14084). National Bureau of Economic ResearchGoogle Scholar
  15. Maglio SJ, Trope Y, Liberman N (2013) The common currency of psychological distance. Psychol Sci 22(4):278–282Google Scholar
  16. Mason W, Suri S (2012) Conducting behavioral research on Amazon’s Mechanical Turk. Behav Res Methods 44(1):1–23CrossRefGoogle Scholar
  17. Matthies M, Giupponi C, Ostendorf B (2007) Environmental decision support systems: Current issues, methods and tools. Environ Model Softw 22(2):123–127CrossRefGoogle Scholar
  18. Meinke H, Stone RC (2005) Seasonal and inter-annual climate forecasting: the new tool for increasing preparedness to climate variability and change in agricultural planning and operations. Clim Chang 70(1–2):221–253CrossRefGoogle Scholar
  19. Nicholson-Cole SA (2005) Representing climate change futures: a critique on the use of images for visual communication. Comput Environ Urban Syst 29(3):255–273CrossRefGoogle Scholar
  20. Noar SM, Benac CN, Harris MS (2007) Does tailoring matter? Meta-analytic review of tailored print health behavior change interventions. Psychol Bull 133(4):673–693CrossRefGoogle Scholar
  21. O’Connor AM, Tugwell P, Wells GA, Elmslie T, Jolly E, Hollingworth G, McPherson R, Bunn H, Graham I, Drake E (1998) A decision aid for women considering hormone therapy after menopause: decision support framework and evaluation. Patient Educ Couns 33(3):267–279CrossRefGoogle Scholar
  22. Olson GM, Olson JS (2003) Human-computer interaction: psychological aspects of the human use of computing. Ann Rev Psychol 54(1):491–516CrossRefGoogle Scholar
  23. Padgett JP, Steinemann AC, Clarke JH, Vandenbergh MPA (2008) Comparison of carbon calculators. Environ Impact Assess Rev 28(2):106–115CrossRefGoogle Scholar
  24. Salvendy G (2012) Handbook of human factors and ergonomics. Wiley, IndianaCrossRefGoogle Scholar
  25. Simao A, Densham PJ, Haklay M (2009) Web-based GIS for collaborative planning and public participation: an application to the strategic planning of wind farm sites. J Environ Manag 90(6):2027–2040CrossRefGoogle Scholar
  26. Spence A, Poortinga W, Pidgeon N (2012) The psychological distance of climate change. Risk Anal 32(6):957–972CrossRefGoogle Scholar
  27. Strauss BH, Ziemlinski R, Weiss JL, Overpeck JT (2012) Tidally adjusted estimates of topographic vulnerability to sea level rise and flooding for the contiguous United States. Environ Res Lett 7(1):014033CrossRefGoogle Scholar
  28. Szwajcer EM, Hiddink GJ, Koelen MA, van Woerkum CM (2009) Written nutrition communication in midwifery practice: what purpose does it serve? Midwifery 25(5):509–517CrossRefGoogle Scholar
  29. Tebaldi C, Strauss BH, Zervas CE (2012) Modelling sea level rise impacts on storm surges along US coasts. Environ Res Lett 7(1):014032CrossRefGoogle Scholar
  30. Ver Steegh N (2002) Yes, no, and maybe: Informed decision making about divorce mediation in the presence of domestic violence. Wm Mary J Women L 9:145Google Scholar
  31. Volk RJ, Spann SJ, Cass AR, Hawley ST (2003) Patient education for informed decision making about prostate cancer screening: a randomized controlled trial with 1-year follow-up. Ann Fam Med 1(1):22–28CrossRefGoogle Scholar
  32. von Winterfeldt D (2013) Bridging the gap between science and decision making. PNAS 110(Supplement 3):14055–14061CrossRefGoogle Scholar
  33. Wong-Parodi, G, Strauss, B (2014) Team science for science communication, PNAS (in press)Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2014

Authors and Affiliations

  • Gabrielle Wong-Parodi
    • 1
    Email author
  • Baruch Fischhoff
    • 2
  • Ben Strauss
    • 3
  1. 1.Department of Engineering and Public PolicyCarnegie Mellon UniversityPittsburghUSA
  2. 2.Departments of Engineering and Public Policy and Social and Decision SciencesCarnegie Mellon UniversityPittsburghUSA
  3. 3.Climate Central, One Palmer SquarePrincetonUSA

Personalised recommendations