Skip to main content

Advertisement

SpringerLink
Log in

Diagnosing errors in climate model intercomparisons

  • Paper in History and Philosophy of Science
  • Published:
European Journal for Philosophy of Science Aims and scope Submit manuscript

Abstract

I examine error diagnosis (model-model disagreement) in climate model intercomparisons including its difficulties, fruitful examples, and prospects for streamlining error diagnosis. I suggest that features of climate model intercomparisons pose a more significant challenge for error diagnosis than do features of individual model construction and complexity. Such features of intercomparisons include, e.g., the number of models involved, how models from different institutions interrelate, and what scientists know about each model. By considering numerous examples in the climate modeling literature, I distill general strategies (e.g., employing physical reasoning and using dimension reduction techniques) used to diagnose model error. Based on these examples, I argue that an error repertoire could be beneficial for improving error diagnosis in climate modeling, although constructing one faces several difficulties. Finally, I suggest that the practice of error diagnosis demonstrates that scientists have a tacit-yet-working understanding of their models which has been under-appreciated by some philosophers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. The examples (and my emphasis in this paper) are focused on multi-model disagreement. For work centered on model-observation discrepancies, including examples of models being used to correct errors in observational and other data, see Lloyd (2012), Abraham et al. (2013), Mann (2018), Weart (2020), and Li (2022).

  2. By “tacit” I have in mind a sort of practice-based knowledge which scientists could perhaps explain to others if pressed but which they typically do not explain to others. Thanks to Matthew Mayernik for prompting me to clarify my use of this term and for pointing me to the work of Schmidt (2012) who discusses how, in many scientific and academic contexts, “tacit” is a “conceptual muddle that mystifies the very concept of practical knowledge” (163).

  3. Lenhard and Winsberg seem to use “sub-model” and “module” interchangeably. In contrast, I adopt climate scientists’ typical usage of these terms, except when directly quoting Lenhard and Winsberg. Effectively this means that sub-models are parameterizations or sub-parameterizations, and the term “modules” is (usually, but not always) reserved for larger pieces of a GCM such as the atmosphere module or ocean module.

  4. Compare with Morrison (2021). Lenhard and Winsberg’s description of model development appears reasonable but may not be accurate to practice.

  5. But see Morrison (2021) for a practice-informed study of how climate modelers prioritize, research, and implement updates to their model over the course of development. Also, large-scale rewrites of GCM code are sometimes done in practice, contrary to Lenhard and Winsberg’s description of climate model development (e.g., see Neale et al., 2012).

  6. Here “attribution” refers to attributing the sources of success and failure in climate models to sub-components of those models. This should not be confused with detection and attribution work in climate science.

  7. Lenhard and Winsberg’s account also implies that scientists cannot attribute sources of model success, however, that is the topic for another paper.

  8. Thank you to an anonymous reviewer for prompting me both to think through these issues more carefully and to explicitly highlight this inconsistency.

  9. This episode has a fairly broad audience, as it was written up at the Wall Street Journal (Hotz, 2022). Additionally, Castillo Brache (2022) uses this example to critique Lenhard and Winsberg’s (2010) account.

  10. Thanks to an anonymous reviewer for prompting me to think more critically about this.

  11. They also do not offer any detailed positive examples of attributing sources of model success.

  12. For further historical reading, see Gates 1979; Arakawa 2000; Washington 2006; Edwards 2010, 2011; Randall et al., 2018; Weart 2020.

  13. For more on climate model hierarchies, see Held (2005) and Jeevanjee et al. (2017).

  14. These two GCMs were configured in a total of five different ways (e.g., varying in terms of how snow and ice were represented, whether a deep ocean was used, and whether seasonal change was represented) to make five distinct projections.

  15. These exploratory activities fall under what Wilson (2021) refers to as “Model dynamic exploration.”

  16. For examples of candid discussions of model tuning by climate scientists, see Mauritsen et al., 2012; Schmidt and Sherwood (2015), Schmidt et al. (2017), Hourdin et al. (2017).

  17. See Steel and Werndl (2013), Frisch (2015), and Schmidt and Sherwood (2015) for a philosophical discussion.

  18. The analysis in Cess et al. (1989) serves as a sort of midpoint between the uncoordinated model intercomparison and the coordinated ones. This intercomparison included some closely related models (i.e., from the same institutions) as well as more distinct models and analyses of the former were more fine-grained than those of the latter (e.g., see their discussion of GFDL I and II on their page 515). Moreover, many of the scientists involved helped develop the models being analyzed.

  19. A list of publications from these diagnostic subprojects can be found here: https://pcmdi.llnl.gov/mips/amip/abstracts/abhme.html

  20. Touzé-Peiffer et al. (2020) also give examples of successful model error diagnosis, saying “In fact, in the literature, we can find many studies investigating the link between the results of a model and its parameterizations (e.g., Hourdin et al., 2013; Notz et al., 2013).” They also mention “studies comparing radiation codes in different climate models, such as Oreopoulos et al. (2012) and Pincus et al. (2015), where the authors analyze not only the model results, but also the corresponding parameterizations and the assumptions they make” (9).

  21. Recall: in AMIP, sea surface temperatures were prescribed. But these scientists still wanted to know what this heat transport would look like because future applications of these models would include coupling them to ocean models.

  22. For philosophical discussion of dynamical sufficiency (which concerns the representation of how a system changes over time) in modeling, see Lloyd et al. (2008) and Kawamleh (2022).

  23. E.g., see Sun et al. (2006), Birch et al. (2015).

  24. Examples of early work on clouds in relation to the Earth’s radiation budget include theoretical work (e.g., Schneider, 1972) and observational work (e.g., Hartmann & Short, 1980).

  25. See Kuo et al. (2020) for a recent statistical analysis of models which differed in their deep convective parameterizations. So-called “process-level” analyses which use statistical methods as well as physical arguments also becoming more common (e.g., see Maloney et al., 2019).

  26. Indeed, the practice of tinkering with a single model over the course of model development and iteratively making changes may also involve error diagnosis (e.g., see Hansen et al., 1983; Danabasoglu et al., 2020; Mayernik, 2021), although such a strategy may only work for single-model evaluations.

  27. A "crucial test” would be superior, i.e., a test which distinguishes between the primary suspected error source in question and the other suspected error sources.

  28. Thanks to Ben Kravitz for inspiring this suggestion.

  29. Thanks to an anonymous reviewer for emphasizing this point.

  30. A big challenge concerns resource availability. When presenting some of these ideas at [omitted for review], a climate modeler asked whether error diagnosis efforts should be focused on errors that have clear solutions vs. errors that are significant but difficult to understand or fix. Even if it is agreed that an error repertoire would be valuable, this doesn’t mean that the resources are available to construct or implement one.

  31. This emphasis on process representations in climate models has also inspired some philosophical accounts, e.g., Lloyd et al. (2021), Kawamleh (2022).

  32. My suggestion here is influenced by Lloyd’s logic of research questions (Lloyd, 2015b) as well as van Fraassen’s pragmatic theory of explanation (van Fraassen, 1980).

  33. Quotation marks pick out quotes from Goodwin (2015), pp. 342–343.

References

  • Abraham, J. P., Baringer, M., Bindoff, N. L., Boyer, T., Cheng, L. J., Church, J. A., Conroy, J. L., Domingues, C. M., Fasullo, J. T., Gilson, J., Goni, G., Good, S. A., Gorman, J. M., Gouretski, V., Ishii, M., Johnson, G. C., Kizu, S., Lyman, J. M., Macdonald, A. M., . . . Willis, J. K. (2013). A review of global ocean temperature observations: Implications for ocean heat content estimates and climate change. Reviews of Geophysics, 51(3), 450–483. https://doi.org/10.1002/rog.20022

  • Annan, J. D., & Hargreaves, J. C. (2010). Reliability of the CMIP3 ensemble. Geophysical Research Letters, 37(2). https://doi.org/10.1029/2009GL041994

  • Annan, J. D., & Hargreaves, J. C. (2017). On the meaning of independence in climate science. Earth System Dynamics, 8(1), 211–224. https://doi.org/10.5194/esd-8-211-2017

    Article  Google Scholar 

  • APE. (2008). Website of the AquaPlanet experiment project, visited Feb 4, 2008. http://www-pcmdi.llnl.gov/projects/amip/ape/index.html

  • Arakawa, A. (2000). A personal perspective on the early years of general circulation modeling at UCLA. In D. A. Randall (Ed.), General circulation model development: Past, present, future (pp. 1–65). Academic Press.

    Google Scholar 

  • Baumberger, C., Knutti, R., & Hadorn, G. H. (2017). Building confidence in climate model projections: An analysis of inferences from fit. WIREs Climate Change, 8(3), e454. https://doi.org/10.1002/wcc.454

    Article  Google Scholar 

  • Birch, C. E., Roberts, M. J., Garcia-Carreras, L., Ackerley, D., Reeder, M. J., Lock, A. P., & Schiemann, R. (2015). Sea-breeze dynamics and convection initiation: The influence of convective parameterization in weather and climate model biases. Journal of Climate, 28(20), 8093–8108. https://doi.org/10.1175/JCLI-D-14-00850.1

    Article  Google Scholar 

  • Bogenschutz, P. A., Gettelman, A., Morrison, H., Larson, V. E., Craig, C., & Schanen, D. P. (2013). Higher-order turbulence closure and its impact on climate simulations in the community atmosphere model. Journal of Climate, 26(23), 9655–9676. https://doi.org/10.1175/JCLI-D-13-00075.1

    Article  Google Scholar 

  • Bukovsky, M. S., McCrary, R. R., Seth, A., & Mearns, L. O. (2017). A mechanistically credible, poleward shift in warm-season precipitation projected for the U.S. Southern Great Plains? Journal of Climate, 30(20), 8275–8298. https://doi.org/10.1175/JCLI-D-16-0316.1

    Article  Google Scholar 

  • Carrier, M., & Lenhard, J. (2019). Climate models: How to assess their reliability. International Studies in the Philosophy of Science, 32(2), 81–100. https://doi.org/10.1080/02698595.2019.1644722

    Article  Google Scholar 

  • Castillo Brache, LA (2022) Fixing high-ECS models: The problem of holism revisited. In Climate sensitivity, paleoclimate data, & the end of model democracy [Symposium]. PSA 28th Biennial Meeting, Nov. 10–13, Pittsburgh, PA, U.S.

  • Cess, R. D., Potter, G. L., Blanchet, J. P., Boer, G. J., Del Genio, A. D., Déqué, M., Dymnikov, V., Galin, V., Gates, W. L., Ghan, S. J., Kiehl, J. T., Lacis, A. A., Le Treut, H., Li, Z.-X., Liang, X.-Z., McAvaney, B. J.. Meleshko, V. P., Mitchell, J. F. B., Morcrette, J.-J., . . . Zhang, M.-H. (1990). Intercomparison and interpretation of climate feedback processes in 19 atmospheric general circulation models. Journal of Geophysical Research: Atmospheres, 95(D10), 16601–16615. https://doi.org/10.1029/JD095iD10p16601

  • Cess, R. D., Potter, G. L., Blanchet, J. P., Boer, G. J., Ghan, S. J., Kiehl, J. T., Le Treut, H., Li, Z. X., Liang, X. Z., Mitchell, J. F., Morcrette, J. J., Randall, D. A., Riches, M. R., Roeckner, E., Schlese, U., Slingo, A., Taylor, K. E., Washington, W. M., Wetherald, R. T., Yagai, I. (1989). Interpretation of cloud-climate feedback as produced by 14 atmospheric general circulation models. Science, 245(4917), 513–516. https://doi.org/10.1126/science.245.4917.513

  • Clark, A. (1987). The kludge in the machine*. Mind & Language, 2(4), 277–300. https://doi.org/10.1111/j.1468-0017.1987.tb00123.x

    Article  Google Scholar 

  • Council, National Research. (2012). A national strategy for advancing climate modeling. https://doi.org/10.17226/13430

  • Danabasoglu, G., Lamarque, J.-F., Bacmeister, J., Bailey, D. A., DuVivier, A. K., Edwards, J., Emmons, L. K., Fasullo, J., Garcia, R., Gettelman, A., Hannay, C., Holland, M. M., Large, W. G., Lauritzen, P. H., Lawrence, D. M., Lenaerts, J. T. M., Lindsay, K., Lipscomb, W. H., Mills, M. J., . . . Strand, W. G. (2020). The community earth system model version 2 (CESM2). Journal of Advances in Modeling Earth Systems 12(2): e2019MS001916. https://doi.org/10.1029/2019MS001916

  • Dethier, C. (2022). When is an ensemble like a sample? ‘Model-Based’ inferences in climate modeling. Synthese, 200(1), 52. https://doi.org/10.1007/s11229-022-03477-5

    Article  Google Scholar 

  • Edwards, P. (2010). A vast machine: Computer models, climate data, and the politics of global warming. MIT Press.

    Google Scholar 

  • Edwards, P. N. (2011). History of climate modeling. Wires Climate Change, 2(1), 128–139. https://doi.org/10.1002/wcc.95

    Article  Google Scholar 

  • Eyring, V., Righi, M., Lauer, A., Evaldsson, M., Wenzel, S., Jones, C., Anav, A., Andrews, O., Cionni, I., Davin, E. L., Deser, C., Ehbrecht, C., Friedlingstein, P., Gleckler, P., Gottschaldt, K. D., Hagemann, S., Juckes, M., Kindermann, S., Krasting, J., . . .  Williams, K. D. (2016). ESMValTool (v1.0) – a community diagnostic and performance metrics tool for routine evaluation of earth system models in CMIP. Geoscientific Model Development, 9(5), 1747–1802. https://doi.org/10.5194/gmd-9-1747-2016

  • van Fraassen, B. C. (1980). The scientific image. Clarendon.

    Book  Google Scholar 

  • Frigg, R., Thompson, E., & Werndl, C. (2015). Philosophy of climate science part II: Modelling climate change. Philosophy Compass, 10(12), 965–977. https://doi.org/10.1111/phc3.12297

    Article  Google Scholar 

  • Frisch, M. (2015). Predictivism and old evidence: A critical look at climate model tuning. European Journal for Philosophy of Science, 5(2), 171–190. https://doi.org/10.1007/s13194-015-0110-4

    Article  Google Scholar 

  • Gates, W. L. (1992). AN AMS CONTINUING SERIES: GLOBAL CHANGE–AMIP: The atmospheric model intercomparison project. Bulletin of the American Meteorological Society, 73(12), 1962–1970. https://doi.org/10.1175/1520-0477(1992)073%3c1962:ATAMIP%3e2.0.CO;2

    Article  Google Scholar 

  • Gates, W. L., Boyle, J. S., Covey, C., Dease, C. G., Doutriaux, C. M., Drach, R. S., Fiorino, M., Gleckler, P. J., Hnilo, J. J., Marlais, S. M., Phillips, T. J., Potter, G. L., Santer, B. D., Sperber, K. R., Taylor, K. E., & Williams, D. N. (1999). An overview of the results of the atmospheric model intercomparison project (AMIP I). Bulletin of the American Meteorological Society, 80(1), 29–56. https://doi.org/10.1175/1520-0477(1999)080%3c0029:AOOTRO%3e2.0.CO;2

  • Gates, W. L. (1979). Report of the JOC Study Conference on Climate Models, Performance, Intercomparison, and Sensitivity Studies (Washington, DC, 3–7 April 1978). 22. International Council of Scientific Unions; [Geneva: obtained from the World Meteorological Organization]

  • Gettelman, A., Hannay, C., Bacmeister, J. T., Neale, R. B., Pendergrass, A. G., Danabasoglu, G., Lamarque, J.-F., Fasullo, J. T., Bailey, D. A., Lawrence, D. M., & Mills, M. J. (2019). High climate sensitivity in the community earth system model version 2 (CESM2). Geophysical Research Letters, 46(14), 8329–8337. https://doi.org/10.1029/2019GL083978

  • Gettelman, A., & Morrison, H. (2015). Advanced two-moment bulk microphysics for global models. Part I: Off-line tests and comparison with other schemes. Journal of Climate, 28(3), 1268–1287. https://doi.org/10.1175/JCLI-D-14-00102.1

    Article  Google Scholar 

  • Gettelman, A., Morrison, H., Santos, S., Bogenschutz, P., & Caldwell, P. M. (2015). Advanced two-moment bulk microphysics for global models. Part II: Global model solutions and aerosol-cloud interactions. Journal of Climate, 28(3), 1288–1307. https://doi.org/10.1175/JCLI-D-14-00103.1

    Article  Google Scholar 

  • Gettelman, A., & Rood, R. B. (2016). Demystifying climate models: A users guide to earth system models. In: A. Gettelman, & R. B. Rood (Eds.), Earth systems data and models. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-48959-8_1

  • Gleckler, P. J., Randall, D. A., Boer, G., Colman, R., Dix, M., Galin, V., Helfand, M., Kiehl, J. T., Kitoh, A., Lau, W. K. M., Liang, X.-Y., Lykossov, V., McAvaney, B. J., Miyakoda, K., Planton, S., & Stern, W. (1995). Cloud-radiative effects on implied oceanic energy transports as simulated by atmospheric general circulation models. Geophysical Research Letters, 22(7), 791–794. https://doi.org/10.1029/95GL00113

  • Goodwin, W. M. (2015). Global climate modeling as applied science. Journal for General Philosophy of Science, 46(2), 339–350. https://doi.org/10.1007/s10838-015-9301-0

  • Hansen, J., Russell, G., Rind, D., Stone, P., Lacis, A., Lebedeff, S., Ruedy, R., & Travis, L. (1983). Efficient three-dimensional global models for climate studies: Models I and II. Monthly Weather Review, 111(4), 609–662. https://doi.org/10.1175/1520-0493(1983)111%3c0609:ETDGMF%3e2.0.CO;2

    Article  Google Scholar 

  • Hartmann, D. L., & Short, D. A. (1980). On the use of earth radiation budget statistics for studies of clouds and climate. Journal of the Atmospheric Sciences, 37(6), 1233–1250. https://doi.org/10.1175/1520-0469(1980)037%3c1233:OTUOER%3e2.0.CO;2

    Article  Google Scholar 

  • Hausfather, Z., Marvel, K., Schmidt, G. A., Nielsen-Gammon, J. W., & Zelinka, M. (2022). Climate simulations: Recognize the ‘Hot Model’ problem. Nature, 605(7908), 26–29. https://doi.org/10.1038/d41586-022-01192-2

    Article  Google Scholar 

  • Hegerl, G. C., Zwiers, F. W., Braconnot, P., Gillett, N. P., Luo, Y., Marengo Orsini, J. A., Nicholls, N., Penner, J. E., Stott, P. A. (2007). Understanding and attributing climate change. In S. Solomon, D. Qin, M. Manning, Z. Chen, M. Marquis, K. B. Avery, M. Tignor, & H. L. Miller (Eds.), Climate change 2007: The physical science basis. Contribution of working group I to the fourth assessment report of the intergovernmental panel on climate change (p. 84). Cambridge University Press.

  • Held, I. M. (2005). The gap between simulation and understanding in climate modeling. Bulletin of the American Meteorological Society, 86(11), 1609–1614. https://doi.org/10.1175/BAMS-86-11-1609

    Article  Google Scholar 

  • Hoesly, R. M., Smith, S. J., Feng, L., Klimont, Z., Janssens-Maenhout, G., Pitkanen, T., Seibert, J. J., Vu, L., Andres, R. J., Bolt, R. M., Bond, T. C., Dawidowski, L., Khold, N., Kurokawa, J., Li, M., Liu, L., Ku, Z., Moura, M. C. P., O'Rourke, P. R., & Zhang, Q. (2018). Historical (1750–2014) anthropogenic emissions of reactive gases and aerosols from the community emissions data system (CEDS). Geoscientific Model Development, 11(1), 369–408. https://doi.org/10.5194/gmd-11-369-2018

  • Hotz, R. (2022). Climate scientists encounter limits of computer models, bedeviling policy. The Wall Street Journal, February 6, 2022.

  • Hourdin, F., Grandpeix, J.-Y., Rio, C., Bony, S., Jam, A., Cheruy, F., Rochetin, N., Fairhead, L., Idelkadi, A., Musat, I., Dufresne, J., Lahellec, A., Lefebvre, M., & Roehrig, R. (2013). LMDZ5B: The atmospheric component of the IPSL climate model with revisited parameterizations for clouds and convection. Climate Dynamics, 40(9), 2193–2222. https://doi.org/10.1007/s00382-012-1343-y

  • Hourdin, F., Mauritsen, T., Gettelman, A., Golaz, J.-C., Balaji, V., Duan, Q., Folini, D., Ji, D., Klocke, D., Qian, Y., Rauser, F., Rio, C., Tomassini, L., Watanabe, M., Williamson, D. (2017). The art and science of climate model tuning. Bulletin of the American Meteorological Society, 98(3), 589–602. https://doi.org/10.1175/BAMS-D-15-00135.1

  • Jebeile, J., & Barberousse, A. (2021). Model spread and progress in climate modelling. European Journal for Philosophy of Science, 11(3), 66. https://doi.org/10.1007/s13194-021-00387-0

    Article  Google Scholar 

  • Jebeile, J., & Crucifix, M. (2020). Multi-model ensembles in climate science: Mathematical structures and expert judgements. Studies in History and Philosophy of Science Part A, 83(October), 44–52. https://doi.org/10.1016/j.shpsa.2020.03.001

    Article  Google Scholar 

  • Jeevanjee, N., Hassanzadeh, P., Hill, S., & Sheshadri, A. (2017). A perspective on climate model hierarchies. Journal of Advances in Modeling Earth Systems, 9(4), 1760–1771. https://doi.org/10.1002/2017MS001038

    Article  Google Scholar 

  • Kawamleh, S. (2022). Confirming (Climate) change: A dynamical account of model evaluation. Synthese, 200(2), 122. https://doi.org/10.1007/s11229-022-03659-1

    Article  Google Scholar 

  • Kravitz, B., & MacMartin, D. G. (2020). Uncertainty and the basis for confidence in solar geoengineering research. Nature Reviews Earth & Environment, 1(1), 64–75. https://doi.org/10.1038/s43017-019-0004-7

    Article  Google Scholar 

  • Kuo, Y.-H., David Neelin, J., Chen, C.-C., Chen, W.-T., Donner, L. J., Gettelman, A., Jiang, X., Maloney, E., Mechoso, C. R., Ming, Y., Schiro, K., Seman, C.J., Wu, C.M., & Zhao, M. (2020). Convective transition statistics over tropical oceans for climate model diagnostics: GCM evaluation. Journal of the Atmospheric Sciences, 77(1), 379–403. https://doi.org/10.1175/JAS-D-19-0132.1

  • Lenhard, J., & Winsberg, E. (2010). Holism, entrenchment, and the future of climate model pluralism. Studies in history and philosophy of science Part B: Studies in history and philosophy of modern physics. Special Issue: Modelling and Simulation in the Atmospheric and Climate Sciences, 41(3), 253–262. https://doi.org/10.1016/j.shpsb.2010.07.001

    Article  Google Scholar 

  • Li, D. (2022). If a tree grows no ring and no one is around: How scientists deal with missing tree rings. Climatic Change, 174(1), 6. https://doi.org/10.1007/s10584-022-03424-w

    Article  Google Scholar 

  • Lloyd, E. A. (2015a). Model robustness as a confirmatory virtue: The case of climate science. Studies in History and Philosophy of Science Part A, 49, 58–68. https://doi.org/10.1016/j.shpsa.2014.12.002

  • Lloyd, E. A. (2015b). Adaptationism and the logic of research questions: How to think clearly about evolutionary causes. Biological Theory, 10(4), 343–362. https://doi.org/10.1007/s13752-015-0214-2

  • Lloyd, E. A. (2012). The role of ‘complex’ empiricism in the debates about satellite data and climate models. Studies in History and Philosophy of Science Part A, 43(2), 390–401. https://doi.org/10.1016/j.shpsa.2012.02.001

    Article  Google Scholar 

  • Lloyd, E. A., Bukovsky, M., & Mearns, L. O. (2021). An analysis of the disagreement about added value by regional climate models. Synthese, 198(12), 11645–11672. https://doi.org/10.1007/s11229-020-02821-x

    Article  Google Scholar 

  • Lloyd, E. A., Lewontin, R. C., & Feldman, M. W. (2008). The generational cycle of state spaces and adequate genetical representation. Philosophy of Science, 75(2), 140–156. https://doi.org/10.1086/590196

    Article  Google Scholar 

  • Maloney, E. D., Gettelman, A., Ming, Y., Neelin, J. D., Barrie, D., Mariotti, A., Chen, C.-C., Coleman, D. R. B., Kuo, Y.-H., Singh, B., Annamalai, H., Berg, A., Booth, J. F., Camargo, S. J., Dai, A., Gonzalez, A., Hafner, J., Jiang, X., Jing, X, . . . Zhao, M. (2019). Process-oriented evaluation of climate and weather forecasting models. Bulletin of the American Meteorological Society, 100(9), 1665–86. https://doi.org/10.1175/BAMS-D-18-0042.1

  • Mann, M. E. (2018). Reconciling climate model/data discrepancies: The case of the ‘Trees That Didn’t Bark’. In E. A. Lloyd & E. Winsberg (Eds.), Climate modelling: Philosophical and conceptual issues (pp. 175–97). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-65058-6_7

  • Marsh, P. T., Brooks, H. E., & Karoly, D. J. (2007). Assessment of the severe weather environment in North America simulated by a global climate model. Atmospheric Science Letters, 8(4), 100–106. https://doi.org/10.1002/asl.159

    Article  Google Scholar 

  • Mauritsen, T., Stevens, B., Roeckner, E., Crueger, T., Esch, M., Giorgetta, M., Haak, H., Jungclaus, J., Klocke, D., Matei, D., Mikolajewicz, U., Notz, D., Pincus, R., Schmidt, H., Tomassini, L. (2012). Tuning the climate of a global model. Journal of Advances in Modeling Earth Systems, 4(3). https://doi.org/10.1029/2012MS000154

  • Mayernik, M. S. (2021). Credibility via coupling: Institutions and infrastructures in climate model intercomparisons. Engaging Science, Technology, and Society, 7(2), 10–32. https://doi.org/10.17351/ests2021.769

    Article  Google Scholar 

  • Mayo, D. G. (1996). Error and the growth of experimental knowledge. University of Chicago Press.

    Book  Google Scholar 

  • Meehl, G. A., Boer, G. J., Covey, C., Latif, M., & Stouffer, R. J. (2000). The coupled model intercomparison project (CMIP). Bulletin of the American Meteorological Society, 81(2), 313–318.

    Article  Google Scholar 

  • Morrison, M. A. (2021). The models are alright: A socio-epistemic theory of the landscape of climate model development. Ph.D., United States -- Indiana: Indiana University. Accessed August 30, 2021. https://www.proquest.com/docview/2489342331/abstract/CDF0E73D2F944EEPQ/1

  • National Academy of Sciences, Climate Research Board. (1979). Carbon dioxide and climate: A scientific assessment (Jule Charney, Chair). National Academy of Sciences.

    Google Scholar 

  • Neale, R. B., Gettelman, A., Park, S., Chen, C.-C., Lauritzen, P. H., Williamson, D. L., Conley, A. J., Kinnison, D., Marsh, D., Smith, A. K., Vitt, F. M., Garcia, R., Lamarque, J.-F., Mills, M. J., Tilmes, S., Morrison, H., Cameron, P., Collins, W. D., Iacono, M. J., . . . Taylor, M. A. (2012). Description of the NCAR community atmosphere model (CAM 5.0). NCAR Technical Note. p. 289.

  • Neale, R. B., & Hoskins, B. J. (2000). A standard test for AGCMs including their physical parametrizations: I: The proposal. Atmospheric Science Letters, 1(2), 101–107. https://doi.org/10.1006/asle.2000.0019

    Article  Google Scholar 

  • Notz, D., Alexander Haumann, F., Haak, H., Jungclaus, J. H., & Marotzke, J. (2013). Arctic sea-ice evolution as modeled by Max Planck institute for meteorology’s earth system model. Journal of Advances in Modeling Earth Systems, 5(2), 173–194. https://doi.org/10.1002/jame.20016

    Article  Google Scholar 

  • Odenbaugh, J. (2018). Building trust, removing doubt? Robustness analysis and climate modeling. In E. A. Lloyd, E. Winsberg (Eds.), Climate modelling: Philosophical and conceptual issues (pp. 297–321). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-65058-6_11

  • O’Loughlin, R. (2021). Robustness reasoning in climate model comparisons. Studies in History and Philosophy of Science Part A, 85(February), 34–43. https://doi.org/10.1016/j.shpsa.2020.12.005

    Article  Google Scholar 

  • Oreopoulos, L., Mlawer, E., Delamere, J., Shippert, T., Cole, J., Fomin, B., Iacono, M., Jin, Z., Li, J., Manners, J., Räisänen, P., Rose, F. G., Zhang, Y., Wilson, M. J., & Rossow, W. B. (2012). The continual intercomparison of radiation codes: Results from phase I. Journal of Geophysical Research: Atmospheres 117(D6). https://doi.org/10.1029/2011JD016821

  • Parker, W. S. (2011). When climate models agree: The significance of robust model predictions. Philosophy of Science, 78(4), 579–600. https://doi.org/10.1086/661566

    Article  Google Scholar 

  • Parker, W. S. (2018a). The significance of robust climate projections. In E. A. Lloyd, E. Winsberg (Eds.), Climate modelling: Philosophical and conceptual issues (pp. 273–96). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-65058-6_9

  • Parker, W. S. (2018b). Climate science. In E. N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University. https://plato.stanford.edu/archives/sum2018b/entries/climate-science/

  • Pincus, R., Mlawer, E. J., Oreopoulos, L., Ackerman, A. S., Baek, S., Brath, M., Buehler, S. A., Cady-Pereira, K. E., Cole, J. N., Dufresne, J. L., Kelley, M., Li, J., Manners, J., Paynter, D. J., Roehrig, R., Sekiguchi, M., & Schwarzkopf, D. M. (2015). Radiative flux and forcing parameterization error in aerosol-free clear skies. Geophysical Research Letters, 42(13), 5485–5492. https://doi.org/10.1002/2015GL064291

  • Pitari, G., Aquila, V., Kravitz, B., Robock, A., Watanabe, S., Cionni, I., De Luca, N., Di Genova, G., Mancini, E., & Tilmes, S. (2014). Stratospheric ozone response to sulfate geoengineering: Results from the geoengineering model intercomparison project (GeoMIP). Journal of Geophysical Research: Atmospheres, 119(5), 2629–2653. https://doi.org/10.1002/2013JD020566

    Article  Google Scholar 

  • Ramanathan, V., Cess, R. D., Harrison, E. F., Minnis, P., Barkstrom, B. R., Ahmad, E., & Hartmann, D. (1989). Cloud-radiative forcing and climate: Results from the earth radiation budget experiment. Science, 243(4887), 57–63. https://doi.org/10.1126/science.243.4887.57

    Article  Google Scholar 

  • Randall, D. A., Bitz, C. M., Danabasoglu, G., Denning, A. S., Gent, P. R., Gettelman, A., Griffies, S. M., Lynch, P., Morrison, H., Pincus, R., Thuburn, J. (2018). 100 years of earth system model development. Meteorological Monographs, 59(1), 12.1–12.66. https://doi.org/10.1175/AMSMONOGRAPHS-D-18-0018.1

  • Schmidt, G. A., & Sherwood, S. (2015). A practical philosophy of complex climate modelling. European Journal for Philosophy of Science, 5(2), 149–169. https://doi.org/10.1007/s13194-014-0102-9

    Article  Google Scholar 

  • Schmidt, G. A., Bader, D., Donner, L. J., Elsaesser, G. S., Golaz, J.-C., Hannay, C., Molod, A., Neale, R. B., & Saha, S. (2017). Practice and philosophy of climate model tuning across six US modeling centers. Geoscientific Model Development, 10(9), 3207–3223. https://doi.org/10.5194/gmd-10-3207-2017

    Article  Google Scholar 

  • Schmidt, K. (2012). The trouble with ‘Tacit Knowledge.’ Computer Supported Cooperative Work (CSCW), 21(2), 163–225. https://doi.org/10.1007/s10606-012-9160-8

    Article  Google Scholar 

  • Schneider, S. H. (1979). Verification of parameterizations in climate modeling. In W. Lawrence Gates (Eds.), Report of the Study Conference on Climate Models: Performance, Intercomparison and Sensitivity Studies (pp. 728–51). World Meteorological Organization, Global Atmospheric Research Program, GARP Publications Series no. 22, 2 vols.

  • Schneider, S. H. (1972). Cloudiness as a global climatic feedback mechanism: The effects on the radiation balance and surface temperature of variations in cloudiness. Journal of the Atmospheric Sciences, 29(8), 1413–1422. https://doi.org/10.1175/1520-0469(1972)029%3c1413:CAAGCF%3e2.0.CO;2

    Article  Google Scholar 

  • Schneider, S. H. (1975). On the carbon dioxide-climate confusion. Journal of Atmospheric Sciences, 32(11), 2060–2066. https://doi.org/10.1175/1520-0469(1975)032%3c2060:OTCDC%3e2.0.CO;2

    Article  Google Scholar 

  • Schneider, S. H., & Dickinson, R. E. (1974). Climate modeling. Reviews of Geophysics, 12(3), 447–493. https://doi.org/10.1029/RG012i003p00447

    Article  Google Scholar 

  • Sengupta, S., & Boyle, J. S. (1998). Using common principal components for comparing GCM simulations. Journal of Climate, 11(5), 816–830. https://doi.org/10.1175/1520-0442(1998)011%3c0816:UCPCFC%3e2.0.CO;2

    Article  Google Scholar 

  • Steele, K., & Werndl, C. (2013). Climate models, calibration, and confirmation. The British Journal for the Philosophy of Science, 64(3), 609–635. https://doi.org/10.1093/bjps/axs036

    Article  Google Scholar 

  • Sun, Y., Solomon, S., Dai, A., & Portmann, R. W. (2006). How often does it rain? Journal of Climate, 19(6), 916–934. https://doi.org/10.1175/JCLI3672.1

    Article  Google Scholar 

  • Tebaldi, C., & Knutti, R. (2007). The use of the multi-model ensemble in probabilistic climate projections. Philosophical Transactions of the Royal Society a: Mathematical, Physical and Engineering Sciences, 365(1857), 2053–2075. https://doi.org/10.1098/rsta.2007.2076

    Article  Google Scholar 

  • Touzé-Peiffer, L., Barberousse, A., & Le Treut, H. (2020). The coupled model intercomparison project: History, uses, and structural effects on climate research. WIREs Climate Change, 11(4), e648. https://doi.org/10.1002/wcc.648

    Article  Google Scholar 

  • Voosen, P. (2021). U.N. climate panel confronts implausibly hot forecasts of future warming. (2021). http://www.science.org/content/article/un-climate-panel-confronts-implausibly-hot-forecasts-future-warming

  • Washington, Warren. 2006. Odyssey in climate modeling, global warming, and advising five presidents. Edited by Mary C. Washington. lulu.com.

  • Weart, S. (2020). The discovery of global warming - A history. The discovery of global warming. 2020. https://history.aip.org/climate/pdf/Gcm.pdf

  • Wilson, J. (2021). Two exploratory uses for general circulation models in climate science. Perspectives on Science, 29(4), 493–509. https://doi.org/10.1162/posc_a_00380

    Article  Google Scholar 

  • Wimsatt, W. C. (2007). Re-engineering philosophy for limited beings. Piecewise approximations to reality. Cambridge, MA and London, England: Harvard University Press.

  • Winsberg, E. (2018a). What does robustness teach Us in climate science: A re-appraisal. Synthese. https://doi.org/10.1007/s11229-018-01997-7

    Article  Google Scholar 

  • Winsberg, E. (2018b). Philosophy and climate science. Cambridge University Press.

    Book  Google Scholar 

  • Zhang, G. J., & McFarlane, N. A. (1995). Sensitivity of climate simulations to the parameterization of cumulus convection in the Canadian climate centre general circulation model. Atmosphere-Ocean, 33(3), 407–446. https://doi.org/10.1080/07055900.1995.9649539

    Article  Google Scholar 

  • Zhang, G. J. (2002). Convective quasi-equilibrium in midlatitude continental environment and its effect on convective parameterization. Journal of Geophysical Research: Atmospheres 107(D14): ACL 12–1-ACL 12–16. https://doi.org/10.1029/2001JD001005

Download references

Acknowledgements

This paper has benefited greatly from extensive discussions with (and feedback from) Lisa Lloyd, Dan Li, Phil Rasch, Evan Arnet, Ben Kravitz, Jutta Schickore, Ann Sophie Barwich, Scott Robeson, Monica Morrison, Matt Mayernik, Siyu Yao, Becca Jackson, and Stu Gluck.

Author information

Authors and Affiliations

  1. Queens College CUNY, Queens, NY, USA

    Ryan O’Loughlin

Authors
  1. Ryan O’Loughlin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ryan O’Loughlin.

Ethics declarations

Ethical approval

As the research for this article did not involve human or non-human animal participants, there is no need for formal institutional ethical approval.

Informed consent

n/a

Conflict of interest

None to report. The manuscript has been blinded for anonymity.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

O’Loughlin, R. Diagnosing errors in climate model intercomparisons. Euro Jnl Phil Sci 13, 20 (2023). https://doi.org/10.1007/s13194-023-00522-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13194-023-00522-z

Keywords

Search

Navigation