Abstract
Lisciandra (2017) poses a challenge for robustness analysis (RA) as applied to economic models. She argues that substituting tractability assumptions risks altering the main mathematical structure of the model, thereby preventing the possibility of meaningfully evaluating the same model under different assumptions. In such cases RA is argued to be inapplicable. However, Lisciandra is mistaken to take the goal of RA as keeping the mathematical properties of tractability assumptions intact. Instead, RA really aims to keep the modeling component while varying the corresponding mathematical formulation. Thus, her argument concerning whether the associated mathematical properties of certain assumptions can be kept intact is irrelevant to the success of RA. Furthermore, we explicate and develop Lloyd’s account of “model robustness” (Lloyd 2015) to provide solutions to Lisciandra’s challenges. Our solutions are, namely, error analysis and independent empirical support. We conclude that although complex economic models do face potential dangers, there are solutions and robustness analysis need not be given up. (157).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
Not applicable.
Code availability
Not applicable.
Notes
In particular, see Woodward’s (2006) derivational robustness.
The epistemic import of a robust model, or of the results of a robustness analysis, are a matter of contention amongst philosophers of science (Lloyd 2010, 2015; Parker 2018; Weisberg 2006; Woodward 2006). We take a stance on this issue in Sect. 3 below, following Lloyd’s (2015) account of model robustness.
Schupbach (2018) emphasizes the diversity of models (or methods or measurements) and their ability to discriminate between explanatory hypotheses. In both the derivational robustness case and in the model robustness framework, such diversity can be understood in terms of the differences between the \(Ai\) s, and in the model robustness case the different sources of evidential support add to this diversity. While Schupbach’s account is claimed to be completely general and all-encompassing, other work has shown it to instead be complementary to Lloyd’s model robustness (O’Loughlin, 2021).
Recently it has been shown that, prior to John Tyndall’s experiments, Eunice Foote showed experimentally that greenhouse gases such as water vapor and CO2 would lead to “solar heating” (Huddleston, 2019).
This is in the spirit of Lloyd’s own suggestion that her account may be fruitfully applied to complex modeling generally (Lloyd 2015, p. 59). A few key differences between economics models and climate models: the latter are often run on supercomputers (they are simulation models) whereas the former are not; the latter employ well understood physical principles whereas the former do not; economists often explicitly desire analytic solutions whereas climate scientists work with numerical results (spatiotemporal gridded data which are output by the models).
Thank you to an anonymous reviewer for pushing us on this.
Kuorikoski et al. specifically mention that different and possibly contradictory assumptions could be made regarding the specific form of customer preference. E.g., constant elasticity substitution functions entail the independence of prices from the fundamentals of the market, while quadratic utility functions entail the opposite, both of which are “literally false” (562).
See also Mäki and Marchionni (2009).
Note she also switched from “iceberg assumption” to “price function”. The latter is a deductive result of the former. See footnote 10 (below) for details.
McCann (2005) aims “to explain the subtle nature of these iceberg properties”, namely the Samuelson function and the Krugman function. Although both aim at representing transport costs, the Samuelson iceberg transport cost function is essentially a step function, characterized by its discontinuity. In contrast, the Krugman function explicitly defines the iceberg transport cost as \({V}_{d}={V}_{o}{e}^{-\tau D}\), where \({V}_{d}\) and \({V}_{o}\) are values of goods at destination and origin respectively, \(D\) is the distance between destination and origin, and \(\tau \) is the decay parameter. Then McCann (2005) deduces three properties of a Krugman function: “the delivered price of the good is (i) convex with respect to the distance, (ii) directly proportional to the original price of the good, and (iii) the transport rate per ton-kilometre is independent of the quantity shipped” (311). These properties are clearly very different from the Samuelson function and some others. E.g., it is not difficult to imagine a geo-economic model that assumes transport economies of scale, meaning that, as the quantity of goods gets larger, the average transport cost per unit gets smaller, which is contradictory to (iii). Thus, McCann concludes that “[t]hese fundamental differences between the properties of the Krugman iceberg assumption and more traditional transport cost functions, therefore lead to something of a problem of interpretation and inference.” (321, emphasis added). Lisciandra seems to interpret McCann’s “problem of interpretation and inference” as a problem for robustness analysis, which is a mistake.
Note that in a recent paper, Lehtinen (2018, p. 556) argues that Lloyd’s (2015) account is weakened “considerably” by the fact that specific model assumptions (that are false if taken literally) such as parameterizations in climate models, are mutually incompatible and thus cannot indirectly confirm the truth of the model output. However, Lehtinen’s account seems to rely on an artificial dichotomy between the falsity and truth of model assumptions. Moreover, Lehtinen portrays Lloyd’s account as being solely concerned with the truth of the model output in virtue of agreement between models. While we do not have the space to defend Lloyd’s account here, we do regard this as a mischaracterization of Lloyd’s account that is at odds with the above description.
See Marchionni (2017) for another appeal to combining empirical strategies of model testing with derivational robustness analysis in economic models.
Thank you to an anonymous reviewer for helping us think further about this issue.
According to McCann (2005): “all of the available evidence suggests that delivered prices tend to be concave rather than convex with distance” (pp. 315–316).
E.g., the greenhouse effect (see Hulme, 2009).
Interestingly, transport costs in older geographical economies models were also supported empirically by various sources (McCann 2005, pp. 311–312).
References
Baba, Y. (2019). Spectral cumulus parameterization based on cloud-resolving model. Climate Dynamics, 52(1), 309–334. https://doi.org/10.1007/s00382-018-4137-z
Bosker, M., & Buringh, E. (2020). Ice(Berg) transport costs. The Economic Journal, 130(629), 1262–1287. https://doi.org/10.1093/ej/ueaa023
Cartwright, N. (2007). The Vanity of Rigour in Economics: Theoretical Models and Galilean Experiments. In Hunting Causes and Using Them: Approaches in Philosophy and Economics. New York: Cambridge University Press. https://doi.org/10.1017/CBO9780511618758
Collins, W. D., V. Ramaswamy, M. D. Schwarzkopf, Y. Sun, R. W. Portmann, Q. Fu, S. E. B. Casanova, et al. (2006). radiative forcing by well-mixed greenhouse gases: estimates from climate models in the intergovernmental panel on climate change (IPCC) Fourth Assessment Report (AR4). Journal of Geophysical Research: Atmospheres 111 (D14). https://doi.org/10.1029/2005JD006713.
Einarsson, B. (2005). Accuracy and Reliability in Scientific Computing. SIAM.
Fillion, N. (2016.) “Demystifying the Applicability of Mathematics.” In Trick or Truth? The Mysterious Connection Between Physics and Mathematics, edited by Anthony Aguirre, Brendan Foster, and Zeeya Merali, 1st ed., 153–144. The Frontiers Collection. Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-27495-9_12.
Fillion, N., & Corless, R. M. (2014). On the epistemological analysis of modeling and computational error in the mathematical sciences. Synthese, 191(7), 1451–1467. https://doi.org/10.1007/s11229-013-0339-4
Gettelman, A., Richard B.R. (2016). Demystifying Climate Models. Vol. 2. Earth Systems Data and Models. Berlin, Heidelberg: Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-662-48959-8.
Hegerl, G., & Zwiers, F. (2011). Use of models in detection and attribution of climate change. Wires Climate Change, 2(4), 570–591. https://doi.org/10.1002/wcc.121
Hindriks, F. A. (2006). Tractability assumptions and the musgrave-mäki typology. Journal of Economic Methodology, 13(4), 401–423. https://doi.org/10.1080/13501780601048733
Huddleston, A. (2019). “Happy 200th Birthday to Eunice Foote, Hidden Climate Science Pioneer.” NOAA Climate - Science & Information for a Climate-Smart Nation - News & Features, July 17, 2019. https://www.climate.gov/news-features/features/happy-200th-birthday-eunice-foote-hidden-climate-science-pioneer.
Hulme, M. (2009). On the origin of ‘the greenhouse effect’: John tyndall’s 1859 interrogation of nature. Weather, 64(5), 121–123. https://doi.org/10.1002/wea.386
Kuorikoski, J., Lehtinen, A., & Marchionni, C. (2010). Economic modelling as robustness analysis. The British Journal for the Philosophy of Science, 61(3), 541–567. https://doi.org/10.1093/bjps/axp049
Lehtinen, A. (2018). Derivational robustness and indirect confirmation. Erkenntnis, 83(3), 539–576. https://doi.org/10.1007/s10670-017-9902-6
Liou, K. N. 1992. “Radiation and cloud processes in the atmosphere. Theory, observation, and modeling. https://www.osti.gov/biblio/7081459.
Lisciandra, C. (2017). Robustness analysis and tractability in modeling. European Journal for Philosophy of Science, 7(1), 79–95.
Lloyd, E. A. (2010). Confirmation and robustness of climate models. Philosophy of Science, 77(5), 971–984. https://doi.org/10.1086/657427
Lloyd, E. A. (2015). Model robustness as a confirmatory virtue: the case of climate science. Studies in History and Philosophy of Science Part A, 49(February), 58–68. https://doi.org/10.1016/j.shpsa.2014.12.002
Marchionni, C. (2017). What is the problem with model-based explanation in economics? Disputatio, 9(47), 603–630. https://doi.org/10.1515/disp-2017-0020
McCann, P. (2005). Transport costs and new economic geography. Journal of Economic Geography, 5(3), 305–318. https://doi.org/10.1093/jnlecg/lbh050
McMullin, E. (1985). Galilean idealization. Studies in History and Philosophy of Science Part A, 16(3), 247–273. https://doi.org/10.1016/0039-3681(85)90003-2
Musgrave, A. (1981). ‘Unreal Assumptions’ in economic theory: The F-twist untwisted. Kyklos, 34(3), 377–387. https://doi.org/10.1111/j.1467-6435.1981.tb01195.x
O’Loughlin, R. (2021). Robustness reasoning in climate model comparisons. Studies in History and Philosophy of Science Part A, 85(February), 34–43. https://doi.org/10.1016/j.shpsa.2020.12.005
Ottaviano, G., Tabuchi, T., & Thisse, J.-F. (2002). Agglomeration and trade revisited. International Economic Review, 43(2), 409–435.
Parker, WS. (2018). “The Significance of Robust Climate Projections.” In Climate Modelling: Philosophical and Conceptual Issues, edited by Elisabeth A. Lloyd and Eric Winsberg, 273–96. Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-65058-6_9.
Schupbach, J. N. (2018). Robustness analysis as explanatory reasoning. The British Journal for the Philosophy of Science, 69(1), 275–300. https://doi.org/10.1093/bjps/axw008
Stensrud, DJ. (2007). Parameterization Schemes: Keys to Understanding Numerical Weather Prediction Models. Cambridge University Press.
Weisberg, M. (2006). Forty Years of ‘The Strategy’: levins on model building and idealization. Biology & Philosophy, 21(5), 623–645. https://doi.org/10.1007/s10539-006-9051-9
Wong, J.-F. (2009). The analysis of a finite element method for the three-species lotka-volterra competition-diffusion with dirichlet boundary conditions. Journal of Computational and Applied Mathematics, 223(1), 421–437. https://doi.org/10.1016/j.cam.2008.01.030
Woodward, J. (2006). Some varieties of robustness. Journal of Economic Methodology, 13(2), 219–240. https://doi.org/10.1080/13501780600733376
Acknowledgements
We would like to thank Lisa Lloyd, Stu Gluck, Evan Arnet, Ann Campbell, Shannon Abelson, Nancy Cartwright, other folks from Lloyd Lab, and two anonymous reviewers for helpful feedback and conversations on various drafts of this work.
Funding
No funding was received to support this research.
Author information
Authors and Affiliations
Contributions
The authors contributed equally to the article.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
O’Loughlin, R., Li, D. Model robustness in economics: the admissibility and evaluation of tractability assumptions. Synthese 200, 32 (2022). https://doi.org/10.1007/s11229-022-03608-y
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11229-022-03608-y