Skip to main content

Some Pitfalls to Beware When Applying Models to Issues of Policy Relevance

Part of the Understanding Complex Systems book series (UCS)


This chapter looks at some of the ways things can go wrong when mathematical or computational models are applied to inform policy on important issues. It looks at some of the pitfalls in the model construction and development phase, including choosing assumptions, the effect of ‘theoretical spectacles’, oversimplified models, not understanding model limitations, and not testing a model enough. It then goes on to discuss the pitfalls that can occur when a model is applied to inform policy, including entrenched policies based on models with little or no evidential support and how models can narrow the evidential base considered. It also looks at confusions concerning model purpose and kinds of question they may answer, when models are used out of context, asking unreasonable things of models, when the uncertainties are too great, when models give a false sense of security, and when the focus should be on values rather than facts. This discussion is then illustrated with two examples, one economic and one from fisheries. It concludes that most of these problems stem from the interface between the modelling and policy worlds. It ends with some simple recommendations to reduce these mistakes.


  • Policy
  • Pitfalls
  • Mental model
  • Abstraction
  • Formal model
  • Transparency
  • Political process
  • Assumptions
  • Reality
  • Theoretical spectacles
  • Oversimplification
  • Model limitations
  • Entrenched models
  • Evidential support
  • Evil models
  • Model purpose
  • Prediction
  • Context of development
  • Context of application
  • Uncertainty
  • False security
  • Austerity policies
  • Newfoundland cod

This is a preview of subscription content, access via your institution.


  1. 1.

    At best, examination is by a few in the same domain as themselves—people who likely have the same assumptions and worldview. Thus, many models are not effectively critiqued in an independent manner.

  2. 2.

    This is shorthand for saying the model’s assumptions make a model useless in terms of its purpose.

  3. 3. (accessed 1 June 2017)

  4. 4.

    Giampietro and Saltelli (2014) provide a discussion on these questions in relation to the ecological footprint.

  5. 5.

    However, model-based explanations of why climate change has been happening are well founded.

  6. 6.

    Whilst models are a tool rather than a picture (see Chap. 4), some are so useless at what they are supposed to do, that it makes sense to call them wrong.


  • Bavington, D. (2009). Managing to endanger: Creating manageable cod fisheries in Newfoundland & Labrador, Canada. Maritime Studies (MAST), 7, 99–119.

    Google Scholar 

  • Bavington, D. (2010). Managed annihilation: An unnatural history of the Newfoundland cod collapse. UBC press.

    Google Scholar 

  • Bavington, D. (2015). Marine and freshwater fisheries in Canada: Uncertainties, conflicts, and hope on the water. In B. Mitchell (Ed.), Resource and environmental management in Canada. Canada: OUP.

    Google Scholar 

  • Benessia, A., Funtowicz, S., Giampietro, M., Guimarães Pereira, A., Ravetz, J., Saltelli, A., et al. (2016). Science on the Verge. Tempe, AZ: The Consortium for Science, Policy and Outcomes at Arizona State University.

    Google Scholar 

  • Black, F., & Scholes, M. (1973). The pricing of options and corporate liabilities. Journal of Political Economy, 81(3), 637–654.

    CrossRef  MathSciNet  MATH  Google Scholar 

  • Bruff, I. (2016). Scandalous obfuscations of crisis in Europe. In: Progress in political economy (PPE). Available from:

  • Cavero, T., & Poinasamy, K. (2013). A cautionary tale: The true cost of austerity and inequality in Europe. Oxford: Oxfam International.

    Google Scholar 

  • Costello, C., Ovando, D., Clavelle, T., Strauss, C. K., Hilborn, R., Melnychuk, M. C., et al. (2016). Global fishery prospects under contrasting management regimes. Proceedings of the National Academy of Sciences, 113(18), 5125–5129.

    CrossRef  Google Scholar 

  • Edmonds, B. (2017). Five different modelling purposes. doi:

    CrossRef  Google Scholar 

  • Finlayson, A. C. (1994). Fishing for truth: A sociological analysis of northern cod stock assessments from 1977 to 1990. St. John’s, Newfoundland: Institute of social and economic research, Memorial University of Newfoundland.

    Google Scholar 

  • Finley, C. (2008). A political history of maximum sustainable yield, 1945–1955. In D. Starkey, P. Holm, & M. Barnard (Eds.), Ocean's past: Management insights from the history of marine animal populations (pp. 1989–1206). London: Earthscan.

    Google Scholar 

  • Giampietro, M., & Saltelli, A. (2014). Footprints to nowhere. Ecological Indicators, 46, 610–621.

    CrossRef  Google Scholar 

  • Glynn, P. D. (2015). Integrated environmental modelling: Human decisions, human challenges. Geological Society, London, Special Publications, 408(1), 161–182.

    CrossRef  Google Scholar 

  • Harris, L. (1990). Independent review of the northern cod stock: Executive summary, and recommendations, Available at:

  • Herndon, T., Ash, M., & Pollin, R. (2013). Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff. Cambridge Journal of Economics, 38(2), 257–279.

    CrossRef  Google Scholar 

  • Holbraad, M., Pedersen, M. A., & de Castro, E. V. (2014). The politics of ontology: Anthropological positions. Cultural Anthropology, 13. Available at:

  • Kuhn, T. (1962). The Structure of Scientific Revolutions. Chicago: University of Chicago Press.

    Google Scholar 

  • Moore, J. W. (2017). The Capitalocene, Part I: On the nature and origins of our ecological crisis. The Journal of Peasant Studies, 44(3), 594–630.

    CrossRef  Google Scholar 

  • Pearce, W., Wesselink, A., & Colebatch, H. (2014). Evidence and meaning in policy making. Evidence & Policy, 10(2), 161–165.

    CrossRef  Google Scholar 

  • Rogoff, K., & Reinhart, C. (2010). Growth in a time of debt. American Economic Review, 100(2), 573–578.

    CrossRef  Google Scholar 

  • Rosewell, B. (2017). Complexity science and the art of policy making. In J. Johnson, P. Ormerod, B. Rosewell, A. Nowak, & Y. C. Zhang (Eds.), Non-equilibrium social science and policy (pp. 159–178). Switzerland: Springer.

    Google Scholar 

  • Saltelli, A., & Funtowicz, S. (2014). When all models are wrong. Issues in Science and Technology, 30(2), 79–85.

    Google Scholar 

  • Saltelli, A., Stark, P. B., Becker, W., & Stano, P. (2015). Climate models as economic guides scientific challenge or quixotic quest? Issues in Science and Technology, 31(3), 79–84.

    Google Scholar 

  • Saltelli, A., & Giampietro, M. (2017). What is wrong with evidence based policy, and how can it be improved? Futures, 91, 62–71, Available at:

  • Scott, J. C. (1998). Seeing like a state: How certain schemes to improve the human condition have failed. Yale University Press.

    Google Scholar 

  • Sterman, J. D. (2002). All models are wrong: Reflections on becoming a systems scientist. System Dynamics Review, 18(4), 501–531.

    CrossRef  Google Scholar 

  • Stern, N. (2016). Economics: Current climate models are grossly misleading. Nature, 530(7591), 407–409.

    CrossRef  Google Scholar 

  • Stewart, I. (2012). The mathematical equation that caused the banks to crash. The Observer, 12 February 2012, Accessed 01 June 2017.

  • Stiglitz, J. E. (2011). Rethinking macroeconomics: What failed, and how to repair it. Journal of the European Economic Association, 9(4), 591–645.

    CrossRef  Google Scholar 

  • Weinberg, A. M. (1972). Science and trans-science. Minerva, 10(2), 209–222.

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Lia ní Aodha .

Editor information

Editors and Affiliations

Further Reading

Further Reading

For further information, we suggest you read the following four reports. Some of these are books, others are articles.

Other useful reading includes the following.

Rights and permissions

Reprints and Permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Aodha, L.n., Edmonds, B. (2017). Some Pitfalls to Beware When Applying Models to Issues of Policy Relevance. In: Edmonds, B., Meyer, R. (eds) Simulating Social Complexity. Understanding Complex Systems. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-66947-2

  • Online ISBN: 978-3-319-66948-9

  • eBook Packages: Computer ScienceComputer Science (R0)