Abstract
The increasing complexity of policy problems, coupled with the political desire to base new policies on the foundation of firm evidence, has accelerated the development of policy assessment tools. These range from complex computer models and cost benefit analysis through simple checklists and decision trees. In the last decade, many governments have established formal policy assessment systems to harness these tools in order to facilitate more evidence-based policy making. These tools are potentially widely available, but to what extent are they used by policy makers and what becomes of the evidence that they generate? This paper addresses these questions by studying the empirical patterns of tool use across 37 cases in three European countries and the European Commission. It uses a simple classification of tools into advanced, formal and simple types. It finds that even when tools are embedded in policy assessment systems, their use is differentiated and on the whole very limited, in particular when it comes to more advanced tools. It then explores these patterns from contrasting theoretical perspectives to shed light on why, when and how different policy assessment tools are used in the policy process.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
As opposed to programmes/plans and project level decisions which are routinely subjected to strategic environmental assessment (SEA) or environmental impact assessment (EIA).
Interview with German official, March 2006.
Interview with UK official, June 2006.
Interview with German official, March 2006.
Interview with Commission official, February 2006.
Interview with UK official, May 2006.
Interview with UK official, June 2006.
Interview with Commission official, January 2006.
Interview with UK official, May 2006.
Interview with UK official, May 2006.
Interview with Swedish official, April 2006.
References
Bartlett, R., & Kurian, P. (1999). The theory of environmental impact assessment: Implicit models of policy making. Policy and Politics, 27, 415–433. doi:10.1332/030557399782218371.
Bendor, J., Moe, T., & Shotts, K. (2001). Recycling the garbage can: An assessment of the research program. The American Political Science Review, 95, 169–190.
BMI. (2000). Moderner Staat—moderne Verwaltung : Gemeinsame Geschäftsordnung der Bundesministerien. Berlin: Bundesministerium des Innern/German Federal Ministry of the Interior.
BMJ (1999). Handbuch der Rechtsförmlichkeit. Empfehlungen des Bundesministeriums der Justiz zur einheitlichen rechtsförmlichen Gestaltung von Gesetzen und Rechtsverordnungen nach § 38 Abs. 3 GGO II. Berlin: Bundesministerium der Justiz/German Federal Ministry of Justice.
Cabinet Office. (2000). Adding it up: Improving analysis and modelling in central government. London: HMSO.
Cabinet Office. (2003). Regulatory impact assessment guidance. London: HMSO.
Clarence, E. (2002). Editorial: Technocracy reinvented: The new evidence based policy movement. Public Policy and Administration, 17, 1–11. doi:10.1177/095207670201700301.
Clark, W. C., & Majone, G. (1985). The critical assessment of scientific enquiries with policy implications. Science, Technology & Human Values, 10, 6–19. doi:10.1177/016224398501000302.
Cohen, M., March, J., & Olsen, J. (1972). A garbage can model of organisational choice. Administrative Science Quarterly, 17, 1–25. doi:10.2307/2392088.
Cronbach, L. J. (Ed.). (1980). Toward reform of programme evaluation. San Francisco: Jossey-Bass.
de Ridder, W., Turnpenny, J., Nilsson, M., & von Raggamby, A. (2007). A framework for tool selection and use in integrated assessment for sustainable development. Journal of Environmental Assessment Policy and Management, 9, 423–441. doi:10.1142/S1464333207002883.
European Commission. (2001). European governance: A white paper. COM (2001) 428. Brussels: European Commission.
European Commission. (2002). Communication: European governance—better law-making. COM (2002) 0275. Brussels: European Commission.
European Commission. (2005). Impact assessment guidelines. Brussels: European Commission.
European Commission. (2006). Communication: A strategic review of better regulation in the European Commission. Brussels: European Commission.
Hanley, N. (2001). Cost benefit analysis and environmental policy making. Environment and Planning C, 19, 103–118.
Harrington, W., & Morgenstern, R. D. (2004). Evaluating regulatory impact analyses. Washington, DC: Resources for the Future.
Hertin, J., Turnpenny, J., Jordan, A., Nilsson, M., Russel, D., & Nykvist, B. (in press). Rationalising the policy mess? Ex ante assessment and the utilisation of knowledge in the policy process. Environment & Planning A.
Huber, G. P. (1991). Organizational learning: The contributing processes and the literatures. Organization Science, 2, 88–115.
Jacobs, S. (2004). RIA and the economic transition to markets. Public Money and Management, 24, 283–290. doi:10.1111/j.1467-9302.2004.00435.x.
Jenkins-Smith, H. (1990). Democratic politics and policy analysis. Belmont, CA: Brooks/Cole.
Jordan, A., & Schout, A. (2006). The coordination of the European Union. Oxford: Oxford University Press.
Jordan, A. J., Wurzel, R. K., & Zito, A. R. (Eds.). (2003). New instruments of environmental governance?. London: Routledge.
Kates, R. W., Clark, W. C., Corell, R., Hall, J., Jaeger, C., Lowe, I., et al. (2001). Sustainability science. Science, 292, 641–642. doi:10.1126/science.1059386.
Kingdon, J. W. (1984). Agendas, alternatives, and public policies. New York, NY: HarperCollins.
Kleindorfer, P., Kunreuther, H., & Schoemaker, P. (1993). Decision sciences: An integrative perspective. Cambridge: Cambridge University Press.
Lascoumes, P., & Le Gales, P. (2007). Introduction: Understanding public policy through its instruments. Governance, 20, 1–21. doi:10.1111/j.1468-0491.2007.00342.x.
Lee, N., & Kirkpatrick, C. (2006). Evidence-based policy making in Europe: An evaluation of European Commission integrated impact assessments. Impact Assessment and Project Assessment, 24, 23–33. doi:10.3152/147154606781765327.
Mitchell, R. B., Clark, W. C., Cash, D., & Dickinson, N. (2006). Global environmental assessments: Information and influence. Cambridge, MA: MIT Press.
Nilsson, M. (2006). The role of assessments and institutions for policy learning: A study on Swedish climate and nuclear policy formation. Policy Sciences, 38, 225–249. doi:10.1007/s11077-006-9006-7.
Olsen, J. (2001). Garbage cans, the new institutionalism, and the study of politics. The American Political Science Review, 95, 191–198.
Owens, S., Rayner, T., & Bina, O. (2004). New agendas for assessment: Reflections on theory, practice, and research. Environment & Planning A, 36, 1943–1959. doi:10.1068/a36281.
Pearce, D. (1998). Cost benefit analysis and environmental policy. Oxford Review of Economic Policy, 14, 84–100. doi:10.1093/oxrep/14.4.84.
Radaelli, C. (1995). The role of knowledge in the policy process. Journal of European Public Policy, 2, 159–183. doi:10.1080/13501769508406981.
Radaelli, C. (2004). The diffusion of regulatory impact analysis: Best practice or lesson-drawing? European Journal of Political Research, 43, 723–747. doi:10.1111/j.0304-4130.2004.00172.x.
Radaelli, C. (2005). Diffusion without convergence: How political context shapes the adoption of regulatory impact assessment. Journal of European Public Policy, 12, 924–943. doi:10.1080/13501760500161621.
Radaelli, C. (2007). Whither better regulation for the Lisbon Agenda? Journal of European Public Policy, 14, 190–207. doi:10.1080/13501760601122274.
Radaelli, C. M., & De Francesco, F. (2007). Regulatory quality in Europe: Concepts, measures and policy processes. Manchester: Manchester University Press.
Regeringskansliet, (2000). Ds 2000:1 Kommittéhandboken. Stockholm: Regeringskansliet.
Rittel, H., & Webber, M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4, 155–169. doi:10.1007/BF01405730.
Sabatier, P. A. (Ed.). (1999). Theories of the policy process. Boulder, CO: Westview Press.
Sabatier, P. A., & Jenkins-Smith, H. C. (1999). The advocacy coalition framework: An assessment. In P. A. Sabatier (Ed.), Theories of the policy process. Boulder, CO: Westview Press.
Sanderson, I. (2006). Complexity, “practical rationality” and evidence-based policy making. Policy and Politics, 34, 115–132. doi:10.1332/030557306775212188.
Shulock, N. (1999). The paradox of policy analysis: If it is not used, why do we produce so much of it? Journal of Policy Analysis and Management, 18, 226–244. doi:10.1002/(SICI)1520-6688(199921)18:2<226::AID-PAM2>3.0.CO;2-J.
Simon, H. (1958). Administrative behaviour. London: Macmillan.
Turnpenny, J., Nilsson, M., Russel, D., Jordan, A., Hertin, J., & Nykvist, B. (2008). Why is integrating policy assessment so hard? A comparative analysis of the institutional capacities and constraints. Journal of Environmental Planning and Management, 51 (in press).
Vibert, F. (2004). The EU’s new system of regulatory impact assessment: A scorecard. Brussels: The European Policy Forum.
Weale, A. (2001). Science advice, democratic responsiveness and public policy. Science & Public Policy, 28, 413–421. doi:10.3152/147154301781781237.
Weiss, C. (1975). Evaluation research in the political context. In E. L. Struening & M. Guttentag (Eds.), Handbook of evaluation research (Vol. 1). London: Sage.
Weiss, C. (1979). The many meanings of research utilization. Public Administration Review, 39, 426–431. doi:10.2307/3109916.
Wildavsky, A. B. (1979). Speaking truth to power: The art and craft of policy analysis. New Brunswick: Transaction.
Acknowledgments
The research was undertaken as part of the EC’s FP6-funded project MATISSE (Methods and Tools for Integrated Sustainability Assessment). The authors thank Anneke von Raggamby and Ingmar von Homeyer whose work on five of the EU cases was funded by the FP6 project Sustainability A-Test. Duncan Russel’s participation was funded by the UK ESRC. We thank the anonymous reviewers for their comments on an earlier version of this paper.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Nilsson, M., Jordan, A., Turnpenny, J. et al. The use and non-use of policy appraisal tools in public policy making: an analysis of three European countries and the European Union. Policy Sci 41, 335–355 (2008). https://doi.org/10.1007/s11077-008-9071-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11077-008-9071-1