The current article questions how experimentation in policy design plays out in practice. In particular, it is interested in understanding how the content and process of policy-design experiments affect their outcomes. The article does so by building on an original study into 31 real-world examples of experimentation in policy design in the building sector in Australia, the Netherlands, and the United States. All examples aim to improve the environmental sustainability of the building sector. The article finds that these 31 examples have attracted moderate to substantial numbers of participants (policy outcome HO.i), but have not achieved substantial numbers of buildings built or retrofitted with high levels of sustainability (policy outcome HO.ii). By carefully unpacking these policy designs into a number of key characteristics, it finds that this mismatch between the two outcomes may partly be explained by flawed policy-design processes. The article concludes with the main lessons learnt and provides some suggestions on how to improve experimentation in policy design.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
The relatively small size of the Netherlands is promising in attracting participants from various backgrounds and from all over the country to a mini-symposium.
The author applies QCA methodology, logic, and tools mainly to gain an insight into potential relevant patterns in the data collected that are difficult, if not impossible, to trace otherwise. These patterns are then used as a starting point for a more descriptive analysis of the data collected. The author is critical of too strong a reliance on QCA tools as it, eventually, forces researchers to break down their rich qualitative data into quantifiable units, which may work well for some qualitative studies or criteria, but surely not for all (Van der Heijden, 2009).
For the fsQCA analysis, these scores were translated as: “++” equals “1;” “+” equals “0.66;” “−”equals “0.33;” and “−−”equals “0” (cf. Ragin et al. 2006).
In line with the custom of qualitative social science, research interviewees provided me with their insights in confidence. As such, the identities of the interviewees (nor the cases studied) cannot be provided. To give the reader insight into the variance of the interviews, voice is given to them by referring to individual interviewees with a number (e.g., “int50”). Please note that some interviewees are referred to with numbers higher than 99 (the number of interviews used for this article). Please note that this article reports on a study that sits in a larger study, which addresses over 50 cases based on over 200 interviews—brief descriptions of all the cases in this larger study can be obtained from www.EnviroVoluntarism.info.
Even more, in many of the cases studied, interviewees were not aware of the other cases included in this study—even those that were implemented in, for instance, the same city.
Abaire, J. (2008). Green buildings: What it means to be “Green” and the evolution of green building laws. The Urban Lawyer, 40(3), 623–632.
Baron, D. P., & Diermeier, D. (2007). Strategic activism and nonmarket strategy. Journal of Economics and Management Strategy, 16, 599–634.
Bastianoni, S., Galli, A., Niccolucci, V., & Pulselli, R. M. (2006). The ecological footprint of building construction. Ecology and the Environment, 93, 345–356.
Beddoes, D., & Booth, C. (2012). Insights and perceptions of sustainable design and construction. In C. Booth, F. Hammond, J. Lamond, & D. Proverbs (Eds.), Solutions for climate change challenges in the built environment (pp. 127–140). Oxford: Wiley-Blackwell.
Biermann, F. (2008). Earth systems governance. In O. Young, L. King, & H. Schroeder (Eds.), Institutions and environmental change (pp. 277–301). Cambridge, MA: MIT Press.
Brown, M. A. (2001). Market failures and barriers as a basis for clean energy policies. Energy Policy, 29(14), 1197–1207.
Campbell, D. (1969). Reforms as experiments. American Psychologist, 24(4), 409–429.
Cashore, B., Auld, G., & Newsom, D. (2004). Governing through markets: Forest certification and the emergence of non-state authority. New Haven: Yale University Press.
Cialdini, R. (2009). Influence. Boston: Pearson.
Cooper, I., & Symes, M. (2009). Sustainable urban development (Vol. 4). Changing professional practice. London: Routledge.
Croci, E. (2005). The handbook of environmental voluntary agreements. Dordrecht: Springer.
Darnall, N., & Sides, S. (2008). Assessing the performance of voluntary environmental programs: Does certification matter? Policy Studies Journal, 36(1), 95–117.
De Burca, G. (2010). New governance and experimentalism: An introduction. Wisconsin Law Review, 227, 227–238.
De Búrca, G., & Scott, J. (2006). New governance and constitutionalism in Europe and the US. Oxford: Hart.
Dewey, J. (1991 ). The public and its problems. Ohio: Swallow Press.
Dunn, W. N. (2003). Public policy analysis an introduction. Harlow: Prentice Hall.
Eichholtz, P., Kok, N., & Quigley, J. (2010). Doing well by doing good? Green office buildings. American Economic Review, 100, 2492–2509.
Evans, B., Joas, M., Sundback, S., & Thobald, K. (2005). Governing sustainable cities. London: Earthscan.
Fielding, N. G., & Fielding, J. L. (1986). Linking data. London: Sage.
Goertz, G., & Mahon, J. (2012). A tale of two cultures. Princeton: Princeton University Press.
Greenberg, D., & Schroder, M. (2003). The digest of social experiments (3rd ed.). Washington: Urban Institute Press.
Gunningham, N. (2009). The new collaborative governance: The localization of regulation. Journal of Law and Society, 36(1), 145–166.
Héritier, A., & Eckert, S. (2008). New modes of governance in the shadow of hierarchy. Journal of Public Policy, 28(1), 113–138.
Heyes, A. G., & Maxwell, J. W. (2004). Private versus public regulation: Political economy of the international environment. Journal of Environmental Economics and Management, 48(2), 978–996.
Hickson, K. (2009). The ABC of carbon. Toowong: ABC Carbon.
Hoffmann, M. (2011). Climate governance at the crossroads. Oxford: Oxford University Press.
IPCC. (2007). Contribution of working group III to the fourth assessment report of the intergovernmental panel on climate change. Cambridge: Cambridge University Press.
Jordan, A., Wurzel, R., & Zito, A. (2005). The rise of new policy instruments in comparative perspective: Has governance eclipsed government? Political Studies, 53(3), 477–496.
Jordan, A., Wurzel, R., & Zito, A. (2010). ‘New’ instruments of environmental governance. Environmental Politics, 12(1), 1–24.
Jordan, A., Wurzel, R., & Zito, A. (2013). Still the century of ‘new’ environmental policy instruments? Environmental Politics, 22(1), 155–173.
Khanna, M., & Anton, W. R. Q. (2002). Corporate environmental management: Regulatory and market-based incentives. Land Economics, 78(4), 539–558.
Lobel, O. (2004). The renew deal: The fall of regulation and the rise of governance in contemporary legal thought. Minesota Law Review, 89(2), 263–293.
Longhurst, R. (2003). Semi-structured interviews and focus groups. In N. J. Clifford & G. Valentine (Eds.), Key methods in geography (pp. 117–132). London: Sage Publishers.
Lyon, T. P., & Maxwell, J. W. (2007). Environmental public voluntary programs reconsidered. The Policy Studies Journal, 35(4), 723–750.
Markusson, N., Atsushi, I., & Stephens, J. (2011). The social and political complexities of learning in carbon capture and storage demonstration projects. Global Environmental Change, 21(2), 293–302.
Marsh, D., & McConnell, A. (2010). Towards a framework for establishing policy success. Public Administration, 88(2), 564–583.
May, P. J. (1992). Policy learning or failure. Journal of Public Policy, 12(4), 331–354.
McCracken, G. (1988). The long interview (Vol. 13). London: Sage.
Newman, P., Beatley, T., & Boyer, H. (2009). Resilient cities. Washington: Island Press.
Overdevest, C., & Zeitlin, J. (2012). Assembling an experimentalist regime: Transnational governance interactions in the forest sector. Regulation and Governance, 6(x), 1–29.
Payne, G., & Williams, M. (2005). Generalization in qualitative research. Sociology, 39(2), 295–314.
Petts, J. (2007). Learning about learning: Lessons from public engagement and deliberation on urban river restoration. The Geographical Journal, 173(4), 300–311.
Pivo, G. (2010). Owner-tenant engagement in sustainable property investing. The Journal of Sustainable Real Estate, 2(1), 183–199.
Potoski, M., & Prakash, A. (2009). Voluntary programs: A club theory perspective. Cambridge: MIT Press.
Ragin, C. C. (1987). The comparative method: Moving beyond qualitative and quantitative strategies. Berkeley: University of California Press.
Ragin, C. C. (2000). Fuzzy-set social science. Chicago: University of Chicago Press.
Ragin, C. C., Rubinson, C., Schaefer, D., Anderson, S., Williams, E., & Giesel, H. (2006). User’s guide to fuzzy-set/qualitative comparative analysis. Tuson: University of Arizona.
Register, R. (1987). Ecocity Berkeley: Building cities for a healthy future. Berkeley: North Atlantic Books.
Richards, D. (1996). Elite interviewing: Approaches and pitfalls. Politics, 16(3), 199–204.
Rihoux, B., Rezsöhazy, I., & Bol, D. (2011). Qualitative comparative analysis in public policy analysis. German Policy Studies, 7(3), 9–82.
Rivera, J., & de Leon, P. (2004). Is greener whiter? Voluntary environmental performance of western ski areas. The Policy Studies Journal, 32(3), 417–437.
Rose, R. (2001). Ten steps in learning lessons from abroad. Hull: Future Governance Programme.
Rose, J. (2011). Diverse perspectives on the groupthink theory. Emerging Leadership Journeys, 4(1), 37–57.
Sabel, C., & Zeitlin, J. (2011). Experimentalist governance. In D. Levi-Faur (Ed.), The Oxford handbook of governance (pp. 169–185). Oxford: Oxford University Press.
Sabel, C., & Zeitlin, J. (2012). Experimentalism in the EU. Regulation and Governance, 6(3), 410–426.
Sanderson, I. (2002). Evaluation, policy learning and evidence-based policy making. Public Administration, 80(1), 1–22.
Schneider, C., & Wagemann, C. (2010). Standards of good practice in qualitative comparative analysis (QCA) and fuzzy-sets. Comparative Sociology, 9(3), 397–418.
Seale, C., Gobo, G., Gubrium, J. F., & Silverman, D. (2004). Qualitative research practice. London: Sage.
Silverman, D. (2001). Interpreting qualitative data (2nd ed.). London: Sage.
Solomon, J. (2008). Law and governance in the twenty-first century regulatory state. Texas Law Review, 86, 819–856.
Stone, D. A. (2002). Policy paradox. The art of political decision making (Revised Edition ed.). New York: Norton.
Termeer, C. (2009). Barriers to new modes of horizontal governance. Public Management Review, 11(3), 299–316.
Van der Heijden, J. (2009). Building Regulatory Enforcement Regimes. Comparative analysis of private sector involvement in the enforcement of public building regulations. Amsterdam: IOS Press.
Van der Heijden, J. (2012). Voluntary environmental governance arrangements. Environmental Policies, 21(3), 486–509.
Van der Heijden, J. (2013). Greening the building sector: Roles for building surveyors. Journal of Building Survey, Appraisal and Valuation, 2(1), 24–32.
Vreugdenhil, H., Taljaard, S., & Slinger, J. (2012). Pilot projects and their diffusion: A case study of integrated coastal management in South Africa. International Journal of Sustainable Development, 15(1/2), 148–172.
Yudelson, J., & Meyer, U. (2013). The world’s greenest buildings. Abingdon: Routledge.
Zimmerman, A., & Martin, M. (2001). Post-occupancy evaluation: Benefits and barriers. Building Research and Information, 29(2), 168–174.
The research presented in this paper is funded through a VENI early career researchers grant by the Netherlands Organization for Scientific Research (451-11-05). I wish to thank my colleagues at RegNet, the participants of the Policy Design Workshop (NUS, February 28 and March 1, 2013, Singapore), and the six anonymous reviewers of this journal for helpful comments to earlier drafts of this article. Finally, I wish to thank all participants for giving me some of their time for our meetings and for sharing valuable insights into their experiences with experimentation in policy design.
About this article
Cite this article
van der Heijden, J. Experimentation in policy design: insights from the building sector. Policy Sci 47, 249–266 (2014). https://doi.org/10.1007/s11077-013-9184-z
- Policy design
- Policy evaluation
- Policy learning