Experimentation in policy design: insights from the building sector

Abstract

The current article questions how experimentation in policy design plays out in practice. In particular, it is interested in understanding how the content and process of policy-design experiments affect their outcomes. The article does so by building on an original study into 31 real-world examples of experimentation in policy design in the building sector in Australia, the Netherlands, and the United States. All examples aim to improve the environmental sustainability of the building sector. The article finds that these 31 examples have attracted moderate to substantial numbers of participants (policy outcome HO.i), but have not achieved substantial numbers of buildings built or retrofitted with high levels of sustainability (policy outcome HO.ii). By carefully unpacking these policy designs into a number of key characteristics, it finds that this mismatch between the two outcomes may partly be explained by flawed policy-design processes. The article concludes with the main lessons learnt and provides some suggestions on how to improve experimentation in policy design.

This is a preview of subscription content, log in to check access.

Notes

  1. 1.

    The relatively small size of the Netherlands is promising in attracting participants from various backgrounds and from all over the country to a mini-symposium.

  2. 2.

    The author applies QCA methodology, logic, and tools mainly to gain an insight into potential relevant patterns in the data collected that are difficult, if not impossible, to trace otherwise. These patterns are then used as a starting point for a more descriptive analysis of the data collected. The author is critical of too strong a reliance on QCA tools as it, eventually, forces researchers to break down their rich qualitative data into quantifiable units, which may work well for some qualitative studies or criteria, but surely not for all (Van der Heijden, 2009).

  3. 3.

    For the fsQCA analysis, these scores were translated as: “++” equals “1;” “+” equals “0.66;” “−”equals “0.33;” and “−−”equals “0” (cf. Ragin et al. 2006).

  4. 4.

    In line with the custom of qualitative social science, research interviewees provided me with their insights in confidence. As such, the identities of the interviewees (nor the cases studied) cannot be provided. To give the reader insight into the variance of the interviews, voice is given to them by referring to individual interviewees with a number (e.g., “int50”). Please note that some interviewees are referred to with numbers higher than 99 (the number of interviews used for this article). Please note that this article reports on a study that sits in a larger study, which addresses over 50 cases based on over 200 interviews—brief descriptions of all the cases in this larger study can be obtained from www.EnviroVoluntarism.info.

  5. 5.

    Even more, in many of the cases studied, interviewees were not aware of the other cases included in this study—even those that were implemented in, for instance, the same city.

References

  1. Abaire, J. (2008). Green buildings: What it means to be “Green” and the evolution of green building laws. The Urban Lawyer, 40(3), 623–632.

    Google Scholar 

  2. Baron, D. P., & Diermeier, D. (2007). Strategic activism and nonmarket strategy. Journal of Economics and Management Strategy, 16, 599–634.

    Article  Google Scholar 

  3. Bastianoni, S., Galli, A., Niccolucci, V., & Pulselli, R. M. (2006). The ecological footprint of building construction. Ecology and the Environment, 93, 345–356.

    Google Scholar 

  4. Beddoes, D., & Booth, C. (2012). Insights and perceptions of sustainable design and construction. In C. Booth, F. Hammond, J. Lamond, & D. Proverbs (Eds.), Solutions for climate change challenges in the built environment (pp. 127–140). Oxford: Wiley-Blackwell.

    Google Scholar 

  5. Biermann, F. (2008). Earth systems governance. In O. Young, L. King, & H. Schroeder (Eds.), Institutions and environmental change (pp. 277–301). Cambridge, MA: MIT Press.

    Google Scholar 

  6. Brown, M. A. (2001). Market failures and barriers as a basis for clean energy policies. Energy Policy, 29(14), 1197–1207.

    Article  Google Scholar 

  7. Campbell, D. (1969). Reforms as experiments. American Psychologist, 24(4), 409–429.

    Article  Google Scholar 

  8. Cashore, B., Auld, G., & Newsom, D. (2004). Governing through markets: Forest certification and the emergence of non-state authority. New Haven: Yale University Press.

    Google Scholar 

  9. Cialdini, R. (2009). Influence. Boston: Pearson.

    Google Scholar 

  10. Cooper, I., & Symes, M. (2009). Sustainable urban development (Vol. 4). Changing professional practice. London: Routledge.

  11. Croci, E. (2005). The handbook of environmental voluntary agreements. Dordrecht: Springer.

    Google Scholar 

  12. Darnall, N., & Sides, S. (2008). Assessing the performance of voluntary environmental programs: Does certification matter? Policy Studies Journal, 36(1), 95–117.

    Article  Google Scholar 

  13. De Burca, G. (2010). New governance and experimentalism: An introduction. Wisconsin Law Review, 227, 227–238.

    Google Scholar 

  14. De Búrca, G., & Scott, J. (2006). New governance and constitutionalism in Europe and the US. Oxford: Hart.

    Google Scholar 

  15. Dewey, J. (1991 [1927]). The public and its problems. Ohio: Swallow Press.

  16. Dunn, W. N. (2003). Public policy analysis an introduction. Harlow: Prentice Hall.

    Google Scholar 

  17. Eichholtz, P., Kok, N., & Quigley, J. (2010). Doing well by doing good? Green office buildings. American Economic Review, 100, 2492–2509.

    Article  Google Scholar 

  18. Evans, B., Joas, M., Sundback, S., & Thobald, K. (2005). Governing sustainable cities. London: Earthscan.

    Google Scholar 

  19. Fielding, N. G., & Fielding, J. L. (1986). Linking data. London: Sage.

    Google Scholar 

  20. Goertz, G., & Mahon, J. (2012). A tale of two cultures. Princeton: Princeton University Press.

    Google Scholar 

  21. Greenberg, D., & Schroder, M. (2003). The digest of social experiments (3rd ed.). Washington: Urban Institute Press.

    Google Scholar 

  22. Gunningham, N. (2009). The new collaborative governance: The localization of regulation. Journal of Law and Society, 36(1), 145–166.

    Article  Google Scholar 

  23. Héritier, A., & Eckert, S. (2008). New modes of governance in the shadow of hierarchy. Journal of Public Policy, 28(1), 113–138.

    Google Scholar 

  24. Heyes, A. G., & Maxwell, J. W. (2004). Private versus public regulation: Political economy of the international environment. Journal of Environmental Economics and Management, 48(2), 978–996.

    Article  Google Scholar 

  25. Hickson, K. (2009). The ABC of carbon. Toowong: ABC Carbon.

    Google Scholar 

  26. Hoffmann, M. (2011). Climate governance at the crossroads. Oxford: Oxford University Press.

    Google Scholar 

  27. IPCC. (2007). Contribution of working group III to the fourth assessment report of the intergovernmental panel on climate change. Cambridge: Cambridge University Press.

    Google Scholar 

  28. Jordan, A., Wurzel, R., & Zito, A. (2005). The rise of new policy instruments in comparative perspective: Has governance eclipsed government? Political Studies, 53(3), 477–496.

    Article  Google Scholar 

  29. Jordan, A., Wurzel, R., & Zito, A. (2010). ‘New’ instruments of environmental governance. Environmental Politics, 12(1), 1–24.

    Article  Google Scholar 

  30. Jordan, A., Wurzel, R., & Zito, A. (2013). Still the century of ‘new’ environmental policy instruments? Environmental Politics, 22(1), 155–173.

    Article  Google Scholar 

  31. Khanna, M., & Anton, W. R. Q. (2002). Corporate environmental management: Regulatory and market-based incentives. Land Economics, 78(4), 539–558.

    Article  Google Scholar 

  32. Lobel, O. (2004). The renew deal: The fall of regulation and the rise of governance in contemporary legal thought. Minesota Law Review, 89(2), 263–293.

    Google Scholar 

  33. Longhurst, R. (2003). Semi-structured interviews and focus groups. In N. J. Clifford & G. Valentine (Eds.), Key methods in geography (pp. 117–132). London: Sage Publishers.

    Google Scholar 

  34. Lyon, T. P., & Maxwell, J. W. (2007). Environmental public voluntary programs reconsidered. The Policy Studies Journal, 35(4), 723–750.

    Article  Google Scholar 

  35. Markusson, N., Atsushi, I., & Stephens, J. (2011). The social and political complexities of learning in carbon capture and storage demonstration projects. Global Environmental Change, 21(2), 293–302.

    Article  Google Scholar 

  36. Marsh, D., & McConnell, A. (2010). Towards a framework for establishing policy success. Public Administration, 88(2), 564–583.

    Article  Google Scholar 

  37. May, P. J. (1992). Policy learning or failure. Journal of Public Policy, 12(4), 331–354.

    Article  Google Scholar 

  38. McCracken, G. (1988). The long interview (Vol. 13). London: Sage.

    Google Scholar 

  39. Newman, P., Beatley, T., & Boyer, H. (2009). Resilient cities. Washington: Island Press.

    Google Scholar 

  40. Overdevest, C., & Zeitlin, J. (2012). Assembling an experimentalist regime: Transnational governance interactions in the forest sector. Regulation and Governance, 6(x), 1–29.

    Google Scholar 

  41. Payne, G., & Williams, M. (2005). Generalization in qualitative research. Sociology, 39(2), 295–314.

    Article  Google Scholar 

  42. Petts, J. (2007). Learning about learning: Lessons from public engagement and deliberation on urban river restoration. The Geographical Journal, 173(4), 300–311.

    Article  Google Scholar 

  43. Pivo, G. (2010). Owner-tenant engagement in sustainable property investing. The Journal of Sustainable Real Estate, 2(1), 183–199.

    Google Scholar 

  44. Potoski, M., & Prakash, A. (2009). Voluntary programs: A club theory perspective. Cambridge: MIT Press.

    Google Scholar 

  45. Ragin, C. C. (1987). The comparative method: Moving beyond qualitative and quantitative strategies. Berkeley: University of California Press.

    Google Scholar 

  46. Ragin, C. C. (2000). Fuzzy-set social science. Chicago: University of Chicago Press.

    Google Scholar 

  47. Ragin, C. C., Rubinson, C., Schaefer, D., Anderson, S., Williams, E., & Giesel, H. (2006). User’s guide to fuzzy-set/qualitative comparative analysis. Tuson: University of Arizona.

    Google Scholar 

  48. Register, R. (1987). Ecocity Berkeley: Building cities for a healthy future. Berkeley: North Atlantic Books.

    Google Scholar 

  49. Richards, D. (1996). Elite interviewing: Approaches and pitfalls. Politics, 16(3), 199–204.

    Article  Google Scholar 

  50. Rihoux, B., Rezsöhazy, I., & Bol, D. (2011). Qualitative comparative analysis in public policy analysis. German Policy Studies, 7(3), 9–82.

    Google Scholar 

  51. Rivera, J., & de Leon, P. (2004). Is greener whiter? Voluntary environmental performance of western ski areas. The Policy Studies Journal, 32(3), 417–437.

    Article  Google Scholar 

  52. Rose, R. (2001). Ten steps in learning lessons from abroad. Hull: Future Governance Programme.

    Google Scholar 

  53. Rose, J. (2011). Diverse perspectives on the groupthink theory. Emerging Leadership Journeys, 4(1), 37–57.

    Google Scholar 

  54. Sabel, C., & Zeitlin, J. (2011). Experimentalist governance. In D. Levi-Faur (Ed.), The Oxford handbook of governance (pp. 169–185). Oxford: Oxford University Press.

    Google Scholar 

  55. Sabel, C., & Zeitlin, J. (2012). Experimentalism in the EU. Regulation and Governance, 6(3), 410–426.

    Article  Google Scholar 

  56. Sanderson, I. (2002). Evaluation, policy learning and evidence-based policy making. Public Administration, 80(1), 1–22.

    Article  Google Scholar 

  57. Schneider, C., & Wagemann, C. (2010). Standards of good practice in qualitative comparative analysis (QCA) and fuzzy-sets. Comparative Sociology, 9(3), 397–418.

    Article  Google Scholar 

  58. Seale, C., Gobo, G., Gubrium, J. F., & Silverman, D. (2004). Qualitative research practice. London: Sage.

    Google Scholar 

  59. Silverman, D. (2001). Interpreting qualitative data (2nd ed.). London: Sage.

    Google Scholar 

  60. Solomon, J. (2008). Law and governance in the twenty-first century regulatory state. Texas Law Review, 86, 819–856.

    Google Scholar 

  61. Stone, D. A. (2002). Policy paradox. The art of political decision making (Revised Edition ed.). New York: Norton.

  62. Termeer, C. (2009). Barriers to new modes of horizontal governance. Public Management Review, 11(3), 299–316.

    Article  Google Scholar 

  63. Van der Heijden, J. (2009). Building Regulatory Enforcement Regimes. Comparative analysis of private sector involvement in the enforcement of public building regulations. Amsterdam: IOS Press.

  64. Van der Heijden, J. (2012). Voluntary environmental governance arrangements. Environmental Policies, 21(3), 486–509.

    Article  Google Scholar 

  65. Van der Heijden, J. (2013). Greening the building sector: Roles for building surveyors. Journal of Building Survey, Appraisal and Valuation, 2(1), 24–32.

    Google Scholar 

  66. Vreugdenhil, H., Taljaard, S., & Slinger, J. (2012). Pilot projects and their diffusion: A case study of integrated coastal management in South Africa. International Journal of Sustainable Development, 15(1/2), 148–172.

    Article  Google Scholar 

  67. Yudelson, J., & Meyer, U. (2013). The world’s greenest buildings. Abingdon: Routledge.

    Google Scholar 

  68. Zimmerman, A., & Martin, M. (2001). Post-occupancy evaluation: Benefits and barriers. Building Research and Information, 29(2), 168–174.

    Article  Google Scholar 

Download references

Acknowledgments

The research presented in this paper is funded through a VENI early career researchers grant by the Netherlands Organization for Scientific Research (451-11-05). I wish to thank my colleagues at RegNet, the participants of the Policy Design Workshop (NUS, February 28 and March 1, 2013, Singapore), and the six anonymous reviewers of this journal for helpful comments to earlier drafts of this article. Finally, I wish to thank all participants for giving me some of their time for our meetings and for sharing valuable insights into their experiences with experimentation in policy design.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Jeroen van der Heijden.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

van der Heijden, J. Experimentation in policy design: insights from the building sector. Policy Sci 47, 249–266 (2014). https://doi.org/10.1007/s11077-013-9184-z

Download citation

Keywords

  • Policy design
  • Policymaking
  • Policy evaluation
  • Experimentation
  • Policy learning