Demonstration projects: in pursuit of transformative change

Powerful national and international organisations (such as the European Union, the World Bank and the United Nations) spend billions of dollars annually on projects that seek to address enduring sustainability challenges, ranging from traffic congestion and energy efficiency to biodiversity loss and disaster risk reduction (Fay et al. 2019; Davies et al. 2021; Torrens and von Wirth 2021). These projects draw together universities, research centres, government agencies, businesses, and non-profits as collaborators, and typically provide support such as funding, expertise, and networking over finite project periods (usually from 2 to 5 years). The goal of these demonstration projects ‘(sometimes called ‘living labs’ or ‘real-world labs’) is usually to demonstrate and validate solutions to societal challenges (Bergmann et al. 2021). As demonstrations are spatially, financially and temporally limited, it is expected that these experiments will later be replicated to realise more substantial benefits (Wamsler et al. 2014; Peng et al. 2019; Cortinovis et al. 2022), although this replication must be understood as a careful translation of these initiatives into new contexts rather than simplistic copying (Schäpke et al. 2018). This process of (sensitive) replication is critical to drive ‘transformative change’, defined as ‘fundamental systems-wide change in the structure and functioning of a system’ (Ferguson et al. 2013, p. 1). For example, a project that successfully delivers a field of ultra-efficient solar panels may be impressive, but it is not impactful on challenges like climate change mitigation unless the demonstration is replicated many times and across various contexts, such that the cumulative impact of multiple projects is reflective of the changes needed.

While demonstration projects can deliver substantial short-term outcomes, their enduring impact is less clear and there is limited evidence that replication occurs, despite the expertise, funding, and diversity of skills involved in these projects (Suškevičs et al. 2018; Davies et al. 2021; Torrens and von Wirth 2021). The establishment of new knowledge, discourses and networks can all be valuable precursors to changes in governance, yet these activities alone are rarely sufficient to break institutional inertia and path dependency; any resulting change is generally incremental rather than transformative (Pahl-Wostl 2009; Arnouts et al. 2012; Suškevičs et al. 2018; King et al. 2023). As project funding concludes, old ways of implementing infrastructure and services often return. As the intervention and scrutiny of international financiers ends, bureaucracies that bent their rules to enable one-off demonstration projects may regress back to the status quo, reapplying familiar rules, processes and norms that often reinforce unsustainable decisions (Dijk et al. 2018; Olejniczak et al. 2020; Davies et al. 2021; Torrens and von Wirth 2021; Fuglsang and Hansen 2022). Showpieces rushed past normal processes and deliberations in the hope of inspiring replication therefore tend to remain one-offs, and ‘learnings’ are absorbed quietly into reports (Torrens and von Wirth 2021). Indeed, the literature highlights how the role of learning in experimental projects pursuing transformative change remains poorly understood (Van Poeck et al. 2020). Repeated experiences suggest that the creation of knowledge and experience within the officer levels of organisations may offer immediate value in the short term, but does not usually equate to ongoing (and eventually transformative) change (Suškevičs et al. 2018).

This paper offers insights into how funding agencies and project teams can design demonstration projects to better improve chances of replication, and thereby progress towards transformative change. Drawing on our experience as practitioners, and using a case study of an EU demonstration project, we contend that three key issues can be addressed in demonstration project design to achieve this outcome. First, we explain how action research can (and should) be much more than just a straightforward collaboration with researchers to gather data about project action. Second, we argue that a theory of change must be transparently stated and rigorously examined—in this case, project teams must convincingly articulate how change via extensive replication is expected to arise from demonstrations. Third, flowing from this requirement, we advocate that institutional change is pursued and tested as part of the project scope, and that projects include leaders that have the power to enact enduring institutional change. This logic is summarised in Fig. 1, and each focus area is explained (with relevant definitions) in greater detail below.

Fig. 1
figure 1

Summary of the argument presented in this paper. On the left, we sketch a generalised status quo of unsuccessful replication. On the right, we present more complete Action Research as a precursor to institutional change that supports replication and ultimately aids progress towards transformative change

Full size image

Action research can be much more than measurement

Being some decades old, action research is a broad family of research approaches rather than a single neat concept, but most broadly it involves a problem-oriented collaboration between practitioners and researchers to both diagnose an issue and also develop and implement solutions to it (McNiff and Whitehead 2006; Byrman 2008). While definitions of the approach are diverse and discipline-specific, action research (AR) is often posited to include three elements that we consider key to the value of this approach in demonstration projects.

Firstly, AR can introduce questions about what will be effective prior to the initiation of actions. These are often questions about what will work, and why it will work, and (crucially) how future actions might be improved (McNiff and Whitehead 2006). So, for example, a team working this way may determine that ‘we think trees will be effective at managing flooding at our project location’, but also ‘we think implementing trees at this location will be difficult under current civil engineering guidelines, so we’re going to trial a modified set of guidelines that we believe will support tree planting and manage engineering risks’.

This links to a second crucial aspect of AR—it can incorporate more interpretive approaches to research and learning, thereby enabling participants to look ‘inwards’ at the systems in which they work, rather than confining research to measurement of project outcomes (McNiff and Whitehead 2006). This additional perspective is very valuable, because the complex organisational and institutional settings that are typical to major demonstration project participants (e.g. government agencies, large corporations) are often barriers to replication (Suškevičs et al. 2018).

Lastly, AR is at its best when it not only integrates diverse skillsets as project partners ‘on paper’, but also brings them out of disciplinary or organisational silos. Practitioners, leaders and researchers should collaborate in both the diagnosis of the problem and the development of solutions based on this diagnosis (Byrman 2008; Cowling et al. 2008; Mell et al. 2022). This leverages the deep knowledge that many practitioners have of their contexts, while also adding the independent perspectives of researchers, without limiting their roles to detached observership (McNiff and Whitehead 2006; Byrman 2008).

The more reflective, collaborative action research we advocate lies close to the concept of Transdisciplinary Research, which also emphasises the inclusion of researchers as participants in problem-oriented, contextually aware collaborations with practitioners (Wickson et al. 2006). Indeed, the blend of both concepts—Transdisciplinary AR—remains a field of active development (Keahey 2021). For the improvement of demonstration projects, the critical elements of value we see in this family of research approaches are the three outlined above.

Theories of change are critical: they must be carefully articulated and examined

Underlying each demonstration project—successful or otherwise—is a theory of change. A theory of change is essentially a rationale for why particular outcomes can be expected (van Tulder and Keen 2018); some definitions add the requirement that this is openly stated, as a means of exposing assumptions (Reinholz and Andrews 2020). Articulation of expectations and visions is posited as a critical early step in urban experiments (Peng et al. 2019). However, in many cases a project’s theory of change is implicit in its design, rather than openly stated, and as such does not receive the same careful examination that is applied to budgets, timeframes and individual deliverables (Archibald et al. 2016). While subtle, an incorrect (or incomplete) theory of change can undermine a project’s chance of delivering lasting improvements (Archibald et al. 2016; Douthwaite and Hoffecker 2017).

Even when assumptions behind a theory of change remain unstated and unexamined, they still play a substantial role in shaping project design (Archibald et al. 2016). For example, project consortia often dedicate intense focus to measurement of demonstration projects, and the impact of these demonstrations in terms of metrics, case studies, and technical expertise. Communication activities are also a common emphasis, and activity to disseminate the acquired knowledge to the public and practitioners can include reports, workshops, and on-site demonstrations of services of infrastructure. However, experiential learning and compelling data rarely equate to lasting change (Suškevičs et al. 2018). The heavy reliance on metrics and case studies in project designs may reflect a tacit theory of change based on the widely-debunked ‘knowledge deficit model’: a persistent notion, particularly in scientific circles, that change is held back primarily by a lack of information (Simis et al. 2016). Project designers may assume that with enough evidence, documentation and demonstration sites, organisations will ‘see sense’ and adopt innovative practices smoothly—but this is rarely the case. Articulating an explicit theory of change allows these kinds of assumptions to be transparently examined and critiqued, and flaws more readily exposed. We contend that well-executed AR creates ideal conditions for formation (and iterative development) of a realistic theory of change by engaging practitioners and researchers in contextual observation and reflection well in advance of project actions (McNiff and Whitehead 2006).

Institutional change must be in scope, and powerful leaders must deliver it

An extensive literature highlights the critical role of institutions in enabling (or blocking) transformative change (Geels 2002; Avelino and Rotmans 2009; Dhakal and Chevalier 2017; Qiao et al. 2018; Peng et al. 2019; Croeser et al. 2021a, b; Davies et al. 2021). There is much more to delivering enduring innovation than writing an ambitious strategy, debuting a new service, or cutting the ribbons on a set of novel demonstration sites. The formal and informal rules and systems that deliver strategies, projects and services must change too, or they can act as significant obstructions (Cowling et al. 2008; Pahl-Wostl 2009). For systems-level change to occur, novel ideas, technologies, and lessons must be diffused across networks and transferred and contextualised in new places; this represents not only a need for on-ground replication of successful sustainability demonstrations, but also a revision of the settings in which replication is intended. For innovations to have enduring influence beyond the life of these urban experiments, this process needs to influence systems of governance, through a realignment of institutional arrangements, resources, and networks of actors (Peng et al. 2019). For simplicity, we refer hereafter to this suite of changes as ‘institutional change’.

Changing these alignments is a substantial task, both within and between organisations (Clement and Mell 2023). These are quite static systems, as holders of power tend to favour stability and avoid confrontation (Flyvbjerg 1998). Existing organisational routines are strongly self-reinforcing or ‘path dependent’; existing actors and rules are coordinated in allocating resources to deliver an existing output, rather than the desired innovation. The status quo enjoys legitimacy, clarity of roles in delivery, and a tailored set of rules and norms that actors have learned to act effectively within, often through considerable repetition. Risk, conflict and costs are low, and delivery is fast, particularly relative to unfamiliar new practices (Uittenbroek 2016). Institutions can provide stability and predictability to governance systems, but they are notoriously resistant to change, even in the face of compelling statistics, case studies or direct experiences that favour transformative change (Clement 2021).

It is vital that projects engage senior leaders as AR partners. These leaders need to be both motivated and collectively powerful enough to enact reforms even in complex, entrenched organisational contexts (Avelino and Rotmans 2009). Knowledge-sharing arrangements established around these projects can support the reflexive learning processes that help challenge norms and identify reform opportunities, and champions in project teams can be empowered by demonstration projects to chip away at informal rules and norms, expand intra-organisational networks and negotiate new projects of work (Wamsler 2015; King et al. 2023). However, the vital work of revising formal institutions and structures—changing regulations, organisational roles, and decision-making procedures—is a task that requires buy-in and action from powerful senior leaders (Cowling et al. 2008; Wamsler 2015). It is important to note that in a leadership context, ‘power’ must constitute more than just formal authority; mobilising support for new rules, norms and processes requires considerable trust from siloed holders of informal power in organisations (Cowling et al. 2008; Avelino and Rotmans 2009; Evans et al. 2015).

Project design as an opportunity to support transformative change: a case study

Urban GreenUP is an EU-funded project demonstrating interventions and building capacity to plan and deliver NBS in cities. The project delivered millions of euros of urban greening in three ‘frontrunner’ cities in Spain (Valladolid), Turkey (Izmir) and England (Liverpool) to plan, implement and test the benefits of NBS in cities. Outcomes include the planting of thousands of trees (Izmir, Liverpool), numerous large green walls (Liverpool, Valladolid), and a major restoration of a concrete canal back to a streambank (Izmir). Project delivery has been facilitated by the creation of a large network of local governments, technical consulting firms, non-profits, research centres and universities, ensuring that project teams in the frontrunner cities were supported as they carried out the planning, procurement, construction and monitoring of novel NBS. Four ‘follower’ cities in Vietnam, Colombia, Italy and Germany have been closely supported with technical guidance and knowledge exchange projects in planning to replicate these demonstration projects. A further network of over twenty additional cities was brought into the consortium to observe and learn from the delivery of the demonstration NBS via reports, webinars and site visits. The authors of this paper that participated in the project directly (TC, SC, IM, SB) are university-based researchers that played active roles in producing both pre-defined project content (plans, metrics, reports) as well as ad-hoc research pieces in the academic literature (e.g. O’Sullivan et al. 2020; Croeser et al. 2021a, b; Clement and Mell 2023); the reflections we offer arose from discussions between the authors over the years of the project, with conversations taking place at meetings, webinars, and during production of project reports and metrics.

Research approaches varied across the cities, with a strong focus on metrics and data collection by researchers. All the greening, data collection and reporting to be undertaken in the project was rigorously and rigidly defined prior to the project, and roles were firmly delineated between project participants. In some cities researchers were participants in each stage of delivering these tasks (e.g. co-development of indicators, active participatory design and delivery of NBS, and monitoring). Data assembled by these activities indicates substantial benefits for many NBS delivered by the project (for example, localised cooling and stormwater retention). Some project teams also engaged in critical reflection and captured ‘lessons learned’ in general terms, and knowledge about barriers were disseminated as part of the project. However, this was largely a descriptive, backward-looking exercise; many individuals built understanding of institutional barriers and prepared reports on these barriers as directed by the contract, but action to address these barriers was not formally within the scope of the project, and the project ‘s design required focus on a pipeline of prescribed deliverables.

As is common in these projects, the theory of change implicit in this project remained tacit and unexamined. The focus on metrics and descriptive reports suggests that there was an expectation that other cities would engage with communication materials, recognise the merit of these investments, and begin delivering NBS. Examination of organisational barriers within the project focused on simply identifying barriers in broad terms (‘social barriers’, legal barriers’ etc.) rather than using the project’s design to require serious critical evaluation of each city’s institutional context, or mandate at least some testing of institutional changes. Similarly, the potentially critical role of executive leaders in the project was not explicitly outlined in the project’s design, with much of the work of NBS delivery being led by project-funded staff at officer level (Croeser et al. 2021a). Nevertheless, project staff had some ad hoc success as policy entrepreneurs, informally renegotiating some processes and roles, but the extent of serious change to alignments of actors, resources or institutions was quite limited (Clement and Mell 2023).

It is evident that this project’s approach to AR could have been significantly expanded, thereby enabling leaders on the project to deliver not only a suite of nature-based solutions, but also a legacy of facilitative institutional changes. With GreenUP leaving a material legacy of NBS interventions, we consider the next frontier in the future design of these types of demonstration projects.

As coming rounds of demonstration project teams pursue transformative change, the three foci advocated in this paper can guide project design to maximise their success in not only producing examples, but also enabling vital large-scale replication. These principles are (1) a deep engagement in action research, (2) an explicitly-stated theory of change and (3) an inclusion of institutional change in the project scope.