Introduction

Cluster policies have featured prominently in the regional competitiveness policy toolkit ever since Porter (1990, 1998) packaged and popularised the concept of clusters as groups of inter-connected firms and related institutions in geographic proximity that give rise to a range of productivity, innovation and competitiveness advantages. The hypothesised benefits of clustering are associated in large part with the cooperative dynamics that emerge among the companies, universities, research centres, training organisations, governmental bodies and other actors that make up clusters. Over the years, this has given rise to significant academic discourse around the relationship between clusters, innovation and competitiveness, via knowledge spill-overs, technology transfer and the cooperative upgrading of business environments (see Breschi & Malerba, 2005; Chatterji et al., 2014; Pitelis et al., 2006; Fornahl & Hassink, 2017; Karlsson, 2008; Ketels et al., 2012 and Lazzeretti et al., 2019, among many others). In this sense, clusters are widely acknowledged as a type of activity-specific system, situated within broader regional innovation systems, where interaction between actors supports quicker diffusion and absorption of knowledge, more effective innovation and efficient solutions to a range of other localised drivers of competitiveness (such as skills and specialised infrastructure).

In this context, policies to support clusters take many shapes and sizes and are difficult to define precisely. Indeed, they are not always explicitly labelled as cluster policies, and they are sometimes driven by political agendas or imitative policy behaviour as opposed to following a clearly stated policy rationale. Nevertheless, they can be broadly defined as those policies designed to address system or network failures—i.e. missing or weak interaction between actors—within activity-specific clusters. They typically seek to strengthen productivity and competitiveness by supporting localised collaborative dynamics, knowledge exchange and mutual learning focused on common challenges facing the companies and other organisations within regional clusters. In their different shapes and sizes, cluster policies have become a permanent feature of the regional competitiveness policy mix and are increasingly leveraged to support the wider regional innovation system, for example through smart specialisation strategies (European Commission, 2013; Foray, 2019; Hassink & Gong, 2019; Saha et al., 2018), and as conduits for addressing social challenges (Alberti & Belfanti, 2019; Konstantynova & Wilson, 2017; OECD, 2016).

Given their popularity and longevity, there are surprisingly few recognised norms for guiding cluster policy evaluation in practice and significant demand among policymakers to address this gap. Yet research on the impacts of cluster policies is relatively scarce and fragmented, due to a combination of a complex policy effect logic, related methodological challenges and the diverse structural characteristics of different cluster policy programmes (Rothgang et al., 2019; Schmiedeberg, 2010; Uyarra & Ramlogan, 2016). The direct outcomes of most cluster policies have significant intangible elements, making it difficult to establish causal relationships with firm-level or regional impacts (which are simultaneously affected by other factors and policies). There is also great variety in cluster policies, rendering the generalisation of results and implications difficult and hampering a holistic overview of the complex patterns of cluster policy effects (Cantner et al., 2019). Finally, data issues complicate cluster policy evaluation research, prompting arguments for blending academic expertise and analysis with the real-time and evolving experience of practitioners that are closer to the most relevant data (Smith et al., 2020).

Cluster policy monitoring and evaluation capacities and practices are consequently extremely heterogenous across different cluster policy programmes, and each specific approach employed highlights different types of results. This both limits understanding of the full range of effects of cluster policies and makes them difficult to benchmark against. It also highlights an important research agenda in terms of establishing a generalised framework for understanding the effects of cluster policy, to promote better monitoring and evaluation practice across the range of different cluster policy contexts and enable learning between those implementing policy in different contexts. The primary contribution of this paper is to establish such a framework, based on analysis of existing literature and cases, combined with an iterative process undertaken to co-design and test a framework with groups of policymakers, practitioners and researchers working in different cluster policy contexts.

Section Two reviews literature on cluster policies and their impacts in the context of the broader regional innovation systems in which they are increasingly implemented. Section Three then reports on a multiyear process of co-creation with two groups of international policymakers, practitioners and researchers. The results of this process are presented in Section Four, in the form of a generalised framework of cluster policy effects that can be used to structure cluster policy evaluation in practice and provide a common focal point for policy learning. Finally, Section Five discusses conclusions and directions for future research.

Cluster policies and their impacts in contemporary context

Porter’s (1990, 1998) initial work on clusters highlighted reasons why clustering helps companies to operate more productively and to innovate, including: better access to employees and suppliers; access to specialised information; complementarities with other actors in the cluster; mutual learning among actors in the cluster; access to specialised institutions and public goods; better motivation and measurement; and the ability to experiment at lower cost and act rapidly.

Indeed, the benefits of clustering are well understood, and were captured in Glaeser et al.’s (1992) influential paper integrating the ideas of Marshall (1890), Arrow (1962) and Romer (1986) to explain the benefits from spatial concentration of firms in the same industry. Alongside Jacobs’ (1969) ideas on the benefits of diversity in urban agglomerations, this M-A-R model has prompted debate around the relative importance of agglomeration and diversification, more recently framed in terms of ‘related’ and ‘unrelated’ variety (Content & Frenken, 2016; Frenken et al., 2007).

Innovation externalities provide a major focus for the benefits of clustering (De Dominicis et al, 2012; Delgado et al., 2014; Gruenz, 2004), with Uyarra and Ramlogan’s (2016) review suggesting on balance that clustering has a positive effect on innovationFootnote 1. Other empirical studies find positive impacts of spatial agglomeration on productivity, especially among small firms (Cainelli et al., 2016; Martin et al., 2011a),Footnote 2 or on regional employment or economic growth (Delgado et al., 2014; Ketels & Protsiv, 2021; Spencer et al., 2010).

Cluster policies: a multidimensional tool in the regional competitiveness toolbox

While the process of clustering itself offers clear and well-acknowledged benefits, there is much less consensus around the suitability and effectiveness of cluster policies. Indeed, the fact that clusters develop everywhereFootnote 3, including in places where there is no cluster policy, raises questions around the circumstances and stages of cluster development (or lifecycles) under which cluster policies can add value to the naturally occurring benefits of clustering. In this regard, Martin and Sunley’s (2003) influential critique of Porter’s work highlighted the dangers of treating clusters as an economic development policy panacea, and the unique challenges of evaluating the effectiveness of cluster policies have since been well documented (Cantner et al., 2019; Rothgang et al., 2019; Schmiedeberg, 2010; Smith et al., 2020; Uyarra & Ramlogan, 2016).

However, to understand these challenges today it is important to consider changes in the ways in which cluster policies are employed since initial experimentation with Porter’s ideas during the 1990s, positioning contemporary cluster policy practice in the context of other developments in regional competitiveness policy thinking. In particular, recent decades have seen a rise in systemic approaches to regional economic development that are most clearly embodied in the vast literature around regional innovation systems and smart specialisation strategies (Aranguren et al., 2017; Asheim et al., 2019; Fagerberg, 2017; Foray, 2018), alongside broad acceptance of the importance of place-specific context for policy design and implementation (Todtling & Trippl, 2005; Coenen et al, 2016).

In this sense clusters are not isolated entities, but activity-specific microsystems that operate within wider regional system dynamics and whose development is mutually dependent with elements of the specific regional systems in which they are embedded (Ashiem et al., 2011; Rypestøl et al., 2021; Trippl et al., 2015). From a policy perspective, therefore, cluster policies form part of a wider regional policy mix (Flanagan et al, 2011) and must be understood in this context. While they interact in practice with other regional competitiveness policies that work in domains such as infrastructure development, skills, internationalisation or innovation, their distinctiveness comes from their targeting of collaborative dynamics within activity-specific microsystems in pursuit of enhancing (combinations of) outcomes in these areas. They typically do so through instruments such as facilitation of cluster-specific collaborative dynamics from a regional development agency, provision of support for formal cluster associations, or funding calls for cluster collaboration projects (Wilson, 2019), and in an era of smart specialisation strategies these instruments have taken on central roles in the entrepreneurial discovery dynamics in many regions (Hassink & Gong, 2019; Foray, 2019; Koschatsky et al., 2017; Perianez Forte & Wilson, 2021; Pugh, 2018).

Recognition of the interconnectedness of cluster policies with a wide range of other regional competitiveness policies, alongside the inherent heterogeneity in approaches to cluster policy within this wider context, has led to cluster policy being labelled as a ‘framework policy’ (European Commission, 2016: 11), an ‘umbrella policy’ (Uyarra & Ramlogan, 2016: 46) or a ‘family of policies’ (Wilson, 2019: 372). Combined with the definitional haziness surrounding cluster policy noted by Martin and Sunley (2003), it has also led to what Njøs et al. (2017) term a ‘stretching’ of the cluster concept in practice, whereby policy towards clusters effectively ‘means different things in different places’ (Uyarra & Ramlogan, 2016: 224–225). In this sense, cluster policies are today a multidimensional and multipurpose tool within a larger regional competitiveness policy toolkit, and their impacts will inevitably depend on both the characteristics of the broader regional innovation system in which they fit and the precise roles that they are designed to play alongside other complementary tools within that specific territorial context.

Assessing the impacts of cluster policies

The nature of the underling rationale of cluster policies in supporting collaborative dynamics means that the impacts of cluster policy in practice intersect with those of whole range of other competitiveness policies (related to innovation, skills, internationalisation, etc.) that may benefit from such collaborative dynamics. Thus, understanding the impacts of cluster policy must be seen in the context of the enormous heterogeneity among cluster policies in terms of their specific objectives, focus, scope, scale, instruments, timescale and intersections with other policies. This makes it extremely challenging to assess and benchmark evidence on the impacts of cluster policies.

As a case in point, Uyarra and Ramlogan (2016) review evidence on the evaluation of 17 cluster policy programmes. Amidst considerable heterogeneity, they found some commonalities in terms of evidence of the capacity of cluster policy programmes to mobilise resources and actors towards innovation, to provide a variety of business support services that boost productivity, and to generate collaborations that otherwise may not have taken place. However, they suggest overall that ‘there is no clear and unambiguous evidence that cluster policy is able to sustainably deliver innovation outcomes, improve levels of entrepreneurship and employment or boost firm productivity’ (p. 226).

With regards to the impacts of cluster programmes on firm-level productivity, studies have shown mixed results. For example, Martin et al. (2011b) found a negative impact on the total factor productivity of firms that received subsidies under France’s Local Production Systems policy. However, Abdesslem and Chiappini (2019) found that the more recent ‘competitiveness clusters’ policy had a positive impact on productivity in the optic/photonic industry. Aranguren et al. (2014) also tested for impacts on labour productivity from the Basque region’s cluster policy, finding ‘weak evidence that the cluster policy has had a positive impact’ (p. 1560).

Studies of the wider regional impacts of cluster policies on growth, exports or employment also exhibit mixed evidence. Doloreux et al. (2016) explore the effects of a series of policies focused on Québec’s coastal region and maritime cluster, finding ‘at best—a marginal impact on the local economy and on the particular sectors targeted’ (p. 208). In their study of the French ‘local production systems’ policy, Martin et al. (2011b) were also unable to identify a statistically significant impact from the policy on firm-level employment, and Fromhold-Eisebith and Eisebith (2008) found only minor aggregate effects on the regional economy in an empirical analysis of two automotive cluster initiatives (in Germany and Austria). Garone et al. (2014), however, did find evidence of positive impacts from Brazil’s cluster development policy on employment growth and export behaviour among SMEs. Moreover, in their review of case studies in South America, Maffioli et al. (2016) conclude that there is solid evidence of a positive impact from cluster development programmes on sales, employment and exports.

There have been several recent studies focused on the German ‘leading-edge cluster competition’. Lehmann and Menter (2017) evaluated treated clusters against a control group, finding a positive impact on regional GDP growth. Moreover, their results highlight the particularly important role of university knowledge and corresponding university-industry collaboration within clusters. This is in line with Audretsch et al.’s (2016) findings related to the impacts of the same policy on new ventures. They suggest that the cluster policy positively affected regional entrepreneurial activities, but that the impact was ‘rather low compared to the impact of local research-intensive universities and the innovative milieu on new venture creation’ (p. 357).Footnote 4

However, based on their experiences with the 5-year evaluation process that formally accompanied the German programme, Rothgang et al. (2019) reflect on the severe methodological challenges in evaluating such programmes, which are structurally complex and strongly linked with other elements of innovation policy systems. They highlight the need to pay more attention to issues of ‘emergence and non-linearity, uncertainty, and time patterns of the observed effects’ and suggest that ‘the task for evaluators consists in adjusting their methodological repertoire to the challenges of complexity’ (Ibid., p. 1673 and p. 1694). These arguments resonate variously with Diez’s (2002, p.298) observation that ‘the most significant effects of these policies are produced in the social, institutional and cultural spheres’, with Wolfe and Gertler’s (2004, p. 081) suggestion that the ‘growth and innovation dynamics of clusters can only be properly captured using qualitative research techniques’, and with Schiemedeberg’s (2010, p. 404) warning that ‘using only a single evaluation method will provide a very limited view on the cluster policy programme’.

Indeed, isolating economic impacts from social and institutional effects is extremely difficult, and a growing number of studies are looking to alternatives that directly analyse the impacts of policy on the social, behavioural and institutional dimensions of clusters. For example, Aragón et al. (2014a, 2014b) explore participatory evaluation approaches and measure social capital and network outcomes, while Etxabe (2018) uses Twitter data to measure social capital. Giuliani et al. (2016) and Calignano and Fitjar (2017) employ social network analysis to evaluate the relationship between cluster policy and inter-organisational networks, while the specific impacts of cluster policies on knowledge networks are the focus of analysis for Calignano et al. (2018), Lucena-Piquero and Vincente (2019) and Graf and Broekel (2020). Moreover, experience with evaluating Latin American cluster development programmes leads Maffioli et al., (2016: 197) to conclude that ‘the ways the networks of linkages develop are intimately related to results’.

Overall, evidence on the impacts of cluster policies can be characterised as mixed and fragmented, reflecting both the heterogeneity of these policies and their inherent relationship to their specific regional context and the other policies that co-exist in that context. What is missing is a common, coherent framework to structure the different potential effects and indicators relevant for cluster policies in a way that can support policymakers in their evaluation practice and facilitate targeted benchmarking. In this regard, Maffioli et al. (2016) have sought to structure different types of expected effects over time on the basis of their experiences in different Latin American contexts, and Rothgang et al. (2019) have distinguished between types of observed and non-observed effects in relation to the different target levels of the German cluster programme. The aim of this paper is to take further steps towards developing a generalised framework that combines different types of effects, experienced at different target levels and over time, that can be applied in a wide range of international contexts.

Co-creating a framework of cluster policy effects

For a generalised framework of cluster policy effects to be practical and useful among a wide range of policymakers, in a way that can facilitate comparative benchmarking and learning dynamics, the process through which it is developed is important. In this sense, the collision between theory and practice is pivotal for the effective evaluation of cluster policies (Smith et al., 2020), which points to the advantages of an interactive research approach (Svensson et al., 2007). Following Culver et al., (2015, pp. 205–206), such a process involves iterative, ongoing interaction and dialogue between relevant stakeholders, who all contribute towards a possible solution through a journey of inquiry ‘where direction, conduct and action are not predetermined, rather they are chosen through observation, reason and evidence, informed by feeling and sensitivity, as the journey progresses’. The chosen methodology thus draws on principles of co-design to bring together a diverse range of people with shared interest and collective motivation, supporting them to collaboratively address a complex challenge through stages of exploration, ideation and iteration (Cruickshank et al., 2013; Manzini, 2015; Norman & Verganti, 2014).

Co-creating a framework with diverse groups of policymakers requires carefully curated inputs to orient the discussion effectively. In this regard, while literature analysis identified a series of areas where cluster policies are typically expected to impact—innovation, productivity, employment, economic growth, exports, and various dimensions of firm behaviour—an additional step was taken to relate more directly with policymakers. Specifically, a detailed comparative analysis of six cluster policy programmes and their associated evaluations was carried out to identify specific effects and indicators characterising recent practice among policymakers themselves.

Given its purpose as an input to frame discussion among a diverse group of cluster policymakers, practitioners and academics with considerable own-context-specific knowledge and a collective interest in better understanding the possibilities for effective cluster evaluation, a heterogeneous selection of cases was targeted. The case selection sought regions or countries with similar core policy objectives—to support collaboration between cluster actors that improves innovation and competitiveness—but a diversity of scopes, specific targeted effects, and methodological approaches in their cluster programmesFootnote 5. For instance, Northern Ireland’s Collaborative Network Programme and Sweden’s Vinnväxt Programme focused on industrial renewal and emerging growth areas, whereas France’s Competitiveness Development Programme focused on existing national, large-scale innovation environments. In terms of evaluation methods, the Danish, French and Norwegian cases included the use of econometric analyses (comparing cluster firms’ performance against that of non-cluster firms), whereas the Swedish case included deep interviews with a variety of stakeholders to explore the system-level effects of long-term collaborative initiatives. For each of the six case studies, a detailed description was prepared, including: a short summary of the cluster programme, an overview of cluster evaluation activities, key results from the most recent programme-level evaluation, and learning points.

This comparative analysis revealed that evaluations were based on similar types of shorter-term effects across comparator regions in terms of firm-level collaboration (e.g. new contacts/partners, new collaborative projects) and innovation behaviour (e.g. collaboration that leads to knowledge development, patents, innovation, investment). All of the evaluations also showed concern with demonstrating longer-term (more indirect) effects in terms of firm-level economic performance (e.g. increased revenue, employment growth, productivity), and most additionally highlighted other longer-term effects related to increased competitiveness and international attractiveness (e.g. increased exports, international collaboration, start-up activity or visibility for both cluster firms and for the sectors or territories in which they are located). Finally, most cases included some other types of broader system-level effects (e.g. increased/more effective use of other innovation schemes, stronger connections with other industries and with policy actors, and strengthened work with environmental sustainability and equality). An overview of key results from across the six cases is included in Table 3 in the Appendix.

By combining the key effects revealed by this comparative analysis of recent cluster policy programme evaluations with dimensions reflected in the academic literature on cluster evaluation, a set of common cluster monitoring and evaluation elements and indicators was elaborated (see Table 1). This was used as an initial input to frame an iterative process of co-creating a generalised framework of effects with two different focus groups of cluster policymakers, practitioners and researchers.

Table 1 Typical elements and indicators in cluster programme monitoring and evaluation
Full size table

The co-creation workshops took place between 2018 and 2020 among two discrete focus groups. The first was TCI Network’s cluster evaluation working group, an open, international group of cluster policymakers, cluster practitioners and researchers that had been meeting regularly since 2013Footnote 6. Four workshops were organised with this group between May 2018 and April 2020 in Ireland, Canada, Sweden and virtually, with between 25 and 40 participants at each workshop. This group provided a broad-based triple helix perspective from a diverse range of national and regional contexts. The second was a group of cluster managers, cluster policymakers and action researchers within one specific cluster policy context, the Swedish Vinnväxt programmeFootnote 7. The framework discussions were introduced into their regular meetings between April 2018 and October 2019 to provide an alternative testing-ground for the framework development in the context of a specific applied policy programme context. The iterative process of co-creation between these two parallel focus groups enabled a progressive structuring, testing and adjusting of a generalised framework of effects for cluster policies.

Through the parallel series of workshops, employing a range of ideation and feedback exercises, the list of elements in Table 1 was gradually developed into a general framework of effects. The authors made notes on the discussion, reactions, ideas, and feedback provided by participants during the initial session, from which they evolved the list of elements into an initial proposed framework. The first articulation of a cluster policy framework of effects highlighted two distinctions: between the direct and indirect effects of cluster policy; and between types of effects felt by different actors (i.e. single actors or the collective group). This was presented for further discussion, feedback and adjustment at the next workshop, and so on, in an iterative process that moved back and forth across the two parallel dynamics.

While the resulting framework is presented in the next section, a couple of examples of key discussion points and refinements are provided here to illustrate the nature of the process. Significant debate focused on whether the effects of cluster policies were best described as ‘direct’ and ‘indirect’ or rather as ‘shorter-term’ and ‘longer-term’. This was partly to reflect that the longer-term systemic effects of cluster policies, whilst being contingent on other external factors (e.g. firm strategic decisions to increase workforce), are strongly influenced by cluster activity. Indirect was felt to suggest little connection and it was widely acknowledged that shorter-term effects can also feed into longer-term effects. Another key discussion point was around the levels at which effects manifested themselves—at the firm or organisational level, across the collaborative group, and/or beyond the cluster—reflecting awareness of the links between clusters themselves and wider regional innovation systems. This highlighted the importance of evidencing not only actor-level effects, but also adopting monitoring and evaluation practices capable of capturing developments in collaborative dynamics and interactions with the broader regional system.

A proposed cluster policy framework of effects

Based on the results from the iterative process set out in the previous section, a generalised framework of effects to be expected from cluster policies is presented here. The framework is structured around two dimensions: the level of effects and the timing of effects (see Fig. 1).

Fig. 1
figure 1

Cluster programme framework of effects

Full size image

Levels of effects

Research evaluating the success of cluster policy programmes has focused above all on the results of participating firms or actors. However, because cluster policies seek to address system (or coordination) failures and contribute to wider territorial economic development, the importance of combining this with results at other levels of analysis became evident in the co-creation process. This mirrors the recent turn in the literature towards analysing behavioural change and networking, as reflected for example in Alfaro Serrano et al. (2016) framework for analysing the effects of cluster development programmes in Latin America. Maintaining a focus on the results of firms, they distinguish four levels of effects among targeted firms: coordination and linkages; resource allocation and investments; business practices and technologies; and, ultimately, business performance. Rothman et al. (2019), on the other hand, move more explicitly beyond the firm as the central actor in their framework for analysing the German cluster policy, which distinguishes effects between the levels of project, actor (firm or research organisation), cluster organisation, cluster (region) and economy.

The practical experiences and inputs of policymakers in our process embraced the need to look beyond the firm in analysing the effects of cluster policies and led ultimately to a simpler distinction between three inter-related levels of effects (A/C/S). These essentially distinguish between where the effects are experienced, at the level of the individual actor (A), of the cluster initiative (or other collaborative group) (C) or of the territorial system (country, region, city, etc.) (S).

The individual actor level (A) encompasses companies, research institutes and other organisations that are participants of the cluster/collaborative initiative. These individual actors may commit themselves to the collaboration through formal mechanisms (e.g. letters of intent, membership fees) or through looser participation and engagement in collaborative activities. Their aim in engaging in collaborative initiatives is assumed to be to strengthen their own capacities and performance.

The cluster initiative level (C) encompasses the set of individual organisations that act together in a collaborative group, with a common purpose. The aim of acting in collaboration is assumed to be to achieve results that cannot be achieved by acting as individual organisations. Thus, it is important to understand how the strength (critical mass), dynamics (networking) and strategic direction of the collaborative initiative evolves over time.

Whereas the activities of a cluster initiative are predominantly focused on the needs and priorities of its members, they can also contribute to changes at the level of the broader territorial system (S). This is likely to occur most immediately among the natural cluster or agglomeration of related activities that exists beyond the members of the collaborative group, but cluster initiatives also connect with other related clusters in the region and are increasingly asked to contribute to regional improvement strategies, for example smart specialisation strategies, thus addressing industrial or societal challenges that extend beyond the cluster itself. This level therefore captures actions stimulated by the cluster policy that create spill-overs (or ripple effects) beyond the cluster and/or imply strategic leadership of other actors in the territorial system. In this sense it is important to evidence how cluster initiatives affect changes or contribute to results at the level of the broader territorial system.

Timing of effects

A second dimension that emerged during the co-creation process concerns the types of effects that would be expected from cluster policies over different time periods in their development. The temporal dimension of constructing the collaborative dynamics that underlie cluster policies is something that is recognised in the academic literature (Fornahl & Hassink, 2017), but often not well-reflected in applied analysis of specific policy programmes that typically have data constraints that make analysing effects over different time periods challenging (Schmiedeberg, 2010). Yet this dimension was acknowledged as critical in the iterative process with policymakers, who are acutely aware of the need to both demonstrate short-term results in the context of political cycles and to be sensitive to the long-term time horizon that is needed for collaborative dynamics to mature and generate wider impacts.

The resulting framework therefore distinguishes between: (i) short-term results (ST) that are experienced primarily by cluster participants 1–3 years after initiation and are more directly attributable to cluster activities; and (ii) long-term effects (LT) that are experienced by cluster participants and others in the broader territorial system as collaborative dynamics become consolidated after the first few years of activity, including more indirect spill-over effects.

Concretely, short-term results (ST) encompass behaviours, perceptions and concrete outputs that are experienced as a direct result of participating in the cluster initiative, during the first 1–3 years of the collaborative initiative. For individual actors (A), these changes include (perceived) increases in competencies, knowledge exchange and capacity to innovate as well as changes in behaviour (e.g. engaging in collaborative activities). On the level of the collaborative grouping (C), one would expect to see indications of (increased) engagement from a diverse group of actors,Footnote 8 new linkages (internally and externally), and the emergence of a shared view of the rationale and value of collective action. One would also expect to see a functioning governance for the collaborative initiative.

At the level of the territorial system (S), a cluster initiative’s activities may contribute to building system resources (knowledge development and dissemination, cluster-specific physical or digital infrastructure or attracting investment or new entrants into the sector) that help to improve the competitiveness and international attractiveness of the innovation ecosystem. In addition, as a cluster initiative matures and builds confidence, critical mass and capacity for collaborative action, it may adopt a strategic leadership role within the territory (or domain). These system leadership roles contribute to the increased effectiveness (of the structures, policies, and institutional arrangements) of the regional innovation system. However, while one may see initial steps towards such system-level effects in the short term, it generally takes more time before collaborative actions can affect or contribute to these more complex and indirect effects.

Long-term effects (LT) encompass changes to performance, as well as changes to behaviours, structures, policies, and institutional arrangements that need time to consolidate. Although there is no well-defined timeframe, the experiences discussed suggest that these longer-term effects may be observed after the first few years of a collaborative initiative. For individual actors (A), longer-term effects include strengthened economic performance, as well as more competitive strategies and behaviours. However, the collaborative dynamics supported by the cluster policy typically play a more indirect role in this long-term performance, influencing through a positive environment rather than being able to lay direct claim to causality. At the level of the collaborative grouping (C), on the other hand, one expects to see continued development over time of the types of effects that began in the short term (critical mass, new linkages, collective will), with their application to more complex challenges. This also includes continual improvements to the quality and professionalisation of the collaborative governance.

It is in the long term where the level of the territorial system (S) really comes into play. With more maturity, critical mass and capacity to drive more complex collaborative actions, the cluster initiative is able adopt a stronger leadership role within the territory or domain. They may act as the voice of industry, influence policy and strategy, amplify the reputation and position of the sector and/or territory and connect with other system leaders in new strategic partnerships. Through its (increasingly ambitious and system-targeted) activities, a cluster initiative could also make other higher-level contributions to broader priorities of importance to the territory, for example playing a leading role in the development of a regional smart specialisation strategy. This could also involve leading responses to key societal challenges, for example related to environmental impacts, issues to promote equality and diversity, or overall cultural vibrancy and well-being agendas.

Monitoring effects

During the process of co-creating the framework represented in Fig. 1 attention was also afforded to the types of concrete indicators that can be used to monitor development and evidence results over time. While the selection of indicators is dependent on the local context, the objectives of the cluster policy programme and the nature of the specific collaborative initiatives being supported, examples of the types of indicators were constructed for the different levels and timing of effects (Fig. 2).

Fig. 2
figure 2

Cluster programme framework of effects—example indicators

Full size image

For individual actors (A), indications of progress in the short term relate to changes in perceptions, capabilities and behaviours (e.g. strengthened knowledge exchange, capacity to innovate, or increased involvement in collaborative innovation), as well as to academic or commercial results (e.g. number of new patent applications, or number of new products, services or improved business processes). In the long term, these are expected to contribute to strengthened performance, using indicators of firm-level economic performance (e.g. growth in revenue, productivity, employment and export, or improved market share).

For the collaborative group/cluster initiative (C), indicators span the short and long term and are related to the evolution of collaborative strength and dynamics among participants in the cluster initiative (or the ‘collaborative journey’ of the cluster). These include changes to the composition of the collaborative initiative (e.g. number and types of actors involved) and changes to collaborative behaviours, capabilities and perceptions (e.g. type and volume of collaborative activities undertaken, depth of collaboration or perceived value of collective action)Footnote 9. A second type of indicator is also identified to capture the changing professionalisation and strategic orientation of the collaboration infrastructure (i.e. the cluster organisation). Here, for example, there are well-established sets of indicators used by the European Cluster Excellence Initiative (ECEI) benchmarking and quality labelling system.Footnote 10

The level of the territorial system (S) is the most challenging (and least explored in current evaluation approaches), where indicators should capture how the collaborative activities supported by the cluster policy contribute to the broader territorial system. These system-level effects include immediate spill-overs and strengthened system resources (e.g. knowledge development and dissemination, entrepreneurship, investment and physical infrastructure), as well as strategic system leadership effects (e.g. source of trusted industry intelligence, influencing policy and strategy, amplifying reputation and position, new strategic partnerships). They may also include contributions to higher-level system effects (e.g. climate action, social inclusion, health and well-being). Typically, changes on the level of the territorial system (involving and affecting not only cluster participants, but also surrounding stakeholders) are complex, context-dependent and long-term processes. As such there are no concrete indicators or measures suggested, but rather characteristics (resources, actors and institutions) and contributions to longer-term strategies of the regional innovation system that may be influenced and upgraded by the collective action of the cluster initiative. In particular, with the leveraging of clusters in the implementation of smart specialisation strategies there is increased attention to understanding and evidencing system-level effects but little concrete progress to date. An important research gap is identified, therefore, in terms of developing approaches capable of tracking and evidencing the contributions that cluster initiatives have on these longer-term system-level change processes.

Discussion and conclusions

The systemic character of clusters (spanning multiple industrial fields and policy areas) and the intangible nature of the results from policy interventions (higher degree of engagement, stronger social capital/trust, increased knowledge spill-overs, collaborative innovation activities, etc.) make the evaluation of cluster policies particularly challenging. Consequently, ‘evaluations of cluster policies are rare and often not very robust’ (OECD, 2015, p. 5). Cluster policies generate results that are difficult to measure quantitatively and may extend into non-economic spheres. In general, this calls for evaluation processes that are formative in nature, to facilitate reflection and continuous policy adaptation, as a complement to summative ex-post measurement of impacts. In practice this requires a smart and flexible combination of indicators and evaluation approaches, adaptable to capturing different types of effects in different contexts. This is particularly important given the heterogeneity of cluster policies and their inherent relationship with the specific regional policy mixes in which they fit, characteristics that have led to only a fragmented understanding of the benefits that cluster initiatives may deliver and limited possibilities for policymakers to benchmark and learn from and with others.

Recent literature on cluster evaluation has made significant advances, both in terms of moving beyond mere performance indicators to capture different types of effects related to behavioural change and networking (Aragón et al. 2014a, 2014b; Calignano & Fitjar, 2017; Calignano et al., 2018; Graf & Broekel, 2020; Lucena-Piquero and Vincente 2019; Maffioli et al., 2016; Smith et al., 2021) and in terms of developing frameworks that distinguish different types of effects for specific contexts (Alfaro Serrano et al., 2016; Rothgang et al., 2019). What remains elusive, however is a generalised framework that could enable policymakers and researchers working on cluster policy evaluation in different contexts to better connect their work and generate greater possibilities for benchmarking and policy learning.

The contribution of this paper is twofold. Firstly, it updates understanding of the effects of cluster policies to reflect their contemporary application as an important part of the regional competitiveness policy toolkit (or policy mix to upgrade/transform regional innovation systems). Secondly, it develops a generalised framework of cluster policy effects by combining literature and policy analysis with a co-creation process involving cluster policymakers working ‘on the ground’ in different contexts. This framework recognises clusters as activity-specific microsystems that operate within wider regional system dynamics and whose development is mutually dependent with elements of the specific regional systems in which they are embedded. As such, it highlights the importance combining different approaches and indicators according to specific cluster policy contexts.

The literature review and exploratory analysis of a set of cluster policy programme evaluations revealed different elements of direct, behavioural and indirect effects. The co-creation of a framework through an iterative series of workshops with two groups of policymakers further highlighted the distinction between effects experienced by participating firms and other actors (A), effects among the collaborative grouping or cluster initiative (C) and effects that extent to the broader territorial system (S). It also led to a distinction between the short-term (ST) and long-term (LT) effects of cluster policies. The overall result is a generalised framework of effects in two dimensions: the level of effects (A/C/S) and the timing of effects (ST/LT). Different types of effects are sought and expected in specific cluster policy contexts in the intersection of these two dimensions, and a suite of indicators that are typically used in practice to capture these effects can be positioned within the framework.

This framework of effects is presented as a starting point from which to design cluster monitoring and evaluation approaches and to compare them across different territorial contexts. Indeed, the framework can be used to tailor evaluation approaches to the context and strategies of each cluster policy, taking into consideration the scope of the evaluation (i.e. a broad cluster programme covering many initiatives, an individual cluster organisation, or particular cluster project), as well as the specific objectives/types of effects the programme aims to achieve and when, and its relationship with the broader regional competitiveness policy mix.

There are, of course, limitations to both the framework itself and the process through which it was generated. The review of literature and policy evaluations that provided initial inputs to the co-creation process was necessarily limited in scope, although it sought to reflect a wide range of existing approaches to evaluate cluster policies. Similarly, the outcomes of the co-creation process were dependent on the insights of the participants. In one of the focus groups these were self-selected from a large international network of cluster practitioners, while in the other group they comprised the full set of policy practitioners working in one specific policy context. The framework resulting from such a process is inevitably broad given the wide range of contexts that it incorporates, and a trade-off is that it lacks specificity, for example in terms of prescribing specific approaches and indicators relevant for a concrete policy context. The distinctions that it makes are also highly simplified, and it is important to also consider the intersections between the different levels and timings of effects (Fløysand et al, 2012).

Nevertheless, the experience of co-creating this framework with a wide range of policymakers suggests that making a series of explicit, albeit broad, distinctions within a common framework will facilitate policymakers in different contexts and at different stages of cluster policy development to relate to one-another in ways that can generate new learning. The practical benefits of relating different types of effects are particularly evident in their culmination at the system level. Here there are large challenges in terms of tracking the contribution of cluster policies (alongside other policies) to smart specialisation strategies, to the upgrading of regional innovation systems, and to territorial leadership.

Indeed, based on this generalised framework of effects for cluster policy programmes, there are many specific avenues for future research. Most existing research on cluster policy effects is focused on firm-level performance dimensions of innovation and productivity. There is growing attention being paid to studying the effects on collaborative dynamics, but still a long way to go in capturing the contributions clusters make in system-level change processes (the bottom quadrants of the framework). This is partly due to the lack of a common understanding of (and attention to) these levels of effects, and partly due to the inherent methodological challenges with evaluating more systemic effects, where direct causality is difficult to evidence and so contribution to change is a more appropriate evaluation approach. Further research is needed to elaborate the characteristics of (successful) collaborative dynamics, as well as the types of system-level effects to which cluster initiatives contribute. Further research is also needed to test alternative methods for data collection and analysis of collaborative and system-level effects that can improve on the indicators used in practice, particularly in the bottom-right quadrant.

By using the framework across different contexts and geographies it can help to structure and ‘set boundaries’ for evaluation, facilitate approaches that allow the aggregation of evaluation evidence from several cluster initiatives to a cluster programme/policy level and the transfer of learning from one cluster evaluation to another through the use of a common framing of indicators. In addition, the framework highlights the importance of collaborative dynamics as a key element of cluster evaluation and acknowledges the role that clusters play in influencing and contributing to system-level change within territories, for example through smart specialisation strategies and through the operationalisation of the United Nations sustainable development goals. By distinguishing these levels of effects, the framework can help strengthen understanding and communication of the range of benefits to which cluster programmes contribute through their support for collaborative dynamics and territorial leadership within activity-specific microsystems that themselves sit within broader regional innovation systems.