1 Introduction

Effective policy design and implementation depend on the quality of institutions, which in turn is reflected in governance arrangements (Rodríguez-Pose 2020). The quality of governance, and more specifically, the capacity to design and implement policy interventions according to envisaged timeframes and budget allocations to achieve the expected results, cannot be taken for granted. Governance is a fundamental enabling condition for policy effectiveness (Meuleman 2015). Not taking it into account deprives policy impact evaluations of explanatory power and, ultimately, of value as tools to guide policy action in practice.

Governance, and more generally the institutional context in which policies are conceived and implemented, acts as a mediating factor in the relationship between ends and means, i.e. in the policy intervention logic. The reality of the implementation phase ought not to be ignored, as it is often done in ex-ante impact assessments (Coenen et al. 2012).

In the context of the European regional innovation policy called Smart Specialisation, evidence shows that seemingly well-designed policies were often not implemented as expected (Gianelle et al. 2020).Footnote 1 This is due to, amongst other things, unclear attribution of responsibilities and lack of political support in the implementation phase, ineffective inter-government coordination, weak interaction with (and engagement of) relevant stakeholders, and lack of adequate skills and resources in public administrations and other partners (Capello and Kroll 2016; Estensoro and Larrea 2016; Guzzo et al. 2018; Marques and Morgan 2018; Guzzo and Peiranez-Forte, 2019; and Guzzo and Gianelle 2021). With its demands on integrating various policy domains, actors, and territorial levels, Smart Specialisation poses burdensome requests in terms of governance, especially in countries and regions with weak institutional capacities and limited public–private relations. Di Cataldo et al. (2022) observe that territories with weak governance structures and quality of government tend to have strategies with a plethora of objectives and a lack of focus, which can undermine policy effectiveness. Also, Smart Specialisation can be challenging to implement even in top innovative and advanced regions with a tradition of good governance (Kristensen et al. 2022).

In this paper, we study the potential macroeconomic impact of the European innovation policy for Smart Specialisation governance. More specifically, we use original empirical data on the governance of the policy in a spatial macroeconomic modelling framework capable of gauging the general equilibrium effects of varying degrees of governance quality. This framework integrates a notion of the observed quality of policy processes, particularly the strategy design and the early implementation phases.

Our contribution narrows the gap between the abstraction of traditional ex-ante impact assessment exercises based on macroeconomic simulations and the often-bumpy reality of how policy interventions may take place. The objective is twofold. On the one hand, we respond to a real and pressing need in the context of multi-annual and complex policy programmes to provide the policymakers with well-timed impact scenarios that consider the factors influencing the policy's success. On the other hand, we address a fundamental shortcoming of most policy impact assessment approaches based on ex-ante simulations: the assumption that the policy will have a good design and will be implemented as planned. This is an arbitrary assumption often disproved by the facts and, therefore, is liable to invalidate the results (Tosun 2014).

Smart Specialisation has been defined as an “ambitious experiment” (Kuznetsov and Sabel 2017, p. 52). It is a policy implemented on a continental scale following a set of common rules and principles, the application of which is guaranteed by the development of specific territorial strategies (Capello and Lenzi 2013). The existence of these strategies was a legally binding requirement (ex-ante conditionality) for accessing cohesion policy funds for research and innovation in the 2014–2020 period (European Union 2013). There is an interest in evaluating this programming period which is coming to an end (the deployment of funds will last until 2023 due to the N + 3 rule) and just prior to the launch of the programmes of the next period.

Smart Specialisation represents an ideal case study for our research, as the implementation of its defining principles depends crucially on governance structures and processes. In particular, the following governance-related requirements and characteristics of the policy are relevant: the ability to carry out selective interventions functional to strategic priorities and pursue them over time (Gianelle et al. 2020); the effective management of broad stakeholder participation in the definition of those priorities through a search and discovery process (Foray 2015; Radosevic and Stancova 2018; Lepore and Spigarelli 2018; Ghinoi et al. 2021); and the operationalization of a monitoring system that ensures continuous feedback of information in the process of policy implementation (Marinelli et al. 2019).

In this paper, we combine two different methods of analysis by using survey data on the nature and quality of Smart Specialisation governance in a spatial general equilibrium model. In particular, we construct a synthetic indicator of the quality of Smart Specialisation governance using the responses to a survey targeted at regional and national administrations responsible for the Smart Specialisation strategies, with data for all the NUTS 2 regions of Italy. We then use the indicator as an input in a spatial dynamic general equilibrium model (based on Lecca et al. 2020, and Barbero et al. 2021) to simulate scenarios quantifying the economic consequences of various levels of governance quality. To the best of our knowledge, there is no available study yet incorporating the quality of governance of Smart Specialisation into policy impact assessments.

The paper is organised as follows. Section 2 presents the conceptual underpinnings of this study. Section 3 introduces the Smart Specialisation governance index. we constructed and the survey data on which it is based. Section 4 presents the modelling framework, and Sect. 5 contains the quantitative results of the analysis. Section 6 concludes.

2 Theoretical background

Policy success and failure depend on many different interrelated components. Some of these elements are internal to the policy context and generally refer to the policy objectives and paradigm and its underlying logic, formulation and implementation. Others are exogenous to the policy and relate to the broader political and governance setting in which the policy is implemented (Peters 2015).

Policy failure occurs in situations where good policy designs are not properly implemented; or where, even in the presence of a rigorous design and good execution, expected results are not achieved due to flaws in the policy paradigm (Howlett et al. 2015). Policies can also have an effect opposite to that intended. Failures also occur when unattainable agendas and goals are set or when policymakers fail to effectively evaluate policy processes and results or fail to learn from present and past policy interventions (Howlett et al. 2015; Hudson et al. 2019).

The broader context in which the policy takes place also matters for the latter’s failure or success. The best policy designs will not yield the expected results if the governance capacity is not conducive to success (Peters 2015). Implementation is highly dependent on the political and institutional context, particularly on the administrative and coordination capacity of bureaucracies, the mechanisms enabling participation and policy learning and the policy capacity of relevant actors. The capacity to engage and negotiate agreements with partners and to coordinate within and across government organisations, along with arrangements to promote multi-level and multi-actor policy making, is as crucial as the capacity to translate the contents of strategy documents into effective implementation procedures, instruments and results (Hudson et al. 2019; May 2015; Peters 2018; Wu et al. 2015).

Accordingly, a better assessment of the overall quality of governance can lead to better estimates of the policy effects compared to the current state of play based on policies assumed to realise their full potential. The assessment of the likely socio-economic impacts of public policies and reforms is an essential component of the policy cycle in the European Union (EU), and it also attracts the attention of academics and scholars. The so-called ex-ante impact assessments are carried out before the implementation of the policy. They are usually based on the assumption that the latter will be implemented smoothly and realise its full potential socio-economic impact (Petrov et al. 2017).

These assessments are necessary to guide strategic policy choices over multi-annual horizons. However, at the same time, they are often based on unrealistic assumptions about the realisation of the policy. For example, several economic models are routinely used for the assessment of European policies, with recent examples including the dynamic stochastic general equilibrium model QUEST used to evaluate the potential impact of the Resilience and Recovery Facility in the EU (Pfeiffer et al. 2021), and the regional computable general equilibrium RHOMOLO used to study the impact and spillovers of cohesion policy (Crucitti et al. 2022; Monfort and Salotti 2021).

In this paper, we relax the standard assumption of the aforementioned impact assessments of perfect policy design and implementation. Thus, we compare the potential impact of the policy assuming that the funds are used to the best of their potential, with the impact which is more reasonable to expect given the actual quality of Smart Specialisation governance arrangements experienced by the regions measured by the survey data at our disposal.

This empirical approach builds on the existing evidence showing how the quality of government (and of institutions in general) may affect economic growth in the EU regions (Rodríguez-Pose and Ketterer 2020), and more specifically on the evidence on how it could affect the administrative performance (Mendez and Bachtler 2022) and the economic returns of the European cohesion policy (Barbero et al. 2022; Rodríguez-Pose and Garcilazo 2015).

Governance structures and processes result from existing formal institutional settings (like the distribution of roles and responsibilities between different government levels), the bureaucratic organisation, administrative traditions and capacity, historical public–private interactions, shared norms and values, and the existence of informal networks and participatory processes. These elements are context-specific, so the resulting governance arrangements vary across countries and territories. Given these differences, it is neither possible nor advisable to define a unique model of Smart Specialisation governance and innovation policy in general that could be universally applied to every region or country (Guzzo and Gianelle 2021). Nonetheless, it is possible to identify two complementary institutional pillars, which are inherent constituents of the Smart Specialisation approach and have general validity across different territories. These two pillars refer to the policy’s management component and inclusiveness.

The Smart Specialisation management authorities oversee the strategy and should guarantee coordination within and across public administrations and agencies to keep the focus on the multi-annual financial horizon. These authorities should have the necessary autonomy and organisational and analytical capacities to turn the “on paper” strategy into actual interventions. They should also have the capacity and authority to coordinate the activities of the multiple actors, administrative entities, and government levels involved in the strategy, often at different spatial scales. As a second pillar, the Smart Specialisation governance requires establishing rules, mechanisms and practices to guarantee the inclusion and actual participation of research and innovation actors and the private sector in the policy process (Foray 2015).

The identification of the specific policy intervention areas should result from an interactive process between policy-makers and the private sector, the so-called entrepreneurial discovery process, which allows to explore and evaluate socio-economic needs, potential benefits, and risks (Content et al. 2022; and Foray 2015). It is therefore fundamental for the relevant stakeholders (higher education and research organisations, businesses, and the civil society) to be involved in the decisions regarding the development and deployment of the strategy, being positively engaged throughout the policy cycle and adequately represented in the formal governance structure.

Finally, a crucial enabling factor for effective strategy management and stakeholder involvement is the presence of adequate skills and resources. This is important for the public administration to design and implement policy measures aligned with the strategy’s aims, reach out the relevant innovation actors, monitor policy development and delivery to timely steer interventions towards expectations, and, more in general, support policy learning. Likewise, the stakeholder groups potentially involved in the strategy process should possess the capacity to guarantee effective contribution to policy processes and long-term commitment to the strategy. The next Section explains the data measuring these aspects of governance using survey data.

3 Measuring Smart Specialisation governance

3.1 The dimensions of Smart Specialisation governance

Our first goal is to build a single, empirically grounded measure of the quality of Smart Specialisation governance to be used in numerical simulations of policy scenarios. To this aim, we follow the characterisation proposed in the previous section and assess each of the pillars of Smart Specialisation governance according to a series of elements that can be used empirically. For each of the selected elements, we collect primary information through a policy maker’s survey targeted at the authorities responsible for the Smart Specialisation strategies in different EU territories (Guzzo et al. 2018).Footnote 2

We define as the management pillar the governance dimension concerned with strategy management along the following six dimensions: (i) existence and effectiveness of a body responsible for setting and revising strategic objectives and priorities (definition and update of the logic of intervention); (ii) existence and effectiveness of a body responsible for the development or deployment of policy instruments (implementation); (iii) existence and effectiveness of a body responsible for coordinating the different governance functions and actors; (iv) adequacy of funding for staff recruitment and training; (v) adequacy of the competences in the area of project planning and implementation; (vi) adequacy of the competences in the area of monitoring.

We define as the inclusiveness pillar the governance dimension concerned with stakeholder involvement and participation using the following six elements: (i) level of stakeholder contribution to the analysis of the national/regional context and potential for innovation; (ii) level of representation of the business, research, and education sectors in the governance system as a whole; (iii) presence of both business and research sector representatives in the group responsible for strategic management; (iv) commitment of the relevant institutional stakeholders in establishing the strategy management team; (v) adequacy of stakeholder engagement overall; (vi) adequacy of skills and capabilities in stakeholder groups. Table 1 summarises the composition of the two Smart Specialisation governance pillars.

Table 1 Composition of the two pillars of Smart Specialisation governance
Full size table

3.2 The survey data

In 2018, the European Commission took a first systematic stock of the state of play of the Smart Specialisation policy experience. This exercise was mainly supported by a survey collecting primary information on the development of Smart Specialisation strategies in EU regions and countries (Guzzo et al. 2018; Marinelli et al. 2019). The objective of the survey was twofold: identifying areas where the adoption of Smart Specialisation triggered a relevant improvement in policy-making practice, as well as understanding emerging critical issues and challenges throughout the policy cycle and drawing lessons and recommendations to feed the debate on the post-2020 cohesion policy.

The survey was sent to all the official Smart Specialisation contact points designated by the EU regions and Member States registered in the European Commission’s Smart Specialisation Platform. These contact points represent regional and national bodies or organisations controlled by (or working for) the public administration, with a direct and primary role in the design and implementation of Smart Specialisation strategies in their respective territories.Footnote 3 They represent the main interlocutors of the European Commission with regard to the development of Smart Specialisation strategies, and they are also a reference for other similarly placed administrations and organisations wishing to establish a form of collaboration. Most of the administrative bodies identified, for instance, coincide with the managing authorities of the European Regional Development Fund (ERDF).

The target population of the survey was constituted by more than 170 regional bodies and 18 national bodies, reflecting the membership of the Smart Specialisation platform at the end of 2017. In order to avoid self-reporting bias and attain a more complete and accurate representation of the Smart Specialisation experience, the survey was designed to gather a single response for each territory. To this end, the contact points receiving the survey request were explicitly instructed to consult and coordinate with different stakeholder groups, individuals, and bodies, and ultimately return a single survey for each territory which ought to be agreed upon by all the relevant actors involved in the strategy governance.

Respondent anonymity was ensured throughout the data collection and analysis process. The survey used multiple-choice questions, mainly aimed at gathering factual information (e.g. regarding the existence and operational status of a body with specific functions or the presence of some specific barriers to policy development) to limit the respondent bias inherent in survey data. The study was part of an open-ended research agenda to develop more precise and objective measures of governance that is based on different sources of information and therefore are less prone to potential bias.

The survey has the indisputable advantage of providing data on the quality of governance that is specific to the policy under scrutiny here, compared with the use of more general surveys and indicators, such as, for example, the European Quality of Government Index (Charron 2021).Footnote 4 By April 2018, 71 valid responses had been collected, referring to 4 national and 67 regional Smart Specialisation strategies. This corresponds to a response rate of about 38% (Guzzo et al. 2018). More in detail, the survey provided full geographical coverage for Italy at the regional level, with 21 responses, one for each of the Italian NUTS2 regions. Given the importance of Italy as a recipient of cohesion policy funds, the spread of its regions along the development scale, and the availability of complete territorial information, we selected it as the case study for our analysis of Smart Specialisation governance and its macroeconomic effects.

Responses were also collected for all but one region in Portugal (6) and Romania (7), half of the regions in Poland (8) and Sweden (4), one third of the Spanish regions (5), and three regions each in Austria, Finland, and France. Lower numbers of responses arrived for the Czech Republic, Greece, Germany, and the Netherlands. The four national strategies for which responses were collected were those of Portugal, Malta, Cyprus, and Bulgaria.

Thus, we selected Italy as the case study for our analysis of Smart Specialisation governance and its macroeconomic effects, based on the following reasons: i) the availability of complete territorial information for all 21 NUTS2 regions, allowing us to fully capture the national dimension of the inter-regional spillovers of the investments channelled through Smart Specialisation; ii) the importance of the country as a recipient of cohesion policy funds (second only to Poland); and iii) the within country diversity in terms of regional development, allowing for the inclusion in the analysis of strategies of both more and less developed regions.

3.3 The quality of Smart Specialisation governance indicator

We introduce here the Quality of Smart Specialisation Governance (QS3GOV) index constructed as a composite indicator that aggregates into a single numerical value the survey results corresponding to the elements and pillars defined in Table 1, where each element is represented by a dichotomic variable (assuming 0/1 values). The two-pillar structure of Smart Specialisation governance allows experimenting with different hypotheses on the aggregation rule when computing the overall indicator of governance quality.

A simple version of the indicator can be constructed by aggregating the scores of all twelve elements in the two pillars using an unweighted arithmetic mean. We denote it as QS3GOV1:

$${{\text {QS3GOV}}}_{1}=\sum_{\theta }\sum_{i}{x}_{pi}$$

θ denotes the two pillars, i denotes the individual elements assessed through the survey, and xpi is a dichotomic variable encoding each specific element. As a result, the QS3GOV1 index can assume integer values in the interval [0, 12]. This purely additive version of the index assumes perfect substitutability between any of the twelve dimensions comprised in the indicator. This means that the elements comprised in the two pillars can compensate for each other. Therefore, a low score obtained, for instance, on a management characteristic, can be offset by a high score obtained on an inclusiveness characteristic, and vice versa. In other words, QS3GOV1 disregards the bipartite structure of Smart Specialisation governance; we described above the complementary nature of the management and inclusiveness pillars and the need for their simultaneous effectiveness.

The objective of QS3GOV1 is not to provide a measurement of governance quality that is the closest possible to our theoretical framework, but rather to provide a relatively low order approximation against which to compare the simulation results of a superior measure that we denote QS3GOV2, where we assume only partial compensability between the two pillars (i.e. lower scores in one of the two cannot be fully offset by higher scores in the other). The QS3GOV2 index is constructed in two steps. First, we aggregate the scores of each dimension within a pillar using an unweighted arithmetic mean; then, we calculate the product of the scores of the two pillars to obtain the final index that can assume integer values in the interval [0, 36]:

$${{\text {QS3GOV}}}_{2}=\sum_{i}{x}_{\theta =1,i}\sum_{i}{x}_{\theta =2,i}$$

Notably, when using QS3GOV2, high overall scores can only be achieved if a high score is obtained in both pillars, management, and inclusiveness. In contrast, a low score in one pillar would result in a low overall score, no matter how high the score is in the other pillar. In the extreme case of a null value in one pillar, the entire index would take the value zero. This version of the indicator incorporates the idea of complementarity between the two pillars of Smart Specialisation governance: good management and proper stakeholder inclusion are necessary conditions, but neither of the two alone is sufficient for good Smart Specialisation governance.

This indicator is closer to the theoretical framework that we introduced earlier. However, it is also more demanding and restrictive, since single positive elements are no longer sufficient to guarantee a good score: all the elements constituting the specific framework of Smart Specialisation governance must be effectively implemented simultaneously.

Due to the confidentiality of the survey responses, we cannot show the exact regional distribution of the QS3GOV indicators that we constructed. We can, however, describe them in aggregate terms. The quality of governance indicators resulting from the Italian data used in this paper does not seem to be a mere reflection of the overall degree of development of the regions, and there is no clear correlation with the amount of EU funds received by the regions. The Pearson correlation between the indicators and regional GDP per capita in 2014, expressed in purchasing power standard, is significant but moderate, at 0.28 for QS3GOV1 and 0.33 for QS3GOV2. This points to the fact that the overall level of development of the regional socio-economic system and its institutions may not be a crucial factor explaining the quality of the Smart Specialisation governance. The correlation between the EU funds allocations and the quality of governance indicators is even lower, -0.11 for QS3GOV1 and − 0.09 for QS3GOV2, and not statistically significant, revealing that the amount of funds available may contribute only to a minor extent to the observed variation in the governance quality of Smart Specialisation strategies.

Finally, the correlation with the European Quality of Government Index mentioned above (Charron et al. 2019; Charron 2021) is 0.06 for QS3GOV1 and 0.15 for QS3GOV2, and neither is statistically significant. This absence of correlation reinforces the importance of using data on the quality of governance specific to the policy under scrutiny rather than a broad multi-dimensional measure based on European citizens’ perceptions about corruption and public services. Overall, we believe the QS3GOV indicators introduced here add valuable information to our understanding of how the policy processes unfold in the real world.

4 Quantifying the economic impact of governance

In this section, we take a well-established model used routinely to assess the impact of EU policies and employ the information extracted from the survey above to design simulations capable of quantifying the economic consequences of varying quality of innovation policy governance.

4.1 The general equilibrium model

We use a spatial dynamic general equilibrium model calibrated with data for all the NUTS 2 regions of the EU. The model is routinely used for the impact assessment of EU policies related to territorial cohesion (Crucitti et al. 2022; Di Comite et al. 2018), research and innovation (Christensen 2022), and the labour market (Sakkas 2018). The complete mathematical representation of the model can be found in Lecca et al. (2018, 2020), and we report here the details of the features directly related to the scenario constructed to analyse the effect of Smart Specialisation governance, namely the production function and private investments.

Smart Specialisation strategies focus on regional research and innovation policies. Therefore, the related investments are modelled using the following transmission mechanisms concerning private investments and capital stock accumulation. Additionally, a crucial point of the analysis is that there may be supply side effects materialising through increased total factor productivity (TFP). The key hypothesis for the analysis is that the magnitude of these productivity effects depends on the governance of the policy.

In each sector j and region r, total production \({Z}_{r,j}\) is a constant elasticity of substitution (CES) combination of the value added \({Y}_{r,j}\) and intermediate inputs \({V}_{r,j}:\)

$${Z}_{r,j}={Ax}_{r,j}{\left[{\delta }_{r,j}^{x}{\cdot V}_{r,j}^{{\rho }_{j}^{x}}+\left(1-{\delta }_{r,j}^{X}\right){\cdot Y}_{r,j}^{{\rho }_{j}^{x}}\right]}^{\frac{1}{{\rho }_{j}^{x}}}$$
(1)

where \({\delta }_{r,j}^{x}\) is the share of intermediate inputs in sector j for region r in total production. \({Ax}_{r,j}\) is a scale parameter, and \({\rho }_{j}^{x}\) is the elasticity parameter obtained from the elasticity of substitution \({\sigma }^{x}\), according to \({\rho }_{j}^{x}=\frac{{\sigma }^{x}-1}{{\sigma }^{x}}\).

\({Y}_{r,j}\) is defined in Eq. (2):

$${Y}_{r,j}={Ay}_{r,j}\left[{\left({K}_{(g)}^{d}\right)}^{\xi }{\left[{\delta }_{r,j}^{Y}{\cdot KD}_{r,j}^{{\rho }_{j}^{y}}+\left(1-{\delta }_{r,j}^{y}\right){\cdot LD}_{r,j}^{{\rho }_{j}^{y}}\right]}^{\frac{1}{{\rho }_{j}^{y}}}\right]-{FC}_{r,j}$$
(2)

\({Y}_{r,j}\), is obtained combining private capital \({KD}_{r,j}\) and employment \({LD}_{r,j}\) in a CES function, net of fixed costs \({FC}_{r,j}\). The scale parameter \({Ay}_{r,j}\) represents the conventional Hicks neutral technical change (TFP) parameter in this production function (in which the elasticity of substitution between capital and labour is set at 0.4).

As for investments, the optimal path of private investments IP is consistent with the neoclassical firm's profit maximisation theory and defined as in Uzawa (1969):

$${I}_{i,r}^{P}={\delta }_{r}{K}_{i,r}^{P}{\left(\frac{{rk}_{i,r}}{{uck}_{r}}\right)}^{v}$$
(3)

v is the accelerator parameter and \(\delta\) is the depreciation rate. Thus, the investment capital ratio (\({\varphi =I}_{r}^{P}\)/\({K}_{r}^{P}\)) is a function of the rate of return to capital (\(rk\)) and the user cost of capital (\(uck\)), allowing the capital stock to reach its desired level smoothly over time, where: \(\frac{\partial \varphi }{\partial rk}>0; \frac{\partial \varphi }{\partial uck}<0\).

The user cost of capital, uck, is derived from Hall and Jorgenson (1967) as a no arbitrage condition, where:

$${\mathrm{uck}}_{r}=(r+{\delta }_{r}){p}_{EU}^{I}+{\Delta p}_{EU}^{I}+{rp}_{r}$$
(4)

r, \({\delta }_{r}\), \({p}_{EU}^{I},\) and \({rp}_{r}\) denote the interest rate, the depreciation rate, the EU investment price index, and an exogenous risk premium, respectively. \({\Delta p}_{\mathrm{EU}}^{I}\) is the change of the investment price index defined between two subsequent periods.

Combining Eqs. (4) and (2), the desired level of capital \({K}_{j,r}^{*}\) is as follows:

$${K}_{j,r}^{*}={N}_{r,j}{\left({\left({\left({K}_{(g)}^{d}\right)}^{\xi }{Ay}_{r,j}\right)}^{{\rho }_{j}^{y}}{\cdot \delta }_{r,j}^{y}\cdot \frac{{uck}_{r,j}}{{Py}_{r,j}}\right)}^{\frac{1}{1-{\rho }_{j}^{y}}}{\cdot Y}_{r,j}$$
(5)

The gap between the desired level of capital and the actual level of capital determines the expected profit in the economy and drives investment in a given period (governed by the differences between uck and rk)\(.\)

The interest and depreciation rates are fixed and equal for all regions (4% and 15%, respectively), whilst the risk premium is a region-specific fixed calibrated parameter. Thus, changes in \(uck\) are only driven by changes in the cost of capital in the whole EU, \({p}_{\mathrm{EU}}^{I}\). This is given as the price index over the sectoral consumer prices, \({P}_{r,i}\), weighted by the capital matrix KM:

$${p}_{\mathrm{EU}}^{I}=\frac{\sum_{i,j,r}{KM}_{i,j, r}{P}_{r,i}}{\sum_{i,j,r}{KM}_{i,j, r}{\overline{P} }_{r,i}}$$
(6)

As in Eq. (3), the allocation of investments between regions is driven by the differences between regional and EU average returns, thus resulting in full capital mobility across regions. In the long run, the capital returns will be the same in all regions.

The private capital stock in each region evolves due to new investments, adjusted by depreciation:

$${\Delta K}_{j,r}^{P}={I}_{j,r}^{P}-{-\delta }_{r}{K}_{j,r}^{P}$$
(7)

The demand for investments \({I}_{j,r}^{P}\) in sector j is translated to the production of investment goods produced by sectors i, \({I}_{j,r}^{S}\), through the capital matrixes \({\mathrm{KM}}_{i,j, r}\) as follows: \({I}_{i,r}^{S}=\sum_{j}{{\mathrm{KM}}_{i,j, r}I}_{j,r}^{P}\)

4.2 Modelling strategy

We simulate the impact of the ERDF resources devoted to the Thematic Objective (TO) 1 “Strengthening research, technological development and innovation” of the 2014–2020 European cohesion policy. Those financial resources can only be accessed by regional authorities in the presence of a Smart Specialisation strategy and must be devoted to the strategy’s implementation. Therefore, we take them as the basis for constructing a scenario investigating how the quality of Smart Specialisation governance affects the economic impact of these investments in Italian regions.

We retrieved financial figures from the official data platform of the European Commission with updated information on financing and achievements under the European Structural and Investment Funds in 2014–2020.Footnote 5 We focus on the resources allocated to the ERDF-TO1 at the beginning of the financial cycle. The first year for which the ERDF regional Operational Programmes—the main strategic documents defining the use of European funds—are available for all 21 Italian regions is 2016. Hence, we took it as reference the year. In the analysis, we only consider the investment financed directly through the EU budget, which for the ERDF-TO1 of Italian regions was almost €2.31 billion. Those resources are entirely devoted to the priorities and interventions provided by the Smart Specialisation strategies through investment in six main fields: (i) enterprise R&I projects, including environmental-transition (30.5% of total investment); (ii) public and private R&I infrastructure (11.2%); (iii) R&I projects in public and private research centres (13%); (iv) technological transfer and university-SME cooperation (16.4%); (v) advanced services and support to business development (12%), (vi) support to business networks linked to the Smart Specialisation priorities (16.4%). In the simulations, the investments are assumed to take place over ten years (2014–2023), with most of them in the second half of the period according to the time profile expected by the financing organisation.

In absolute terms, the distribution of the EU funds allocated to regional Smart Specialisation strategies in Italy for the whole financial cycle 2014–2020 appears to be quite dispersed as a consequence of a combination of factors: the level of regional development, the overall allocation of EU resources to each region, the thematic concentration rules set by the cohesion policy, and the autonomous decision of the regional administration on how much resources to allocate to research and innovation policies compared to other competing policy areas within the ERDF. The southern, less developed regions of Campania, Sicily, and Apulia receive the most more than 300 million euros each; on the opposite side of the spectrum, the Autonomous Province of Bolzano, Molise, and Aosta Valley receive the least amount of funds, less than 20 million euros each, mostly because of their small size.

Figure 1 shows the allocations of EU funds to ERDF-TO1 for the whole financial cycle as percentage of 2014 regional GDP, divided by ten, corresponding to the ten-year horizon over which investment is assumed to take place. The data hence are an approximation of the average annual investment over GDP, showing a clear territorial pattern: less developed regions, primarily located in the southern and insular part of the country, exhibit the highest investment intensity, with Apulia, Calabria, Basilicata, and Sicily scoring higher than 0.04; more developed regions, mainly in the northern part of the country, have the least investment intensity, with Lombardy, Emilia-Romagna, Veneto, and the capital region of Lazio scoring less than 0.005, almost one tenth of the figures in less developed regions.

Fig. 1
figure 1

Source: DG REGIO, cohesion policy portal, and own calculations

Smart Specialisation strategies fund allocation (annual average as % of 2014 GDP).

Full size image

Regional governments use ERDF-TO1 research and innovation investments to support investors who want to engage in risky activities that can have high growth potential. The effects on private investments, and therefore on private capital stock, are simulated through a change in the risk premium, affecting the user cost of capital presented in Eq. (4). The information on research and development investments is translated into a change in the risk premium in the model by starting from the relationship with the ratio between private investments and the capital stock:

$$\frac{{I}^{P}}{{k}^{P}}=\delta \cdot {\left(\frac{rk}{uck}\right)}^{\rho }$$
(8)

where \(\rho\) is is an elasticity parameter that governs the magnitude of the gap between the rate of the return to capital, \(rk\) and the user cost of capital, \(uck\). When investments increase due to the policy shock \(x\), we obtain a new value for the user cost of capital which can be calculated as follows:

$$\frac{{I}^{P}+x}{{k}^{P}}=\delta \cdot {\left(\frac{rk}{uck\mathrm{^{\prime}}}\right)}^{\rho }$$
(9)
$${\left({\mathrm{uck}}^{\mathrm{^{\prime}}}\right)}^{\rho }={\left(rk\right)}^{\rho }\cdot \frac{\delta {K}^{P}}{{I}^{P}+x}$$
(10)

The difference between \({\mathrm{uck}}^{\mathrm{^{\prime}}}\) and \(\mathrm{uck}\) yields the change in the risk premium, which is introduced in the model to obtain the desired increase in investments due to the Smart Specialisation interventions.

This constitutes the baseline scenario for our analysis, producing an economic impact on the Italian economy solely based on the increased private investments (which temporarily increase the private capital stock). We hypothesise that good policy governance may yield additional supply side effects via TFP-enhancing effects (increasing the parameter A in Eq. (2)). In order to translate the money injection into TFP shocks in RHOMOLO, we use a simple accounting approach according to which the amount of investments directly augments the total output of the economy. The TFP improvement is then calculated as follows:

$$\dot{A}=\frac{\delta }{Y}x$$
(11)

where \(\dot{A}\) represents the change in TFP, the scale parameter of the production function, \(x\) is the R&D expenditure of the policy, Y is the output, and \(\delta\) is the R&D output elasticity. We base the values of the parameter \(\delta\) on the study on Italian regions made by Bronzini and Piselli (2009). In particular, we assume an elasticity of research and development investments to productivity of 0.026 when the policy is well implemented (equal to the baseline estimate contained in Table 3 of Bronzini and Piselli 2009, p. 192), which can get as high as 0.065 (that is, the highest estimate reported in Table 3, p. 192) when the policy is implemented in an exceptionally good way. We distinguish between the regions implementing well/exceptionally well based on the quantitative indicator on governance obtained with the survey data as explained above. On the other hand, we assume that when the Smart Specialisation policy suffers from poor governance, no TFP-enhancing effects materialise at all.

We devise three scenarios depending on the simulated TFP effects related to Smart Specialisation and its governance. The first scenario has no TFP effects and is equivalent to saying that the elasticity of investments to productivity is zero. In this scenario, the policy only has investment effects on the demand side and via a temporary increase in the private capital stock, but no structural (productivity) effects. Then, we simulate a Hypothetical scenario in which the maximum elasticity is assigned to all regions irrespective of their Smart Specialisation governance to have an admittedly unrealistic scenario in which the policy is implemented exceptionally well everywhere, maximising the productivity effects.

Finally, the Governance scenario uses the survey scores to assign the maximum elasticity of 0.065 to the best performers within the country, a 0.026 elasticity to the middle group of regions, and zero to the worst performers. There are two different governance scenarios depending on the indicator used for the simulations: QS3GOV1 or QS3GOV2, as shown in Table 2.

Table 2 Scenario-specific RnD investments elasticity to productivity
Full size table

Out of the 21 NUTS 2 regions of Italy, in the QS3GOV1 Governance scenario, five have indicator values above 8 (and therefore are characterised by the maximum elasticity). For eleven more, the values of the indicator lie between 5 and 8 (and they are assigned a 0.026 elasticity). Consequently, only five regions do not enjoy any TFP effect in this scenario. Things are different in the case of the more demanding QS3GOV2 Governance scenario, in which the maximum elasticity is assigned to only three regions, the medium elasticity is assigned to six regions, and no TFP effects are assumed for the remaining 12 regions, whose policy governance was not simultaneously effective in the two governance pillars. We expect these differences amongst scenarios to be reflected in the quantitative results of the analysis presented in the next section.Footnote 6

5 The results of the analysis

Figure 2 shows the impact of ERDF-TO1 investments on the Italian GDP according to the four scenarios described above. The results of the simulations are presented as discounted percentage deviations from the baseline values in the absence of the policy—that is, in the absence of any Smart Specialisation-related investment.Footnote 7

Fig. 2
figure 2

Source: RHOMOLO simulations

Policy impact on Italian GDP—four different scenarios.

Full size image

The green dotted-and-dashed line refers to the scenario in which only investment effects are associated with the regional structural investments, with increased demand due to the policy investment temporarily increasing the private capital stock. The impact on GDP is the lowest of the four scenarios, with a peak reached towards the end of the implementation period (+ 0.033% of GDP, equivalent to €516 million, eight years after the start of the policy implementation). The effects slowly disappear as the accumulated private capital stock gets reabsorbed through depreciation. By 2043 (year 30 of the simulation), the policy hardly impacts the Italian GDP. Cumulatively, the Italian GDP is higher than in the absence of the policy by 0.29% over twenty years, or €4.58 billion. This makes it for a discounted GDP multiplier of 1.98 after twenty years, which means that each euro invested in ERDF-TO1 generates €1.98 of GDP.

However, this scenario is notably conservative in estimating the potential impact of the policy since investments in research and development are likely to generate increases in TFP. The two Governance scenarios introduce TFP effects with the hypothesis that a good policy implementation may lead to additional beneficial effects of the investments. The red dashed line refers to the QS3GOV1 indicator and shows that, in this case, the GDP impact is much higher than in the no TFP effects scenario, with a peak of + 0.040% (€640 million) in 2021, and a cumulative impact of + 0.40% (€6.28 billion) over twenty years. The latter implies a discounted multiplier of 2.72, meaning that GDP increases by almost three euros for every euro invested. Moreover, by the end of the thirty years simulated here, the Italian GDP is above the no policy scenario GDP by almost 0.003%, since TFP effects last longer than the mere investment effects (we assume that the TFP improvements decay at a yearly rate of 5%).

Using the stricter definition, the governance indicator, that is QS3GOV2, reduces the estimated benefits associated with good governance with respect to the previous case: + 0.037% in 2021 (€593 million), for a cumulative impact of + 0.36% (€5.64 billion) after twenty years.

It is interesting to compare the Governance scenarios not only with a scenario in which there are no supply side effects associated with the policy but also with the Hypothetical scenario in which the productivity-enhancing effects of the policy are maximised in all regions (when the 0.065 elasticity is applied everywhere). The blue line represents the GDP impact in the Hypothetical scenario in Fig. 1, and it is clearly above the other two. In this case, the twenty-year cumulative impact is + 0.54% (€8.55 billion), with an implied discounted GDP multiplier of 3.69.

These numbers suggest that the way the Smart Specialisation policy was implemented in the Italian regions could generate between €1.06 and €1.70 billion of GDP over twenty years (that would be the difference between the Governance scenarios and the No TFP effects one), increasing the pure investment-related impact of the policy by 23 to almost 40%. At the same time, we could say that between €2.26 billion and €2.91 billion of potential GDP gains over twenty years will not materialise due to the insufficiently good governance of the policy in some of the regions (calculated as the difference between the Hypothetical scenario and the Governance ones).

These aggregate results mask significant regional heterogeneity in the GDP impacts of the policy intervention depending on the quality of the Smart Specialisation governance. Table 3 reports the differences in the cumulated GDP impact in three different points in time between the results of the No TFP scenario and those of the three scenarios in which TFP is allowed to increase depending on how well the policy has been implemented.

Table 3 Cumulated GDP impact differences from the No TFP scenario
Full size table

The numbers in Table 3 show that, at the end of the policy implementation period, the average Italian region is gaining between 0.05 and 0.08% thanks to the way it implemented Smart Specialisation. These numbers increase to 0.14% and 0.21%, and 0.21% and 0.30% by 2033 and 2043, respectively (notice that the difference between the Hypothetical scenario and the baseline one is 0.87% by 2043). The median is in all cases below the average, meaning that there are some regions characterised by large differences, making the distribution skewed to the right. This is confirmed by the relatively high standard deviations in all scenarios and at all points in time showed in the table.

Finally, we present in Fig. 3 two scatterplots showing the relationship between the measure of the quality of Smart Specialisation governance coming from the survey, and the difference in the 2023 GDP impact measured as the ratio between the impact obtained in the Governance scenario and the one of the No TFP effects scenario (so that a value of 1.5, for instance, indicates a GDP impact 50% above that of the latter scenario). There are two scatterplots: one for QS3GOV1 (left panel) and the other for QS3GOV2 (right panel). The results show a positive relationship between these two variables: the regions which implemented well their Smart Specialisation strategies (according to the dimensions captured by the survey we used) enjoy higher returns in terms of GDP impact from the TO1 investments of the ERDF than the regions with poor governance, leading to a sub-optimal implementation of the policy.

Fig. 3
figure 3

Source: own elaborations

Quality of Smart Specialisation governance and GDP impact (difference between Governance scenario and No TFP scenario).

Full size image

This type of evidence is consistent with the findings of other studies. Barbero et al. (2022) show a positive correlation between the quality of government and public capital and in turn, a positive correlation of the latter with the impact of cohesion policy investments. Rodríguez-Pose and Garcilazo (2015) find a positive association between the returns of the European cohesion policy and the quality of government. It should be noted, however, that the latter variable is different from the one we use in our study, which is specific to how Smart Specialisation was deployed rather than being a more general measure of the institutional quality of the territories under analysis.

6 Conclusions

According to the literature, governance is a key condition for policy effectiveness. This is especially true in the case of research and innovation policy, which takes place in a highly volatile environment, where the mean-end relationship that characterises policy action is crucially mediated by governance and by the institutional context in which policies are conceived and implemented. Not considering the reality of policy governance therefore deprives any innovation policy impact evaluation of explanatory power and, ultimately, of value as a tool to guide policy action in practice.

In this paper, we systematically link the empirical assessment of the quality of innovation policy governance with policy impact assessment based on macroeconomic modelling. We apply our methodology to the case of the European regional innovation policy for Smart Specialisation.

We construct a synthetic indicator of the quality of Smart Specialisation governance using the responses to a survey targeted at regional and national administrations, using data for all the NUTS 2 regions of Italy. We then use the indicator as an input in a spatial dynamic general equilibrium model to simulate scenarios quantifying the economic consequences of various levels of governance quality.

We find that the measured quality of Smart Specialisation governance in Italian regions could increase the pure investment-related impact of the policy by 23 to almost 40 percent over the entire time horizon that we consider. At the same time, we estimate that further potential GDP gains—in the order of an additional 40–50 percent over what was achieved with the current levels of governance—would not materialise because of the comparatively low quality of governance in some regions. The gains stemming from the policy governance are heterogeneous across the Italian regions, with a distribution skewed to the right characterised by a large standard deviation.

These results hint at a dramatic variation in policy outcomes depending on the quality of governance. Our contribution hence narrows the gap between the abstraction of traditional ex-ante impact assessment exercises based on macroeconomic simulations and the reality of how policy interventions take place. Our results highlight the importance of all the phases of the policy cycle, from planning to implementation and monitoring. They also call for improvements in how standard macroeconomic policy assessments are carried out since the assumption of perfect policy implementation may often be unrealistic.

At the same time, the results suggest that the margins for increasing the impact of innovation policy through improving governance quality are substantial. This begs the crucial question of whether and how it is possible to increase the quality of innovation policy governance in the least-performing regions in order for them to converge towards the more virtuous models already experienced in some territories.

We argue that achieving such convergence would be helped by the interplay of three factors: (i) an improved knowledge about how innovation policies operate in their systemic contexts and institutional environment (Rodríguez-Pose and Di Cataldo 2015), allowing for a more customised design of intervention measures and programmes; (ii) mechanisms favouring trans-regional and transnational policy transfer (Wink 2010; Stone et al. 2020), which can be public initiatives and services; (iii) the build-up of policy capacity (Howlett 2015; Wu et al. 2015) both in the territorial public administrations and in the network of innovation actors and intermediary bodies that participate in the development of the territory.

These factors tend to be addressed in different strands of literature, ranging from the economics and policy of research and innovation to regional science through political science and administrative studies. To the best of our knowledge, they have seldom been treated in an integrated manner. For example, policy transfer and policy capacity have been touched only marginally in the mainstream research and innovation policy literature. An interesting avenue for future work might be the attempt to create a more systematic bridge between these research areas.