Keywords

1 Introduction

When it comes to innovation, SMEs have a variety of practical challenges. Accessibility to finance may be challenging to get for SMEs, particularly when risky initiatives are involved (Lee et al., 2010; Romero-Martínez et al., 2010; Van de Vrande et al., 2009). The level to which this is an obstacle differs depending on the age of the organization, the company size, the intensity of the investigation, the growth orientation (Zimmermann & Thomä, 2016), and, in many circumstances, the geographic location (Hölzl & Janger, 2014). Additional hurdles may include problems in hiring highly trained individuals (Belitz & Lejpras, 2016; Bianchi et al., 2010; Dahlander & Gann, 2010; Duarte et al., 2017; Gardocka-Jałowiec & Wierzbicka, 2019), issues with management (Zhou et al., 2021), lower adsorption ability (Müller et al., 2021), and challenges in capturing value (Bouncken et al., 2020). Nonetheless, the most fundamental hurdles to innovation are perceived to be economic (García-Quevedo et al., 2018). Over the 2014–2020 programmatic period, the ERDF provided around 66 billion Euros to boost innovation and productivity, particularly in the European Union (EU) SMEs (Gramillano et al., 2018). Despite the evaluation of the implementation of these funds being mandatory, according to Ortiz and Fernandez (2022), policymakers still face major challenges in their assessment and control stages, owing to the absence of useful information, comparative studies, and organizational qualifications. Moreover, evaluation mechanisms during the 2014–2020 programmatic cycle focused heavily on evaluating procedure results, with hardly any data on the criteria to measure the immediate benefits of the initiatives funded (Ortiz & Fernandez, 2022). Furthermore, in the case of R&I policies, the assessment technique plays an important role in assisting the national/regional authorities in the enhancement of upcoming policy tools by identifying the strengths and weaknesses of previous policy stages (Neto & Santos, 2020). In this context, there are numerous techniques for appraising cohesion policy (Lopez-Rodríguez & Faíña, 2014). Macroeconomic and econometric modeling are commonly used approaches for analyzing the effect of cohesion policy (Henriques et al., 2022a, b). Computable General Equilibrium Models along with input–output models and econometric techniques are normally employed in the context of R&I socioeconomic effect evaluation (e.g., Di Comite et al., 2018; Diukanova et al., 2022; Barbero et al., 2022). Even though these approaches allow for the evaluation and study of the major effects of EU funds on economic growth, they do not allow evaluating management failures (Marzinotto, 2012). Moreover, they ignore the allocation of EU funding within every region to different thematic objectives (TO). The research mainstream is based on econometric studies (see, for example, Stojčić et al., 2020; Radicic & Pugh, 2017; Santos et al., 2019; Thum-Thysen et al., 2019; Fattorini et al., 2020; Sein & Prokop, 2021). Nevertheless, it produces contradictory results (Berkowitz et al., 2019), prompting some experts to dispute its use (Durlauf, 2009; Wostner & Šlander, 2009; Berkowitz et al., 2019). Other methods can also be employed, but with the same intrinsic shortcomings (e.g., Bedu & Vanderstocken, 2020; Gustafsson et al., 2020). The evaluation procedures generally available do not allow comparing any regional or national OP against its peers. These do not enable the identification of the adjustments that should occur to enhance the efficiency of OPs’ execution (Gouveia et al., 2021). Moreover, these methods often require fulfilling statistical hypotheses (namely, normality, absence of multicollinearity, and homoscedasticity). Therefore, the adoption of nonparametric methodologies can be valuable and appropriate, particularly as the data freely available on the European Commission website can be used in conjunction with DEA models. The efficient production frontier is usually derived through stochastic approaches (Gouveia et al., 2021). These, nevertheless, can just accommodate an output level at a time (Gouveia et al., 2021). Contrastingly, DEA can easily handle many inputs (resources) and outputs (outcomes) and can also be applied to determine the efficient production frontier. Furthermore, contrary to stochastic techniques, DEA does not rely on any production function form or error term. According to DEA, the greater the divergence from the production efficient frontier, the greater the inefficiency of the decision-making unit (DMU) (in this case, the OPs) under appraisal. Also, the DEA methodology can be particularly valuable for management authorities (MA) because it enables the detection of best practices, and also identifies the changes that need to occur to improve the performance of the OPs under evaluation.

In this framework, Athanassopoulos (1996) used DEA to determine the relative geographical weaknesses of the EU’s Level II territories. Gómez-García et al. (2012) assessed the pure and global technical efficiencies regarding Thematic Objective 1 (TO1) in the deployment of EU structural funds from 2000 to 2006. They employed labor and productivity levels as outputs, and the Stochastic Frontier Analysis (SFA) together with the DEA methodology. Anderson & Stejskal, (2019) employed DEA to evaluate the efficiency of innovation diffusion in EU MS based on their European Innovation Scoreboard scores. Furthermore, Gouveia et al. (2021) employed the Value-Based DEA technique, considering the primary elements that can impair the efficient execution of structural funds in different OPs devoted to SMEs’ competitiveness. Henriques et al. (2022a) used the SBM approach in conjunction with cluster analysis to evaluate 102 OPs from 22 EU MS focused on the implementation of a low-carbon economy in SMEs. Finally, Henriques et al. (2022b) evaluated the efficiency of 53 R&I OPs from 19 countries utilizing the Network SBM technique in combination with cluster analysis for appraising the implementation of EU funds devoted to promoting R&I in SMEs. Nevertheless, their work did not accommodate for the influence of contextual variables and random errors in efficiency evaluation. Therefore, this work aims to fill this gap by suggesting an approach that combines a three-stage SBM model and SFA, which to the best of our knowledge has not hitherto been used in this context. Through this method it is possible to further understand if the efficiency results attained are mainly related to management failures or the contextual environment of the OPs or statistical noise, also providing information on the contextual factors with the greatest effect on the OPs’ inefficiencies.

Insofar, the main research questions that we seek to address with this work are given below:

  • RQ1: “Which contextual variables show a relevant effect on the inefficiencies of the OPs committed to boosting R&I in SMEs?”

  • RQ2: “What are the impacts of considering contextual factors on the efficiency of the OPs?”

This article is organized as follows. Section 2 explains the basic assumptions underlying the techniques suggested to assess the execution of the OPs evaluated. Section 3 addresses the key rationale for choosing the inputs and outputs utilized in this study, as well as some statistics on the data that instantiates the SBM and SFA models. Section 4 delves into the major findings. Section 5 summarizes the major results, discusses potential policy recommendations, identifies the main shortcomings, and proposes further work advances.

2 Methodology

Classical DEA techniques, like the CCR (Charnes et al., 1978) and BCC (Banker et al., 1984), are radial, which means that they can simply manage proportional adjustments in the inputs or outputs used in the assessment. Therefore, the CCR and BCC efficiency ratings produced indicate the highest proportionate input (output) contraction (expansion) rates for all inputs (outputs). Nevertheless, owing to factor substitutions, this sort of premise is frequently not met in practice.

As a result, in opposition to the CCR and BCC approaches, we employ the SBM approach (Tone, 2001), which allows for a broader study of efficiency due to its non-radial nature (i.e., inputs and outputs can vary non-radially), also enabling to consider non-oriented models (i.e., address simultaneous variations of the inputs and outputs).

2.1 The SBM Model

The generalized SBM model of Tone (2001) may be presented (by taking m inputs, s outputs, and n DMUs into account) as follows:

$$ \begin{gathered} \begin{array}{*{20}c} {Min} \\ {{\mathbf\lambda} ,{\mathbf{s}}^- , {\mathbf{s}}^+ } \\ \end{array} \rho = \frac{{1 - \frac{1}{m}\sum_{i = 1}^m s_i^- /x_{ik} }}{{1 + \frac{1}{s}\sum_{r = 1}^s s_r^+ /y_{rk} }} \hfill \\ {\text{s.t.}} \hfill \\ x_{ik} = \mathop \sum \limits_{j = 1}^n x_{ij} \lambda_j + s_i^- ,i = 1, \, ...,m \hfill \\ y_{rk} = \mathop \sum \limits_{j = 1}^n y_{rj} \lambda_j - s_r^+ ,r = \, 1, \, ...,s \hfill \\ \mathop \sum \limits_{j = 1}^n \lambda _j = 1,\lambda _j \ge 0,j = 1,{\text{ }} \ldots ,n, \hfill \\ \lambda_j \ge 0, j = 1, ..., n, \hfill \\ s_i^- \ge 0,i = 1, \, ...,m, \hfill \\ s_i^+ \ge 0,r = \, 1, \, ...,s, \hfill \\ \end{gathered} $$
(1)

where X = [xij, i = 1, 2, …, m, j = 1, 2, …, n] is the (m × n) matrix of inputs, Y = [yrj, r = 1, 2, …, s, j = 1, 2, …, n] is the matrix of outputs (s × n) and the rows of these matrices for DMUk are, respectively, \({\mathbf{x}}_k^T\) and \({\mathbf{y}}_k^T\), where T is the transpose of a vector. Also, we presume a Variable Returns to Scale technology with the imposition of \(\sum_{j = 1}^n {\lambda _j } \) = 1, \(_j\) ≥ 0 (∀j). The value of 0 < ρ < 1 can be seen as the ratio of average inefficiencies of inputs and outputs.

A DMUk is SBM-efficient if \(\rho^* = 1\), , meaning that the slacks (\(s_i^-\) and \( s_i^+\)) are null for all the inputs and outputs.

Problem (1) can be converted into a linear problem, by applying a positive scalar variable t (see Tone (2001)). Further details on this modeling approach can be found in Tone (2001) and regarding SBM superefficiency in Tone (2002).

2.2 Stochastic Frontier Analysis

Fried et al. (2002) proposed a three-stage DEA model. In the first stage, the SBM model is applied to calculate the technical efficiency of each DMU, and the necessary changes required to the inputs and outputs to turn inefficient DMUs into efficient ones (i.e., the slacks). In the second stage, the slacks are grouped into three types: contextual variables, inefficient management, and statistical noise. The slacks are the dependent variables, while the contextual variables are the independent variables. The objective is to remove the influence of contextual factors and random errors. SFA is then used to modify the input and output factors (Aigner et al., 1977; Meeusen & Broeck, 1977).

Therefore, the slack of each input obtained for every inefficient DMUj (\(j = 1, \ldots ,p\)) is:

$$ s_{ij} = f\left( {X_j ,{ }\beta^i } \right) + v_{ij} + u_{ij} ,{ }i = 1, \ldots ,m;{ }j = 1, \ldots ,p, $$
(2)

where \(s_{ij}\) is the slack of input i of DMUj, \(f\left( {X_j ,{ }\beta^i } \right)\) is the slack frontier, and \(\beta^i\) corresponds to the coefficients related to the contextual variables. Expression \(v_{ij} + u_{ij}\) is the mixed error, \(v_{ij}\) is the statistical noise and \(u_{ij}\) is the management inefficiency. Generally, it is presumed that \(v_{ij} \sim N\left( {0;{\text{~}}\sigma _v^2 } \right) \) and \( u_{ij} \sim N^ + \left( {\mu ^i ;\sigma _u^2 } \right), \) where \(v_{ij}\) and \(u_{ij}\) are independent variables.

Consider that \(\gamma = \frac{\sigma_u^2 }{{\sigma_u^2 + \sigma_v^2 }}\). If \(\gamma\) is near 1, it implies that the majority of the adjustment necessary to reach efficiency is related to management inefficiency. If \(\gamma \) is near 0, the random error is the prevalent factor.

Subsequently, the adjusted input and output slacks are obtained by splitting the mixed error. According to Jondrow et al. (1982), the conditional inefficiency is given as:

$$ E\left( {u_{ij} |u_{ij} + v_{ij} { }} \right) = \frac{\sigma \delta }{{1 + \delta^2 }}\left[ {\frac{{\varphi \left( {\frac{\varepsilon_j \delta }{\sigma }} \right)}}{{\emptyset \left( {\frac{\varepsilon_j \delta }{\sigma }} \right)}} + \frac{\varepsilon_j \delta }{\sigma }} \right], $$
(3)

where \(\delta = \frac{\sigma_u }{{\sigma_v }},{ }\varepsilon_j = v_{ij} + u_{ij},{ }\sigma^2 = \sigma_u^2 + \sigma_v^2 ,{ }\varphi { }\) and \(\emptyset\) are, correspondingly, the density and distribution functions of the standard normal distribution. Hence, the expected value of random error is:

$$ E\left( {v_{ij} |u_{ij} + v_{ij} { }} \right) = s_{ij} - f\left( {Z_j ,{ }\beta^i } \right) - E\left( {u_{ij} |u_{ij} + v_{ij} { }} \right), $$
(4)

Secondly, the input and output factors of each DMU are changed according to the SFA outcomes by removing the significant contextual effects and statistical noises.

According to Tone and Tsutsui (2009), we begin by employing these formulas:

$$ x_{ij}^A = x_{ij} - f\left( {Z_j , \hat{\beta }^i } \right) - \hat{v}_{ij} \left( {input} \right) $$
(5)
$$ y_{rj}^A = y_{rj} + f\left( {Z_j , \hat{\beta }^r } \right) + \hat{v}_{rj } \left( {output} \right). $$
(6)

The input data are adjusted using (5) as follows (Tone & Tsutsui, 2009):

$$ x_{ij}^{AA} = \frac{{x_{imax} - x_{imin} }}{{x_{imax}^A - x_{imin}^A }}\left( {x_{ij}^A - x_{imin}^A } \right) + x_{imin} , i = 1, \ldots ,m ;j = 1, \ldots ,p $$
(7)

where

$$ x_{imin} = \mathop {\min }\limits_k \left\{ {x_{ik} } \right\}; x_{imax} = \mathop {\max }\limits_k \left\{ {x_{ik} } \right\}; x_{imin}^A = \mathop {\min }\limits_k \left\{ {x_{ik}^A } \right\} \;{\text{and}}\; x_{imax}^A = \mathop {\max }\limits_k \left\{ {x_{ik}^A } \right\}. $$

Analogously, the outputs are changed using (6) as (Tone & Tsutsui, 2009):

$$ y_{rj}^{AA} = \frac{{y_{rmax} - y_{rmin} }}{{y_{rmax}^A - y_{rmin}^A }}\left( {y_{rj}^A - y_{rmin}^A } \right) + y_{rmin} , r = 1, \ldots ,s ;j = 1, \ldots ,p $$
(8)

where

$$ y_{rmin} = \mathop {\min }\limits_k \left\{ {y_{rk} } \right\}; y_{rmax} = \mathop {\max }\limits_k \left\{ {y_{rk} } \right\}; y_{rmin}^A = \mathop {\min }\limits_k \left\{ {y_{rk}^A } \right\} \;{\text{and}}\; y_{rmax}^A = \mathop {\max }\limits_k \left\{ {y_{rk}^A } \right\}. $$

Then again, the efficiency scores are computed through SBM by employing the previously adjusted inputs and outputs.

3 Data

3.1 Input and Output Factors

This work is a follow-up of the work published by Henriques et al. (2022b) and, therefore, we have employed mostly the same input and output factors chosen therein for evaluating the efficiency of the execution of ERDF allotted to boost R&I in SMEs—see Table 1 and Fig. 1. All the information regarding these data is obtainable from Henriques et al. (2022b).

Table 1 External and intermediate inputs and outputs selected for instantiating the SBM model
Full size table
Fig. 1
A bar graph A has a y-axis, number, 4 sets of parameters of 6 columns each, and a table below detailing all the data, and here, the column maximum exhibits the highest values across all parameters. Bar graph B of euros versus total eligible spending has the column labeled maximum with the highest value at 2,654,718,365.00.

Source Authors’ computation based on data from Henriques et al. (2022a, b)

Descriptive statistics of Inputs and Outputs for output and process indicators a and for financial indicators b.

Full size image

3.2 Contextual Factors

The regional GDP at purchasing power parity per capita (GDPPPpc) was considered a contextual variable being used as a proxy to measure economic activity (Barbero et al., 2022; Diukanova et al., 2022; Hervás-Oliver et al., 2021). Besides, Barbero et al. (2022) concluded that the achievement of regional targets related to the ERDF TO1 has a positive impact on all economic indicators, including the GDP, in the selected regions (Greece, Italy, Portugal, and Spain).

According to Diukanova et al. (2022), R&I and low-carbon European structural funds can exert substantial positive effects on the indicator of tertiary education attainment, thus the percentage of the population aged 25–34 who have finished university education was also considered in this set of contextual factors.

Anderson and Stejskal (2019) used variables that fall into the category of human resource (lifelong learning, employment in knowledge-intensive activities), finance (public sector R&D expenditure, private sector R&D expenditure, sales of new-to-market and new-to-firm innovations) and non-financial innovation structures (non-R&D innovation expenditure). Additionally, as referred in Hervás-Oliver et al. (2021), the variation in the development of EU regions affects the innovation capacity of SMEs located in each territory and consequently, it is important to incorporate the variables that better capture innovation in SMEs (e.g., innovation activities like public and private R&D expenditures, non-R&D innovation expenditures, innovative SMEs collaborating with others).

Therefore, we have used similar variables that were reported in the latest European Innovation Scoreboard (Hollanders, 2021). Finally, Sein and Prokop (2021) stress the key role of a firm’s R&D, which has proven to be a mediator of the effects of public funding and triple- and quadruple-helix cooperation on the product and process innovation activities of Norwegian firms. In this study, variables such as SMEs with product innovations and SMEs with business process innovations were used. Therefore, we considered, in this context, sales of new-to-market and new-to-firm innovation. All the contextual variables shown in Table 2 (apart from GDPPPPpc whose data were obtained from the OECD website) were extracted from the European Innovation Scoreboard (Hollanders, 2021), allowing to capture differences in SMEs innovation across regions. All these indicators are normalized between 0 and 1 at origin, to produce a composite indicator integrating variables from different scales. Table 2 shows the main descriptive statistics of the contextual variables.

Table 2 Descriptive statistics of the contextual variables
Full size table

4 Discussion of Results

The initial results were computed with the help of the Max DEA software and their descriptive statistics are depicted in Table 3.

Table 3 Descriptive statistics for efficient and inefficient OPs
Full size table

From Table 3, it can be seen, in general, that the variability of the efficiency scores is bigger for efficient OPs than for inefficient ones (with the standard deviation varying between 0.25 and 0.15, for the first and the latter, respectively). Besides, inefficient OPs present very low mean efficiency scores (with an average potential improvement of efficiency of 94%). Figure 2 illustrates the number of OPs at several subintervals for the efficiency scores.

Fig. 2
A bar graph of the number of operational programs versus efficiency scores. The highest column has a value of 38 with an efficiency score of 0.0 to 0.1, followed by columns with values of 10 and 2 with efficiency scores of greater than 1, and 0.3 to 0.4, respectively.

Source Authors’ own elaboration

Number of OPs at different subintervals of efficiency scores.

Full size image

The number of OPs classified as efficient is 10 (Fig. 2).

Out of the ten efficient OPs, the three most chosen as benchmarks are “Brussels Capital Region—ERDF” (41 times), “Aragón—ERDF” (17 times), and “Toscana—ERDF” (18 times)—see Table 4. The OP most frequently viewed as a benchmark is characterized as an “Innovation Leader” and manages to score in all the outputs examined in the assessment—see Table 4.

Table 4 Characteristics of efficient OPs
Full size table

The SBM model also offers an outline of the changes that really should be made to inputs and outputs to convert inefficient OPs into efficient ones—see Fig. 3.

Fig. 3
A 3 D bar graph of average original factors versus their projections for inefficient O Ps has 5 bars plotted. Under total eligible spending, original factors exhibit their highest value of 157,674,529 versus projection at 33,945,543, and under improved R and I and infrastructures, projection is at its highest at 11,112, versus original at 489.

Source Authors’ own elaboration

Average original factors versus their projections for inefficient OPs.

Full size image

The ‘number of researchers working in improved R&I infrastructures’ has the largest potential for improvement (2174%), followed by ‘total eligible spending’ (−78%), the ‘number of enterprises supported for new-to-the-market products’ (71%), the ‘number of enterprises supported’ (46%) and the ‘number of enterprises working with R&I institutions’—see Fig. 3. All in all, like other studies, our findings also show the importance of the lack of skills as an obstacle to R&I OPs’ implementation (e.g., Belitz & Lejpras, 2016; Duarte et al., 2017; Gardocka-Jałowiec & Wierzbicka, 2019). These results also highlight the need to foster the cooperation and networking of SMEs with research institutions, thus corroborating Hervás-Oliver et al. (2021) findings. Besides, additionally, since there seems to be an overuse of the EU funding (because of the required reduction on eligible spending), our results suggest the validation of the ‘European paradox’ since there seems to exist an ‘innovation gap’ in that supporting innovation inputs through public funding does not necessarily lead to innovation outputs (Hammadou et al., 2014; Radicic & Pugh, 2017).

4.1 Results Obtained with SFA

To remove the potential effects of contextual factors and random errors, the functional forms given in (4) were estimated. The slacks of the outputs were considered as dependent variables, originating four regressions models. The multicollinearity was evaluated through the variance inflation factor (VIF), which measures the strength of correlation between the independent variables. This indicator is always greater than or equal to 1. Table 5 illustrates the values of VIF considering three sets of variables. When VIF is higher than 10, there is significant multicollinearity that needs to be corrected, thus, in the first step, we began by removing the four contextual variables that verify such condition. Afterward, the VIF values for the remaining variables were calculated (see Table 5, second step). Usually, values within 1 and 5 are not deemed relevant to cause concern (Belsley, 1991; James et al., 2013), but it seemed prudent to recalculate the VIF values without the variables “Non-R&D innovation expenditures” and “Innovation expenditures per person employed” and “PCT patent applications”. The small values of VIF presented in Table 5 reveal that, when considering these variables, there is no problem of collinearity.

Table 5 VIF values
Full size table

To run the SFA regression models, the R software, version 4.0.5 (RStudio Team, 2021), particularly, the sfaR package version 0.1.1 was used (Dakpo et al., 2022). The final regression models are shown in Table 6.

Table 6 SFA analysis results
Full size table

In model (1), the value of γ is very close to zero, thus the statistical noise is in a dominant position. Furthermore, statistical noise and the contextual variables GDPPPPpc, Lifelong learning, and R&D expenditures of the public sector explain virtually all the variation that occurred in the slack of the output “Researchers working in improved infrastructures”. For the models (2), (3), and (4), the values of γ are near one and statistically significant (1%), this means that management problems are the principal cause of the achieved technical (in)efficiency. The contextual variables considered in model (2) cause a relevant effect on the slack since all the regression coefficients associated are significant (at the 1% level). Likewise, in model (3), we found statistically significant variables to explain the required adjustments in “Enterprises Working with Research Institutions.” Concerning model (4), since there are no statistically significant variables, no adjusted values are required for this output.

According to Table 6, a rise in GDPPPPpc contributes to a larger necessary increase of “Researchers Working in Improved Infrastructures”, “Enterprises Supported” and “Enterprises Working with Research Institutions”. On the one hand, regarding the two latter indicators, these findings seem to suggest that richer regions do not show a better use of ERDF targeted to strengthen R&I in SMEs. Bukvić et al. (2021) arrived at similar conclusions regarding the underuse of ERDF by SMEs in the Information and Communication Technologies sector in Croatia from 2014–2020. They ascertained that the difficulties and time required to submit, produce, and assess project proposals were a probable justification for these findings. Furthermore, Martinez-Cillero et al. (2020) reported that SMEs’ investments are poorer than would be anticipated by standard economic models, proposing that these firms are particularly sensitive to funding difficulties. Another possible explanation might be related to the use of further financing opportunities in the framework of other funding programs (outside ERDF). On the other hand, regarding the first indicator, these outcomes also highlight the need to handle the lack of skilled researchers, a major hurdle to innovation also identified in more developed regions (Hölzl & Janger, 2014).

Additionally, a higher percentage of the population with tertiary education does not lead to an efficient number of “Enterprises Supported” and “Enterprises Working with Research Institutions” supported. These results might suggest that higher education institutions should be further contributing to the actual needs of the economy. In this framework, initiatives should be promoted to increase the relationship of SMEs with higher education institutions, since this type of linkage can be beneficial for the innovation environment (Kobarg et al., 2018; Rajalo & Vadi, 2017).

Lifelong learning seems to be positively associated with a better performance of the outputs “Researchers Working in Improved Infrastructures” and “Enterprises Supported” because it is negatively related to their required improvements. These outcomes may be explained by the abundance of the population involved in lifelong learning activities in the generality of MS (Anderson & Stejskal, 2019). Besides, these findings also suggest that coordinated lifelong learning policies play a pivotal role in propelling innovation and progress among MS and regions.

R&D expenditures within the public sector seem to have a positive contribution to the required enhancement of the adjustments on “Researchers Working in Improved Infrastructures”, with the opposite effect on the number of “Enterprises Supported” and “Enterprises Working with Research Institutions”. On the one hand, these results highlight the positive effect of public R&D spending on education attainment (i.e., a higher number of skilled researchers) since these are also linked with expenses in the higher education public sector. However, the two latter findings may imply that increased R&D expenditures within the public sector are not a viable strategy to mitigate SMEs’ inability to engage in R&D (Hervás-Oliver et al., 2021). This might also suggest that EU SMEs cannot absorb the spillover effects from public R&D (Rodríguez-Pose & Wilkie, 2019). Furthermore, since the R&D expenditures within the business sector show a positive effect on the required enhancement of “Enterprises Working with Research Institutions” (i.e., a reduction of the necessary adjustment to become efficient), these findings appear to demonstrate the lesser influence of public R&D in SME innovation compared to private R&D. Similarly, results were also attained by Hervás-Oliver et al. (2021).

In what concerns the innovative SMEs collaborating with others as a percentage of SMEs, there is a positive effect both on the number of “Enterprises Supported” and the “Enterprises Working with Research Institutions” (the enhancement required in these two outputs is negative). In a similar context, Hervás-Oliver et al. (2021) concluded that SME collaboration with exterior sources of knowledge (either supply-chain actors and competitors or universities or other sources of research) is positively related to regional SME innovation.

Finally, sales of new-to-market and new-to-firm innovations require a further enhancement of the number of “Enterprises Working with Research Institutions” (the enhancement required in this output is positive). These findings might be influenced by the fact that this contextual variable does not make a distinction between incremental and radical innovation, also considering non-technological innovations (Apa et al., 2021).

4.2 Results Obtained with the Adjusted Factors

Table 7 shows that efficient OPs hardly change their average efficiency scores with the adjusted factors (the standard deviation is the same, i.e., 0.27). Besides, the efficiency scores are bounded within the same interval, i.e., [1.04, 1.88], demonstrating efficiency scores bigger than 1.24 for more than 50% of the efficient OPs. Also, inefficient OPs decrease the variability of their efficiency scores (with a standard deviation of 0.11 against the previous 0.15, with more than 50% of inefficient OPs having efficiency values just under 0.06) and increased their average efficiency from 0.06 to 0.10 (underlining the importance of the contextual variables).

Table 7 Descriptive statistics of the results obtained for efficient and inefficient OPs with adjusted factors
Full size table

Figure 4 depicts the difference in the technical efficiency of the OPs with and without adjusted factors.

Fig. 4
A line graph of efficiency score versus the efficient O Ps obtained has 2 lines plotted. The efficiency with adjusted factors line begins at 1.9 at competitiveness, drops, exhibits a peak of 0.9, drops, and plateaus from there, and the efficiency with original values line exhibits the same trend but without notable peaks.

Source Authors’ own elaboration

Efficiency scores for the efficient OPs obtained with adjusted and non-adjusted factors.

Full size image

Figure 5 illustrates the greatest efficiency gains attained with the adjusted factors. When contrasted with the first step of the analysis, “Competitiveness and Cohesion–HR—ERDF/CF” demonstrated the greatest gain in efficiency, with values going from 0.0003 to 0.0676. Overall, these outcomes suggest that the inefficiencies originally computed for these OPs were not solely the result of their low technical level but were also related to their contextual factors.

Fig. 5
A graph of efficiency score and change versus the efficient O Ps obtained has a line, and columns plotted. The line begins at an efficiency of 1.19 at 19000.00%, steeply drops, decreases gradually to 0, and plateaus, and the efficiency with adjusted factors, and original values columns are highest at Castille y Leon at efficiency scores of 1.1.

Source Authors’ own elaboration

Efficiency scores for the efficient OPs with the greatest efficiency gains obtained with adjusted and non-adjusted factors.

Full size image

Then again, out of the ten efficient OPs, the three most chosen as benchmarks are “Brussels Capital Region—ERDF” (39 times), “Aragón—ERDF” (20 times), and “Toscana—ERDF” (17 times) and “Extremadura—ERDF” (15 times).

5 Conclusions and Further Research

The primary purpose of this article was to evaluate the reasons behind the inefficiency of the OPs devoted to boosting R&I in SMEs. With this aim, we assessed 53 OPs within TO1 from 19 EU MS. To begin with, the SBM modeling approach is utilized to calculate the technical efficiency of every OP. At this stage, important data about the overall adjustments that should be made to reduce any disparities between inefficient OPs and their corresponding benchmarks are obtained.

Unlike other commonly used techniques applied in comparable situations, such as reference cases, econometric and statistical methods, and macroeconomic and microeconomic analyses, the SBM model can be particularly useful for MA, as it enables them to identify the references of best practices and the required changes to improve the OPs’ implementation performance, also contemplating their performance in two different stages. The second phase consists of employing SFA to the slacks of inefficient OPs to change the inputs and outputs after removing environmental effects and statistical noise. At this stage, information is extracted about how environmental factors may influence the efficiency of ERDF deployment in distinct OPs devoted to the promotion of R&I in SMEs, as well as the magnitude of management flaws. Finally, the previously corrected factors are employed in the SBM model to obtain new efficiency ratings.

Our main conclusions are discussed next.

RQ1: “Which contextual variables show a relevant effect on the inefficiencies of the OPs committed to boosting R&I in SMEs?”

Our results indicate that more developed regions do not make efficient use of ERDF aimed at promoting R&I in SMEs. The difficulty and time necessary to submit, develop and evaluate the project proposals, and the higher vulnerability of these types of enterprises to financial issues are possible explanations for these poor results. Alternatively, these findings can also be attributed to the utilization of additional financing options within the context of other funding programs. Furthermore, these results also demonstrate the need of addressing the shortage of trained researchers, which has been recognized as a key barrier to innovation in more developed regions. Besides, a larger percentage of the population with university education does not result in an adequate number of “Enterprises” and “Enterprises Working with Research Institutions” supported. These findings may imply that higher education institutions should contribute more to the economy's genuine demands. Initiatives should be pushed in this framework to strengthen SMEs’ relationships with higher education institutions since this form of collaboration can be advantageous to the innovation environment.

Lifelong learning appears to be favorably correlated with the higher performance of the outputs “Researchers Working in Improved Infrastructures” and “Enterprises Supported”. Hence, our results indicate that integrated lifelong learning strategies are critical in accelerating innovation among MS and regions.

R&D expenditures in the public sector appear to contribute positively to the needed enhancement of the adjustments on “Researchers Working in Improved Infrastructures” but have the opposite effect on the number of “Enterprises Supported” and “Enterprises Working with Research Institutions”. On the one hand, these findings indicate the favorable impact of these expenditures on educational attainment because they are also connected to expenditures in the public higher education sector. The two latter findings, however, may suggest that greater R&D expenditures in the public sector are not a realistic option for mitigating SMEs’ incapacity to engage in R&D. This might also imply that SMEs cannot absorb spillover effects from governmental R&D.

Additionally, because R&D expenditures in the business sector have a beneficial impact on the required improvement of “Enterprises Working with Research Institutions”, these findings appear to show that public R&D has a lesser influence on SME innovation compared to private R&D.

Concerning innovative SMEs cooperating with others as a proportion of SMEs, there is a favorable influence of this contextual variable both on the number of “Enterprises” and on “Enterprises Working with Research Institutions” supported. Therefore, it might be ascertained that the collaboration of SMEs with external sources of knowledge is positively connected to regional SME innovation.

Finally, sales of new-to-market and new-to-firm innovations have a negative effect on the number of “Enterprises Working with Research Institutions” (the enhancement required in this output is positive). These findings might be impacted by the fact that this contextual variable does not distinguish between incremental and radical innovation, as well as non-technological breakthroughs.

RQ2: “What are the impacts of considering contextual factors on the efficiency of the OPs?”

If the factors are adjusted according to Tone and Tsutsui (2009), 19% of OPs (10) manage to attain technical efficiency in any case, indicating that the effects of contextual factors are more visible on inefficient OPs. The biggest gap in efficiency was found in inefficient OPs that showed an average efficiency gain of 67%. The most efficient regions regardless of the adjustments were “Competitiveness Entrepreneurship and Innovation—GR—ERDF/ESF”, “England—ERDF” and “Multi-regional Spain—ERDF”, with values of efficiency ranging between 1.25 and 1.88. In general, it can be concluded that the technical efficiency of the OPs classified as efficient was mostly driven by good management practices.

Although this work gave new perspectives on the evaluation of OPs dedicated to R&I in EU SMEs, it had limitations. Though the performance framework made available by the European Commission includes a set of procedural indicators, there is no full correspondence between the data collected for the OPs’ accomplishments and their financial execution. Secondly, since the data available is often sparse, our evaluation was applied to a reduced number of OPs.

Whereas our work focused on an assessment technique for use throughout the reporting phases of the programmatic cycle, future work should address ex-post assessment, with a special emphasis on the spillover effects of the OPs under TO1.