Keywords

1 Introduction

Various efforts have been undertaken over the last ten years to investigate the significance of ICT use in SMEs, leading to a huge array of published studies. Taylor (2015) explored two theoretical foundations in this respect: ‘diffusion of innovation theory’ (Rogers et al., 2014) and ‘technology, organization, and environment structure’ (Tornatzky and Fleischer, 1990) to provide a thorough solid background for SMEs’ usage of ICT. This comprehensive framework contains a taxonomy that covers several of the most important internal and external variables influencing SMEs’ ICT adoption. Mbuyisa and Leonard (2017), on the other hand, studied the link between ICT, SMEs, and poverty alleviation. To begin, this study emphasizes the significance of ICT deployment in SMEs. Following that, it investigates how SMEs might use ICTs to help them eliminate poverty. Disparities in ICT availability among organizations were also investigated in a study on the digital divide (DD) among firms conducted by Bach et al (2013). This assessment looked at the region, firm type, and research period, and the impact and causes of DD. Another study examined the key reasons, repercussions, and barriers to ICT adoption in SMEs (Consoli, 2012). Taruté and Gatautis (2014) evaluated the possible impact of ICT on SMEs’ competitiveness. Lehner and Sundby (2018) examined the importance of ICT skills and competencies for SMEs from several perspectives.

Oberländer et al. (2020) assessed the subject of workplace digital competencies. Isensee et al. (2020) developed a conceptual model to emphasize the connections among three domains, to present the first comprehensive picture of organizational behavior, sustainability, and digitalization levels in SMEs, as well as their interconnections. They found that their strategies, organizational key competencies, governance, and mindsets are by far the most explored cultural features. Another recent study (Ramdani et al, 2022) looked at digital innovation in SMEs and discovered that it is driven by past experiences, progresses throughout many stages of innovation, and results in organizational and corporate ongoing learning. Similarly, Thrassou et al. (2020) did a literature review to help identify the major problems and possibilities for SMEs in the framework of digitization and ICT advancements.

Overall, none of the previous studies focuses on the evaluation of ICT policy. According to Reggi & Gil-Garcia (2021), there has been a lack of scholarly attention on how to select the best approaches for allocating funding amongst diverse ICT initiatives. Moreover, limited researchers have attempted to determine if this sort of funding is delivered in line with some of the most pressing requirements of each place (Kleibrink et al., 2015; Reggi & Scicchitano, 2014). Ex-post assessments are frequently utilized in research evaluating European structural funding awarded to ICT (Kleibrink et al., 2015; Reggi & Scicchitano, 2014). There is additional research that provides an ex-ante assessment of the factors influencing funding distribution among various ICT activities (Reggi & Gil-Garcia, 2021).

Nonetheless, no research has been undertaken so far that compares the implementation of OPs connected to ICT policies to their peers throughout the same period, or that indicates the modifications that must be applied to achieve the OPs’ efficiency.

Consequently, the objective of this paper is to contribute to the existing literature by employing a mathematical method that allows Management Authorities (MA) to better assess the implementation of OPs focused on ICT deployment in SMEs by using two non-parametric methodologies, specifically the SBM and the WRDD models. In this context, the research questions (RQ) that this work seeks to answer are as follows:

RQ1: “What indicators restrict the successful use of ERDF allocated to enhance ICT adoption in EU SMEs?”

RQ2: “What OPs are generally elected as benchmarks across the assessed time frame?”

RQ3: “What OPs demonstrate more resilience in terms of their efficiency classification in the context of changes in the indicators used?”

The following is the structure for this paper. Section 2 describes the fundamental principles behind the methodologies proposed to evaluate the implementation of the OPs under consideration. Section 3 discusses the main reasons for selecting the inputs and outputs, also presenting their descriptive statistics. The main results are discussed in Sect. 4. Section 5 summarizes the important findings, examines prospective political implications, identifies critical weaknesses, and suggests future research directions.

2 Methodology

Classical DEA techniques, such as the CCR (Charnes et al., 1978) and BCC (Banker et al., 1984), are radial, which means that they can only manage proportional changes in the inputs (resources) or outputs (outcomes). Consequently, the CCR and BCC efficiency ratings produced indicate the highest proportionate input (output) contraction (expansion) rates for all inputs (outputs). Unfortunately, this sort of premise is frequently not met. As a result, we utilize the SBM model (Tone, 2001), which provides for a much more extensive evaluation of efficiency since it is non-radial (i.e., inputs and outputs can adapt non-proportionally) and non-oriented (i.e., enables addressing simultaneous changes of the inputs and outputs). In addition, we applied the WRDDM (Chen et al., 2015), which allows us to decompose inefficiencies.

2.1 The SBM and the WRDD Models

Consider a set of \(n\) DMUs \(\left( {{\text{DMU}}_1 ,{\text{DMU}}_2 , \ldots ,{\text{DMU}}_{\text{n}} } \right)\) with X = [xij, i = 1, 2, …, m, j = 1, 2, …, n] the (m × n) matrix of inputs, Y = [yrj, r = 1, 2, …, s, j = 1, 2, …, n] the matrix of outputs (s ×  n) and the rows of these matrices for DMUk are, respectively, \({\varvec{x}}_k^T\) and \({\varvec{y}}_k^T ,\), where T is the transpose of a vector. Also, assume a Variable Returns to Scale (VRS) technology with the imposition of \( \sum_{j = 1}^n {\lambda _j = {\text{1}},\;\lambda _j \ge 0\left( {\forall _j } \right)} . \).

The SBM model

The WRDDM

\( \begin{array}{*{20}c} {Min} \\ {{\mathbf{\lambda }},{\varvec{s}}^ - ,{\varvec{s}}^ + } \\ \end{array} \rho = \frac{{1 - \frac{1}{m}\sum_{i = 1}^m {{{s_i^ - } / {x_{ik} }}} }}{{1 + \frac{1}{s}\sum_{r = 1}^s {{{s_r^ + } / {y_{rk} }}} }} \)

\({\text{s.t.}}\)

\( x_{ik} = \sum_{j = 1}^n {x_{ij} \lambda _j } + s_i^ - ,\;i = {\text{1}}, \ldots ,m \)

\( y_{rk} = \sum_{j = 1}^n {y_{rj} \lambda _j } - s_r^ + ,\;r = {\text{1}}, \ldots ,s \)

\(\mathop \sum \limits_{j = 1}^n \lambda_j = 1, \, \lambda_j \ge 0,j = 1, \ldots ,n\)

\(\lambda_j \ge 0,j = 1, \ldots ,n\)

\( s_i^- \ge 0,i = 1, \ldots ,m\)

\(s_i^+ \ge 0,r = 1, \ldots ,s\)

(1)

\({\text{max}} \beta_k^R = max \left( {w_y \left( { \sum \limits_{r{ }} \varpi_y^r \alpha_k^r } \right) + w_x \left( { \sum \limits_{i{ }} \varpi_x^i \zeta_k^i } \right)} \right)\)

\({\text{s.t.}}\mathop \sum \limits_{j = 1}^n \lambda_j y_{rj} \ge y_{rk} + \alpha_k^r g_{yr} ,r = 1, \ldots ,s,\)

\(\mathop \sum \limits_{j = 1}^n \lambda_j x_{ij} \le x_{ik} - \zeta_k^i g_{xi} , \, i = 1, \ldots ,m\)

\(\mathop \sum \limits_{j = 1}^n \lambda_j = 1, \, \lambda_j \ge 0, \, j = 1, \ldots ,n,\)

(2)

Where the value of 0 < ρ < 1 can be seen as the ratio of average inefficiencies of inputs and outputs. A DMUk is SBM-efficient if \(\rho^* = 1\), meaning that the slacks (\(s_i^-\) and \( s_i^+\)) are null for all the inputs and outputs

 

Where \(\alpha_o^r\) and \(\zeta_o^i\) are the inefficiency values for every output and input, respectively, and \( \beta_k^R\) is the overall inefficiency. \(\zeta_o^i g_{xi}\) is the reduction needed on \(x_{ik}\) and \(\alpha_o^r g_{yr}\) is the needed increase on \(y_{rk}\) to turn DMUk into an efficient DMU. The directional vectors \({\varvec{g}}_x\) and \({\varvec{g}}_y\) are such that \(\left( { - {\varvec{g}}_x ,{\varvec{g}}_y } \right) = \left( { - {\varvec{x}}_k , {\varvec{y}}_k } \right)\)

 
  

A DMUk is efficient when \(\beta_k^R = 0\). \(w_y\) and \(w_x\), with \(w_y + w_x = 1\), are weights that assign the importance of the outputs and inputs. The importance of the inefficiencies of every factor is defined such that \(\sum \limits_{r{ }\epsilon O} \varpi_y^r = 1\), \(\sum \limits_{i{ }\epsilon I} \varpi_x^i = 1\)

 

Model (1) can be transformed into a linear problem, by using a positive scalar variable t (for further details see Tone (2001)).

The set of benchmark DMUs for each inefficient DMUk in both models is \(E_k = \left\{ {j: \lambda_j^* > 0,j = 1, \ldots ,n} \right\}\) and the reference point for each inefficient DMUk is obtained as:

$$ \left( {{\varvec{\hat{x}}}_k ,{\varvec{\hat{y}}}_k } \right) = \left( {{\varvec{x}}_k - {\varvec{s}}^{ - *} ,{\varvec{y}}_k + {\varvec{s}}^{ + *} } \right) = \left( {\sum_{j\epsilon E_k } {\lambda _j^* {\varvec{x}}_{\varvec{j}} } ,\sum_{j\epsilon E_k } {\lambda _j^* {\varvec{y}}_j } } \right). $$
(3)

The WRDDM model’s measure of inefficiency may be translated into a slack based measure departing from problem (4):

$$ \begin{gathered} {\text{max}}\left( {w_y \left( { \sum \limits_{r{ }} \varpi_y^r \frac{{s_r^{ +^{\prime} } }}{{g_{yr} }}} \right) + w_x \left( { \sum \limits_{i{ }} \varpi_x^i \frac{{s_i^{ -^{\prime} } }}{{g_{xi} }}} \right)} \right) \hfill \\ {\text{s.t.}}\mathop \sum \limits_{j = 1}^n \lambda_j y_{rj} = y_{ro} + s_r^{ +^{\prime} } ,r = {1}, \ldots ,s, \hfill \\ \mathop \sum \limits_{j = 1}^n \lambda_j x_{ij} = x_{io} - s_i^{ -^{\prime} } ,i = {1}, \ldots ,m, \hfill \\ \mathop \sum \limits_{j = 1}^n \lambda_j = 1, \, \lambda_j \ge 0, \, j = 1, \ldots ,n, \hfill \\ s_r^{ +^{\prime} } \ge 0\left( {\forall_r } \right),s_i^{ -^{\prime} } \ge 0\left( {\forall_i } \right). \hfill \\ \end{gathered} $$
(4)

Let \(\left( {s_r^{ + {*}^{\prime} } , \, s_i^{ - {*}^{\prime} } , \, \lambda_j^{*} } \right)\) be the optimal solution to problem (4). The WRDDM inefficiency measure can be decomposed from:

$$ \left( {w_y \left( { \sum \limits_{r{\text{~}}} \alpha _o^{r^{*^{\prime} } } } \right) + w_x \left( { \sum \limits_{i{\text{~}}} \zeta _o^{i*^{\prime} } } \right)} \right),\;{\text{where}}\;\alpha _o^{r*^{\prime} } = \varpi _y^r \frac{{s_r^{ + ^{\prime} } }}{{g_{yr} }} \cdot {\text{e}} \cdot \zeta _o^{i*^{\prime} } = \varpi _x^i \frac{{s_i^{ - ^{\prime} } }}{{g_{xi} }} $$
(5)

3 Data

The input and output indicators used for measuring the efficiency of ERDF dedicated to ICT adoption in SMEs were picked from a set of measures mandated by the EU.Footnote 1

3.1 Input and Output Factors

3.1.1 “Total Eligible Costs Decided” and “Total Eligible Spending”

The criteria used to measure the capacity of the OPs’ absorption are “total eligible spending” and “total eligible costs decided”. The former concerns qualified costs that have been documented and verified by a decision authority. Therefore, this component is employed as an output since the more the value assigned to it, the greater the financial implementation of each operation. The latter is regarded as an input since it relates to the resources that are available to the initiatives selected for support, which must be maintained as low as possible.

3.1.2 Number of Operations Supported

The “number of operations supported” alludes to the number of projects financed by the ERDF. The greater the number of projects funded, the greater the prospect of improving organizational ICT usage. Consequently, this indicator is an output.

Table 1 shows the descriptive statistics of the inputs and outputs used in the OPs’ efficiency appraisal.

Table 1 Descriptive statistics of the inputs and outputs
Full size table

4 Discussion of Results

Results for the SBM model were obtained using Max DEA software, whereas for the WRDD model were obtained through an Excel Visual Basic-based application created by the authors for solving our DEA models that uses Excel Solver as the backend. The descriptive statistics thus obtained with both models are shown in Fig. 1.

Fig. 1
Four separate illustrations depict the results obtained with S B M and W R D D models. The mean, median, standard deviation, minimum, maximum, and count values are given for all the results.

Source Authors’ own elaboration

Descriptive statistics of the results obtained with the SBM and WRDD models.

Full size image

Figure 1 shows, as expected, that efficient OPs have larger mean scores than inefficient ones. Nevertheless, there is a lower variability of the scores obtained for efficient OPs with the WRDD model when compared to those computed with the SBM model (efficiency within the range [1, 1.31] and [1.00, 1.71] and at least 50% of efficient OPs having efficiency values greater 1.07 and 1.16 according to the WRDD and SBM models, respectively). Besides, inefficient OPs have a wide variety of efficiency ratings, particularly with WRDD model scores ranging from −205.37 to 0.98 against those of the SBM model which goes from 0 to 0.96. On the one hand, the WRDD model has a weighted additive objective function (that weights the average percentual inefficiency of the inputs and outputs factors) whose inefficiency scores can be higher than one (i.e., it might be necessary for an increase higher than 100% on the outputs), and thus the inherent efficiency scores \(\left( {1 - \beta_k^R } \right)\) can be negative. Overall, these findings show the highest discriminatory power of inefficient OPs in the WRDD model.

The top four OPs elected as benchmarks are “Extremadura—ERDF” (27 and 29, with SBM and WRDD models, respectively), “País Vasco—ERDF” (26 and 24, with SBM and WRDD models, respectively), “Provence-Alpes-Côte d’Azur—ERDF/ESF/YEI” (20 with both models) and “Multi-regional Spain—ERDF” (14 and 21, with SBM and WRDD models, respectively)—see Fig. 2. Remarkably, each of these regions is located in MS that are among the biggest players in ICT support for SMEs (Pellegrin et al., 2018). Furthermore, the results of the Spanish regional OPs are in line with the conclusions of Ruiz-Rodríguez et al. (2018). For these researchers, Spanish regions have an intermediate or greater degree of digital innovation than their EU peers, as well as a reduced digital divide (DD) (i.e., a smaller disparity in firm digitization) than that of the other European MS. Interestingly, in the research reported by Ruiz-Rodríguez et al. (2018), Greece and Bulgaria were identified as the countries whose enterprises were last placed concerning digital adoption (based on 2015 data), highlighting the measures undertaken by these MS in the implementation of ICT by SMEs in the latest program time frame.

Fig. 2
Two stacked histograms depict the number of times as benchmark and the ranking of S B M and W R D D for ten various characteristics.

Source Authors’ own elaboration

Characteristics of efficient OPs according to the SBM and WRDD models.

Full size image

Both the SBM and the WRDD models also deliver information on the required changes that inputs and outputs should undergo to turn inefficient OPs into efficient ones (Fig. 3).

Fig. 3
Two separate stacked histograms depict the total eligible spending and total eligible cost decided in Euros and percentage, and the number of operations in percentage for S B M and W R D D models.

Source Authors’ own elaboration

Improvement potential for the OPs with the SBM and WRDD models.

Full size image

The ‘number of operations supported’ has the largest potential for improvement in both models (313% and 376%, according to the SBM and WRDD models, respectively), while ‘eligible cost decided’ necessitates a higher decrease with the SBM model (−22% against −5% with the WRDD model) and ‘eligible spending’ necessitates smaller changes with this same model (2% against 17% with the WRDD model)—see Fig. 3. Overall, the WRDD ends up being more demanding about the outputs whereas the SBM model is more demanding regarding the input. Finally, with the decomposition of inefficiency (with the WRDD model) it is also possible to conclude that the ‘Number of operations’ supported has an average effect on the inefficiency of 93%, while ‘Total eligible spending’ only has an impact of about 6%.

4.1 Robustness Study

Deterministic values are used both for inputs and outputs in conventional DEA approaches. However, the values considered to represent the input and output data are sometimes uncertain. Usually, the original DEA model is converted into two models, thus allowing to attain the upper and lower bounds of the efficiency scores. In the first model, all DMUs’ outputs are raised while their inputs are lowered, except for the DMU under examination (i.e., DMUk worsens its efficiency performance while the remaining DMUs improve their efficiency performance). In the second scenario, the opposite case occurs (Henriques & Marcenaro, 2021). The robustness assessment was conducted by employing data perturbations of 5% and 10% on the original data.

4.1.1 Robustness Study

According to our findings in Figs. 4a and b, only four OPs managed to hold efficiency for both data change scenarios according to both DEA models: “Enterprise and Innovation for Competitiveness—CZ—ERDF” (1st in terms of robustness with the SBM and 3rd with the WRDD models, respectively), “Provence-Alpes-Côte d’Azur—ERDF/ESF/YEI” (2nd in terms of robustness with both models), “Multi-regional Spain—ERDF” (3rd in terms of robustness with the SBM and 4th with the WRDD models, respectively), and “Extremadura—ERDF” (4th in terms of robustness with the SBM and 1st with the WRDD models, respectively). Additionally, while with the WRDD model 41% of the OPs are robustly inefficient, with the SBM model this value rises to 55%. Ultimately, these data indicate a weak utilization of ERDF in SMEs’ ICT adoption. These findings support those of Pellegrin et al. (2018), who established that the EU lags behind its competitors (United States, Japan, and South Korea) in respect of ICT usage and connectivity, especially among SMEs.

Fig. 4
Two line graphs depict the efficiency, efficiency U 5 percent, efficiency L 5 percent, efficiency L 10 percent, and efficient U 10 percent for S B M and W R D D models. a) There is a little fluctuation in the trend. b) The trend remains the same for many categories.

Source Authors’ own elaboration

Robustness analysis per OP with the SBM a and WRDD b models.

Full size image

5 Conclusions and Further Research

The main objective of this paper is to contrast the efficiency reached in the procedural efficiency of 51 OPs related to ICT assistance in SMEs from 16 EU MS according to two DEA models. We employ both the SBM and the WRDD models, which are both non-radial and non-oriented, to evaluate if the results obtained were robust.

Our main conclusions are provided next.

RQ1: “What indicators restrict the successful use of ERDF allocated to enhance ICT adoption in EU SMEs?”

The ‘number of operations supported’ has the greatest room to improve in both models (313% and 376%, as per SBM and WRDD models, respectively), whereas ‘eligible spending decided’ usually requires a significant decline with the SBM model (−22% versus −5% with the WRDD model) and ‘eligible spending’ requires minor changes with the very same model (2% versus 17% with the WRDD model). Essentially, the WRDD model is much more demanding of outputs, while the SBM model is much more demanding of inputs. Finally, based on the breakdown of inefficiency (using the WRDD model), it is reasonable to determine that the ‘Number of operations’ supported has a 93% average influence on inefficiency, whilst ‘Total eligible spending’ only has a 3% impact.

RQ2: “What OPs are generally elected as benchmarks across the assessed time frame?”

The four leading benchmarks are “Extremadura—ERDF” (27 and 29, using SBM and WRDD models, respectively), “País Vasco—ERDF” (26 and 24, using SBM and WRDD models, respectively), “Provence-Alpes-Côte d’Azur—ERDF/ESF/YEI” (20 using both models), and “Multi-regional Spain—ERDF” (14 and 21, using SBM and WRDD models, respectively). Surprisingly, every one of these regions is situated in MS that have a major participation in ICT assistance for SMEs (Pellegrin et al., 2018). In this regard, it is worth noting that two of these OPs are in Spain (Pellegrini et al., 2018), where the use of “vouchers” has proven to be an effective means of engaging SMEs and giving them the help that is simple to administer and specific to their requirements.

RQ3: “What OPs demonstrate more resilience in terms of their efficiency classification in the context of changes in the indicators used?”

Just four OPs continued to preserve efficiency for both data changes under both DEA models: “Enterprise and Innovation for Competitiveness—CZ—ERDF” (1st in terms of robustness with the SBM and 3rd with the WRDD models, respectively), “Provence-Alpes-Côte d’Azur—ERDF/ESF/YEI” (2nd in terms of robustness with both models), “Multi-regional Spain—(4th in terms of robustness with the SBM and 1st with the WRDD models, respectively). Furthermore, whereas the WRDD model has 41% of the OPs being robustly inefficient, the SBM model has 55%.

All in all, the results obtained with both models are generally consistent. In the end, it can be stated that SMEs’ recourse to ESIF (particularly ERDF) is restricted. These findings might be related to their lack of organizational knowledge to interact with the numerous technicalities involved in applying for and executing ERDF operations. As regards ICT, this challenge becomes more urgent when compared to ‘conventional’ SME operations. As a result, operations in a sector known for sudden growth, such as ICT, need greater flexibility and competence. Therefore, MA should seek ways to give extra support that simplify operations while meeting the needs of SMEs.

Additionally, our research highlights the shortage of measures for measuring the performance of ESIF funding committed to ICT help in SMEs. Future work should address the impact of contextual variables on the outcomes herein obtained to understand the true reasons behind these poor results.