1 Introduction

Proper measurement of digital development is a key factor in understanding the further strategies and elaboration of innovation development plans for sustainable development.

Digital transformation is the process of changing the economic and social level of development of businesses and countries through digitalization. While some of its components can only be measured at the firm level, they will ultimately have a crucial impact on the economic and social development of whole industries and even countries, and can spell the difference between success stories (winners) and failures (losers) for both firms and societies.

Moreover, despite the obvious advantages brought about by rapid digitalization, there are also disadvantages (such as the deepening of the digital divide and increasing disparity in employment opportunities), and it is important to provide the necessary balance between technological development and human activity in the process of digital transformation.

In our earlier papers [Bánhidi et al. 2019 and 2021], we used a dataset comprising scores in these five principal dimensions for the 28 EU countries and the Russian Federation, compiled from the European Commission’s I-DESI report. This paper continues the discussion of digital development of Russia and other constituent countries of the former Soviet Union using the Portulans Institute’s Network Readiness Index.

The Network Readiness Index (NRI) is a composite index based on 4 main pillars, 12 sub-pillars and 60 indicators (Portulans Institute, 2021). The pillars and sub-pillars are presented in Table 1, the full description of indicators and their sources is given in Dutta and Lanvin (2020:287–300).

The NRI measures the digital development of countries based on the development of technology, people, governance, impact, which are the four main pillars.

Table 1 Pillars and sub-pillars of Network Readiness Index
Full size table

Technology is the backbone of the network economy and therefore is the first pillar of Network Readiness Index, which consists of three sub-pillars for Access, Content and Future technologies. The People pillar refers to measurement of ICT usage at three levels of analysis: Individuals, Businesses and Governments. This measurement reflects the main stakeholders’ access to technology resources. The Governance pillar reflects the effectiveness of institutions that provide and regulate activity within the network economy. Finally, the Impact pillar measures the economic, social and human impacts of digitalization.

A key advantage of the NRI over the previously used I-DESI database is that it covers a much wider range of countries (134 countries, listed in our Appendix), including not only European and other developed countries, but also developing and least-developed (LDC) countries. This makes it possible, for example, to compare the successor states of the Soviet Union, which is one of the main objectives of our current paper. However, this quest for completeness may come with the trade-off that the granularity and reliability of the data may not be as high, especially with respect to less developed countries. In addition, some of the basic data on internet domain registrations are not available in the public database itself due to trade secrets.

Our primary research objective was to use this comprehensive database to assess and compare countries’ digital competitiveness using DEA methodology and various objective weighting methods. In the absence of the explicit input criteria required by classical DEA, we sought to provide a viable alternative framework to the equal weights scheme of the original NRI scoring model for ranking the Central and Eastern European member states of the European Union and the now independent constituent republics of the former Soviet Union.

In the next sections, we provide a brief literature review and present our proposed alternatives to the NRI scoring model using the Data Envelopment Analysis (DEA) Without Explicit Input (WEI) method and the Common Weight Analysis (CWA) method. We then present the main features of our country rankings and our conclusions.

2 Literature review

The literature on country rankings is very broad and diverse, so the following is intended to be a brief, non-exhaustive review of works on composite indicators that are roughly similar to the one we use, with a particular focus on those that applied the Data Envelopment Analysis methodology. The works are presented in chronological order (except where thematically related).

Despotis (2005b) applies a DEA-like index-maximizing model to produce a new measure of human development akin to the Human Development Index (HDI), but without equal weighting of the three main dimensions. The author uses this methodology to assess the relative human development performance of countries in the Asia-Pacific region. Global estimates of human development based on optimal joint weights are then obtained using a goal programming model. The proposed measure has a high correlation with the HDI, but it is claimed to be superior based on its objective (optimized) weights. An important difference between the approach of Despotis and our own is that we use a DEA model without explicit weights, while they use real GDP per capita as an input.

Bérenger and Verdier-Chouchane (2007) propose a measure of well-being (as an alternative to the well-known HDI) consisting of two components, standard of living and quality of life, separating measures of resource availability and functioning. Their empirical results for 170 countries are based on two multidimensional analyses, the Totally Fuzzy Analysis (TFA) and the Factorial Analysis of Correspondences (FCA), focusing on the situation of African countries. Importantly for our paper, they also find that the chosen methods and weighting systems have a noticeable impact on the rankings, but note that the two methods are complementary as they each use a different type of aggregation (for FAC, the weights are objectively calculated, similar to our own weight vectors). However, by looking at the rank correlations, they find that the differences between the two methods are not statistically significant.

Cherchye et al. (2008a) investigate the sensitivity of rankings to weighting and aggregation. They propose robust, consistent ranking methods that consider multiple weighting methods together and can be trained using linear programming techniques (similar to our DEA weights). They demonstrate the viability of their proposed method using the HDI and examine the robustness of country classification and ranking using it. The authors conclude that while the classification of countries on the basis of the HDI is likely to be robust, the rankings themselves are not.

Somarriba and Pena (2009) evaluate the advantages and disadvantages of three possible methodologies (PCA, DEA and distance measure “P2”) for obtaining synthetic indicators in the areas of welfare and quality of life, applying their methodology to the EU Member States. The authors find that the DEA methodology could be useful for constructing synthetic indicators because it can guarantee impartiality in the weights, but it can also allow the introduction of expert opinions. On the other hand, they note that in their study this method likely overestimates the level of quality of life and lacks discriminatory power to classify countries. They also argue that using this method to construct a measure of quality of life could be difficult because the classification of certain variables of interest as inputs or outputs by the investigator could be arbitrary and have a significant impact on the results. Dobos and Vörösmarty (2023) also illustrate this issue, and the potential bias introduced by categorizing criteria as inputs or outputs, using the illustrative example of a supplier selection problem. However, while this issue of criterion classification is indeed potentially problematic, we would like to emphasize that our proposed approach is not affected by it, since we use a specific DEA methodology that does not require the classification of any sub-pillar as an input criterion.

Karagiannis and Karagiannis (2020) propose a weighting scheme for the construction of composite indicators based on Shannon entropy. The claimed advantages of their methodology are that it provides a set of common (or group-specific) weights across decision units, allowing full comparison and ranking, and is easy to implement. The authors explore the potential of their proposed methodology by using it to re-estimate the HDI, and conclude that it is superior to the equal weights scheme employed by the original HDI scoring model.

Omrani et al. (2020) also propose a new approach to calculate HDI-like scores based on the BWM (best worst method) and MULTIMOORA methods and apply their proposed approach to evaluate the provinces of Iran. The authors incorporate new criteria for each of the main dimensions of the HDI in their index. The BWM is used to assign weights to the criteria in each dimension and then the Iranian provinces are ranked using the MULTIMOORA method.

Van Puyenbroeck and Rogge (2020) set out to compare regional human development using frontier difference indices. The authors propose an alternative approach to assessing intergroup differences by applying a chained and an intertemporal version of the Global Frontier Index to regional categories of countries in the construction of the HDI. The authors use a robust version of DEA to deal with potential biases in certain composite HDI estimates.

Petróczy (2021) proposes an alternative ranking based on World Bank data on bilateral remittances for the period 2010–2015 using a least-squares method. The author compares their alternative measure with the HDI and World Happiness Report rankings based on pairwise Spearman’s rank correlations, which show a modestly strong relationship between the indices. The proposed least squares method is judged to have favorable axiomatic properties, but appears to be somewhat sensitive to aggregation, requiring properly disaggregated data for valid, unbiased results.

Hosseinzadeh et al. (2023) apply DEA as a nonparametric method to evaluate the efficiency of assets in large-scale portfolio problems. The authors considered multiple asset performance criteria in a linear DEA model, reducing the dimensionality of portfolio problems. They first introduce several reward/risk criteria, commonly found in portfolio literature, to identify features of financial returns. Then they suggest DEA input/output sets to preselect efficient assets in a large-scale portfolio framework. Finally, they evaluate how these preselected assets impact various portfolio optimization strategies. Their empirical analysis finds that DEA preselection was superior to classic PCA factor models for large-scale portfolio selection problems, and also outperformed the S&P 500 index.

2.1 The models: DEA without explicit inputs and common weight analysis

The first full-fledged Data Envelopment Analysis (DEA) model was introduced by Charnes et al. (1978), which was followed by countless model versions and extensions, developed by various authors. However, methodological development in this area is still very active, also in the Central European region (cf. e.g. Jablonský et al. (2022).

When we evaluate the NRI data, for each of the twelve sub-pillars, the ideal value is as high as possible, so these can all be considered as output criteria in a DEA-type model. As there are no criteria to be minimized, no input criteria exist. This type of model is called either the DEA model Without Explicit Inputs (DEA/WEI) or the DEA-type Composite Indicators (DEA/CI) method in the literature. (Cherchye et al. 2008b; Dobos and Vörösmarty 2014) All of the models used are input-oriented (CCR-I), but it should be noted that in this case the orientation of the models does not play a decisive role, because no input criteria are explicitly included in the models. The Data Envelopment Analysis models used are all constant return to scale (CRS) CCR-I/CRS models, so these models will be used in the rest of the paper. Of course, the analyses could also be performed with variable return to scale (VRS) DEA/VRS models. The only change would be that the non-negative sum of the weights would have to be equal to one.

The DEA/WEI model was first applied by Fernandez-Castro and Smith (1994) to a real-world problem and then further examined by Despotis (2005a) and Liu et al. (2011). Due to the shape of the model, it was also used to create composite indicators, as mentioned above.

The basic model of DEA cannot be used to obtain a total order (unambiguous ranking) of countries, which is our goal now, because the efficiency of several decision making units (DMUs, in our case countries) can reach the maximum, i.e. value one, therefore these countries cannot be ranked. However, we may also face the problem that each DMU achieves efficiencies that can be calculated with different weights. Therefore, a method must be found to evaluate all possible decision units with equal weight.

One of the first Common Weight Analysis (CWA) models was proposed by Podinovski and Athanassopoulos (1998). The model is called the Maximin DEA model, where we first find the DMU that gives the lowest efficiency for a given weight vector, and then look for the weight vector for which we maximize this minimum. The method got its name from this procedure.

To solve this problem, a DEA procedure must be found for which all possible DMUs are evaluated with equal weights. This method is called the common weights procedure. The simplest form of the method was proposed by Liu and Peng (2008). The model traces the search for common weights to solve a linear programming problem whose constraint conditions contain linear constraints that include efficiency.

Finally, the third DEA model using common weights determines weights based on compromise programming. This procedure was proposed by Kao and Hung (2005). The objective function here is a distance function, which can be the Manhattan, the Euclidean or the Chebyshev distance function. To use one of these, however, an anti-ideal or ideal point must be defined in the space of decision-making units’ efficiency. Anti-ideal efficiency can be e.g. zero efficiency, while the ideal is either the previously defined DEA efficiency, or the maximum efficiency to be achieved for each DMU (the all-ones vector, hereinafter: vector 1). Only the two ideal efficiencies are used in this paper. For the ideal point, find the weight at which the ideal point is closest to the set of weights. For anti-ideal points, we look for the furthest efficiency. Vector E* comprises the DEA efficiencies, which show the best efficiencies for each DMU.

The vector of possible weights of the DEA model can be determined by the system of Eq. (1) to (2). Inequalities (1) shows the upper limit of DEA efficiency, i.e. one, while inequality (2) defines the non-negativity of weights. The number of decision-making units is p, and vector yj comprises the criteria values for the jth decision making unit, in this case country. The vectors yj can be summarized in the Y matrix. Vector u is the DEA weights. The DEA/WEI weighs are equal to vector u·Y.

$${\bf{u}}\;{{\bf{y}}_j}\; \le \;1;\;j\; = \;1,2,...,p$$
(1)
$${\bf{u}}\; \ge \;0$$
(2)

The objective functions of possible DEA models are listed in Table 2.

Table 2 The DEA models used by the authors to determine the optimal weights
Full size table

Based on our studies, we establish six rankings. This is because the result of the Common Weight Analysis for Manhattan distances gives the same result for both ideal vectors, i.e., the DEA efficiency E* and the summation vector 1 as well. (Toloo 2014)

The distance functions of compromise programming are as follows:

Euclidean distance (k = 2):

$${{\rm{d}}_2}\;({\rm{u}} \cdot {\rm{Y;}}\;{\rm{E)}}\;{\rm{ = }}\;\sqrt {\sum\limits_{j = 1}^p {{{(u \cdot {y_j}\; - \;{E_j})}^2}} }$$

Chebyshev distance (k = +∞): \({{\rm{d}}_{ + \infty }}\;({\rm{u}} \cdot {\rm{Y;}}\;{\rm{E)}}\;{\rm{ = }}\;\mathop {\max }\limits_{1 \le j \le p} \left| {u \cdot {y_j}\; - \;{E_j}} \right|\),

where vector E is a possible ideal efficiency vector, which can be equal to efficiency vectors E* or 1.

2.2 Ranking with the DEA/WEI and CWA methods

Before presenting the results, we show that for the DEA / WEI model, the Manhattan distance gives the same result for all ideal points, so the DEA efficiency and maximum efficiency vectors are the same. Therefore, it is sufficient to determine the efficiency obtained by the common weight analysis method. Since the sum of the given efficiencies does not depend on the weights, it is sufficient to minimize the expression \(- \varvec{u}\bullet \mathbf{Y}\bullet 1\), which means that the minus of the linear function should be maximized. This also means that we got the CWA model back. Therefore, minimizing the Manhattan distance in the DEA / WEI models also leads to the optimization problem of the CWA model.

The mathematical form of the six common weights problems can be described as follows:

u·yj ≤ 1; j = 1,2,…,p. (1’)

u ≥ 0. (2’)

Fi(u) → min/max, i = 1,2,…,6. (3’)

Problems (i) 1, 2 should be maximized, while problems 3, 4, 5 and 6 should be minimized due to distance. The analytical form of the functions is given in Table 2.

2.3 Common weights and ranks of sub-pillars in the DEA models

Table 3 shows the NRI sub-pillar weights and the optimal weights for the six DEA models. Only three indicators have non-negative values in the weights. These are Regulation, Inclusion and SDG Contribution. Interestingly, the Content and Economy sub-pillars do not affect DEA efficiency because they have zero weight in all six DEA models.

Table 3 NRI and common weights for the six models
Full size table

The appendix contains the solution to the efficiency obtained by the six common weights methods. Consider the linear stochastic relationship, i.e. correlations, between the efficiencies determined by the common weights methods and the efficiency of DEA. Two types of correlations can be considered: Pearson’s and Kendall’s tau-b correlations.

Table 4 presents the Pearson correlation coefficients. Since these efficiencies are continuous variables, Pearson correlation can be used. Correlations show a high value. The question can also be asked whether it is worthwhile to calculate efficiencies without defining DEA efficiencies. If there are p DMUs (in our case: countries), then if DEA efficiency is considered to be an ideal virtual DMU, then these efficiencies must be determined first, which means solving p linear programming (LP) problems. Determining common weights is a solution to an additional LP. Conversely, if only the maximum available efficiency, i.e. efficiency one, distance from a possible efficiency is determined, then only a single linear programming problem needs to be solved.

Table 4 Pearson correlation between DEA efficiencies and common weights
Full size table

The result in Table 4 shows that the optimal solution of the Chebyshev distance by minimizing the distance from vector 1 (hereinafter: Cheb-1) has a correlation of 0.943 with the DEA distance, which can be considered strong. Although the correlation is even stronger (0.960) if we consider E* instead of the all-ones vector, the two different computational efficiencies do not show significantly different results. This may suggest that it is not necessary to determine all p DEA efficiencies, which can lead to time and cost savings.

Table 5 Kendall’s tau-b correlation between DEA efficiencies and common weights
Full size table

Kendall’s tau-b correlation shows the correlation between rankings in Table 5. This correlation coefficient shows a strong stochastic correlation when it is higher than 0.7 and close to 1 for the NRI data. The result obtained with the Chebyshev distance mentioned above exhibits the strongest correlation (0.831, 0.807) for both efficiency vectors.

2.4 Clustering countries with common weights

In this section, we present our country rankings, starting with the ranking of all 134 countries (in the dataset) based on the Cheb-1 model, which we consider the best of the six models, due to its high correlation coefficients and the fact that it is not necessary to solve p linear programming models, only a single one. Figure 1 presents seven country clusters of equal size (based on the Cheb-1 efficiencies of their constituent countries), which by and large, shows the disparities between the continents that we could expect based on the well-known geographic divides (e.g. North-South, West-East). The full ranking and the efficiencies are presented in the Appendix, together with the efficiencies and rankings of the other five models.

Fig. 1
figure 1

Source: Own compilation with Mapchart.net

The ranks of 134 countries in the Cheb-1 model (darker shade: higher rank).

Full size image

In the following, we also present some regional rankings, starting with the Central and Eastern European (hereinafter: CEE) Member States of the European Union (10 countries). Figure 2 shows the best and worst rankings and Fig. 3 illustrates the average efficiency of the CEE countries in our six models.

Fig. 2
figure 2

Source: Own compilation

The highest and lowest ranks of the 10 countries in the CEE region.

Full size image
Fig. 3
figure 3

Source: Own compilation with Mapchart.net

The average efficiencies of CEE countries (darker: higher average efficiency).

Full size image

The ranking is topped by Estonia in all six, while Bulgaria or Romania are the worst performers (but the country at the bottom changes according to the model used). On the other hand, the ranking of the other CEE countries shows more marked variation: for instance, the Czech Republic and Poland may be ranked as high as third or as low as 8th depending on the chosen model, while Hungary is ranked 7th to 9th.

The next regional rankings (Fig. 4) are presented for the former Soviet republics (13 countries that declared their independence from the Soviet Union during 1990–1991). Among these countries, the rankings are topped by the three Baltic countries, Estonia, Lithuania and Latvia (in this particular order) in all six models, while Tajikistan is the least developed country regardless of the model used. Russia is ranked as high as 4th in some models, but as low as 11th in one model. The ranking of Azerbaijan is also fairly sensitive to the model chosen, as they may be ranked as high as 5th or as low as 12th.

Fig. 4
figure 4

Source: Own compilation

The highest and lowest ranks of the 13 former Soviet republics.

Full size image

3 Conclusions

In this paper, we demonstrated how the DEA/WEI and DEA/CWA method can be used to provide a viable framework for ranking the former member states (“republics”) of the Soviet Union in the absence of explicit input criteria and predetermined weights that are required by the classical DEA and with predetermined weights for the NRI scoring model. These methods can eliminate the need for a pre-defined weighting system used by the original composite index, and instead use an intrinsic one based on the statistical properties of the dataset.

According to our rankings, the Baltic countries that are Estonia, Lithuania, Latvia demonstrate respectable results in digital development relative to Slavic countries, Caucasian and Central Asian countries of the former Soviet Union, while the position of Russia or Azerbaijan is fairly sensitive to the model chosen. The case of Estonia is an excellent example of a nation that, despite its initial peripheral position and low economic potential, has become a digital leader and an example of best practice that could be followed by other countries in the two regions.

Of course, our research has some limitations, some of which are related to the NRI database itself, which is fairly comprehensive and well maintained, but has some data availability issues with respect to the internet domain registrations that are affected by trade secrets. Other limitations could be related to the peculiar characteristics of the DEA/WEI method, which led to the exclusion of the “Content” and “Economy” sub-pillars from the ranking results, although we should note that the implications of this may not be too dramatic due to the significant multicollinearity among the NRI dimensions, which we plan to explore in a future paper.

Future research could also investigate the sub-pillars with a number statistical methods, such as correlational analysis, principal components method or cluster analysis. The analysis could also be extended to the level of indicators and to the previous and subsequent reports of the NRI, and a comparison with the somewhat similar I-DESI database could also be made.