Composite indices, alternative weights, and comparison robustness

Abstract

Composite indices are widely used in development economics and can often be highly influential. Yet most remain controversial owing to inter alia the arbitrary selection of component weights. Several studies have proposed testing the robustness of rankings generated by composite indices with respect to alternative weights but have not provided sufficient guidance on the choice of these alternatives. This paper proposes a holistic yet theoretically novel approach for selecting sets of alternative weights and assessing comparison robustness that is applicable to linear composite indices with any finite number of dimensions. Our approach is founded on the main normative assumption that a consensus has been reached on the minimum and the maximum allowable weights that should be assigned to the components. This approach is applied to robustness testing of inter-temporal country improvements generated by arguably the world’s most influential composite development index, the UNDP Human Development Index.

1 Introduction

Composite indices might not rule our world, but they are highly influential in it. This is particularly the case with composite international indices that seek to assess the achievements of countries by various criteria. As Høyland et al. (2012, pp. 1) observe, “one can hardly open a newspaper without finding a reference to an international index;” they also refer to the “tyranny” of the rankings produced by these indices. The vast majority of composite indices fall into a class of what Ravallion (2011) describes as ‘mashups’. Ravallion (2011, pp. 1) defines a mashup as a “composite index for which existing theory and practice provides little or no guidance for its design ... (with) an unusually large number of moving parts, which the producer is essentially free to set.” Ravallion (2011, pp. 1) points to a number of pitfalls in the use of these indices, stressing that “clearer warning signs are needed for users” of them.

A key moving part of most composite indices, which producers are free to set, is their component weights. Most are set arbitrarily, with the most common practice being to set equal weights for each component. This choice of weights is due to an uncertainty about the correct weights arising from a lack of theoretical or other guidance.¹ Uncertainty about the setting of weights has clear implications for the interpretation of a composite index. Perhaps the most obvious implication is for index rankings.² Index rankings are a function inter alia of the weights, and, if there is uncertainty about the correct weights, it must follow that there is uncertainty over the veracity of these rankings.

Uncertainty over composite indices’ weights has been acknowledged in a number of previous studies. These studies have sought to analyze the robustness of ranks provided by equally weighted composite indices to alternative weights (Cahill 2005; Cherchye et al. 2008; Foster et al. 2009, 2013; Permanyer 2011; Zheng and Zheng 2015). Each study looked at the well-known UNDP Human Development Index (HDI), although their analyses are applicable to most composite indices. They did not propose replacing equal weights with alternative weighting schemes, instead advocating tests for the robustness of composite indices rankings to their assigned equal weights so as to aid interpretation, in broadly the same way that significance tests are used in statistical analysis.

Our paper contributes to research on the robustness of linear composite index comparisons of indices that fall into the Ravallion mashup class. Its fundamental premise is that in the absence of rigorous scientific guidance on the setting of weights for the components of composite indices, assigning equal weight to their components is broadly defensible provided that, in Ravallion’s words, ‘warning signs’ are provided as to the implications of this for the rankings they yield.

The prime objective of this paper is to address a difficulty encountered by previous studies: the selection of a set of alternative weighting schemes for assessing rank robustness. This selection is a requirement of the tests proposed by these studies, yet none provide sufficient guidance for such selection. We propose a general yet theoretically novel approach that allows selecting a set of alternative weighting schemes as well as assessing relevant comparison robustness. Our approach is founded on the main normative assumption that a consensus has been reached on the minimum and the maximum allowable weights that should be assigned to the components. In other words, there is a consensus on an upper bound and a lower bound on weights, which then yields a particular set of alternative weighting schemes with respect to which the robustness of pairwise comparisons should be tested. We consider two variants. One is where we allow the weight for every dimension to vary independently yet uniformly between a common upper bound and a common lower bound. The other is where we allow the weights to vary, but not necessarily uniformly for every dimension.

The approach that we propose is applicable to linear composite indices or their monotonic transformations with any finite number of dimensions. We show an application of our approach to the well-known HDI, whose annual publication in the Human Development Reports is eagerly awaited and receives enormous attention in the media, policy circles, and elsewhere. The HDI is a composite index that combines country achievements in health, education, and income. We evaluate the prevalence of robust country-specific inter-temporal HDI comparisons for six successful sub-periods during the years 1980–2013. To the best of our knowledge, testing the robustness of inter-temporal comparisons of the HDI or other composite indices of its general type has not previously been attempted. This is an important exercise as the HDI is also used to assess country-specific changes over time.³

The paper is structured as follows. Section 2 introduces the notation and framework. Section 3 outlines the theoretical contribution of our paper in comparison to the existing literature. Section 4 develops and presents the approach, where the weight on each dimension is allowed to vary uniformly. Section 6 applies this approach to assess the inter-temporal robustness of the country HDIs. Section 5 extends the approach developed in Sect. 4 to certain cases, where weights are allowed to vary non-uniformly. Section 7 provides concluding remarks.

2 Notation and framework

We assume that there are a fixed number of $D \in \mathbb {N} \setminus \{ 1 \}$ dimensions or components, where $\mathbb {N}$ is the set of positive integers. Let $\mathcal {X} \subseteq \mathbb {R}^D$ denote the non-empty set of performance vectors to be ranked. A performance vector $x \in \mathcal {X}$ summarises the normalised performances in D dimensions. We denote any dimension by subscript d and the weight assigned to the dimension by $w_d$. The weight assigned to one dimension in comparison to any other dimension represents the relative importance of the former compared to the latter. For example, if all dimensions are equally weighted (i.e., $w_d=1/D$ $\forall d$), then they are considered equally important relative to each other.

Weights assigned to D dimensions are summarized by vector $w=(w_1,\ldots ,w_D)$. We refer to a vector of weights as a weighting scheme. We make two natural assumptions about the weights that they (i) are non-negative (i.e., $w_d \ge 0$ $\forall d$) and (ii) sum to one (i.e., $\sum \nolimits _{d=1} ^D w_d =1$). We denote all possible D-dimensional weighting schemes by $\mathcal {W}$, such that $\mathcal {W}= \{ (w_1,\ldots ,w_D) \ | \ w_d \ge 0 \ \forall d, \sum \nolimits _{d=1} ^D w_d=1 \}$.

Using a performance vector $x \in \mathcal {X}$ and a weighting scheme $w \in \mathcal {W}$, a composite index is defined as $C(x;w)= \sum \nolimits _{d=1} ^D w_d x_d$. For any two performance vectors $x,y \in \mathcal {X}$, y has an equal or higher composite index value than x at an initial weighting scheme $w^0 \in \mathcal {W}$, whenever $C(y;w^0) \ge C(x;w^0)$. This initial comparison between y and x is stated to be robust with respect to a non-empty set of weighting schemes $\Delta$ such that $w^0 \in \Delta \subseteq \mathcal {W}$ and is denoted by $y C_ \Delta x$, if and only if $C(y;w) \ge C(x;w)$ for all $w \in \Delta$.⁴

We will be using the following additional notation in our paper. We denote a D-dimensional doubly stochastic matrix, which is a non-negative square matrix with every row and every column summing to one, by $\mathbf{B}$. The set of all D-dimensional doubly stochastic matrices is denoted by $\mathcal {B}$. A D-dimensional permutation matrix, which is also a non-negative square matrix with every row and every column having only one element equal to one and the rest of the elements equal to zero, is denoted by $\mathbf{P}$. We denote the factorial of any $n \in \mathbb {N}$ by n!, such that $n! = n \times (n-1) \times \cdots \times 1$. Finally, we denote an n-dimensional vector of ones by $\mathbf {1}_n =( \underbrace{1,\ldots ,1}_n )$.

3 Set of reasonable alternative weighting schemes

How should a reasonable set of alternative weighting schemes be defined for testing the veracity of composite index comparisons? This issue has been well recognized in the academic literature, with a number of corresponding tests having been proposed and applied, but none provide sufficient guidance for selecting a reasonable set of alternative weighting schemes.

Cherchye et al. (2008), for instance, propose a test for the robustness of pair-wise comparisons to simultaneous changes in the index’s weights, component variables’ normalization, and aggregation methods, obtaining conditions that kept the comparison under the original weighting scheme preserved. The variation in weight of each dimension for testing robustness in their framework is determined by the quantiles of the raw data figures. Although the robustness tests are conducted for different quantiles, no guidance is provided on how a particular quantile should be selected.

Foster et al. (2009, 2013) propose an epsilon-contamination model to devise an approach for choosing a set of alternative weighting schemes to be used in assessing composite index comparison robustness. In this approach, one is assumed to have only partial confidence that the initial weighting scheme is correct and any other weighting schemes could be feasible alternatives. The level of confidence one places on the initial weight determines the size and the shape of the set of alternative weighting schemes. The approach determines only a particular shape of alternative weighting schemes that are merely homothetic contractions of the entire set of weighting schemes $\mathcal {W}$. Furthermore, it can be practically difficulty to determine the level of confidence one places on the initial weighting scheme. Permanyer (2011) also envisages the need for robustness testing by considering a set of alternative weighting schemes around the initial weighting scheme and as an example applies the Foster et al. (2009, 2013) approach.

A normative framework for determining an alternative set of weighting schemes requires a process of strong justifications. Zheng and Zheng (2015) sought to avoid this requirement. Using a fuzzy set theoretical framework, they avoided starting with any initial weighting scheme by considering all possible weighting schemes to be potential alternatives while proposing a robustness measure for gauging the strength of pairwise comparisons. It should however be noted that all possible weighting schemes include those that assign the entire weight to one dimension and zero weight to the remaining dimensions. In these cases, the entire ranking is determined by any one dimension. Allowing such possibilities (entire weight to one dimension and zero weight to the remaining) however goes against the spirit of multidimensionality, which should reflect strictly positive contributions of multiple dimensions to the final index score. There is strong justification, therefore, for not considering these extreme weighting schemes as meaningful alternatives. There can also be arguments against assigning excessively high or low weight to any dimension by any alternative weighting scheme.

In our paper, we provide a novel, flexible and holistic approach for determining a set of feasible alternative component weighting schemes. Our primary assumption is that the process of choosing a set of reasonable alternative weighting schemes is subject to a general consensus that the weight on any dimension should not be allowed to be higher than a particular value (maximum weight or upper bound) and should not be allowed to be lower than a particular value (minimum weight or lower bound). We refer to this new approach of choosing a set of alternative weights for testing robustness as max–min bound approach. We develop two variants of this approach: One is referred to as the uniform max–min bound approach and the other is referred to as the non-uniform max–min bound approach. The motivation for the uniform max–min bound approach is the most frequent practice of equally weighting every dimension with the argument that there is no a priori reason or justification for doing otherwise. Similarly, one may argue that there is no a priori reason for allowing weights on different dimensions to vary to different extents. On the other hand, in some cases if there exists strong justifications for allowing weights to vary in different extents, then that should motivate the non-uniform max–min bound approach. In the next section, we formally introduce the uniform max–min bound approach; whereas in Sect. 5, we present certain cases of the non-uniform max–min bound approach.

4 The uniform max–min bound approach

We first show how one may determine a set of alternative weighting schemes $\Delta$ for checking the robustness of pairwise comparisons when there is neither any a priori reason for treating different dimensions with different importance nor any a priori reason for allowing weights to vary in different extents. In this case, the initial weighting scheme $w^0$ assigns equal weight to all dimensions. The approach yields a continuum of feasible alternative weighting schemes with respect to which the robustness of pairwise comparisons should be evaluated. We next formally present the approach.

Suppose there is a consensus that the weight on any dimension should not be lower than $\alpha \in [0,1/D)$ and the weight on any dimension should not be higher than $\beta \in (1/D,1]$. Then, $\Delta = \{ w_1,\ldots ,w_d \ | \ \alpha \le w_d \le \beta \ \forall d \ \mathrm {and} \ \sum \nolimits _{d=1} ^D w_d=1 \}$. Note that the restrictions $\alpha <1/D$ and $\beta >1/D$ prevent $\Delta$ to be singleton. It turns out that $\Delta \in \mathcal {W}$ is bounded and is a convex hull of a finite number of weighting schemes. Once these finite number of weighting schemes are obtained, then one may need to compare a pair of composite index values at these weighting schemes in order to conclude robustness of an underlying pair-wise comparison with respect to all weights in $\Delta$. What are these finite number of weighting schemes then? We answer this question resorting to the majorization theory of measurement.

Definition 1

Any weighting scheme $w' \in \mathcal {W}$ is not more unequal than any other weighting scheme $w \in \mathcal {W}$ if and only if $w'=w \mathbf{B}$ for any $\mathbf{B} \in \mathcal {B}$ (Marshall and Olkin 1979, pp. 22).

A D-dimensional doubly stochastic matrix $\mathbf{B} \in \mathcal {B}$ is the convex hull of a maximum possible D! permutation matrices. Technically, ${\mathbf{B}}= \sum \nolimits _{m=1}^{D!} \omega _m {\mathbf{P}}_m$, where ${\mathbf{P}}_m$ is the mth D-dimensional permutation matrix and $\omega _m$ is a weight attached to ${\mathbf{P}}_m$ such that $\omega _m \ge 0$ and $\sum \nolimits _{m=1} ^{D!} \omega _m =1$. It follows that the relation between $w'$ and w in Definition 1 can be expressed as $w' = w {\mathbf{B}} = w \sum _{m=1} ^{D!} \omega _m {\mathbf{P}}_m = \sum \nolimits _{m=1} ^{D!} \omega _m w {\mathbf{P}}_m$, where $w {\mathbf{P}}_m$ is the mth permutation of the weighting scheme w, which leads to the following definition.

Observation 1

Any weighting scheme $w' \in \mathcal {W}$ that is not more unequal than any $w \in \mathcal {W}$ by Definition 1 is an element in the convex hull of D! permutations of weighting scheme w.

This concept can be used to identify the finite number of weighting schemes that create the convex hull of $\Delta$. In order to do so, we must identify the most unequal weighting scheme in $\Delta$. Starting from a particular weighting scheme, the most unequal weighting scheme can be obtained by a finite number of regressive transfers—a concept that is frequently used in the inequality measurement literature.

Definition 2

For any $w, w' \in \mathcal {W}$, weighting scheme w is obtained from a more equal weighting scheme $w'$ by a regressive transfer if $w'_l \le w'_h$, $w_l=w'_l - \epsilon$ and $w_h=w'_h + \epsilon$ for any $\epsilon > 0$ and $w_d=w'_d \ \forall d \ne h,l$.

Let us denote the most unequal weighting scheme in $\Delta$ by $\bar{w}$ and so the mth permutation of the weighting scheme is $\bar{w} {\mathbf{P}}_m$. Following Observation 1 and Definition 2, then $\Delta$ should be the convex hull of D! weighting schemes $\{ \bar{w} {\mathbf{P}}_m \}_{m=1} ^{D!}$. In practice however the number of distinct permutations of $\bar{w}$ may be equal or less than D!. For example, when the most unequal weighting scheme has weights of 0.5, 0.3, and 0.2, then there are six (3!) unique permutations: (0.5, 0.3, 0.2), (0.5, 0.2, 0.3), (0.3, 0.5, 0.2), (0.3, 0.2, 0.5), (0.2, 0.5, 0.3), and (0.2, 0.3, 0.5). When the most unequal weighting scheme has weights of 0.5, 0.25, and 0.25, then there are only three unique permutations: (0.5, 0.25, 0.25), (0.25, 0.5, 0.25), and (0.25, 0.25, 0.5). For any arbitrary number of dimensions D, let us denote the number of unique permutations by $\bar{D}$, such that $D \le \bar{D} \le D!$. Then $\Delta$ is a convex hull of $\bar{D}$ distinct weighting schemes $v_1,v_2,\ldots ,v_{\bar{D}}$, such that $v_m \ne v_{m'}$ for any $m \ne m'$. The following theorem determines the value of $\bar{D}$ as well as the corresponding vertices.

Theorem 1

For any $D \in \mathbb {N} {\setminus } \{ 1 \}$ and for any $\alpha \in [0,1/D)$, $\beta \in (1/D,1]$, $\tilde{\beta } = \min \{ \beta ,1-(D-1)\alpha \}$ and $\tilde{\alpha } = \max \{ \alpha ,1-(D-1)\beta \}$, the polytope $\Delta = \{ w_1,w_2,\dots ,w_D \ | \ \alpha \le w_d \le \beta \ \forall d \ \mathrm {and} \ \sum \nolimits _{d=1} ^D w_d =1 \}$ is a convex hull of $\bar{D}$ distinct vertices $v_1,v_2,\ldots ,v_{\bar{D}}$, such that

• (a) $\bar{D}={D!}/{[(D-d)! \times d!]}$ whenever $d \tilde{\alpha } + (D-d) \tilde{\beta } = 1$ for some $d \in \{1,2,\ldots ,D-1 \}$. The $\bar{D}$ vertices are unique permutations of the D-dimensional vector $( \tilde{\alpha } \mathbf {1}_d , \tilde{\beta } \mathbf {1}_{D-d} )$.

• (b) $\bar{D}=D!/[(D-d-1)! \times d!]$ whenever $d \tilde{\alpha } + (D-d) \tilde{\beta } > 1$ and $(d+1) \tilde{\alpha } + (D-d-1) \tilde{\beta } < 1$ for some $d \in \{1,2,\ldots ,D-2 \}$. The $\bar{D}$ vertices are unique permutations of the D-dimensional vector $( \tilde{\alpha } \mathbf {1}_d , \gamma , \tilde{\beta } \mathbf {1}_{D-d-1} )$, where $\gamma = 1 - d \tilde{\alpha } - (D-d-1)\tilde{\beta }$.

Proof

We already know that $\Delta = \{ w_1,\ldots ,w_d \ | \ \alpha \le w_d \le \beta \ \forall d \ \mathrm {and} \ \sum \nolimits _{d=1} ^D w_d =1 \}$, $\alpha \in [0,1/D)$ and $\beta \in (1/D,1]$. Note that the constraints $w_d \ge \alpha$ and $w_d \le \beta$ may not always be binding due to the additional constraint $\sum \nolimits _{d=1} ^D w_d =1$. Whenever $\beta$ is set to be larger than $1-(D-1) \alpha$ or $\alpha$ is set to be lower than $1-(D-1) \beta )$, this additional constraint may be violated. Given the joint restriction on $\alpha$ and $\beta$, the maximum feasible upper bound on each weight is $\tilde{\beta } = \min \{ \beta , 1 - (D-1) \alpha \}$ and the minimum feasible lower bound on each weight is $\tilde{\alpha } = \max \{ \alpha , 1 - (D-1) \beta \}$ for all $d = 1,\ldots ,D$.

Given that $\tilde{\alpha } \in [0,1/D)$ and $\tilde{\beta } \in (1/D,1]$, clearly $w^0 \in \Delta$, which is the least unequal weighting scheme. Starting from the D-dimensional equal weight vector $w^0$, the most unequal weighting scheme $\bar{w} \in \Delta$ can be obtained by a finite number of regressive transfers among element weights following Definition 2, until the maximum possible number of elements reach either $\tilde{\alpha }$’s or $\tilde{\beta }$’s. Note thus that there must be at least one $\tilde{\alpha }$ and at least one $\tilde{\beta }$ in $\bar{w}$. In addition, the weights within $\bar{w}$ should satisfy the constraint $\sum \nolimits _{d=1} ^D w_d =1$.

Given that $\sum \nolimits _{d=1} ^D w_d <1$ whenever $w_d = \tilde{\alpha }$ for all d and $\sum \nolimits _{d=1} ^D w_d >1$ whenever $w_d = \tilde{\beta }$ for all d, there exists an $a \in (0,1)$ such that:

$$\begin{aligned} a \sum _{d=1} ^D \tilde{\alpha } + (1-a) \sum _{d=1} ^D \tilde{\beta } =1 \implies a D \tilde{\alpha } + (D-aD) \tilde{\beta } = 1. \end{aligned}$$

Depending on whether aD is an integer value or not, we may have the following two cases.

• (a) Suppose, $aD = d$ for some $d \in \{ 1,2,\ldots ,D-1 \}$. In this case, the values of $\tilde{\alpha }$ and $\tilde{\beta }$ are such that $d \tilde{\alpha } + (D-d) \tilde{\beta } = 1$, which clearly satisfies the constraint $\sum \nolimits _{d=1} ^D w_d =1$. Thus, the most unequal weighting vector $\bar{w}$ consists of d number of $\tilde{\alpha }$’s and $(D-d)$ number of $\tilde{\beta }$’s, i.e., $\bar{w} = ( \tilde{\alpha } \mathbf {1}_d , \tilde{\beta } \mathbf {1}_{D-d} )$. Given that $\tilde{\beta }$ is repeated $(D-d)$ times and $\tilde{\alpha }$ is repeated d number of times, the number of unique permutations is $\bar{D}=D!/[(D-d)! \times d!)]$.

• (b) Suppose, $aD \ne d$ for all $d \in \{ 1,2,\ldots ,D-1 \}$. Instead $aD = d+\varepsilon$ for some $d \in \{ 1,\ldots ,D-2 \}$ and for some $\varepsilon \in (0,1)$. Note that we set the restriction on d here as $d \ge 1$ and $d \le D-2$ because there must be at least one $\tilde{\alpha }$ and at least one $\tilde{\beta }$ in $\bar{w}$. We then have:

  $$\begin{aligned} (d + \varepsilon ) \tilde{\alpha } + (D-d-\varepsilon ) \tilde{\beta } = 1 \implies d \tilde{\alpha } + (D-d) \tilde{\beta } + \varepsilon (\tilde{\alpha } - \tilde{\beta }) = 1. \end{aligned}$$

  Since, $\tilde{\alpha } - \tilde{\beta } < 0$ and $\varepsilon \in (0,1)$, it must be the case that $d \tilde{\alpha } + (D-d) \tilde{\beta }>1$ and $(d+1) \tilde{\alpha } + (D-d-1) \tilde{\beta }<1$ for some $d \in \{ 1,2,\ldots ,D-2 \}$. In other words, if we assign weight $\tilde{\beta }$ to $D-d$ dimensions and weight $\tilde{\alpha }$ to d dimensions, then $\sum \nolimits _{d=1} ^D \bar{w}_d>1$; whereas, if we assign weight $\tilde{\beta }$ to one less dimension (i.e., $D-d-1$ dimensions) and assign weight $\tilde{\alpha }$ to an additional dimension (i.e., $d+1$ dimensions), then $\sum \nolimits _{d=1} ^D \bar{w}_d <1$. Both cases violate the restriction $\sum \nolimits _{d=1} ^D w_d =1$. Hence, there must be some $\gamma \in (\tilde{\alpha },\tilde{\beta })$ such that $d \tilde{\alpha } + (D-d-1) \tilde{\beta } + \gamma =1$ for $d \in \{1,2,\ldots ,D-2 \}$. Thus, $\bar{w}$ must have $(D-d-1)$ elements that are equal to $\tilde{\beta }$, d elements that are equal to $\tilde{\alpha }$, and the remaining element is equal to $\gamma$, i.e., $\bar{w} = ( \tilde{\alpha } \mathbf {1}_d , \gamma , \tilde{\beta } \mathbf {1}_{D-d-1} )$. As $\tilde{\beta }$ is repeated $(D-d-1)$ times and $\tilde{\alpha }$ is repeated d times, the number of unique permutations of $\bar{w}$ is $\bar{D}=D!/((D-d-1)! \times d!)$.

This completes our proof. $\square$

Theorem 1 is quite powerful in the sense that it determines the number of distinct permutations of the most unequal weights in $\Delta$ for any arbitrary number of D dimensions once a consensus on the values of $\alpha$ and/or $\beta$ is reached. The minimum number of unique permutations is obtained when the most unequal weighting scheme is such that all $(D-1)$ dimensions receive the same weight while the remaining dimension receives a different weight. In this case, the number of unique permutations is $[D!/(D-1)!]=D$. It follows from Theorem 1 that this case occurs when $d \alpha + (D-d)\beta =1$ and either $d=1$ or $d=(D-1)$ for any $\alpha \in [0,1/D)$.

There are three cases where the number of unique permutations of the most unequal weighting scheme is only D. The first case is when weights are allowed to vary to the fullest extent. In this extreme case, $\beta =1$ and $\alpha =0$. The most unequal weight $\bar{w} \in \Delta$ is obtained when all $(D-1)$ dimensions are assigned a weight of zero and the remaining dimension is assigned a weight of one. The second case is when one only chooses the value of $\alpha \in (0,1/D)$. Implicitly, in this case, $\beta =1$. The most unequal weighting scheme $\bar{w} \in \Delta$ is obtained when all $(D-1)$ dimensions are assigned weight $\alpha$ and the remaining dimension is assigned weight $\tilde{\beta }=1-(D-1) \alpha < \beta$. The third case is when one chooses only $\beta \in (1/D,1/(D-1)]$. Implicitly, in this case, $\alpha =0$. The most unequal weighting scheme $\bar{w} \in \Delta$ is obtained when all $(D-1)$ dimensions are assigned weight $\beta$ and the remaining dimension is assigned weight $\tilde{\alpha }=[1-\beta (D-1)] > \alpha$.

4.1 Examples

Let us provide certain examples involving three dimensions ($D=3$).

First, suppose $\alpha =1/6$ and $\beta =1/2$. In this case, $\tilde{\alpha } = \max \{ \alpha , 1-(D-1) \beta \} = \max \{ 1/6,0 \} = \alpha$ and $\tilde{\beta } = \min \{ \beta , 1-(D-1)\alpha \} = \min \{ 1/2, 2/3 \}$ $= \beta$. It can be easily checked that there does not exist any $d \in \mathbb {N}$ such that $d \tilde{\alpha } + (D-d) \tilde{\beta } = 1$. This example, thus, corresponds to case b. in Theorem 1. At the respective $\bar{w}$, one dimension is assigned $\tilde{\beta } = 1/2$, the second dimension is assigned $\tilde{\alpha } = 1/6$ and the remaining dimension is assigned $\gamma =1/3$. The number of unique permutations of $\bar{w}$ is $\bar{D}=D!/([D-d-1]!d!)$ for $d=1$. Thus, $\bar{D}=6$ and the six distinct permutations are: $v_1=(1/2,1/3,1/6)$, $v_2=(1/2,1/6,1/3)$, $v_3=(1/3,1/6,1/2)$, $v_4=(1/6,1/3,1/2)$, $v_5=(1/6,1/2,1/3)$, and $v_6=(1/3,1/2,1/6)$. We denote their convex hull by $\Delta _1$. The shape of $\Delta _1$ is depicted in panel (a) of Fig. 1. To check if $y \Delta _1 x$, we simply need to compare y and x at these six weighting schemes: $v_1,\dots ,v_6$.

Fig. 1

Examples of sets of alternative weights $\Delta$

Second, suppose $\alpha =1/6$ and no additional restriction is assumed on $\beta$ and so implicitly $\beta = 1$. We denote the corresponding set of weighting schemes as $\Delta _2= \{ w_1,\ldots ,w_D \ | \ 1/6 \le w_d \le 1 \ \forall d, \sum \nolimits _{d=1} ^D w_d =1 \}$. In this case, $\tilde{\alpha } = \max \{ \alpha , 1-(D-1)\beta \} = \max \{ 1/6, -1 \} = 1/6$ and $\tilde{\beta } = \min \{ \beta , 1-(D-1)\alpha \} = \min \{ 1, 2/3 \} = 2/3$. Clearly, $d \tilde{\alpha } + (D-d) \tilde{\beta } = 1$ for $d = 2$. This example corresponds to case a. in Theorem 1. There are only $\bar{D} = 3!/(1! \times 2!)=3$ distinct permutations. The corresponding $\bar{w}$ in this case assigns $\tilde{\alpha }=1/6$ to two dimensions and assigns $\tilde{\beta }=2/3$ to the remaining dimension. We present the shape of $\Delta _2$ in Panel (b), where $\Delta _2$ is a convex hull of $v_1=(2/3,1/6,1/6)$, $v_2=(1/6,2/3,1/6)$, and $v_3=(1/6,1/6,2/3)$. Note that setting just a lower bound on weights yields the same set of weights proposed by Foster et al. (2009) for a particular level of confidence with respect to the initially chosen equal weight through the epsilon-contamination model.

Third, suppose $\beta = 0.4$ and there is no additional restriction on $\alpha$ and so implicitly $\alpha = 0$. We denote the corresponding set of weighting schemes by $\Delta _3= \{ w_1,\ldots ,w_D \ | \ 0 \le w_d \le 0.4 \ \forall d \ \mathrm {and} \ \sum \nolimits _{d=1} ^D w_d=1 \}$. In this case, $\tilde{\alpha }= \max \{ \alpha , 1-(D-1) \beta \} = \max \{ 0, 0.2 \} = 0.2$ and $\tilde{\beta } = \min \{ \beta , 1- (D-1) \alpha \} = \min \{ 0.4,1 \}=0.4$. The $\bar{w}$ in this case assigns 0.4 to any two dimensions and assigns 0.2 to the remaining dimension. The number of unique permutations is $\bar{D}=3$ and $\Delta _3$ is a convex hull of $v_1=(0.4,0.4,0.2)$, $v_2=(0.4,0.2,0.4)$, and $v_3=(0.2,0.4,0.4)$. The shape of $\Delta _3$ is depicted in panel (c) of Fig. 1. To assess robustness, we need to compare composite index scores at these three vertices.

5 The non-uniform max–min bound approach

The uniform max–min bound approach in Sect. 4 may be justified when there is no strong reason for allowing weights to vary to different extents. However, there may be cases when this is not true. For example, while designing Mexico’s official multidimensional poverty measure, half of the total weight is assigned to the monetary dimension, while the rest of the weights were distributed to the rest of the non-monetary indicators (Foster 2007). Mexican government may have a strong reason behind assigning the particular weight to the monetary dimension, but may not be fully confident about how the rest of the weight should be distributed across the non-monetary dimensions. Although Mexico’s poverty measurement methodology was not based on a composite index, yet these types of situations may arise in practice for composite indices. In this section, we extend the Uniform Max–Min Bound approach developed in Sect. 4 to three non-uniform cases.

5.1 Case I

Let us consider the situations where weights are fixed for some dimensions, but robustness needs to be checked for the rest of the dimensions under the condition that the weights on these dimensions are allowed to vary uniformly. Suppose, without loss of generality, that robustness needs to be checked for the first $D' \ge 2$ of the D dimensions, while the weights for the remaining $D-D' \ge 1$ dimension(s) are fixed. The total weight assigned to the first $D'$ dimensions is $\lambda \in (0,1)$; whereas the fixed weights assigned to the remaining dimensions are $w_d = \hat{w}_d$ $\forall d=D'+1,\ldots ,D$ and they sum up to $1-\lambda$.

The maximum possible weight that may be assigned to any of the first $D'$ dimensions is $\beta \in (1/D',\lambda ]$; whereas, the minimum possible weight that may be assigned is $\alpha \in [0,1/D')$. The set of alternative weights in this case is: $\Delta '= \{ w_1,\ldots ,w_{D} \ | \ \alpha \le w_d \le \beta \ \forall d=1,\ldots ,D'; \sum \nolimits _{d=1} ^{D'} w_d =\lambda ; w_d = \hat{w}_d \in (0,1) \ \forall d=D'+1,\ldots ,D; \sum \nolimits _{d=D'+1} ^{D} \hat{w}_d =1-\lambda \}$. This case is a clear extension of the uniform max–min bound approach but applied to only $D'$ dimensions.

5.2 Case II

Let us now consider the situation where the maximum possible weight should not be larger than $\beta \le 1$ and the minimum possible weight should not be smaller than $\alpha \ge 0$, but all dimensions are ordered according to their importance. Without loss of generality, we assume that $w_1 \le w_2 \le \cdots \le w_D$. The set of alternative weights for checking robustness in this case is: $\Delta ^*= \{ w_1,\ldots ,w_{D} \ | \ \alpha \le w_1 \le \cdots \le w_D \le \beta , \sum \nolimits _{d=1} ^{D} w_d =1 \}$. Note that $\Delta ^*$ is a subset of the set of alternative weights $\Delta$ defined in Sect. 4 and is a convex hull of $\bar{D}^*$ weighting schemes as presented in Proposition 1. Note that unlike in case of Theorem 1, permutation of the most unequal weighting scheme may not be possible because of the constraint $w_1 \le w_2 \le \cdots \le w_D$.

Proposition 1

For any $D \in \mathbb {N} {\setminus } \{ 1 \}$ and for any $\alpha \in [0,1/D)$, $\beta \in (1/D,1]$, $\tilde{\beta } = \min \{ \beta ,1-(D-1) \alpha \}$ and $\tilde{\alpha } = \max \{ \alpha ,1-(D-1) \beta \}$, the polytope $\Delta ^* = \{ w_1,w_2,\ldots ,w_D \ | \ \alpha \le w_1 \le \cdots \le w_D \le \beta \ \forall d \ \mathrm {and} \ \sum \nolimits _{d=1} ^D w_d =1 \}$ is a convex hull of $\bar{D}^*$ distinct vertices $v_1,v_2,\ldots ,v_{\bar{D}^*}$, such that:

• (a) $\bar{D}^* =D$, whenever $d \tilde{\alpha } + (D-d) \tilde{\beta } = 1$ for some $d \in \{1,2,\ldots ,D-1 \}$. One vertex is $( \frac{1}{D}\mathbf {1}_{D} )$. The $\delta$th vertex of the remaining $D-1$ vertices is $( \tilde{\alpha }_\delta \mathbf {1}_d, \tilde{\beta }_\delta \mathbf {1}_{D-d} )$, where $\tilde{\alpha }_\delta = \max \{ \tilde{\alpha },[1-(D-d) \tilde{\beta }]/ d \}$ and $\tilde{\beta }_\delta = \min \{ \tilde{\beta },(1- d \tilde{\alpha }) / (D-d) \}$ for all $\delta =1,\ldots ,D-1$ and for $d=\delta$.

• (b) $\bar{D}^* =D+1$, whenever $d \tilde{\alpha } + (D-d) \tilde{\beta } > 1$ and $(d+1) \tilde{\alpha } + (D-d-1) \tilde{\beta } < 1$ for some $d \in \{1,2,\ldots ,D-2 \}$. One vertex is $( \tilde{\alpha } \mathbf {1}_d , \gamma , \tilde{\beta } \mathbf {1}_{D-d-1} )$, where $\gamma = 1 - d \tilde{\alpha } - (D-d-1)\tilde{\beta }$. Another vertex is $( \frac{1}{D}\mathbf {1}_{D} )$. The $\delta$th vertex of the remaining $D-1$ vertices is $( \tilde{\alpha }_\delta \mathbf {1}_d , \tilde{\beta }_\delta \mathbf {1}_{D-d} )$, where $\tilde{\alpha }_\delta = \max \{ \tilde{\alpha },[1-(D-d) \tilde{\beta }]/d \}$ and $\tilde{\beta }_\delta = \min \{ \tilde{\beta },(1- d \tilde{\alpha }) / (D-d) \}$ for all $\delta =1,\ldots ,D-1$ and for $d = \delta$.

Proof

We are given that $\Delta ^*= \{ w_1,\ldots ,w_{D} \ | \ \alpha \le w_1 \le \cdots \le w_D \le \beta \ \mathrm {and} \ \sum \nolimits _{d=1} ^{D} w_d =1 \}$, which is a subset of $\Delta = \{ w_1,\ldots ,w_{D} \ | \ \alpha \le w_d \le \beta \ \forall d \ \mathrm {and} \ \sum \nolimits _{d=1} ^{D} w_d =1 \}$. We are already aware from Theorem 1 that the feasible upper and lower bounds for $\Delta$ are $\tilde{\beta } = \min \{ \beta ,1-(D-1) \alpha \}$ and $\tilde{\alpha } = \max \{ \alpha ,1-(D-d) \beta \}$, respectively, which also apply to its subset $\Delta ^{*}$. So, $\Delta ^*= \{ w_1,\ldots ,w_{D} \ | \ \tilde{\alpha } \le w_1 \le \cdots \le w_D \le \tilde{\beta } \ \mathrm {and} \ \sum \nolimits _{d=1} ^{D} w_d =1 \}$.

We also know from Theorem 1 that $\Delta$ is a convex hull of unique permutations of the weighting scheme $( \tilde{\alpha } \mathbf {1}_d , \tilde{\beta } \mathbf {1}_{D-d} )$, whenever $d \tilde{\alpha } + (D-d) \tilde{\beta } = 1$ for some $d \in \{1,2,\ldots ,D-1 \}$; whereas, $\Delta$ is a convex hull of unique permutations of the weighting scheme $( \tilde{\alpha } \mathbf {1}_d , \gamma , \tilde{\beta } \mathbf {1}_{D-d-1} )$ where $\gamma = 1 - d \tilde{\alpha } - (D-d-1)\tilde{\beta }$ and $\gamma \in (\tilde{\alpha },\tilde{\beta })$, whenever $d \tilde{\alpha } + (D-d) \tilde{\beta } > 1$ and $(d+1) \tilde{\alpha } + (D-d-1) \tilde{\beta } < 1$ for some $d \in \{1,2,\ldots ,D-2 \}$. Among all permutations, only $\bar{u} = ( \tilde{\alpha } \mathbf {1}_d , \tilde{\beta } \mathbf {1}_{D-d} )$ in the first case and $\bar{u} ^\prime = ( \tilde{\alpha } \mathbf {1}_d , \gamma , \tilde{\beta } \mathbf {1}_{D-d-1} )$ in the second case satisfy the restriction: $w_1 \le \cdots \le w_D$. Thus, each of $\bar{u}$ and $\bar{u} ^\prime$ form a vertex of $\Delta ^*$ in the respective case. Any other permutation of $\bar{u}$ and $\bar{u} ^\prime$ violates the restriction and so cannot form a vertex of $\Delta ^*$.

The D weights are bounded both from above and from below as: $\tilde{\alpha } \le w_1 \le w_2$, $w_{d-1} \le w_d \le w_{d+1}$ for all $d = 2,\ldots ,D-1$, and $w_{D-1} \le w_D \le \tilde{\beta }$. These restrictions lead to the following D binding constraints, which should form D additional vertices or extreme points $v_1 ^{*}, \ldots , v_D ^{*}$ for $\Delta ^*$:

$$\begin{aligned} v_1 ^{*}:&\tilde{\alpha } \le w_1< w_2 = \cdots = w_D \le \tilde{\beta } \\ v_2 ^{*}:&\tilde{\alpha } \le w_1 = w_2< w_3 = \cdots = w_D \le \tilde{\beta } \\ \vdots&\\ v_{D-1} ^{*}:&\tilde{\alpha } \le w_1 = \cdots = w_{D-1} < w_D \le \tilde{\beta } \\ v_{D} ^{*}:&\tilde{\alpha } \le w_1 = \cdots = w_{D} \le \tilde{\beta }. \end{aligned}$$

For the rest of the proof, we shall refer to any of these D vertices by subscript $\delta$. Any of the D elements within a vertex will continue to be denoted by subscript d. The constraints for the vertices $v_1 ^*, \ldots , v_D ^*$ are however not always binding in practice either from above by $\tilde{\beta }$ or from below by $\tilde{\alpha }$ due to the additional constraint $\sum \nolimits _{d=1} ^D w_d = 1$. Let us denote the maximum feasible upper bound and the minimum feasible lower bound for $v_\delta ^*$ by $\tilde{\alpha } _\delta$ and $\tilde{\beta } _\delta$ for some $\delta \in \{ 1,2,\ldots ,D-1 \}$. For any $v_\delta ^{*}$, $\tilde{\alpha } _\delta = \tilde{\alpha }$ but $\tilde{\beta } _\delta < \tilde{\beta }$ whenever $d \tilde{\alpha } + (D-d) \tilde{\beta } > 1$ for $d=\delta$, and $\tilde{\beta } _\delta = \tilde{\beta }$ but $\tilde{\alpha } _\delta > \tilde{\alpha }$ whenever $d \tilde{\alpha } + (D-d) \tilde{\beta } < 1$ for $d=\delta$. Thus, $\tilde{\alpha } _\delta = \max \{ \tilde{\alpha },[1 - (D-d) \tilde{\beta }] /d \}$ and $\tilde{\beta } _\delta = \min \{ \tilde{\beta },(1 - d \tilde{\alpha }) /(D-d) \}$, such that:

$$\begin{aligned} d {\tilde{\alpha }}_\delta + (D-d) \tilde{\beta } _\delta = 1 \end{aligned}$$

(1)

for every $\delta \in \{ 1,\ldots ,D-1 \}$ and for $d=\delta$. The $\delta$th vertex of $\Delta ^*$ is $( \tilde{\alpha }_\delta \mathbf {1}_d, \tilde{\beta }_\delta \mathbf {1}_{D-d} )$ for $\delta \in \{ 1,2,\ldots ,D-1\}$. For $v_D ^*$, all the D elements are equal to 1 / D, i.e., $v_D ^* = (\frac{1}{D} \mathbf {1}_{D})$.

We next show that $v_1 ^{*}, \ldots , v_D ^{*}$ are indeed extreme points. If $v_\delta ^*$ for some $\delta \in \{ 1,\ldots ,D \}$ is not an extreme point, then there must exist some $w^\prime , w^{\prime \prime } \in \Delta ^*$ such that, for some $a \in (0,1)$:

$$\begin{aligned} a w^\prime + (1-a) w^{\prime \prime } = v_\delta ^*. \end{aligned}$$

(2)

We shall show that if $w^\prime \in \Delta ^*$, then $w^{\prime \prime } \notin \Delta ^*$. First, suppose, $w^\prime \in \Delta ^*$ and $w^\prime \ne v_\delta ^*$ for some $\delta \in \{ 1,2,\ldots ,D-1 \}$, such that either $w_d ^\prime > \tilde{\alpha }_\delta$ or $w_d ^\prime < \tilde{\beta }_\delta$ for some $d \in \{ 1,2,\ldots ,D \}$. Clearly, by Eq. (2), it must be the case that either $w_d ^{\prime \prime } < \tilde{\alpha }_\delta$ or $w_d ^{\prime \prime } > \tilde{\beta }_\delta$, violating the constraint on $\Delta ^*$. So, $w^{\prime \prime } \notin \Delta ^*$. Second, suppose $w^\prime \in \Delta ^*$ and $w^\prime \ne v_\delta ^*$ for some $\delta \in \{ 1,2,\ldots ,D \}$, such that the dth element and the $(d+1)$th element in $v_\delta ^*$ are equal but $w_d ^\prime < w_{d+1} ^\prime$. It then must be the case that $w_d ^{\prime \prime } > w_{d+1} ^{\prime \prime }$ by Eq. (2), which also violates the constraint on $\Delta ^*$. Thus, again, $w^{\prime \prime } \notin \Delta ^*$. The second part includes the case for $v_D ^{*}$.

We finally show that $\bar{D}$ may take the value of D (in part a. of the proposition) or $D+1$ (in part b. of the proposition). First, suppose, $d \tilde{\alpha } + (D-d) \tilde{\beta } = 1$ for some $d \in \{1,2,\ldots ,D-1 \}$. Then $\bar{u}$ form a vertex of $\Delta ^*$ as discussed earlier. However, the elements in $\bar{u}$ satisfy Eq. (1), which leads to $\tilde{\alpha } = \tilde{\alpha }_\delta$ and $\tilde{\beta } = \tilde{\beta }_\delta$ and so $\bar{u} = v_\delta ^*$ for some $\delta \in \{ 1,\ldots ,D-1 \}$. Hence, $\Delta ^*$ in this case consists of D vertices: $v_1 ^*, \ldots , v_D ^*$. Second, suppose $d \tilde{\alpha } + (D-d) \tilde{\beta } > 1$ and $(d+1) \tilde{\alpha } + (D-d-1) \tilde{\beta } < 1$ for some $d \in \{1,2,\ldots ,D-2 \}$. We have already discussed that $\bar{u} ^\prime$ forms a vertex of $\Delta ^*$. Now, $\bar{u} ^\prime$ is an additional vertex besides $v_1 ^*, \ldots , v_D ^*$ since $\bar{u} ^\prime \ne v_\delta ^*$ for all $\delta \in \{ 1,\ldots ,D \}$. Therefore, $\Delta ^*$ in this case consists of $(D+1)$ vertices: $v_1 ^*, \ldots , v_D ^*,\bar{u} ^\prime$. $\square$

5.3 Examples

We present three examples in Fig. 2 based on each of our earlier illustrations in Sect. 4.

First, recall the example with $\alpha =1/6$ and $\beta =1/2$ and additionally $w_1 \le w_2 \le w_3$. In this case, $\tilde{\alpha } = \alpha = 1/6$ and $\tilde{\beta } = \beta = 1/2$, but there does not exist any $d \in \mathbb {N}$ such that $d \tilde{\alpha } + (D-d) \tilde{\beta } = 1$. This example thus corresponds to part b. of Proposition 1 and so $\bar{D} = D+1 = 4$. The feasible upper bound and the feasible lower bound for the first two vertices are obtained as follows:

• $v_1 ^*$: $\tilde{\alpha } _1 = \max \left\{ \tilde{\alpha },\dfrac{1 - (3-1) \times \tilde{\beta }]}{1} \right\} = \dfrac{1}{6}$ and $\tilde{\beta } _1 = \min \left\{ \tilde{\beta },\dfrac{1 - 1 \times \tilde{\alpha }}{3-1} \right\} = \dfrac{5}{12}$.

• $v_2 ^*$: $\tilde{\alpha } _2 = \max \left\{ \tilde{\alpha },\dfrac{1 - (3-2) \times \tilde{\beta }]}{2} \right\} = \dfrac{1}{4}$ and $\tilde{\beta } _2 = \min \left\{ \tilde{\beta },\dfrac{1 - 2 \times \tilde{\alpha }}{3-2} \right\} = \dfrac{1}{2}$.

Thus, $\Delta _1 ^*$ is a convex hull of $v_1 ^* =(1/6,5/12,5/12)$, $v_2 ^*=(1/4,1/4,1/2)$, $v_3 ^*=(1/3,1/3,1/3)$, and $v_4 ^*=\bar{u} ^\prime =(1/6,1/3,1/2)$ as shown in panel (a) of Fig. 2.

Fig. 2

Examples of sets of alternative weights $\Delta ^*$

Second, recall the example with $\alpha =1/6$ and $\beta = 1$ and additionally, $w_1 \le w_2 \le w_3$. Recall further, in this case, that $\tilde{\alpha } = \alpha = 1/6$ but $\tilde{\beta } = 2/3 < \beta$. This example corresponds to part a. of Proposition 1 since $2 \tilde{\alpha } + \tilde{\beta } = 1$ and thus $\bar{D} = D = 3$. The feasible upper bound and the feasible lower bound for the first two vertices are obtained as follows:

• $v_1 ^*$: $\tilde{\alpha } _1 = \max \left\{ \tilde{\alpha },\dfrac{1 - (3-1) \times \tilde{\beta }]}{1} \right\} = \dfrac{1}{6}$ and $\tilde{\beta } _1 = \min \left\{ \tilde{\beta },\dfrac{1 - 1 \times \tilde{\alpha }}{3-1} \right\} = \dfrac{5}{12}$.

• $v_2 ^*$: $\tilde{\alpha } _2 = \max \left\{ \tilde{\alpha },\dfrac{1 - (3-2) \times \tilde{\beta }]}{2} \right\} = \dfrac{1}{6}$ and $\tilde{\beta } _2 = \min \left\{ \tilde{\beta },\dfrac{1 - 2 \times \tilde{\alpha }}{3-2} \right\} = \dfrac{2}{3}$.

Thus, $\Delta _2 ^*$ is a convex hull of three vertices $v_1 ^* =(1/6,5/12,5/12)$, $v_2 ^*=(1/6,1/6,2/3)$, and $v_3 ^*=(1/3,1/3,1/3)$ as depicted in panel (b) of Fig. 2.

Third, recall the example with $\beta =0.4$ and $\alpha = 0$, with the additional restriction $w_1 \le w_2 \le w_3$. In this case, $\tilde{\alpha } = 0.2 > \alpha$ and $\tilde{\beta } = \beta = 0.4$. This example also corresponds to part a. of Proposition 1 since $\tilde{\alpha } + 2\tilde{\beta } = 1$ and so $\bar{D} = D = 3$. The feasible upper bound and the feasible lower bound for the first two vertices are obtained as follows:

• $v_1 ^*$: $\tilde{\alpha } _1 = \max \left\{ \tilde{\alpha },\dfrac{1 - (3-1) \times \tilde{\beta }]}{1} \right\} = 0.2$ and $\tilde{\beta } _1 = \min \left\{ \tilde{\beta },\dfrac{1 - 1 \times \tilde{\alpha }}{3-1} \right\} = 0.4$.

• $v_2 ^*$: $\tilde{\alpha } _2 = \max \left\{ \tilde{\alpha },\dfrac{1 - (3-2) \times \tilde{\beta }]}{2} \right\} = 0.3$ and $\tilde{\beta } _2 = \min \left\{ \tilde{\beta },\dfrac{1 - 2 \times \tilde{\alpha }}{3-2} \right\} = 0.4$.

Thus, $\Delta _3 ^*$ is the convex hull of $v_1 ^* =(0.2,0.4,0.4)$, $v_2 ^*=(0.3,0.3,0.4)$, and $v_3 ^*=(1/3,1/3,1/3)$ as presented in panel (c) of Fig. 2.

5.4 Case III

Finally, there may be a general situation where weights in different dimensions may be allowed to vary to different extents, such that $\alpha _d \in [0,1)$ and $\beta _d \in (0,1]$ for all $d=1,\ldots ,D$. Therefore, $\Delta ^{**} = \{ w_1,\ldots ,w_D \ | \ \alpha _d \le w_d \le \beta _d \ \forall d, \sum \nolimits _{d=1} ^D w_d = 1 \}$ is the set of weighting schemes, which is a convex hull of the hyper planes $w_d \ge \alpha _d$ and $w_d \le \beta _d$ for all $d=1,\ldots ,D$, consisting of $\bar{D}^{**}$ vertices, such that $D \le \bar{D}^{**} \le D!$. The value of $\bar{D}^{**}$ depends on the complex relationship between the 2D parametric bounds: $\alpha _1,\ldots ,\alpha _D$ and $\beta _1,\ldots ,\beta _D$. In order to show how the value of $\bar{D}^{**}$ is affected by the selection of bounds, we present an illustration involving three dimensions.

5.5 An illustration with three dimensions

In this case, $D=3$ and the bounded weights are: $w_1 \in [\alpha _1,\beta _1]$, $w_2 \in [\alpha _2,\beta _2]$ and $w_3 \in [\alpha _3,\beta _3]$. The set of alternative weights $\Delta ^{**}$ is bounded by the six hyperplanes $w_d \ge \alpha _d$ and $w_d \le \beta _d$ for $d=1,2,3$. Thus, $\Delta ^{**}$ is a convex hull of a maximum of $3!=6$ vertices or extreme points, but due to the additional restriction $\sum \nolimits _{d=1} ^D w_d = 1$, as earlier, these restrictions are not necessarily binding. It is possible, for example, that $\alpha _1 + \beta _2 + \beta _3 <1$ or, say, $\alpha _1 + \alpha _2 + \beta _3 >1$. Let us denote the minimum feasible weight and the maximum feasible weight that may be assigned to the dth dimension by $\tilde{\alpha }_d = \max \{ \alpha _d, 1 - \sum \nolimits _{d^\prime \ne d} \beta _{d ^\prime } \}$ and $\tilde{\beta }_d = \min \{ \beta _d, 1 - \sum \nolimits _{d^\prime \ne d} \alpha _{d ^\prime } \}$ for $d,d^\prime \in \{1,2,3 \}$. For example, $\tilde{\alpha }_1 = \max \{ \alpha _1, 1 - \beta _2 - \beta _3 \}$ and $\tilde{\beta }_1 = \min \{ \beta _1, 1 - \alpha _2 - \alpha _3 \}$.

Each of the six vertices may be obtained by assigning the maximum feasible weight to one dimension, by assigning the minimum feasible weight to a second dimension, and then by assigning the rest of the weight to the remaining third dimension. Let us denote by $\bar{u}_{d \delta }$, for $d,\delta \in {1,2,3}$ and $d \ne \delta$, the vertex where the dth element is equal to $\tilde{\beta }_d$, the $\delta$th element is equal to $\tilde{\alpha }_\delta$ and the remaining element is equal to $\gamma _{d \delta } = 1 - \tilde{\beta }_d - \tilde{\alpha }_\delta$. Thus, the six vertices are:

$$\begin{aligned} \bar{u}_{12}= & {} (\tilde{\beta }_1,\tilde{\alpha }_2,\gamma _{12});\quad \bar{u}_{21} = (\tilde{\alpha }_1,\tilde{\beta }_2,\gamma _{21});\quad \bar{u}_{13} = (\tilde{\beta }_1,\gamma _{13},\tilde{\alpha }_3); \\ \bar{u}_{31}= & {} (\tilde{\alpha }_1,\gamma _{31},\tilde{\beta }_3);\quad \bar{u}_{23} = (\gamma _{23},\tilde{\beta }_2,\tilde{\alpha }_3);\quad \bar{u}_{32} = (\gamma _{32},\tilde{\alpha }_2,\tilde{\beta }_3). \end{aligned}$$

We now show how different values of the bounds may determine the number of unique vertices $\bar{D} ^{**}$ of $\Delta ^{**}$. Suppose, the upper and lower bounds of $w_1$ and $w_2$ are $\alpha _1=0.1$, $\beta _1=0.4$, $\alpha _2=0.25$, and $\beta _2=0.45$, respectively. We also suppose that the lower bound of $w_3$ is $\alpha _3=0.3$, but we choose different values of the upper bound of $w_3$ to demonstrate how it affects the value of $\bar{D}^{**}$.

First, suppose $\beta _3=0.5$. Then, $\tilde{\alpha }_1 = 0.1$, $\tilde{\alpha }_2 = 0.25$, $\tilde{\alpha }_3 = 0.3$, $\tilde{\beta }_1 = 0.4$, $\tilde{\beta }_2 = 0.45$, and $\tilde{\beta }_3 = 0.5$. Thus, $\gamma _{12} =0.35$, $\gamma _{21}=0.45$, $\gamma _{13}=0.3$, $\gamma _{31}=0.4$, $\gamma _{23}=0.25$ and $\gamma _{32}=0.25$. Using these values, we obtain:

$$\begin{aligned} \bar{u}_{12}= & {} (0.40,0.25,0.35); \quad \bar{u}_{21} = (0.10,0.45,0.45);\quad \bar{u}_{13} = (0.40,0.30,0.30); \\ \bar{u}_{31}= & {} (0.10,0.40,0.50);\quad \bar{u}_{23} = (0.25,0.45,0.30);\quad \bar{u}_{32} = (0.25,0.25,0.50). \end{aligned}$$

Note that all six vertices are different from each other in this case and so $\Delta _1 ^{**}$ is a convex hull of six vertices: $v_1 ^{**} =\bar{u}_{12}$, $v_2 ^{**}=\bar{u}_{13}$, $v_3 ^{**}=\bar{u}_{23}$, $v_4 ^{**}=\bar{u}_{21}$, $v_5 ^{**}=\bar{u}_{31}$ and $v_6 ^{**}=\bar{u}_{32}$ as depicted in panel (a) of Fig. 3. Thus, $\bar{D}_1 ^{**}=6$.

Fig. 3

Examples of sets of alternative weights $\Delta ^{**}$

Second, suppose instead that $\beta _3=0.7$. In this case, $\tilde{\alpha }_1 = 0.1$, $\tilde{\alpha }_2 = 0.25$, $\tilde{\alpha }_3 = 0.3$, $\tilde{\beta }_1 = 0.4$, $\tilde{\beta }_2 = 0.45$, and $\tilde{\beta }_3 = 0.65$. Thus, $\gamma _{12} =0.35$, $\gamma _{21}=0.45$, $\gamma _{13}=0.3$, $\gamma _{31}=0.25$, $\gamma _{23}=0.25$ and $\gamma _{32}=0.1$. Using these values, we obtain:

$$\begin{aligned} \bar{u}_{12}= & {} (0.40,0.25,0.35);\quad \bar{u}_{21} = (0.10,0.45,0.45);\quad \bar{u}_{13} = (0.40,0.30,0.30);\\ \bar{u}_{31}= & {} (0.10,0.25,0.65);\quad \bar{u}_{23} = (0.25,0.45,0.30);\quad \bar{u}_{32} = (0.10,0.25,0.65). \end{aligned}$$

Note that there are only five unique vertices and so $\Delta _2 ^{**}$ is a convex hull of five vertices: $v_1 ^{**} =\bar{u}_{12}$, $v_2 ^{**}=\bar{u}_{13}$, $v_3 ^{**}=\bar{u}_{23}$, $v_4 ^{**}=\bar{u}_{21}$, and $v_5 ^{**}=\bar{u}_{31}=\bar{u}_{32}$ as depicted in panel (b) of Fig. 3. Thus, $\bar{D}_2 ^{**}=5$.

Third, suppose $\beta _3=0.35$. In this case, $\tilde{\alpha }_1 = 0.2$, $\tilde{\alpha }_2 = 0.25$, $\tilde{\alpha }_3 = 0.3$, $\tilde{\beta }_1 = 0.4$, $\tilde{\beta }_2 = 0.45$, and $\tilde{\beta }_3 = 0.35$. Thus, $\gamma _{12} =0.35$, $\gamma _{21}=0.35$, $\gamma _{13}=0.3$, $\gamma _{31}=0.45$, $\gamma _{23}=0.25$ and $\gamma _{32}=0.4$. Using these values, we obtain:

$$\begin{aligned} \bar{u}_{12}= & {} (0.40,0.25,0.35);\quad \bar{u}_{21} = (0.20,0.45,0.35);\quad \bar{u}_{13} = (0.40,0.30,0.30);\\ \bar{u}_{31}= & {} (0.20,0.45,0.35);\quad \bar{u}_{23} = (0.25,0.45,0.30);\quad \bar{u}_{32} = (0.40,0.25,0.35). \end{aligned}$$

Here, $\Delta _3 ^{**}$ is a convex hull of only four vertices: $v_1 ^{**} =\bar{u}_{12}=\bar{u}_{32}$, $v_2 ^{**}=\bar{u}_{13}$, $v_3 ^{**}=\bar{u}_{23}$, and $v_4 ^{**}=\bar{u}_{21}=\bar{u}_{31}$, as shown in panel (c) of Fig. 3. Hence, $\bar{D}_3 ^{**}=4$.

It is thus evident how the value of $\bar{D} ^{**}$ may be affected by even only one upper bound. In this general approach, we may neither need to restrict $\alpha _d$’s above by 1 / D nor need we restrict $\beta _d$’s below by the 1 / D. What restrictions however do we need to impose on $\alpha _d$’s and $\beta _d$’s in order to make sure that the set of alternative weights $\Delta ^{**}$ is not empty? Proposition 2 provides an answer to this question.

Proposition 2

For any $D \in \mathbb {N} {\setminus } \{ 1 \}$ and for $\alpha _d \in [0,1)$ and $\beta _d \in (0,1]$ such that $\alpha _d \le \beta _d \ \forall d=1,\ldots ,D$, $\Delta ^{**}= \{ w_1,\ldots ,w_{D} \ | \ \alpha _d \le w_d \le \beta _d \ \forall d, \sum \nolimits _{d=1} ^{D} w_d =1 \}$ is non-empty if and only if $\sum \nolimits _{d=1} ^{D} \alpha _d \le 1$ and $\sum \nolimits _{d=1} ^{D} \beta _d \ge 1$.

Proof

The sufficiency part is straightforward. Consider the weighting vector $\tilde{w}$ such that $\tilde{w}_d = \alpha _d \ \forall d$ and $\sum \nolimits _{d=1} ^{D} \alpha _d = 1$. Clearly, $\tilde{w} \in \Delta ^{**}$ and so $\Delta ^{**}$ is non-empty.

Let us now prove the necessity part. Suppose that either (i) $\sum _{d=1} ^{D} \alpha _d > 1$ or (ii) $\sum \nolimits _{d=1} ^{D} \beta _d < 1$. Consider any weighting vector $\hat{w}$ such that $\hat{w}_d \ge \alpha _d \ \forall d$. Then, certainly $\sum \nolimits _{d=1} ^{D} \hat{w} _d > 1$ and so $\hat{w} \notin \Delta ^{**}$. Similarly, consider any weighting vector $\dot{w}$, such that $\dot{w}_d \le \beta _d \ \forall d$. In what follows, $\sum \nolimits _{d=1} ^{D} \dot{w} _d < 1$ and so $\dot{w} \notin \Delta ^{**}$. So, $\Delta ^{**}$ in both cases is empty. This completes our proof. $\square$

Proposition 2 requires that the upper and lower bounds on weights should be selected in such a way that $\sum \nolimits _{d=1} ^{D} \alpha _d \le 1$ and $\sum \nolimits _{d=1} ^{D} \beta _d \ge 1$. The set of alternative weights would be empty, otherwise.

6 An inter-temporal illustration with the Human Development Index

The HDI is a composite index that combines country performances in three dimensions: a long and healthy life, access to knowledge, and a decent material standard of living. The HDI has been revised many times since 1990, but each of the three dimensions has consistently been equally weighted. This feature has remained controversial from the moment the HDI was first released.

Between 1994 to 2009, the HDI had been formed by the following aggregation formula:

$$\begin{aligned} \textit{HDI}_A = \dfrac{1}{3} \sum _{d=1} ^3 x_d, \end{aligned}$$

(3)

where $x_1$, $x_2$, and $x_3$ are the normalised country performances in the three dimensions, respectively. The use of an equally weighted arithmetic mean assumes perfect substitutability among normalized performances.⁵ This assumption has been questioned since the first release of the HDI and the assumption has been relaxed in the 2010 Human Development Report (United Nations Development Programme 2010) through formulating the HDI as a geometric mean, as follows:

$$\begin{aligned} \textit{HDI}_G = \prod _{d=1} ^3 x _d ^{\frac{1}{3}}. \end{aligned}$$

(4)

For testing the robustness of the $HDI_G$, we use its linearised logarithmic transformation:

$$\begin{aligned} \ln \textit{HDI}_G = \dfrac{1}{3} \sum _{d=1} ^3 \ln {x_d}. \end{aligned}$$

(5)

6.1 Debate on HDI weights and comparison robustness

Hopkins (1991) commented that the UNDP essentially invoked Occam’s razor in the selection of weights, taking the simplest possible alternative that is likely to attract the least disagreement. Kelley (1991) argued for a higher weight for income on the grounds that it provides a capacity to choose among many other dimensions of human development. While concerns over the HDI weights have simmered over time since the early 1990s, the UNDPs adoption of the geometric mean formulation of the HDI in 2010 returned attention more firmly to it. Ravallion (2011, pp. 12–13) commented that the “equality of the weights was, of course, an arbitrary judgment, and it might have been hoped that the weights would evolve in the light of the subsequent public debate. But that did not happen. The weights on the three components of the HDI (health, education, and income) have not changed in 20 years, and it is hard to believe that the HDI got it right first go.”

Kelley (1991), however, acknowledged that a priori it is difficult to justify any set of weights and for this reason calls for testing the sensitivity of the HDI to alternative weights. It is this issue that has motivated various rank robustness studies for the HDI. Cahill (2005), for example, used a simple approach to conclude that HDI rankings were robust. Using six alternative weighting schemes, Cahill found the six country rankings to be statistically indistinguishable from the original HDI ranking. Unlike Cahill, Foster et al. (2009, 2013) and Permanyer (2011) proposed relatively sophisticated normative frameworks for determining a set of alternative weighting schemes. Like Cahill, however, Foster et al. (2009) found similar conclusions on HDI robustness: nearly $70 \%$ of cross-sectional pairwise HDI country rankings were robust between 1998 and 2004 regardless of how the three achievements were weighted, whereas more than $90 \%$ of cross-sectional pairwise comparisons during the same period were robust when the weight on each of the three dimensions was allowed to vary between 1 / 4 and 1 / 2. Cherchye et al. (2008) found that nearly $75 \%$ of pairwise comparisons of 2002 HDI scores were reversible to the simultaneous application of alternative normalizations, aggregation methods, and weights. Zheng and Zheng (2015) found that seven of the 45 pair-wise comparisons of the top ten HDI countries in the 2014 Human Development Report (United Nations Development Programme 2014) were fully robust or had a ‘truth value’ of unity.

6.2 How robust are inter-temporal changes in the HDI?

We now address how robust the changes in HDI scores for individual countries are between 1980 and 2013, using the dimensional performances published by the UNDP. These data are available for all indicators for most years from 1990 but are not updated annually, despite HDI values being annually published since this year. For this reason, we select data for every five years in the period 1980 to 2005, plus that for 2013. This selection provides us with data on all three dimensions for 123 countries.⁶ We commence our investigation by using these data to calculate both the geometric formulation in Eq. (3) and the arithmetic mean formulation in Eq. (4). We consider both formulations primarily to understand whether the UNDP’s move to the geometric mean matters in terms of rank robustness.

Table 1 HDI scores and dimension achievements between 1980 and 2013

Table 1 presents the change in mean HDIs and dimensional performances for the 123 countries between 1980 and 2013. Columns 2 and 4 of the table show that the means of both HDIs have steadily improved between 1980 and 2013, with the corresponding standard deviation remaining between 0.160 and 0.178. Performances across years in each dimension are reported in columns 6, 8, and 10 of Table 1. Mean performances in all three dimensions have gradually increased, especially in education.

Table 2 The changes in component indices and HDIs across 123 countries between 1980 and 2013

Improvement in dimensional performances and in the overall HDI scores have not, however, been observed for all countries as it is evident from Table 2. Columns 2–4 of the table report the number of countries whose performances in each dimension improved between each period. For example, between 1980 and 1985, the health performance improved or did not change in 117 countries, while education and income performances improved or did not change in 113 and 69 countries, respectively. The fifth (seventh) column presents the number of countries with improved (deteriorating) performances simultaneously in all three dimensions, while the sixth (eighth) column presents the number of countries that did not have deteriorating (improved) performances simultaneously in all dimensions.

The remaining columns in Table 2 present aggregate results. Columns 9 and 13 present the number of countries in each period for which $\textit{HDI}_A$ and $\textit{HDI}_G$ improved, respectively. Columns 11 and 15 present the number of countries in each period for which $\textit{HDI}_A$ and $\textit{HDI}_G$ deteriorated, respectively. HDIs for all countries improved only between 1980 and 2013, between 2000 and 2010, between 2000 and 2013, and between 2005 and 2013.

If we compare column 5 with columns 9 and 13, large discrepancies are observed. Between 1985 and 1990, $\textit{HDI}_A$ improved for 106 countries and $\textit{HDI}_G$ improved for 108 countries, but only for 70 countries was it the case that none of the three dimensions deteriorated. For the rest of the 36–38 odd countries, the HDI improvement was accompanied by deterioration in at least one dimension, which means any alternative weighting scheme might have reversed the direction of improvement. In fact, it was never the case that all three indicators improved or declined together across years. We observe from the fourth column that income was the most volatile dimension and unlike health and education, it did not increase systematically.

The question that arises from the preceding observations is how robust were the observed improvements and reductions in the HDIs? The answer certainly depends on how different sub-indices have changed over time as well as on the set of alternative weighting schemes subject to which we check robustness. For this exercise, we allow the weights to vary non-uniformly for three dimensions, similar to our illustration with three dimensions under Case III in Sect. 5. Note that a much simpler exercise could have been to allow weights to vary uniformly for all three dimensions within a common upper bound and a common lower bound, as presented in Sect. 4.

We allow the weights for both the health ($w_1$) and the education ($w_2$) dimension to vary between 0.1 and 0.7; whereas we allow the weight on the income dimension ($w_3$) to vary between 0.05 and 0.9. Thus, $\alpha _1 = \alpha _2 =0.1$, $\beta _1 = \beta _2 =0.7$, $\alpha _3 =0.05$ and $\beta _3= 0.9$. Clearly, $\tilde{\alpha }_1 = 0.10$, $\tilde{\alpha }_2 = 0.10$, $\tilde{\alpha }_3 = 0.05$, $\tilde{\beta }_1 = 0.7$, $\tilde{\beta }_2 = 0.7$, and $\tilde{\beta }_3 = 0.8$. So, $\gamma _{12} =0.20$, $\gamma _{21}=0.20$, $\gamma _{13}=0.25$, $\gamma _{31}=0.10$, $\gamma _{23}=0.25$ and $\gamma _{32}=0.10$. Using the values of $\tilde{\alpha }$’s, $\tilde{\beta }$’s and $\gamma$’s, we obtain: $\bar{u}_{12} = (0.70,0.10,0.20)$; $\bar{u}_{21} = (0.10,0.70,0.20)$; $\bar{u}_{13} = (0.70,0.25,0.05)$; $\bar{u}_{31} = (0.10,0.10,0.80)$; $\bar{u}_{23} = (0.25,0.70,0.05)$; and $\bar{u}_{32} = (0.10,0.10,0.80)$. There are five unique vertices (since $\bar{u}_{31} = \bar{u}_{32}$). The set of alternative weights $\Delta ^{**}$ is a convex hull of five weighting schemes: (0.70, 0.10, 0.20), (0.70, 0.25, 0.05), (0.10, 0.70, 0.20), (0.25, 0.70, 0.05), and (0.10, 0.10, 0.80). The shape of $\Delta ^{**}$ should resemble the shape in panel (b) of Fig. 3.

Columns 10, 12, 14, and 16 of Table 2 present the number of robust changes in $\textit{HDI}_A$ and $\textit{HDI}_G$ with respect to $\Delta _1$. For example, between 1980 and 1985, only 78 of the 111 $\textit{HDI}_A$ increases and one of the 12 $\textit{HDI}_A$ reductions were robust. Similarly, 82 of the 116 $\textit{HDI}_G$ increases were robust during the same period. The largest number of robust changes in HDIs were observed between 2000 and 2013: 115 for $\textit{HDI}_A$ and 117 for $\textit{HDI}_G$. The lowest number of robust changes in HDIs were observed between 1980 and 1985: 79 for $\textit{HDI}_A$ (78 increases and one decrease) and 83 for $\textit{HDI}_G$ (82 increases and one decrease).

Table 3 The number of periods in which robust changes in HDI occurred

We finally present an interesting result by asking how many changes were robust with respect to $\Delta _1$ across a spell of six periods: 1980–1985, 1985–1990, 1990–2000, 2000–2005, 2005–2010, and 2010–2013. The answer can be found in Table 3. The first column reports the number of periods ranging between zero and six. Columns 2 and 6 report the number of countries with robust improvements in $HDI_A$ and $HDI_G$, respectively, for the respective number of periods. We surprisingly find that only 33 of the 123 countries had robust $HDI_A$ improvements and only 36 of the 123 countries had robust $HDI_G$ improvements across all six periods.⁷ A similar number of $HDI_A$ and $HDI_G$ improvements were robust across four or five periods; whereas only three countries had robust $HDI_A$ improvements for one period and one country had robust $HDI_G$ improvement for one period.

7 Concluding remarks

This paper looked at the robustness of comparisons of composite indices with respect to a set of alternative weighting schemes. The initial weighting schemes are typically chosen arbitrarily and as such there is ambiguity over the comparison of index scores, be they in relation to cross-section rankings or inter-temporal comparisons of index scores for the units of analysis under consideration.

In the paper, we addressed a difficulty encountered by several previous studies: the selection of alternative weighting schemes for assessing the robustness of comparisons. This selection is a requirement of the tests proposed by these studies, yet none provide sufficient guidance for such selection. We proposed a general yet theoretically novel approach for this purpose. This approach is founded on the normative assumption that a consensus has been reached on the minimum and the maximum allowable weights that should be assigned to each component. This consensus then yields a particular set of alternative weights against which the robustness of comparisons is tested. We considered two variants of this approach. In one, we allowed weights on all dimensions to vary within common maximum and minimum possible values; in the other, we relaxed this assumption and the weights were allowed to vary to different extents.

In order to show the applicability of our approach, we evaluated the prevalence of robust country-specific inter-temporal comparisons of the influential HDI. Testing the robustness of inter-temporal comparisons of the HDI or other composite indices has not previously been attempted. The results of this evaluation were striking. It found that less than one-third of the inter-temporal HDI comparisons were robust across six sub-periods between 1980 and 2013. This has obvious and serious implications for the use of the HDI in incisively assessing changes in human development over time.

We end this paper by adding voice to previous calls for greater warning signals to be attached to the use of composite indices for which there is insufficient guidance, theoretical or otherwise, in their design. A great risk is that unless greater care and sophistication are used in the reporting of composite indices, their ability to inform could be compromised. It is commonplace in reporting the results of econometric analysis to provide a range of diagnostic and other statistics, including t-ratios, so the reader can make judgments about the veracity of these results. No equivalent statistics presently accompany the reporting of composite index scores. It is high time that they did, and the reporting of comparison robustness information would be a useful starting point. We further post that robustness approaches should be such that they are intuitive and practically amenable to adaptations and implementations. Our paper proposed a normative yet intuitive approach that seeks to fulfil this criterion.