Keeping things simple: why the Human Development Index should not diverge from its equal weights assumption
- First Online:
- Cite this article as:
- Stapleton, L.M. & Garrod, G.D. Soc Indic Res (2007) 84: 179. doi:10.1007/s11205-006-9081-3
- 201 Views
Using a range of statistical criteria rooted in Information Theory we show that there is little justification for relaxing the equal weights assumption underlying the United Nation’s Human Development Index (HDI) even if the true HDI diverges significantly from this assumption. Put differently, the additional model complexity that unequal weights add to the HDI more than counteracts the improvement in goodness-of-fit. This suggests that, in some cases, there may be limited validity in increasing the complexity of a range of other composite sustainability indices.
KeywordsComplexityComposite indicesHuman Development IndexInformation TheorySustainable developmentWell-being
Although there are many definitions of sustainable development perhaps the most famous and widely cited is that put forward by the World Commission on Environment and Development (WCED, 1987) which states that development is sustainable if it “meets the needs of the present without compromising the ability of future generations to meet their own needs”. Following this, Agenda 21 was presented at the United Nations Conference on Environment and Development (UNCED, 1992) in Rio de Janerio as a blueprint for action, at various spatial scales from local to global, to facilitate a move towards sustainable development. The kind of sustainable development that Agenda 21 was aiming to facilitate is that articulated by the WCED (1987); interestingly, however, the actual text of Agenda 21 does not define sustainable development or sustainability despite using these terms a total of 373 times. In this context sustainability is only distinct from sustainable development in a linguistic sense; for an argument in favour of decoupling these two terms see Sneddon (2000).
Pannell and Glenn (2000) suggest that the stimulus for developing sustainability indicators was the fact that sustainability cannot be condensed to a single simple definition. Of equal importance was Chapter 40 of Agenda 21 which highlighted the need for a more systematic approach to the identification and utilisation of sustainable development indicators in order to ascertain whether or not development was becoming more, or less, sustainable over time. This call for action stimulated and continues to stimulate a myriad of research ventures, at different scales, within the broad area of indicators of sustainable development.
Indicators of sustainable development need to account for and address the fact that there are economic, environmental, social and institutional dimensions to sustainability. The number and array of indicators that have been proposed since the publication of Agenda 21 reflects these different dimensions of sustainable development with the exception of indicators covering the institutional dimension of sustainability which are currently hampered by methodological issues and a lack of data. Different criteria have been suggested to guide the selection of indicators of sustainable development (e.g. Hardi and Zdan, 1997) to try and ensure that a given indicator set is: adequate given the time-frame and spatial-scale under consideration; reflective of the different dimensions of sustainability; sensitive to intergenerational and intragenerational concerns. To this end, indicator frameworks have been employed to systematize the selection of indicators so that important elements are covered by the indicator set (e.g. Bossel, 1999).
Composite indices of sustainable development are, superficially at least, an attractive option relative to simple, non-aggregated, indicators of sustainable development; if the state of the world can be represented by a few key numbers why would this not be preferred to a longer list of numbers? In this paper we provide: a brief overview of composite indices of sustainable development (Sect. 2); a more specific discussion of those composite indices which have been formulated to measure well-being (Sect. 3); an examination of what an information theoretic approach to composite indices of sustainable development tells us in terms of the appropriateness of such indicators using a well-known indicator of well-being––the United Nation’s Human Development Index––as an illustrative example (Sect. 4); conclusions from this analysis (Sect. 5).
1.1 Composite indices of sustainable development: overview
A composite index is an aggregation of individual indicators which can be weighted to reflect the relative importance of each indicator (Nardo, Saisana, Saltelli, Tarantola, 2005a). According to Stevens (2005), the rationale for developing and using such composite indices to inform public policy is that they integrate a mass of information into easily understood formats for a general audience. Stevens (2005) also notes how “... their construction is not straightforward, they can provide misleading information...”. Similarly, Bossel (1999) notes how composite indices can hide serious deficits. To elaborate, a composite index could show positive increases over time suggesting that development is becoming more sustainable but this aggregate rise could mask declines in some components of the index which is obviously converse to the notion of sustainability. Not surprisingly therefore, the development and use of composite indices of sustainable development has proponents and opponents.
1.2 Composite indices of sustainable development: well-being
There has been a significant research effort internationally to define and operationalise measures of well-being. This is at least partly due to the broad nature of the concept and thus the wide array of interpretations that can be placed upon it. In practice, measures of well-being tend to concentrate explicitly on economic and social dimensions; many also incorporate the environmental dimension. Here we focus on the most commonly used in the literature. Three essentially identical indicators involve the estimation of a range of economic, social and environmental benefits in monetary terms but with different names: the Genuine Progress Indicator (GPI; Cobb, Halstead, & Rowe, 1995); the Index of Sustainable Economic Welfare (ISEW; Daly & Cobb, 1989); and the Sustainable Net Benefit Index (SNBI; Lawn & Sanders, 1999). A fourth indicator, the United Nations Development Programme’s Human Development Index (HDI; UNDP, 1990) aggregates life expectancy, adult literacy combined with years of education (hereafter education) and GDP per capita into one measure. In terms of the first three measures, three identical indices which go by different names are far from ideal and can cause confusion. Healy and Côté (2001), for example, describe the GPI and ISEW as distinct indices which they are not. Further research into these indices needs to converge upon on agreed nomenclature for what is being measured. Lawn (2006), who despite coining the SNBI nomenclature with a colleague in 1999 now opts for “the Genuine Progress Indicator as the best name so far devised”. However, Herman Daly who co-coined the ISEW term in 1989 is less positive about the GPI name: “Index of Sustainable Economic Welfare has the advantage of being the most explicitly descriptive of what it is [...]; Sustainable Net Benefit Index is a close second; and Genuine Progress Indicator is a distant third, in my opinion. But for appeal to the general public maybe Genuine Progress Indicator is simpler” (H. Daly, Pers. Comm. 12 May 2006). We would argue that, to the academic, ISEW is the most appropriate nomenclature because welfare is a well-established concept in economics whereas the actual meaning of the net benefits in the SNBI and the genuine progress in the GPI are not immediately apparent. However, non-academics may have a semantic problem with the term welfare, associating it with financial aid from the government for example––particularly in an American context. Therefore, although ISEW is perhaps the most appropriate name from an academic viewpoint, all three names are problematic from the perspective of wider stakeholders.
There is a substantial literature on these three measures and a controversy surrounding their use. Atkinson (1995) and Neumayer (1999) for example offer methodological critiques which are answered by Lawn (2006) in terms of “illuminating a sound theoretical foundation” for these measures. However, because these three measures are expressed in monetary units their accuracy depends upon the quality of valuation methods used for this purpose which Lawn (2006) calls into question. Therefore, even if the various methodological critiques regarding the measures themselves do not stand up to scrutiny, their dependence upon problematic valuation methodologies is not in doubt.
Since its proposition in 1990, the HDI has gained noteworthy prominence. HDIs are published annually in UN Human Development Reports (HDRs) which, according to Sagar and Najam (1999), represent “the flagship publication not only of the UNDP, but possibly of the entire UN system”. Not surprisingly, given the prominence of this measure to the UN, it has received significant attention in the literature. The nomenclature––Human Development Index––does not appear to raise obvious semantic problems within the context of sustainable development (however outside of this context, human development could be taken to mean the evolution of Homo sapiens) and because it does not depend on the valuation of non-market goods or services it avoids the problems associated with the three measures of well-being discussed above. This aside, critiques of the methodology abound questioning the quality of the data employed (e.g. Murray, 1991), the failure of the indicator to acknowledge within country differences (Sagar and Najam, 1998) and the method used to aggregate the indicator’s three components (Srinivasan, 1994; Booysen, 2002). However, these issues are not unique to the HDI and could equally be levied against an array of composite indices. The HDI is also problematic because the methodology used for its calculation has evolved since its inception in 1990 making comparisons over time difficult (Morse, 2003). The UN is explicit about the shortcomings of the HDI and, therefore, implicit in its continued use is the statement that the index is at least as good as other related composite indices of sustainable development.
1.3 The Human Development Index: an information theoretic analysis
Information Theory is a branch of applied mathematics that has been applied to problems in many fields over the last 50 years such as electrical engineering, physics and psychology. More recently it has been applied within the environmental sciences to determine, for example, whether more complex models of terrestrial nutrient flux are justified compared to simpler, nested alternatives (Stapleton et al., 2006). Specifically, instead of comparing models on the basis of their goodness-of-fit to a dataset, statistics rooted in Information Theory can be used which include a goodness-of-fit component whilst penalising complexity. The Root Mean Squared Deviation (RMSD), the Akaike Information Criterion (AIC; Akaike, 1974) and the Schwarz Information Criterion (SIC; Schwarz, 1978) take into account the number of adjustable (free) parameters in a given model. The Minimum Description Length (MDL; Rissanen, 1987) and the Information-Theoretic Measure of Complexity (ICOMP; Bozdogan, 1990) take into account the number of adjustable model parameters in a model as well as the complexity of its functional form. Therefore, a model which is selected using one or more of these statistics (Eq. 1) could be described as parsimonious (striking a better balance between goodness-of-fit and complexity) relative to alternative models.
Composite indices of sustainable development could be regarded as models but models which fit the data perfectly; it’s not possible to go out into the field and measure these theoretical constructs directly in order to determine whether such models are an accurate representation of reality. The existing HDI attaches equal weights to its three components and although a survey of 1,547 researchers recently concluded that this was optimal relative to differentiating these weights, this equal weighting approach has been heavily criticised (Chowdhury & Squire, 2006). Instead of an equal weighting approach let us assume that the true HDI lies about unequal coefficients. A HDI based on equal coefficients will necessarily have a lower goodness-of-fit to this true case but differentiating such coefficients adds complexity to the HDI model so that this improvement in goodness-of-fit might not justify this additional complexity according to model selection statistics. Assume therefore:
Goodness of fit and model selection statistics where hypothetical data were generated around the 2003 Human Development Index (HDI) and then tested against the existing HDI (Model 1) and a more complex alternative with adjustable coefficients (Model 2)
α = 0.333
α1 = 0.510
α2 = 0.302
α3 = 0.186
α = 0.333
α1 = 0.057
α2 = 0.853
α3 = 0.087
Pearson’s correlation matrix for the life expectancy, education and GDP indicators used to calculate the Human Development Index for 177 countries in 2003
Although it is not possible to go out into the field and directly measure the HDI and conduct the kind of statistical analysis outlined above, the nature of the results have important practical implications for the HDI. The nature of the data generated in this work suggests that alternative HDIs with different coefficients provided a better goodness-of-fit to these datasets when compared to the current HDI which attaches equal weights to life expectancy, education and GDP. However, attaching such equal weights implies only one parameter compared to three parameters if the weights associated with life expectancy, education and GDP are differentiated. Statistics rooted in Information Theory suggest that, even if the true weights are significantly different from each other, there is a lack of justification for acknowledging this in the functional form of the HDI, not least because the three variables (indicators) which constitute the HDI are collinear. Put differently, although the sensitivity of composite indicator outputs to different changes associated with the construction of such indicators has been examined previously (e.g. Morse, 2003; Nardo et al., 2005a; Nardo, Saisana, Saltelli, Tarantola, Hoffman, & Giovannini, 2005b) the work presented here goes further by illustrating that if such changes increase the complexity of the indicator under consideration then the additional assumptions this brings forth may not be parsimonious relative to simpler alternatives.
The authors are working on a European Union Sixth Framework Project (System for Environmental and Agricultural Modelling: Linking European Science and Society; SEAMLESS) which helped provide inspiration for this work.