Abstract
The paper examines the issue of weights and importance in composite indices of development. Building a composite index involves several steps, one of them being the weighting of variables. The nominal weight assigned to a variable often differs from the degree to which the variable affects the scores of the overall index. The newly suggested notion of importance is based on the idea that an important indicator, if omitted from the index, causes large changes in countries’ results. We propose a method of measuring the importance and apply it to inequality variables in composite indices of development. The results show a low importance for most inequality variables, and for some of them, a large discrepancy between the nominal weights and the importance. We argue that the importance of variables should be considered in the process of index construction. This may imply a modification of the index when there is a large discrepancy between the nominal weights and importance and when the importance of some variables is extremely low. Whether any such modification is justified must be decided within the context of the particular index.
Similar content being viewed by others
Notes
Many composite indices have a two-layer hierarchical structure of variables composed of several components (dimensions) which include several indicators. We use the term “indicator” for the lower hierarchical level and “component” for the higher hierarchical level. We call the resulting structure “index,” “composite index,” and “overall index.”
The motivation behind the focus on inequality indicators stems from the growing interest in inequality in academia and public discourse (e.g. see Wilkinson and Pickett 2009; Piketty 2014) as well as in global policy (the Sustainable Development Goals include Goal 10 which aims to reduce inequalities).
Equal weights can be applied on various levels in the structure of composite index. For example, if the composite index is composed from three components, with five indicators in each, it is possible to assign equal weights to each of the three components and/or to each of the five indicators. If the components have a different number of indicators, the equal weights of components do necessarily mean different weights of indicators and the equal weights of indicators necessarily imply different weights of components.
For the HDI see United National Development Programme (UNDP 2016), for the Multidimensional Poverty Index see Alkire and Robles (2017), and for the CDI see Käppeli et al. (2017). Interestingly, equal weights in the HDI and CDI were later corroborated by other studies. Chowdhury and Squire (2006) asked a group of experts around the world to set the weights for components of the two indices; the weights derived from experts’ opinion were not substantially different from equal weights, especially for the HDI. Nguefack-Tsague et al. (2011) showed that principal component analysis based on the correlation matrix of the HDI components leads to practically the same weights.
It should be noted that the tradeoffs are affected not just by “equal” weights, but also by other methodological choices. While this specific critique applies to the HDI, Ravallion (2011) also made a more general case against composite indices of development.
The modification of variability is common but not a necessary feature of normalization. For example, the former methodology of the CDI normalized indicators linearly to a set of transformed values with a fixed average, preserving the variability of the original data. The consequence was that indicators (components) with a low variability had a low influence on the scores of the component (index), while highly variable indicators (components) had a large influence (see Syrovátka and Hák 2015).
By the relative distance, we mean a gap in scores. For example, the gap between countries on the first and the second position in comparison with the gap between the second and the third place according to the overall index.
Fixing values of a certain indicator decreases the variability of the composite index not just because one source of variability disappears, but also because other sources of variability (other indicators) are in general correlated with this indicator and thus the reduction of variability in one indicator brings at least a partial reduction of variability from other sources.
In the case of the SSI and WHI, inequality is an indicator included in only one component. We investigate the importance of inequality in this component. In the case of the IEWB, inequality itself is one component of the index. In other cases (the IHDI and IAH), we assess the importance of indicators in a composite index.
As all examined indices (components) assign the same nominal weights to all indicators, the factor of nominal weight does not contribute to differences in relative importance between indicators within an index. For this reason, we do not comment on the factor of nominal weights in this section.
The average income can be increased by increasing the income at the lower and upper parts of the income distribution (e.g. for a person who has 1 USD per day and for a person with 1000 USD per day), while it is not possible to increase life expectancy by significantly raising the length of life of those who already have reached a very high age (it is practically impossible to raise the length of life of 90-year-old person by 50 years; the same is possible for infants). Therefore, to increase the life expectancy significantly, inequality in length of life must be reduced.
The IEWB is inconsistent in that it mixes data for countries and for provinces in two datasets, but we want to raise a more general issue that is practically relevant for indicators calculated at a country level.
For example, if a few new countries are now added to the IHDI (currently calculated for 151 countries) the importance of indicators will probably change negligibly; adding new countries to the IEWB (calculated for 14 selected OECD countries) will probably have a higher effect on the importance as the original country base is small and new countries will increasingly be structurally different (especially if new countries are outside the OECD). We suppose that indices that do not have large universal coverage at their inception will be first calculated for richer countries (with available data) and the newer countries will be increasingly poorer countries, likely more different than the original set of countries.
In some cases, the difference between the nominal weight and the importance is striking: for example, the inequality indicator in the IAH has s nominal weight of 50%, while its MRI is lower than 5%.
In the case when the nominal weights are determined by statistical methods, they are also dependent on data. One can even argue that nominal weights set by participatory methods are also indirectly dependent on data as the experts do not assign weights without considerations of the realities (captured in the actual data structure).
We note that the number of countries included in different IHDI editions slightly fluctuates (ranging from 132 countries in 2012 up to 152 countries in 2014). Year-by-year differences in importance presented in Table 13 are, therefore, partly driven also by changes in the number and structure of countries included.
Abbreviations
- CDI:
-
Commitment to Development Index
- EPI:
-
Environmental Performance Index
- HDI:
-
Human Development Index
- IAH:
-
Index of Inequality-Adjusted Happiness
- IEWB:
-
Index of Economic Well-being
- IHDI:
-
Inequality-adjusted Human Development Index
- MAI:
-
Measure of absolute importance
- MCI:
-
Modified composite index
- MRI:
-
Measure of relative importance
- OCI:
-
Original composite index
- PCA:
-
Principal component analysis
- OECD:
-
Organisation for Economic Co-operation and Development
- SSI/HW:
-
Sustainable Society Index/Human Well-being component
- UNDP:
-
United Nations Development Programme
- WHI/QL:
-
World Happiness Index/Quality of Life component
References
Alkire, S., & Foster, J. (2010). Designing the inequality-adjusted human development index. OPHI working paper 37. Oxford: Oxford Poverty & Human Development Initiative.
Alkire, S., & Robles, G. (2017). Multidimensional poverty index summer 2017: Brief methodological note and results. MPI methodological notes 44. Oxford: Oxford Poverty & Human Development Initiative.
Anderberg, M. R. (1973). Cluster analysis for applications. New York: Academic Press Inc.
Atkinson, A. B. (1970). On the measurement of inequality. Journal of Economic Theory,2(3), 244–263.
Bandura, R. (2008). A survey of composite indices measuring country performance: 2008 update. UNDP/ODS working paper. New York, NY: Office of Development Studies, United Nations Development Programme.
Becker, W., Saisana, M., Paruolo, P., & Vandecasteele, I. (2017). Weights and importance in composite indicators: Closing the gap. Ecological Indicators,80, 12–22.
Becker, W., & Saltelli, A. (2015). Design for sensitivity analysis. In A. Dean, M. Morris, J. Stufken, & D. Bingham (Eds.), Handbook of design and analysis of experiments (pp. 627–674). Boca Raton: Chapman & Hall/CRC Press.
Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.
Chowdhury, S., & Squire, L. (2006). Setting weights for aggregate indices: An application to the commitment to development index and human development index. The Journal of Development Studies,42(5), 761–771.
Cobham, A., Schlogl, L., & Sumner, A. (2016). Inequality and the tails: The Palma proposition and ratio. Global Policy,7(1), 25–36.
de Kruijk, H., & Rutten, M. (2007). Weighting dimensions of poverty based on people’s priorities: Constructing a composite poverty index for the maldives. IIDE discussion paper 2007 08-01. Rotterdam: Institute for International & Development Economics.
Decancq, K., & Lugo, M. A. (2010). Weights in multidimensional indices of well-being: An overview. https://lirias.kuleuven.be/bitstream/123456789/262767/1/DPS1006.pdf. Accessed January 2, 2018.
Ding, Y., Fu, Y., Lai, K. K., & Leung, W. K. J. (2017). Using ranked weights and acceptability analysis to construct composite indicators: A case study of regional sustainable society index. Social Indicators Research. https://doi.org/10.1007/s11205-017-1765-3.
Gan, X., Fernandez, I. C., Guo, J., Wilson, M., Zhao, Y., Zhou, B., et al. (2017). When to use what: Methods for weighting and aggregating sustainability indicators. Ecological Indicators, 81, 491–502.
Gini, C. (1912). Variabilità e Mutuabilità. Contributo allo Studio delle Distribuzioni e delle Relazioni Statistiche. Bologna: C. Cuppini.
Ginsberg, N., Osborn, J., & Blank, G. (1986). Geographic perspectives on the wealth of nations. Department of Geography research paper no. 220. Chicago: University of Chicago.
Globeco. (2001). L´indice Globeco du Bonheur Mondial. http://www.globeco.fr/images/PDF/BMB/bonheur-mondial-2001.pdf. Accessed February 21, 2017.
Globeco. (2016). Bonheur mondial. Edition 2016. http://globeco.fr/images/PDF/BMB/bonheur-mondial-2016.pdf. Accessed January 22, 2017.
Gorsuch, R. L. (1974). Factor analysis. Philadelphia: W.B Saunders.
Greco, S., Ishizaka, A., Matarazzo, B., & Torrisi, G. (2017). Stochastic multi-attribute acceptability analysis (SMAA): An application to the ranking of Italian regions. Regional Studies. https://doi.org/10.1080/00343404.2017.1347612.
Greco, S., Ishizaka, A., Tasiou, M., & Torrisi, G. (2018). On the methodological framework of composite indices: A review of the issues of weighting, aggregation, and robustness. Social Indicators Research. https://doi.org/10.1007/s11205-017-1832-9.
Hagerty, M. R., & Land, K. C. (2006). Constructing summary indices of quality of life: A model for the effect of heterogenous importance weights. Sociological Methods & Research,35(4), 455–496.
Hsu, A., Johnson, L. A., & Lloyd, A. (2013). Measuring progress: A practical guide from the developers of the environmental performance index (EPI). New Haven: Yale Center for Environmental Law & Policy.
Hume, D. (1740). A treatise of human nature. London: Penguin Books. (reprinted in 1985).
Jolliffe, I. T. (1986). Principal component analysis. New York: Springer.
Kalmijn, W., & Veenhoven, R. (2014). Index of inequality-adjusted happiness (IAH) improved: A research note. Journal of Happiness Studies,15(6), 1259–1265.
Käppeli, A., Hillebrandt, H., & Mitchell, I. (2017). The commitment to development index: 2017 edition. Methodological overview paper. Washington, D.C.: Center for Global Development.
Krishnakumar, J., & Nagar, A. L. (2008). On exact statistical properties of multidimensional indices based on principal components, factor analysis, MIMIC and structural equation models. Social Indicators Research,86(3), 481–496.
Kuznets, S. (1955). Economic growth and income inequality. American Economic Review,XLV(1), 1–28.
Nguefack-Tsague, G., Klasen, S., & Zucchini, W. (2011). On weighting the components of the Human Development Index: A statistical justification. Journal of Human Development and Capabilities,12(2), 183–202.
Nord, E. (2013). Disability weights in the Global Burden of Disease 2010: Unclear meaning and overstatement of international agreement. Health Policy,111(1), 99–104.
OECD. (2008). Handbook on constructing composite indicators: Methodology and user guide. Paris: OECD Publications.
Osberg, L., & Sharpe, A. (1998). An index of economic well-being for Canada. Research report. Human Resources Development Canada. http://www.csls.ca/iwb/articles.asp. Accessed January 2, 2017.
Osberg, L., & Sharpe, A. (2016a). Database of the index of economic well-being for selected OECD countries and Alberta, 1980–2014. Centre for the Study of Living Standards. http://www.csls.ca/iwb/oecd.asp. Accessed February 21, 2017.
Osberg, L., & Sharpe, A. (2016b). Database of the index of economic well-being for Canada and the provinces, 1981–2014. Centre for the Study of Living Standards. http://www.csls.ca/iwb/prov.asp. Accessed February 21, 2017.
Paruolo, P., Saisana, M., & Saltelli, A. (2013). Ratings and rankings: Voodoo or science? Journal of the Royal Statistical Society,176(3), 609–634.
Pearson, K. (1905). On the general theory of skew correlation and non-linear regression. London: Dulau & Co.
Piketty, T. (2014). Capital in the twenty-first century. Cambridge: The Belknap Press of Harvard University Press.
Ravallion, M. (1997). Good and bad growth: The human development reports. World Development,25(5), 631–638.
Ravallion, M. (2011). Mashup indices of development. World Bank Research Observer,27(1), 1–32.
Saisana, M., Saltelli, A., & Tarantola, S. (2005). Uncertainty and sensitivity analysis techniques as tools for the quality assessment of composite indicators. Journal of the Royal Statistical Society,168(2), 307–323.
Saisana, M., & Tarantola, S. (2002). State-of-the-art report on current methodologies and practices for composite indicator development. Ispra: Joint Research Centre.
Salomon, J. A., Vos, T., Hogan, D. R., Gagnon, M., Naghavi, M., Mokdad, A., et al. (2012). Common values in assessing health outcomes from disease and injury: Disability weights measurement study for the Global Burden of Disease Study 2010. Lancet,380(9859), 2129–2143.
Saltelli, A., & Tarantola, S. (2002). On the relative importance of input factors in mathematical models: Safety assessment for nuclear waste disposal. Journal of the American Statistical Association,97(459), 702–709.
Salzman, J. (2003). Methodological choices encountered in the construction of composite indices of economic and social well-Being. Research paper. http://www.csls.ca/events/cea2003/salzman-typol-cea2003.pdf. Accessed February 23, 2017.
Sustainable Society Foundation. (2016a). Calculation formulas of the Sustainable Society Index, SSI-2016. http://www.ssfindex.com/ssi2014/wp-content/uploads/pdf/calculation_formulas-2016.pdf. Accessed February 23, 2017.
Sustainable Society Foundation. (2016b). SSI 2016. http://www.ssfindex.com/ssi2016/wp-content/uploads/pdf/ssi2016.pdf. Accessed February 23, 2017.
Syrovátka, M., & Hák, T. (2015). Measuring rich-country policies toward the global environment: A critical analysis of the environmental component of the Commitment to Development Index. Journal of Environment & Development,24(1), 105–128.
UNDP (United Nations Development Programme). (1990). Human development report 1990. New York, NY: Oxford University Press.
UNDP (United Nations Development Programme). (2010). Human development report 2010. The real wealth of nations: Pathways to human development. New York, NY: Palgrave Macmillan.
UNDP (United Nations Development Programme). (2011). Human development report 2011. Sustainability and equity: A better future for all. New York, NY: Palgrave Macmillan.
UNDP (United Nations Development Programme). (2013). Human development report 2013. The rise of the south: Human progress in a diverse world. Washington DC: Communications Development Incorporated.
UNDP (United Nations Development Programme). (2014). Human development report 2014. Sustaining human progress: Reducing vulnerabilities and building resilience. Washington DC: Communications Development Incorporated.
UNDP (United Nations Development Programme). (2015). Human development report 2015: Work for human development. Washington DC: Communications Development Incorporated.
UNDP (United Nations Development Programme). (2016). Human development report 2016: Human development for everyone. Washington DC: Communications Development Incorporated.
van de Kerk, G., & Manuel, A. (2008). A comprehensive index for a sustainable society: The SSI—the Sustainable Society Index. Ecological Economics,66(2–3), 228–242.
Veenhoven, R. (2003). Equality-adjusted happiness in nations. In Paper presented at the conference of the International Society for Quality of Life Studies (ISQOLS), Frankfurt (DE).
Veenhoven, R. (2017). Inequality-adjusted happiness in 159 nations 2005–2014. World Database of Happiness. https://worlddatabaseofhappiness.eur.nl/hap_nat/findingreports/RankReport_InequalityHappiness.php. Accessed March 23, 2017.
Veenhoven, R., & Kalmijn, W. M. (2005). Inequality-adjusted happiness in nations: Egalitarianism and utilitarianism married in a new index of societal performance. Journal of Happiness Studies,6, 421–455.
Wilkinson, R., & Pickett, K. (2009). The spirit level: Why greater equality makes societies stronger. New York, NY: Bloomsbury.
Xu, C., & Gertner, G. Z. (2008a). A general first-order global sensitivity analysis method. Reliability Engineering and System Safety,93(7), 1060–1071.
Xu, C., & Gertner, G. Z. (2008b). Uncertainty and sensitivity analysis for models with correlated parameters. Reliability Engineering and System Safety,93(10), 1563–1573.
Yang, L. (2014). An inventory of composite measures of human progress. United Nations Development Programme, Human Development Report Office. http://hdr.undp.org/sites/default/files/inventory_report_working_paper.pdf. Accessed 23 Mar 2017.
Funding
Funding was provided by Univerzita Palackého v Olomouci (Grant No. IGA_PrF_2019_025).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
1.1 Derivation of Formula (3)
Let us show that the correlation coefficient \( r_{k} \) used in the definition of measure of absolute importance (MAI) of the k-th indicator can be expressed in form (3).
The equalities above are based on the definition of correlation coefficient and properties of variance and covariance of a sum.
Now, we express \( cov\left( {OCI,x_{k} } \right) \) by means of correlation coefficient \( \rho_{k} = corr\left( {x_{k} ,OCI} \right) \).
Since \( \rho_{k} = \frac{{cov\left( {OCI,x_{k} } \right)}}{{SD\left( {OCI} \right)SD\left( {x_{k} } \right)}} \), it holds \( cov\left( {OCI,x_{k} } \right) = \rho_{k} SD\left( {OCI} \right)SD\left( {x_{k} } \right) \).
Using this expression, the formula for \( r_{k} \) can be rewritten in the following form:
The final step consists in dividing both nominator and denominator by \( var\left( {OCI} \right) = SD\left( {OCI} \right)^{2} \). Denoting \( RV_{k} = SD\left( {x_{k} } \right)/SD\left( {OCI} \right) \), it immediately follows that
1.2 Derivation of Formula (4)
In the case of uncorrelated indicators \( x_{1} , \ldots ,x_{p} \), it holds
and therefore
By substituting this term into expression (3), one obtain the following equality
Rights and permissions
About this article
Cite this article
Schlossarek, M., Syrovátka, M. & Vencálek, O. The Importance of Variables in Composite Indices: A Contribution to the Methodology and Application to Development Indices. Soc Indic Res 145, 1125–1160 (2019). https://doi.org/10.1007/s11205-019-02125-9
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11205-019-02125-9