Advertisement

Weights and Importance in Composite Indicators: Mind the Gap

  • William Becker
  • Paolo Paruolo
  • Michaela Saisana
  • Andrea Saltelli
Reference work entry

Abstract

Multidimensional measures (often termed composite indicators) are popular tools in the public discourse for assessing the performance of countries on human development, perceived corruption, innovation, competitiveness, or other complex phenomena. These measures combine a set of variables using an aggregation formula, which is often a weighted arithmetic average. The values of the weights are usually meant to reflect the variables importance in the index. This paper uses measures drawn from global sensitivity analysis, specifically the Pearson correlation ratio, to discuss to what extent the importance of each variable coincides with the intentions of the developers. Two nonparametric regression approaches are used to provide alternative estimates of the correlation ratios, which are compared with linear measures. The relative advantages of different estimation procedures are discussed. Three case studies are investigated: the Resource Governance Index, the Good Country Index, and the Financial Secrecy Index.

Keywords

Composite indicators Nonlinear regression Sensitivity analysis 

References

  1. 1.
    Anholt, S., Govers, R.: The good country index. Tech. rep., The Good Country Party. http://www.goodcountry.org/ (2014)
  2. 2.
    Athanasoglou, S., Weziak-Bialowolska, D., Saisana, M.: Environmental performance index 2014: Jrc analysis and recommendations. Tech. rep., European Commission, Joint Research Centre (2014)Google Scholar
  3. 3.
    Bandura, R.: Composite indicators and rankings: inventory 2011. Tech. rep., United Nations Development Programme – Office of Development Studies (2011)Google Scholar
  4. 4.
    Bowman, A., Azzalini, A.: Applied Smoothing Techniques for Data Analysis: The Kernel Approach with S-Plus Illustrations, vol 18. Oxford University Press, New York (1997)zbMATHGoogle Scholar
  5. 5.
    Cobham, A., Jansky, P., Christensen, J., Eichenberger, S.: Financial Secrecy Index 2013: Methodology. Tech. rep., The Tax Justice Network. http://www.financialsecrecyindex.com/ (2013)
  6. 6.
    Cobham, A., Janský, P., Meinzer, M.: The financial secrecy index: shedding new light on the geography of secrecy. Econ. Geogr. 91(3), 281–303 (2015)CrossRefGoogle Scholar
  7. 7.
    Da Veiga, S., Wahl, F., Gamboa, F.: Local polynomial estimation for sensitivity analysis on models with correlated inputs. Technometrics 51(4), 452–463 (2009)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Decancq, K., Lugo, M.A.: Weights in multidimensional indices of wellbeing: an overview. Econ. Rev. 32(1), 7–34 (2013)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Data Mining, Inference and Prediction. Springer, New York (2001)CrossRefzbMATHGoogle Scholar
  10. 10.
    Kelley, J.G., Simmons, B.A.: Politics by number: indicators as social pressure in international relations. Am. J. Pol. Sci. 59(1), 55–70 (2015)CrossRefGoogle Scholar
  11. 11.
    Li, G., Rabitz, H., Yelvington, P.E., Oluwole, O.O., Bacon, F., Kolb, C.E., Schoendorf, J.: Global sensitivity analysis for systems with independent and/or correlated inputs. J. Phys. Chem. A 114(19), 6022–6032 (2010)CrossRefGoogle Scholar
  12. 12.
    Paracchini, M.L., Zulian, G., Kopperoinen, L., Maes, J., Schägner, J.P., Termansen, M., Zandersen, M., Perez-Soba, M., Scholefield, P.A., Bidoglio, G.: Mapping cultural ecosystem services: a framework to assess the potential for outdoor recreation across the EU. Ecol. Indic. 45, 371–385 (2014)CrossRefGoogle Scholar
  13. 13.
    Paruolo, P., Saisana, M., Saltelli, A.: Ratings and rankings: voodoo or science? J. R. Stat. Soc. Ser. A. (Stat. Soc.) 176(3), 609–634 (2013)Google Scholar
  14. 14.
    Pearson, K.: On the General Theory of Skew Correlation and Non-linear Regression. Volume XIV of Mathematical Contributions to the Theory of Evolution, Drapers’ Company Research Memoirs. Dulau & Co., London (1905). Reprinted in: Early Statistical Papers, Cambridge University Press, Cambridge (1948)Google Scholar
  15. 15.
    Quiroz, J.C., Lintzer, M.: The 2013 resource governance index. Tech. rep., The Revenue Watch Institute (2013)Google Scholar
  16. 16.
    Rasmussen, C., Williams, C.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)zbMATHGoogle Scholar
  17. 17.
    Ruppert, D., Wand, M., Carroll, R.: Semiparametric Regression, vol. 12. Cambridge University Press, Cambridge (2003)CrossRefzbMATHGoogle Scholar
  18. 18.
    Saisana, M., Saltelli, A.: Joint Research Centre statistical audit of the 2014 Global Innovation Index. Tech. rep., European Commission, Joint Research Centre (2014)Google Scholar
  19. 19.
    Saisana, M., Saltelli, A., Tarantola, S.: Uncertainty and sensitivity analysis techniques as tools for the quality assessment of composite indicators. J. R. Stat. Soc. A 168(2), 307–323 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Saisana, M., d’Hombres, B., Saltelli, A.: Rickety numbers: volatility of university rankings and policy implications. Res. Policy 40(1), 165–177 (2011)CrossRefGoogle Scholar
  21. 21.
    Saltelli, A., Tarantola, S., Campolongo, F.: Sensitivity analysis as an ingredient of modelling. Stat. Sci. 15(4), 377–395 (2000)CrossRefGoogle Scholar
  22. 22.
    Saltelli, A., Tarantola, S.: On the relative importance of input factors in mathematical models: safety assessment for nuclear waste disposal. J. Am. Stat. Assoc. 97, 702–709 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M., Tarantola, S.: Global Sensitivity Analysis – The Primer. John Wiley & Sons, Hoboken (2008)zbMATHGoogle Scholar
  24. 24.
    Saltelli, A., Ratto, M., Tarantola, S., Campolongo, F.: Update 1 of: Sensitivity analysis for chemical models. Chem. Rev. 112(5), 1–25 (2012)CrossRefzbMATHGoogle Scholar
  25. 25.
    Storlie, C., Helton, J.: Multiple predictor smoothing methods for sensitivity analysis: Description of techniques. Reliab. Eng. Syst. Saf. 93(1), 28–54 (2008)CrossRefGoogle Scholar
  26. 26.
    Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B (Methodol.) 58, 267–288 (1996)MathSciNetzbMATHGoogle Scholar
  27. 27.
    Yang, L.: An inventory of composite measures of human progress. Tech. rep., United Nations Development Programme Human Development Report Office (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2017

Authors and Affiliations

  • William Becker
    • 1
  • Paolo Paruolo
    • 1
  • Michaela Saisana
    • 1
  • Andrea Saltelli
    • 2
    • 3
  1. 1.European Commission Joint Research CentreIspra (VA)Italy
  2. 2.Centre for the Study of the Sciences and the Humanities (SVT)University of Bergen (UIB)BergenNorway
  3. 3.Institut de Ciencia i Tecnologia Ambientals (ICTA)Universitat Autonoma de Barcelona (UAB)BarcelonaSpain

Personalised recommendations