Skip to main content
SpringerLink
Log in

Robust Distribution-Based Winsorization in Composite Indicators Construction

  • Original Research
  • Published:
Social Indicators Research Aims and scope Submit manuscript

Abstract

Composite indicators are widely used to determine the ranking of countries, organizations or individuals in terms of overall performance on multiple criteria. Their calculation requires standardization of the individual statistical criteria and aggregation of the standardized indicators. These operations introduce a potential propagation effect of extreme values on the calculation of the composite indicator of all entities. In this paper, we propose robust composite indicators for which this propagation effect is limited. The approach uses winsorization based on a robust estimate of the distribution of the sub-indicators. It is designed such that the winsorization affects only the composite indicator rank but has no effect on the entities ranking in each sub-indicator. The simulation study documents the benefits of distribution-based winsorization in the presence of outliers. It leads to a ranking that is closer to the clean data ranking when compared to the ranking obtained using either no winsorization or the traditional winsorization based on empirical quantiles. In the empirical application, we illustrate the use of winsorization for ranking countries based on the United Nations Industrial Development Organization’s Competitive Industrial Performance index. We show that even though the sub-indicator ranking does not change, the robust winsorization approach has a material impact on the ranking of the composite indicator for countries with large discrepancies in the scores of the sub-indicators.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. A related but different literature on composite indicator robustness studies the sensitivity of the composite indicator-based ranking to the choice of standardization, aggregation and imputation methods. Booysen (2002) provides an overview of dimensions to classify and evaluate the various implementations. Saisana and Saltelli (2011) use Monte Carlo techniques to quantify the sensitivity of the composite indicator rankings to the choice of standardization, aggregation and imputation method. Their technique is used in UNIDO (2013) to evaluate the robustness of the Composite Industrial Performance index. Davino and Romano (2014) describe how multivariate statistical techniques such as analysis of variance (ANOVA) and principal components analysis (PCA) can be used to analyze the variability due to the different composite indicator construction methods. Cherchye et al. (2008) propose a linear programming technique to obtain a robust ranking, which holds for a wide set of normalization and/or aggregation procedures. Permanyer (2011) studies the lack of ranking robustness by choosing specific weights scheme for the variables included in composite indicators.

  2. Alternative approaches to reduce the sensitivity of composite indicators to outliers include the use of data transformation (e.g., taking logarithms, Box-Cox function) (see, e.g., Correia 2014; OECD 2008b; Parente 2019) and data trimming (see, e.g., Cherchye et al. 2007; Saisana and Saltelli 2008).

  3. It should be noted that other complex distributions could be considered to model the data, but the Weibull distribution is chosen because of its simple form and because it is applicable to the indicators used in the empirical application. Inspired by Alfons et al. (2013), we replace the extreme observations with quantiles of the fitted Weibull distribution of sub-indicators.

  4. For instance, when we extend the dimension of sub-indicators to 8 or use data generated from a Weibull distribution with other parameters, we still reach similar conclusions.

References

  • Alfons, A., Templ, M., & Filzmoser, P. (2013). Robust estimation of economic indicators from survey samples based on Pareto tail modelling. Journal of the Royal Statistical Society: Series C (Applied Statistics), 62, 271–286.

    Article  Google Scholar 

  • Boudt, K., Caliskan, D., & Croux, C. (2011). Robust explicit estimators of Weibull parameters. Metrika, 73(2), 187–209.

    Article  Google Scholar 

  • Booysen, F. (2002). An overview and evaluation of composite indices of development. Social Indicators Research, 59(2), 115–151.

    Article  Google Scholar 

  • Cherchye, L., Moesen, W., Rogge, N., & Van Puyenbroeck, T. (2007). An introduction to “benefit of the doubt” composite indicators. Social Indicators Research, 82, 111–145.

    Article  Google Scholar 

  • Cherchye, L., Ooghe, E., & Van Puyenbroeck, T. (2008). Robust human development rankings. The Journal of Economic Inequality, 6, 287–321.

    Article  Google Scholar 

  • Correia, C. M. (2014). Measuring creativity in the EU member states. Investigaciones Regionales: Journal of Regional Research, 30, 7–26.

    Google Scholar 

  • Davino, C., & Romano, R. (2014). Assessment of composite indicators using the ANOVA model combined with multivariate methods. Social Indicators Research, 119, 627–646.

    Article  Google Scholar 

  • Hubert, M., Rousseeuw, P. J., & Verdonck, T. (2012). A deterministic algorithm for robust location and scatter. Journal of Computational and Graphical Statistics, 21(3), 618–637.

    Article  Google Scholar 

  • Khan, J. A., Van Aelst, S., & Zamar, R. H. (2007). Robust linear model selection based on least angle regression. Journal of the American Statistical Association, 102(480), 1289–1299.

    Article  Google Scholar 

  • Maggino, F. (2017). Complexity in society: From indicators construction to their synthesis (Vol. 70). Berlin: Springer.

    Book  Google Scholar 

  • Moll de Alba, J., & Todorov, V. (2018). Green industrial performance: The GIP index. World Review of Science, Technology and Sustainable Development, 14(2–3), 266–293.

    Article  Google Scholar 

  • Moll de Alba, J., & Todorov, V. (2019). Measurement of green industrial performance: An enhanced GIP index. International Journal of Environment and Sustainable Development (forthcoming).

  • OECD. (2008). Glossary of statistical terms.

  • OECD. (2008). Handbook on constructing composite indicators: Methodology and user guide. Paris: OECD.

  • Parente, F. (2019). A multidimensional analysis of the EU regional inequalities. Social Indicators Research, 143(3), 1017–1044.

    Article  Google Scholar 

  • Permanyer, I. (2011). Uncertainty and robustness in composite indices rankings. Oxford Economic Papers, 64(1), 57–79.

    Article  Google Scholar 

  • Rousseeuw, P. J. (1984). Least median of squares regression. Journal of the American Statistical Association, 79(388), 871–880.

    Article  Google Scholar 

  • Rousseeuw, P. J. (1985). In W. Grossmann, G. Pflug, I. Vincze and W. Wertz (Eds.), Mathematical statistics and applications (Vol. B, pp. 283–297). Dordrecht: Reidel.

  • Rousseeuw, P. J., & Driessen, K. (1999). A fast algorithm for the minimum covariance determinant estimator. Technometrics, 41, 212–223.

    Article  Google Scholar 

  • Saisana, M. (2010). ELLI-index: A sound measure for lifelong learning in the EU European Commission. EUR 24529 EN.JRC, IPSC, Ispra, Italy (pp. 15–16).

  • Saisana, M., & Saltelli, A. (2008). Expert panel opinion and global sensitivity analysis for composite indicators. Computational methods in transport: Verification and validation (pp. 251–275). Dordrecht: Springer.

    Chapter  Google Scholar 

  • Saisana, M., & Saltelli, A. (2011). Rankings and ratings: Instructions for use. Hague Journal on the Rule of Law, 3(2), 247–268.

    Article  Google Scholar 

  • Saltelli, A. (2007). Composite indicators between analysis and advocacy. Social Indicators Research, 81, 65–77.

    Article  Google Scholar 

  • Tarabusi, E. C., & Guarini, G. (2013). An unbalance adjustment method for development indicators. Social Indicators Research, 112(1), 19–45.

    Article  Google Scholar 

  • Todorov, V., & Filzmoser, P. (2009). An object-oriented framework for robust multivariate analysis. Journal of Statistical Software, 32(3), 1–47.

    Article  Google Scholar 

  • UNIDO. (2002). Industrial development report 2002/2003: Competing through innovation and learning. Vienna: UNIDO.

  • UNIDO. (2013). Competitive industrial performance index 2012/2013: The industrial competitiveness of nations: Looking back, forging ahead. Vienna: UNIDO.

  • UNIDO. (2017). Competitive industrial performance report 2016 (Vols. 1–2). Vienna: UNIDO.

  • UNIDO. (2019). Competitive industrial performance report 2018. Vienna: UNIDO.

Download references

Author information

Authors and Affiliations

  1. Department of Economics, Ghent University, 9000 Ghent, Belgium

    Kris Boudt

  2. Solvay Business School, Vrije Universiteit Brussel, 1050, Brussel, Belgium

    Kris Boudt & Wenjing Wang

  3. School of Business and Economics, Vrije Universiteit Amsterdam, 1081, Amsterdam, the Netherlands

    Kris Boudt

  4. United Nations Industrial Development Organization (UNIDO), 1400 Vienna, Austria

    Valentin Todorov

  5. School of Statistics and Mathematics, Central University of Finance and Economics, 100081, Beijing, China

    Wenjing Wang

Authors
  1. Kris Boudt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Valentin Todorov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenjing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenjing Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The views expressed herein are those of the author(s) and do not necessarily reflect the views of the United Nations Industrial Development Organization.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Boudt, K., Todorov, V. & Wang, W. Robust Distribution-Based Winsorization in Composite Indicators Construction. Soc Indic Res 149, 375–397 (2020). https://doi.org/10.1007/s11205-019-02259-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11205-019-02259-w

Keywords

Search

Navigation