Abstract
In the context of the multidimensional measurement of complex phenomena, the major focus of the recent literature has been on the choice of the dimensions’ weights and the shape of the aggregation function, while few studies have concentrated on how normalisation influences the results. With the aim of building a measure of Social Inclusion for 63 European regions in 2012, we adopt a standard linear aggregation framework and compare three alternative normalisation approaches: a data-driven min-max function and a data-driven Z-score, whose parameters depend solely on the available data, and an expert-based function, whose parameters are elicited through a survey at the University of Venice Ca’ Foscari. Regardless of the adopted strategy, we show that normalisation plays a crucial part in defining variables’ weighting. The data-driven strategies allocate a large relative weight to the longevity dimension, whereas the survey-driven results in a rather equal distribution of weights. Data-driven approaches produce trade-offs that are hard to interpret in economic terms and debatable from a social desirability perspective, thus constituting a positive analysis of Social Inclusion. Conversely, the expert-based normalisation is heavily affected by elicitation techniques, and allows for a normative interpretation of the resulting index. Furthermore, the three strategies lead to substantially different conclusions in terms of levels (both between and within countries) and distribution of Inclusion: numerous rank-reversals occur when switching the normalisation methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The act of synthesizing a composite latent phenomenon encompasses methodological issues that have economic, philosophical (as well as psychological) and political connotations. Indeed, these issues arise from a fundamental mismatch between the kind of multiplicity inherent in the latent concept and the multiplicity characterizing the forged measure (the result of the researcher’s work). In a sense, the latent multidimensional concept (e.g., Well-being or Social Inclusion) is an un-synthesized multiplicity, in that it is composite by nature and perceived as a whole by the human sensibility. Since the phenomenon is unmeasurable per se, the researcher is forced to separate it, operationally, in numerous measurable components, in order to aggregate them back to provide a proxy of the latent phenomenon. In other words, building a synthetic index of Well-being requires that the indeterminate nature of multiplicity is made determinate through a specification of its contents, and of their relationship.
- 2.
- 3.
The concept of Social Inclusion/exclusion should not be confused with the variable ‘at risk of poverty or social exclusion’ in the Eurostat database, which defines an individual as at risk of poverty or social exclusion when at least one of the following conditions hold: (a) equivalent household income below 60% of national median; (b) households with at least 4 of the following 9 issues: (i) impossibility to bear unexpected expenses, (ii) cannot afford a week holiday, (iii) issues with the mortgage, rent, bills; (iv) cannot afford a proper meal every 2 days; (v) not able to adequately heat the house; (vi) not able to afford a washing machine (vii) a color TV (viii) a phone (ix) an automobile; (c) living in families whose members aged 18–59 work less than a fifth of their time.
- 4.
- 5.
As a robustness check we enlarged the sample with data for statistical-regions for Czech Republic, Greece, Norway and the Netherlands, without any significant change to the results of the analysis. Besides, as stated in the introduction, the purpose of this chapter is to offer a methodological discussion that can be applied to composite analyses in various fields and from various data-selections.
- 6.
In the words of Martinetti and von Jacobi (2012), the implicit assumption for equal weighting is that “in absence of any objective mechanism for determining the relative importance of the considered dimensions, the most neutral method is assigning an equal weight to each of them”. Indeed, both Chowdhury and Squire (2006) and Nguefack-Tsague et al. (2011) provide evidence in favour of equal weighting after collecting expert preferences.
- 7.
The MRS between two observed dimensions will be equal to their “weights” also if the derivatives of their normalisation functions are equal, i.e., if \( {v}_k^{\prime}\left({x}_k\right)/{v}_j^{\prime}\left({x}_j\right)=1 \)
- 8.
The choice of multiplying by 100 eases readability of the results in the remaining of the paper, and does not affect any result.
- 9.
The autonomous cities of Ceuta and Medilla, located on the Mediterranean coast of Morocco but belonging to Spain since fifteenth century, are substantially different from other Spanish regions. Given that their values for school-dropouts, long-term unemployment and poverty rate are sensibly higher than the rest of the sample, we prudently decided to treat them as outliers and exclude them from the computation of the thresholds. This decision has no significant consequences on the results of the paper, nor on its implications. Including them in the sample would raise the maximum values for early school-leaving rate to 54.2% (Ceuta 2005), for long-term unemployment to 18.2% (Ceuta 2012), and for poverty rate to 48.9% (Ceuta 2008). A graphical distribution of the data used for the min-max normalisation is reported in Fig. 11.6 (Appendix A)
- 10.
A common specification for the Z-score normalisation in time-dependent studies (as detailed, e.g., in OECD and European Commission (2008)) adopts as references the averages and the standard deviations across countries for a given reference year. We chose to compute both the references across countries and time, to be consistent with strategy followed in designing the data-driven min-max (where the same aforementioned OECD report suggests to adopt minima and maxima across countries and time). As a robustness test, we also computed the Z-score values using as references the averages and the standard deviations across countries for the year 2012. Such change has no consequences on the results and implications of our analysis.
- 11.
Although, in principle, it would be of interest to widen the Survey population to professors of other Departments (Asian and North African Studies, Environmental Sciences, Humanities, Linguistic, Molecular Sciences and Philosophy), we were led by time and resources constraints to focus on those Faculty more specifically connected to the issues of Social Inclusion and to the disciplines related to the four indicators over which a judgment was asked.
- 12.
For further details, please refer to http://www.qualtrics.com/
- 13.
No territories in our sample reach 5% poverty-rate or 73 years in longevity-at-birth, so no “positive” capping occurs.
- 14.
- 15.
- 16.
Atkinson et al. (2004) argue that the ultimate concern of the policy-maker should be casted on performance levels, since rankings might conceal the actual distances between territorial units, thus leading the reader to misleading conclusions.
- 17.
Still, we know from Fig. 11.3 that their distance from the more virtuous territories is lower under the data-driven approaches
- 18.
The Kendall-τ test is a non-parametric method that allows to measure the degree of correspondence between two rankings. In particular, the Kendall-τ b allows for the possibility of ties in the rankings. Command in STATA: ktau. A resulting test-value of zero would indicate that no correlation exist between the two rankings, while a value of 1 would indicate perfect correlation. Conversely, negative values (down to a minimum of −1) would indicate that rankings are inverted.
- 19.
Fixed intervals of values were imposed in order to avoid extreme and implausible choices (like 0 years old of longevity as “harmful” threshold). The predetermined intervals were: [90–60 years] for longevity; [0%, 50%] for early-school-leaving; [0%, 50%] for long-term unemployment; and [0%, 50%] for poverty-rate. No respondents chose one of the non-zero extremes as their preferred threshold.
- 20.
The disclaimer aimed at avoiding inconsistent choices, e.g., a respondent who would choose, say, 81 years old as a harmful threshold, and subsequently choose 80 as a favourable threshold. No such patterns occurred.
- 21.
We chose the median response as a measure of central tendency to summarize a representative answer, as often done in the literature (e.g., Hoskins and Mascherini (2009)) because of its lower sensitivity to outliers, especially when the sample size is small.
References
Anand, S., & Sen, A. (1994). Human development index: Methodology and measurement. Human Development Report Office (HDRO). New York: United Nations Development Programme (UNDP).
Atkinson, A. B., Cantillon, B., Marlier, E., & Nolan, B. (2002). Social indicators: The EU and social inclusion: The EU and social inclusion. Oxford: Oxford University Press.
Atkinson, A. B., Marlier, E., & Nolan, B. (2004). Indicators and targets for social inclusion in the European Union. JCMS: Journal of Common Market Studies, 42(1), 47–75.
Bertin, G., Carrino, L., & Giove, S. (2016). The Italian regional well-being in a multi-expert non-additive perspective. Social Indicators Research, 3, 1–37.
Blackorby, C., & Donaldson, D. (1982). Ratio-scale and translation-scale full interpersonal comparability without domain restrictions: Admissible social-evaluation functions. International Economic Review, 23, 249–268.
Boarini, R., & D’Ercole, M. M. (2013). Going beyond GDP: An OECD perspective*. Fiscal Studies, 34(3), 289–314. doi:10.1111/j.1475-5890.2013.12007.x.
Bourguignon, F., & Chakravarty, S. R. (2003). The measurement of multidimensional poverty. The Journal of Economic Inequality, 1(1), 25–49.
Cherchye, L., Knox Lovell, C. A., Moesen, W., & Van Puyenbroeck, T. (2007). One market, one number? A composite indicator assessment of EU internal market dynamics. European Economic Review, 51(3), 749–779. doi:http://dx.doi.org/10.1016/j.euroecorev.2006.03.011.
Chowdhury, S., & Squire, L. (2006). Setting weights for aggregate indices: An application to the commitment to development index and human development index. Journal of Development Studies, 42(5), 761–771.
Decancq, K., & Lugo, M. A. (2008). Setting weights in multidimensional indices of well-being. Oxford: Oxford Poverty & Human Development Initiative.
Decancq, K., & Lugo, M. A. (2009). Measuring inequality of well-being with a correlation-sensitive multidimensional Gini index. University of Oxford Department of Economics Discussion Paper Series, 459.
Decancq, K., & Lugo, M. A. (2013). Weights in multidimensional indices of wellbeing: An overview. Econometric Reviews, 32(1), 7–34.
European Commission. (2009). Portfolio of indicators for the monitoring of the European strategy for social protection and social inclusion Bruxelles.
European Commission (2010). Combating poverty and social exclusion. A statistical portrait of the European Unio. European Union.
European Communities Commission. (1992). Towards a Europe of solidarity. Intensifying the fight against social exclusion, fostering integration. Communication from the commission. COM(92) 542 final, 23 December 1992. Bruxelles: Commission of the European Communities.
European Council. (2000). Lisbon European Council 23–24 March 2000. Presidency conclusions.
European Council. (2001). Laeken European Council. 14–15 December 2001. Presidency conclusions and annexes.
Giovannini, E., Nardo, M., Saisana, M., Saltelli, A., Tarantola, A., & Hoffman, A. (2008). Handbook on constructing composite indicators: Methodology and user guide. Paris: OECD.
Hoskins, B., & Mascherini, M. (2009). Measuring active citizenship through the development of a composite indicator. Social Indicators Research, 90(3), 459–488. doi:10.1007/s11205-008-9271-2.
Kasparian, J., & Rolland, A. (2012). OECD’s ‘better life index’: Can any country be well ranked? Journal of Applied Statistics, 39(10), 2223–2230.
Kim, Y., Kee, Y., & Lee, S. (2015). An analysis of the relative importance of components in measuring community wellbeing: Perspectives of citizens, public officials, and experts. Social Indicators Research, 121(2), 345–369. doi:10.1007/s11205-014-0652-4.
Klugman, J., Rodríguez, F., & Choi, H.-J. (2011). The HDI 2010: New controversies, old critiques. The Journal of Economic Inequality, 9(2), 249–288.
Lefebvre, M., Coelli, T., & Pestieau, P. (2010). On the convergence of social protection performance in the European Union. CESifo Economic Studies, 56(2), 300–322.
Maggino, F., & Nuvolati, G. (2012). Quality of life in Italy: Research and Reflections. 48: Springer.
Martinetti, E. C., & von Jacobi, N. (2012). Light and shade of multidimensional indexes: How methodological choices impact on empirical results. Quality of life in Italy: Research and Reflections. Social Indicators Research Series, 48.
Mascherini, M., & Hoskins, B. (2008). Retrieving expert opinion on weights for the Active Citizenship Composite Indicator. European Commission–Institute for the protection and security of the citizen–EUR JRC46303 EN.
Mazziotta, M., & Pareto, A. (2015). On a generalized non-compensatory composite index for measuring socio-economic phenomena. Social indicators research, 1–21. doi:10.1007/s11205-015-0998-2.
Meyer, P., & Ponthière, G. (2011). Eliciting preferences on multiattribute societies with a Choquet integral. Computational Economics, 37(2), 133–168.
Murias, P., Novello, S., & Martinez, F. (2012). The regions of economic well-being in Italy and Spain. Regional Studies, 46(6), 793–816.
Nguefack-Tsague, G., Klasen, S., & Zucchini, W. (2011). On weighting the components of the human development index: A statistical justification. Journal of human development and capabilities, 12(2), 183–202.
OECD, & European Commission, J. R. C. E. (2008). Handbook on constructing composite indicators: Methodology and user guide. Paris: OECD Publishing.
Pestieau, P. (2009). Assessing the performance of the public sector. Annals of public and cooperative economics, 80(1), 133–161.
Pinar, M., Cruciani, C., Giove, S., & Sostero, M. (2014). Constructing the FEEM sustainability index: A Choquet integral application. Ecological Indicators, 39, 189–202.
Ravallion, M. (2011). On multidimensional indices of poverty. The Journal of Economic Inequality, 9(2), 235–248.
Ravallion, M. (2012a). Mashup indices of development. World Bank Research Observer, 27(1), 1–32.
Ravallion, M. (2012b). Troubling tradeoffs in the human development index. Journal of Development Economics, 99(2), 201–209.
Saisana, M., Saltelli, A., & Tarantola, S. (2005). Uncertainty and sensitivity analysis techniques as tools for the quality assessment of composite indicators. Journal of the Royal Statistical Society: Series A (Statistics in Society), 168(2), 307–323.
Sen, A., & Anand, S. (1997). Concepts of human development and poverty: A multidimensional perspective. Poverty and human development: Human development papers. NewYork: UNDP.
Silva, R., & Ferreira-Lopes, A. (2013). A regional development index for Portugal. Social Indicators Research, 1–31.
Stiglitz, J. E., Sen, A., & Fitoussi, J.-P. (2010). Report by the commission on the measurement of economic performance and social progress.
Giovanni Bertin, Ludovico Carrino, Silvio Giove, The Italian Regional Well-Being in a Multi-expert Non-additive Perspective. Social Indicators Research
Acknowledgement
The author wishes to thank Michele Bernasconi, Giovanni Bertin, Eric Bonsang, Agar Brugiavini, Stefano Campostrini, Roberto Casarin, Koen Decancq, Silvio Giove, Filomena Maggino, Sergio Perelman, Pierre Pestieau, Dino Rizzi, Maurizio Zenezini, for their valuable comments to previous versions of this paper. The paper benefited as well from comments by participants to seminars at Ca’ Foscari University of Venice, University of Trieste, as well as to the conference “Complexity in Society” at University of Padova and “Data Science and Social Research” at University of Napoli Federico II. The author acknowledges the financial support of Fondazione Ca’ Foscari.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendices
Appendices
Appendix A: Data-Description
Variables’ distribution for data-driven normalisation for Belgium, Denmark, Germany, Italy and Spain from 2004 to 2012
Appendix B: Description of the Survey
The Survey was structured as follows:
-
An introductory section discussed the topics, the purpose and the contents of the survey.
-
Respondents were asked to select the variables (amongst the four described in section “Social Inclusion, definition and sample selection”) for which they would be willing to perform an evaluation.
-
A randomization led the respondent to a page devoted to one of the selected variables. All pages were homogeneously designed with a consistent phrasing.
-
The EUROSTAT definition of the variable at hand was offered, and descriptive statistics were shown through a bar graph, for 25 European countries (years 2000 and 2012).
-
The main task of the survey was then detailed. Respondents should identify, according to their own opinion, two main thresholds for the variable at hand: a negative threshold, defined as a “value of the selected variable which conveys a certainly undesirable and problematic condition”, and a positive defined as “a value conveying a certainly desirable and virtuous condition”. The threshold had to be chosen by dragging a slider (using the mouse left-click) on a predetermined discrete interval of values,Footnote 19 and releasing it to identify the preferred value (see Fig. 11.7 for a snapshot of the negative-threshold choice for life expectancy).
-
An example involving a mock variable “X” explained how to deal with the Qualtrics layout in order to identify the thresholds.
-
After choosing the positive and the negative thresholds, a confirmation was required by clicking on “confirm and proceed” button, which would lead the respondent to the next variable-specific page, or to the last section of the survey (if no variables were left).
-
The last section of the survey included questions on respondents’ age, gender and affiliation (either Economics or Management).
As an example, let us consider the survey-page devoted to the life-expectancy-at-birth indicator. First, a definition of life expectancy was provided. Then, data for 25 European countries (years 2000 and 2012) were shown. At this point, respondents are faced with the summary of what they will be asked to do, i.e., identifying both a favourable and a harmful threshold for life-expectancy-at-birth, according to their own opinion. The harmful threshold is defined as a “level of longevity which represents a certainly negative and undesirable condition”. The favour threshold is defined as a “level of longevity which represents a certainly positive and desirable condition”. Before reaching the actual question, a full example was provided with a generic variable “X”. Respondents had, then, to determine the harmful threshold by dragging a slider on an interval of values (with the left mouse-click), and dropping it at the point that corresponded to their view of a certainly undesirable level of longevity. Figure 11.7 illustrates the choice that respondents were facing for the harmful threshold of longevity. The choice was not entirely free, since we constrained respondents to select a level of life expectancy within a predetermined interval ranging from 60 to 90 years old, in order to avoid extremely implausible choices (like 0 years old). Similar steps characterized the choice of the favourable threshold, where respondents had to select their answer in the same interval between 60 and 90 years old. A cautionary disclaimer was emphasized at this point, stressing the fact that the favourable threshold should, by construction, be higher than- or equal to- the harmful threshold previously selected.Footnote 20
Choice of the negative threshold for life expectancy
Out of 149 invitations, we received 88 responses. 59 were faculty members of the Department of Economics, 29 from the Department of Management. The following table provide brief descriptive statistics on our sample.
Median responses and interquartile range are reported in Table 11.15,Footnote 21 while Fig. 11.8 illustrates the histograms for the responses’ distribution. The blue thick-dashed lines represent the answers for the favourable thresholds.
Distribution of survey responses
Appendix C: Results for Administrative Regions
The following coefficients are obtained by implementing the LD model (11.9)
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Carrino, L. (2017). The Role of Normalisation in Building Composite Indicators. Rationale and Consequences of Different Strategies, Applied to Social Inclusion. In: Maggino, F. (eds) Complexity in Society: From Indicators Construction to their Synthesis. Social Indicators Research Series, vol 70. Springer, Cham. https://doi.org/10.1007/978-3-319-60595-1_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-60595-1_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-60593-7
Online ISBN: 978-3-319-60595-1
eBook Packages: Social SciencesSocial Sciences (R0)