Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Measuring well-being dispersion on discrete rating scales

  • 70 Accesses


Average levels of well-being across countries and time periods have been reported widely over the years as one of the measures that go beyond the Gross Domestic Product. It is relevant for policy makers to also have information on well-being dispersion to identify groups that lag behind with respect to their ability to pursue happiness. In addition, there is an inherent moral appeal to not only maximize wellbeing but also equalize well-being among people. In this paper, we try to answer the following research question: which measure(s) should be used to gain insight into wellbeing dispersion? We review sixteen measures and their properties, study their behavior in over 92 thousand simulated distributions, and apply them to 4 years of the Dutch Social Cohesion and Well-Being Survey, using bootstrapping to quantify their precision. Our inventory shows that when applied to a discrete ordinal rating scale, common measures such as the standard deviation and Gini coefficient do not show any advantage over the less restrictive index of ordinal variation (IOV). Only the generalized entropy and Atkinson index adhere to additional principles, most notably of diminishing transfers, at the expense of full scale invariance. The simulation study illustrates that dispersion measures are positively correlated but do not rank distributions the same. The field study shows that only the Atkinson index with high inequality aversion provides additional insight. We recommend using the index of ordinal variation, supplemented if needed with the Atkinson index using a high value for the inequality aversion parameter.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6


  1. Allison, P. D. (1978). Measures of inequality. American Sociological Review,43(6), 865–880.

  2. Atkinson, A. (1970). On the measurement of inequality. Journal of Economic Theory,2, 244–263.

  3. Balakrishnan, N., & Ma, C. W. (1990). A comparative study of various tests for the equality of two population variances. Journal of Statistical Computation and Simulation,35(1–2), 41–89.

  4. Berger, Y. G., & Muñoz, J. F. (2015). On estimating quantiles using auxiliary information. Journal of Official Statistics,31(1), 101–119.

  5. Berry, K. J., & Mielke, P. W. (1992). Assessment of variation in ordinal data. Perceptual and Motor Skills,74(1), 63–66.

  6. Bickel, P. J., & Lehmann, E. L. (1976). Descriptive statistics, for nonparametric models. III dispersion. The Annals of Statistics,4(6), 1139–1158.

  7. Biewen, M. (2002). Bootstrap inference for inequality, mobility and poverty measurement. Journal of Econometrics,108, 317–342.

  8. Blair, J., & Lacy, M. G. (2000). Statistics of ordinal variation. Sociological Methods & Research,28(3), 251–280.

  9. Boos, D. D., & Brownie, C. (2004). Comparing variances and other measures of dispersion. Statistical Science,19(4), 571–578.

  10. Box, G. E. P. (1953). Non-normality and tests on variances. Biometrika,40(3–4), 318–335.

  11. Brown, M. B., & Forsythe, A. B. (1974). Robust tests for the equality of variances. Journal of the American Statistical Association,69(346), 364–367.

  12. Calaway, R., & Weston, S. (2015). Foreach: provides foreach looping construct for R. Accessed January 2019.

  13. Calaway, R., Weston, S., & Tenenbaum, D. (2015). DoParallel: foreach parallel adaptor for the ‘parallel’ package. Accessed January 2019.

  14. Conover, W. J., Johnson, M. E., & Johnson, M. M. (1981). A comparative study of tests for homogeneity of variances, with applications to the outer continental shelf bidding data. Technometrics,23(4), 351–361.

  15. Cowell, F. A. (2009). Measuring inequality. Oxford: Oxford University Press.

  16. Cowell, F. A., & Flachaire, E. (2017). Inequality with ordinal data. Economica,84, 290–321.

  17. Cummins, R. A. (2005). The domains of life satisfaction: an attempt to order chaos. In A. C. Michalos (Ed.), Citation classics from social indicators research (Vol. 26, pp. 559–584)., Social indicators research series Dordrecht: Springer.

  18. Cummins, R. A., & Gullone, E. (2000). Why we should not use 5-point Likert scales: The case for subjective quality of life measurement. In Proceedings of the second international conference on quality of life in cities (pp. 74–93). Singapore: National University of Singapore.

  19. Curran, J., Williams, J., Kelleher, J., & Barber., D. (2016). Multicool: permutations of multisets in cool-lex order. Accessed January 2019.

  20. Diener, E., Wirtz, D., Tov, W., Kim-Prieto, C., Choi, D., Oishi, S., et al. (2010). New well-being measures: Short scales to assess flourishing and positive and negative feelings. Social Indicators Research,97(2), 143–156.

  21. Duclos, J.-Y., Esteban, J., & Ray, D. (2004). Polarization: Concepts, measurements, estimation. Econometrica,72(6), 1737–1772.

  22. Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap (Vol. 57)., Monographs on statistics and applied probability New York: Chapman and Hall.

  23. Emerson, J. W., Green, W. A., Schloerke, B., Crowley, J., Cook, D., Hofmann, H., et al. (2012). The generalized pairs plot. Journal of Computational and Graphical Statistics,22, 79–91.

  24. Esteban, J.-M., & Ray, D. (1994). On the measurement of polarization. Econometrica,62(4), 819–851.

  25. Eurostat (2017). Final report of the expert group on quality of life indicators. Luxembourg: Publications Office of the European Union.

  26. Fordyce, M. W. (1988). A review of research on the happiness measures: A sixty second index of happiness and mental health. Social Indicators Research,20(4), 355–381.

  27. Geng, S., Wang, W. J., & Miller, C. (1979). Small sample size comparisons of test for homogeneity of variances by Monte-Carlo. Communications in Statistics—Simulation and Computation,8(4), 379–389.

  28. Gini, C. (1921). Measurement of inequality of incomes. The Economic Journal,31(121), 124–126.

  29. Hall, I. J. (1972). Some comparisons of tests for equality of variances. Journal of Statistical Computation and Simulation,1(2), 183–194.

  30. Hankin, R. K. S. (2017). Partitions: Additive partitions of integers. Accessed January 2019.

  31. Hoover, E. M., Jr. (1936). The measurement of industrial localization. The Review of Economics and Statistics,18(4), 162–171.

  32. Hoskins, B., Saisana, M., & Villalba, C. M. H. (2015). Civic competence of youth in Europe: Measuring cross national variation through the creation of a composite indicator. Social Indicators Research,123(2), 431–457.

  33. Jenkins, S. P., & Van Kerm, P. (2009). The measurement of economic inequality. In B. Nolan, W. Salverda, & T. M. Smeeding (Eds.), The Oxford handbook of economic inequality (pp. 40–70). Oxford: Oxford University Press.

  34. Kalmijn, W., & Veenhoven, R. (2005). Measuring inequality of happiness in nations: In search for proper statistics. Journal of Happiness Studies,6, 357–396.

  35. Kendall, M. G., & Smith, B. B. (1939). The problem of m rankings. The Annals of Mathematical Statistics,10, 275–287.

  36. Kovačević, M. S., & Binder, D. A. (1997). Variance estimation for measures of income inequality and polarization—the estimating equations approach. Journal of Official Statistics,13(1), 41–58.

  37. Lambert, P. J. (2001). The distribution and redistribution of income (3rd ed.). Manchester: Manchester University Press.

  38. Legendre, P. (2005). Species associations: The Kendall coefficient of concordance revisited. Journal of Agricultural, Biological, and Environmental Statistics,10(2), 226–245.

  39. Leik, R. K. (1966). A measure of ordinal consensus. Pacific Sociological Review,9(2), 85–90.

  40. Lim, T. S., & Loh, W. Y. (1996). A comparison of tests of equality of variances. Computational Statistics & Data Analysis,22, 287–301.

  41. Linacre, J. M. (1999). Investigating rating scale category utility. Journal of Outcome Measurement,3(2), 103–122.

  42. Lumley, T. (2010). Complex surveys: A guide to analysis using R. Hoboken: Wiley.

  43. Lyubomirsky, S., & Lepper, H. L. (1999). A measure of subjective happiness: Preliminary reliability and construct validation. Social Indicators Research,46(2), 137–155.

  44. Marozzi, M. (2011). Levene type tests for the ratio of two scales. Journal of Statistical Computation and Simulation,81(7), 815–826.

  45. Marozzi, M. (2014). Testing for concordance between several criteria. Journal of Statistical Computation and Simulation,84(9), 1843–1850.

  46. Marozzi, M. (2015). Measuring trust in European public institutions. Social Indicators Research,123(3), 879–895.

  47. Marozzi, M. (2016). Construction, robustness assessment and application of an index of perceived level of socio-economic threat from immigrants: A study of 47 European countries and regions. Social Indicators Research,128(1), 413–437.

  48. Mills, A. M., & Zandvakili, S. (1997). Statistical inference via bootstrapping for measures of inequality. Journal of Applied Econometrics,12, 133–150.

  49. NEF. (2006). The happy planet index. An index of human well-being and environmental impact. Accessed January 2019.

  50. Norman, G. (2010). Likert scales, levels of measurement, and the “laws” of statistics. Advances in Health Sciences Education,15(5), 625–632.

  51. OECD. (2011). How’s life? Measuring well-being. Paris: OECD. Accessed January 2019.

  52. Ruedin, D. (2016). Agrmt: Calculate agreement or consensus in ordered rating scales. Accessed January 2019.

  53. Salverda, W., & Checchi, D. (2015). Labor market institutions and the dispersion of wage earnings. In A. B. Atkinson & F. Bourguignon (Eds.), Handbook of income distribution (Vol. 2A, pp. 1535–1728). Amsterdam: Elsevier.

  54. Schoder, J. (2014). Inequality with ordinal data. cross-disciplinary review of methodologies and application to life satisfaction in Europe. University of Salzburg, working paper, 12/2014, No. 5. Accessed January 2019.

  55. Shorrocks, A. F. (1980). The class of additively decomposable inequality measures. Econometrica,48(3), 613–625.

  56. Stevens, S. S. (1946). On the theory of scales of measurement. Science,103(2684), 677–680.

  57. Stiglitz, J. E., Sen, A., & Fitoussi, J.-P. (2009). Report by the commission on the measurement of economic performance and social progress. Accessed January 2019.

  58. Tastle, W. J., & Wierman, M. J. (2006). An information theoretic measure for the evaluation of ordinal scale data. Behavior Research Methods,38(3), 487–494.

  59. UN. (2015). Sustainable development goals. 17 Goals to transform our world. Accessed Jan 2019.

  60. van Beuningen, J., Jol, C., & Moonen, L. (2015). De persoonlijke welzijnsindex. De ontwikkeling van een index voor subjectief welzijn [The personal well-being index. The development of an index for subjective well-being]. Heerlen: Statistics Netherlands. Accessed Jan 2019.

  61. van Beuningen, J., & Schmeets, H. (2013). Developing a social capital index for the Netherlands. Social Indicators Research,113, 859–886.

  62. Van Der Eijk, C. (2001). Measuring agreement in ordered rating scales. Quality and Quantity,35, 325–341.

  63. van Ruth, F., Schouten, B., & Wekker, R. (2005). The Statistics Netherlands’ business cycle tracer. Methodological aspects; concept, cycle computation and indicator selection. Heerlen: Statistics Netherlands. Accessed January 2019.

  64. Veenhoven, R. (2005a). Inequality of happiness in nations. Journal of Happiness Studies,6, 351–355.

  65. Veenhoven, R. (2005b). Return of inequality in modern society. Journal of Happiness Studies,6, 457–487.

  66. Veenhoven, R. (2017). World database of happiness. Rotterdam: Erasmus University Rotterdam. Accessed January 2019.

  67. Velleman, P. F., & Wilkinson, L. (1993). Nominal, ordinal, interval, and ratio typologies are misleading. The American Statistician,47(1), 65–72.

  68. Western, B., & Bloome, D. (2009). Variance function regressions for studying inequality. Sociological Methodology,39, 293–326.

  69. Wolfson, M. C. (1997). Divergent inequalities: Theory and empirical results. Review of Income and Wealth,43(4), 401–421.

  70. World Economic Forum. (2017). The inclusive growth and development report 2017. Accessed January 2019.

  71. Yitzhaki, S. (1983). On an extension of the Gini inequality Index. International Economic Review,24(3), 617–628.

  72. Yitzhaki, S. (2013). More than a dozen alternative ways of spelling Gini. In S. Yitzhaki & E. Schechtman (Eds.), The Gini methodology (pp. 11–31). New York: Springer.

Download references


We thank Kees Zeelenberg and Linda Moonen for discussion and useful comments on an earlier version of this paper. We also thank the anonymous reviewers for their helpful comments. The views expressed in this paper are those of the authors and do not necessarily reflect the policies of Statistics Netherlands.

Author information

Correspondence to Jacqueline van Beuningen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.


Appendix 1 dispersion measures defined

The dispersion measures are defined below (Table 2). Let \(Y\) be a discrete random variable with possible values \(\left\{ {1, \ldots ,K} \right\}\), probability mass \(p_{k} = P\left( {Y = k} \right)\), \(\hat{p}_{k} = \frac{{\hat{N}_{k} }}{N}\) and cumulative distribution function \(F_{k} = P\left( {Y \le k} \right) = \sum\nolimits_{l = 1}^{k} {p_{l} }\), where \(N_{k}\) is the number of population units in category \(k\) and \(N = \sum\nolimits_{k = 1}^{K} {N_{k} }\) the total population size. For instance, \(Y\) is subjective well-being on a ten point scale with possible scores \(\left\{ {1, \ldots ,K = 10} \right\}\). In a sample survey, the distribution can be estimated using \(\hat{N}_{k} = \sum\nolimits_{i = 1}^{{r_{k} }} {w_{i} }\), where \(w_{i}\) is the calibration weight of respondent \(i\) and \(r_{k}\) the number of respondents in category \(y_{i} = k\). In a homogeneous population or extreme unimodal distribution (no dispersion or a maximally concentrated distribution in which case everyone has the same score), \(p_{k} = 1\) for one category and 0 for the other categories; in a uniform distribution (maximum dispersion or the scores are equally distributed across answer categories), \(p_{k} = \frac{1}{K}\) for all categories; in an extreme bimodal distribution (maximum dispersion with scores on the two extreme categories), \(p_{1} = p_{K} = 0.5\) and 0 for the other categories.

Table 2 Measures describing the distribution of a variable measured on a discrete rating scale

Appendix 2 convergence

See Fig. 7.

Fig. 7

Effect of number of bootstraps on a the mean and b the standard error of the estimated happiness dispersion according to IOV. Red line shows the total sample, gray lines show different domains. Happiness data from the Social Cohesion and Well-Being Survey 2016. (Color figure online)

Appendix 3 performance of bootstrapped confidence intervals

The interpretation of a frequentist 95% confidence interval is that if we would repeat the sample survey an infinite number of times, 95% of the confidence intervals would contain the true population parameter. To verify whether this holds for our bootstrapped approximation of the confidence interval, a simulation study was conducted. An artificial population of about 13 million persons was created from the Social Cohesion and Well-Being Survey, using the rounded calibration weight of each 18 + respondent as the number of population units it represents. Happiness dispersion in this population was taken as the true population value. From the artificial population, we drew 1000 simple random samples of a size equal to the number of respondents in the survey (about 7 thousand). For each sample, happiness dispersion was estimated, including the 95% confidence interval estimated from 1000 bootstrap resamples. For each sample we checked whether the 95% bootstrapped confidence interval contained the true population value. The proportion of intervals containing the true population value is shown in Fig. 8. The simulation was performed independently for 4 years. Six measures were used to quantify happiness dispersion. Happiness dispersion was estimated for the total population and for two age classes. See e.g. Berger and Muñoz (2015) for a more elaborate simulation study on coverage rates of a bootstrap variance estimator.

Fig. 8

Performance of bootstrapped 95% confidence intervals of estimated happiness dispersion in the Social Cohesion and Well-Being Surveys 2013–2016. Proportion of 1000 intervals containing the true population value. Happiness dispersion was quantified according to six measures (panels) and estimated for the total population and two age classes (legend). The gray horizontal line is the 0.95 reference. (Color figure online)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Burger, J., van Beuningen, J. Measuring well-being dispersion on discrete rating scales. Soc Indic Res (2020).

Download citation


  • Beyond GDP
  • Income inequality axioms
  • Ordinal rating scale
  • Probability mass distribution
  • Bootstrap