Social Indicators Research

, Volume 110, Issue 1, pp 89–110 | Cite as

How much Confidence can we have in EU-SILC? Complex Sample Designs and the Standard Error of the Europe 2020 Poverty Indicators

  • Tim GoedeméEmail author


If estimates are based on samples, they should be accompanied by appropriate standard errors and confidence intervals. This is true for scientific research in general, and is even more important if estimates are used to inform and evaluate policy measures such as those aimed at attaining the Europe 2020 poverty reduction target. In this article I pay explicit attention to the calculation of standard errors and confidence intervals, with an application to the European Union Statistics on Income and Living Conditions (EU-SILC). The estimation of accurate standard errors requires among others good documentation and proper sample design variables in the dataset. However, this information is not always available. Therefore, I complement the existing documentation on the sample design of EU-SILC and test the effect on estimated standard errors of various simplifying assumptions with regard to the sample design. It is shown that accounting for clustering within households is of paramount importance. Although this results in many cases in a good approximation of the standard error, taking as much as possible account of the entire sample design generally leads to more accurate estimates, even if sample design variables are partially lacking. The effect is illustrated for the official Europe 2020 indicators of poverty and social exclusion and for all European countries included in the EU-SILC 2008 dataset. The findings are not only relevant for EU-SILC users, but also for users of other surveys on income and living conditions which lack accurate sample design variables.


Europe 2020 poverty reduction target Complex sample design Incomplete sample design variables Standard error Confidence interval EU-SILC Clustering within households Variance estimation 



This article has benefited from comments by Karel Van den Bosch, Guillaume Osier, Ulrich Kohler, Lina Salanauskaite, Vincent Corluy, Joris Ghysels, Rudi Van Dam, Koen Decancq and an anonymous referee. I am grateful to Fabienne Montaigne and Pascal Wolff for offering me the opportunity to run my Stata programmes on the EU-SILC data available to Eurostat. The paper has been presented at the Final Equalsoc Conference, June 2010 in Amsterdam, the EU-LFS and EU-SILC 2nd European User Conference, March 2011 in Mannheim as well as the 4th Conference of the European Survey Research Association, July 2011 in Lausanne. Comments and suggestions from the participants are gratefully acknowledged. This research has been funded by the Research Foundation—Flanders (FWO). All opinions expressed in this paper as well as any remaining errors and shortcomings are my own. Findings, opinions and suggestions presented in this article do not necessarily reflect those of Eurostat.


  1. Alfons, A., Temple, M., & Filzmoser, P. (2009). On the influence of imputation methods on Laeken indicators: Simulations and recommendations. In Conference of European statisticians, Neuchatel, Switzerland, October 57, 2009 (pp. 9).Google Scholar
  2. Araar, A., & Duclos, J.-Y. (2007). DASP: Distributive analysis stata package. PEP, CIRPÉE and World Bank, Université Laval.Google Scholar
  3. Araar, A., & Duclos, J.-Y. (2009). DAD: A software for poverty and distributive analysis. Journal of Economic and Social Measurement, 34(2/3), 175–189.Google Scholar
  4. Atkinson, A. B., Cantillon, B., Marlier, E., & Nolan, B. (2002). Social indicators: The EU and social inclusion. Oxford: Oxford University Press.Google Scholar
  5. Berger, Y. G., & Skinner, C. J. (2003). Variance estimation for a low income proportion. Journal of the Royal Statistical Society. Series C (Applied Statistics), 52(4), 457–468.CrossRefGoogle Scholar
  6. Biewen, M. (2002). Bootstrap inference for inequality, mobility and poverty measurement. Journal of Econometrics, 108(2), 317–342.CrossRefGoogle Scholar
  7. Biewen, M., & Jenkins, S. P. (2006). Variance estimation for generalized entropy and Atkinson inequality indices: The complex survey data case. Oxford Bulletin of Economics and Statistics, 68(3), 371–383.CrossRefGoogle Scholar
  8. Cochran, W. G. (1977). Sampling techniques. New York: Wiley.Google Scholar
  9. Davidson, R., & Duclos, J.-Y. (2000). Statistical inference for stochastic dominance and for the measurement of poverty and inequality. Econometrica, 68(6), 1435–1464.CrossRefGoogle Scholar
  10. Davidson, R., & Flachaire, E. (2007). Asymptotic and bootstrap inference for inequality and poverty measures. Journal of Econometrics, 141(1), 141–166.CrossRefGoogle Scholar
  11. de Vos, K., & Zaidi, A. M. (1998). Poverty measurement in the European Union: Country-specific or union-wide poverty lines? Journal of Income Distribution, 8(1), 77–92.CrossRefGoogle Scholar
  12. del Mar Rueda, M., & Muñoz, J. F. (2011). Estimation of poverty measures with auxiliary information in sample surveys. Quality and Quantity, 45(3), 687–700.CrossRefGoogle Scholar
  13. Dewilde, C. (2004). The multidimensional measurement of poverty in Belgium and Britain: A categorical approach. Social Indicators Research, 68(3), 331–369.CrossRefGoogle Scholar
  14. Dewilde, C. (2008). Individual and institutional determinants of multidimensional poverty: A European comparison. Social Indicators Research, 86(2), 233–256.CrossRefGoogle Scholar
  15. Duclos, J.-Y., & Araar, A. (2006). Poverty and equity. Measurement, policy, and estimation with DAD. New York: Springer.Google Scholar
  16. Efron, B., & Tibshirani, R. J. (1998). An introduction to the bootstrap. Boca Raton: Chapman & Hall/CRC.Google Scholar
  17. European Commission. (2006). Portfolio of overarching indicators and streamlined social inclusion, pensions, and health portfolios. Brussels: European Commission.Google Scholar
  18. European Council. (2010). European Council 17 June 2010 conclusions. Brussels: European Council.Google Scholar
  19. Eurostat. (2002). Monographs of official statistics. Variance estimation methods in the European Union. Luxembourg: Office for Official Publications of the European Communities.Google Scholar
  20. Eurostat. (2010a). 2008 comparative EU intermediate quality report. Version 2June 2010. Luxembourg: Eurostat.Google Scholar
  21. Eurostat. (2010b). Combating poverty and social exclusion. A statistical portrait of the European Union 2010. Luxembourg: Publications Office of the European Union.Google Scholar
  22. Eurostat. (2010c). Description of target variables: Cross-sectional and longitudinal 2008 operation (Version January 2010). Brussels: European Commission.Google Scholar
  23. Foster, J., Greer, J., & Thorbecke, E. (1984). A class of decomposable poverty measures. Econometrica, 52(3), 761–766.CrossRefGoogle Scholar
  24. Frazer, H., Marlier, E., Natali, D., Van Dam, R., & Vanhercke, B. (2010). Europe 2020: Towards a more social EU? In E. Marlier, D. Natali, & R. Van Dam (Eds.), Europe 2020. Towards a more social EU? (pp. 15–44). Brussels: P.I.E. Peter Lang.Google Scholar
  25. Goedemé, T. (2010a). The construction and use of sample design variables in EU-SILC. A user’s perspective. Report prepared for Eurostat, Antwerp: Herman Deleeck Centre for Social Policy, University of Antwerp.Google Scholar
  26. Goedemé, T. (2010b). The standard error of estimates based on EU-SILC. An exploration through the Europe 2020 poverty indicators. CSB Working Paper Series, WP 10/09 (p. 36). Antwerp: Herman Deleeck Centre for Social Policy, University of Antwerp.Google Scholar
  27. Goedemé, T., & Rottiers, S. (2011). Poverty in the Enlarged European Union. A discussion about definitions and reference groups. Sociology Compass, 5(1), 77–91. doi: 10.1111/j.1751-9020.2010.00350.x.
  28. Groves, R. M., Fowler, F. J. J., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2009). Survey methodology (2nd ed.). New Jersey: Wiley.Google Scholar
  29. Guio, A.-C. (2009). What can be learned from deprivation indicators in Europe? In Paper presented at the indicator subgroup of the Social Protection Committee, February 10, 2009 (p. 33).Google Scholar
  30. Heeringa, S. G., West, B. T., & Berglund, P. A. (2010). Applied survey data analysis. Boca Raton: Chapman & Hall/CRC.CrossRefGoogle Scholar
  31. Howes, S., & Lanjouw, J. O. (1998). Does sample design matter for poverty rate comparisons? Review of Income and Wealth, 44(1), 99–109.CrossRefGoogle Scholar
  32. Jolliffe, D., Datt, G., & Sharma, M. (2004). Robust poverty and inequality measurement in Egypt: Correcting for spatial-price variation and sample design effects. Review of Development Economics, 8(4), 557–572.CrossRefGoogle Scholar
  33. Jolliffe, D., & Semykina, A. (1999). sg117—robust standard errors for the Foster–Greer–Thorbecke class of poverty indices. Stata Technical Bulletin, STB, 51, 34–36.Google Scholar
  34. Kakwani, N. (1993). Statistical inference in the measurement of poverty. The Review of Economics and Statistics, 75(4), 632–639.CrossRefGoogle Scholar
  35. Kalton, G. (1983). Introduction to survey sampling (Vol. 35, quantitative applications in the social sciences). Beverly Hills: Sage Publications.Google Scholar
  36. Kangas, O., & Ritakallio, V.-M. (2007). Relative to what? Cross national pictures of European poverty measured by regional, national and European standards. European Societies, 9(2), 119–145.CrossRefGoogle Scholar
  37. Kish, L. (1965). Survey sampling. New York: Wiley.Google Scholar
  38. Kolenikov, S. (2010). Resampling variance estimation for complex survey data. Stata Journal, 10(2), 165–199.Google Scholar
  39. Lee, E. S., & Forthofer, R. N. (2006). Analyzing complex survey data. Second edition (Vol. 71, quantitative applications in the social sciences). Thousand Oaks: Sage Publications.Google Scholar
  40. Lohmann, H. (2011). Comparability of EU-SILC survey and register data: The relationship among employment, earnings and poverty. Journal of European Social Policy, 21(1), 37–54. doi: 10.1177/0958928710385734.Google Scholar
  41. Marlier, E., Atkinson, A. B., Cantillon, B., & Nolan, B. (2007). The EU and social inclusion. Facing the challenges. Bristol: The Policy Press.Google Scholar
  42. Mooney, C. Z., & Duval, R. D. (1993). Bootstrapping: A nonparametric approach to statistical inference (Vol. 95, quantitative applications in the social sciences). Newbury Park: Sage Publications.Google Scholar
  43. OECD. (2008). Growing unequal? Income distribution and poverty in OECD countries. Paris: OECD.Google Scholar
  44. Osier, G. (2009). Variance estimation for complex indicators of poverty and inequality using linearization techniques. Survey Research Methods, 3(3), 167–195.Google Scholar
  45. Preston, I. (1995). Sampling distributions of relative poverty statistics. Journal of the Royal Statistical Society. Series C (Applied Statistics), 44(1), 91–99.Google Scholar
  46. Rodgers, J. R., & Rodgers, J. L. (1993). Chronic poverty in the United States. The Journal of Human Resources, 28(1), 25–54.CrossRefGoogle Scholar
  47. Shao, J. (1996). Invited discussion paper resampling methods in sample surveys. Statistics: A Journal of Theoretical and Applied Statistics, 27(3), 203–237.Google Scholar
  48. Shao, J., & Chen, Y. (1998). Bootstrapping sample quantiles based on complex survey data under hot deck imputation. Statistica Sinica, 8(4), 1071–1085.Google Scholar
  49. Sturgis, P. (2004). Analysing complex survey data: Clustering, stratification and weights. Social Research Update, 43, 1–6.Google Scholar
  50. Thuysbaert, B. (2008). Inference for the measurement of poverty in the presence of a stochastic weighting variable. Journal of Economic Inequality, 6(1), 33–55.CrossRefGoogle Scholar
  51. Trede, M. (2002). Bootstrapping inequality measures under the null hypothesis: Is it worth the effort? Journal of Economics, 77(Supplement 1), 261–282.CrossRefGoogle Scholar
  52. Van Kerm, P. (2002). Inference on inequality measures: A Monte Carlo experiment. Journal of Economics, 77(Supplement 1), 283–306.CrossRefGoogle Scholar
  53. Van Kerm, P. (2007). Extreme incomes and the estimation of poverty and inequality indicators from EU-SILC. IRISS Working Paper Series (p. 51). Luxembourg: CEPS-Instead.Google Scholar
  54. Verma, V., Betti, G., & Gagliardi, F. (2010). An assessment of survey errors in EU-SILC. Eurostat Methodologies and Working Papers (p. 70). Luxembourg: Eurostat.Google Scholar
  55. Whelan, C. T., & Maître, B. (2007). Income, deprivation and economic stress in the enlarged European Union. Social Indicators Research, 83(2), 309–329.CrossRefGoogle Scholar
  56. Wolff, P. (2010). 17% of EU citizens were at-risk-of-poverty in 2008. Statistics in focus (p. 8). Luxembourg: Eurostat.Google Scholar
  57. Wolter, K. M. (2007). Introduction to variance estimation. New York: Springer.Google Scholar
  58. Zheng, B. (2001). Statistical inference for poverty measures with relative poverty lines. Journal of Econometrics, 101(2), 337–356.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2011

Authors and Affiliations

  1. 1.Herman Deleeck Centre for Social PolicyUniversity of AntwerpAntwerpBelgium
  2. 2.Research Foundation—FlandersBrusselBelgium

Personalised recommendations