Survey mode effects on measured income inequality

Abstract

We study the effect of interview modes on estimates of economic inequality which are based on survey data. We exploit variation in interview modes in the Austrian EU-SILC panel, where between 2007 and 2008 the interview mode was switched from personal interviews to telephone interviews for some but not all participants. We combine methods from the program evaluation literature with methods from the distributional decomposition literature to obtain causal estimates of the effect of interview mode on estimated inequality. We find that the interview mode has a large effect on estimated inequality, where telephone interviews lead to a larger downward bias. The effect of the mode is much smaller for robust inequality measures such as interquantile ranges, as these are not sensitive to the tails of the distribution. The magnitude of effects we find are of a similar order as the differences in many international and intertemporal comparisons of inequality.

This is a preview of subscription content, log in to check access.

References

  1. Angrist, J., Pischke, J.-S.: Mostly Harmless Econometrics. Princeton University Press, Princeton (2008)

    Google Scholar 

  2. Atkeson, L.R., Adams, A.N., Alvarez, R.M.: Nonresponse and Mode Effects in Self- and Interviewer-Administered Surveys. Political Analysis (2014)

  3. Cowell, F.A., Victoria-Feser, M.-P.: Robustness properties of inequality measures. Econometrica 64(1), 77–101 (1996)

    Article  Google Scholar 

  4. de Leeuw, E.D.: Data Quality in Mail, Telephone and Face to Face Surveys. TT-Publikaties, Amsterdam (1992)

    Google Scholar 

  5. Dillman, D.A., Smyth, J.D., Christian, L.M.: Internet, Phone, Mail, and Mixed-Mode Surveys: the Tailored Design Method. Wiley, New York (2014)

    Google Scholar 

  6. DiNardo, J., Fortin, N., Lemieux, T.: Labor market institutions and the distribution of wages, 1973-1992: A semiparametric approach. Econometrica 64, 1001–1044 (1996)

    Article  Google Scholar 

  7. Firpo, S., Fortin, N.M., Lemieux, T.: Unconditional quantile regression. Econometrica 77(3), 953–973 (2009)

    Article  Google Scholar 

  8. Huber, P.J.: Robust Statistics. Wiley Series in Probability and Statistics. Wiley, New York (2003)

    Google Scholar 

  9. Iacus, S.M., King, G., Porro, G.: Matching for Causal Inference without Balance Checking. Working paper series, Harvard (2008)

    Google Scholar 

  10. Imbens, G.W.: Nonparametric estimation of average treatment effects under exogeneity: a review. Rev. Econ. Stat. 86(1), 4–29 (2004)

    Article  Google Scholar 

  11. Imbens, G.W., Rubin, D.B.: Causal Inference fo Statistics, Social and Biomedical Sciences an Introduction. Cambridge University Press, Cambridge (2015)

    Google Scholar 

  12. Jäckle, A., Roberts, C., Lynn, P.: Assessing the effect of data collection mode on measurement. Int. Stat. Rev. 78(1), 3–20 (2010)

    Article  Google Scholar 

  13. Klausch, T., Hox, J.J., Schouten, B.: Assessing the Mode-Dependency of Sample Selectivity Across the Survey Response Process. Technical Report 2013-03, Statistics Netherlands (2013a)

  14. Klausch, T., Hox, J.J., Schouten, B.: Measurement effects of survey mode on the equivalence of attitudinal rating scale questions. Sociol. Methods Res. 42(3), 227–263 (2013b)

    Article  Google Scholar 

  15. Klausch, T., Hox, J.J., Schouten, B.: Evaluating bias of sequential mixed-mode designs against benchmark surveys. Sociol. Methods Res. 46(3), 456–489 (2015a)

    Article  Google Scholar 

  16. Klausch, T., Hox, J.J., Schouten, B.: Selection error in single- and mixed mode surveys of the Dutch general population. J. R. Stat. Soc.: Ser. A (Statistics in Society) 178(4), 945–961 (2015b)

    Article  Google Scholar 

  17. Lohmann, H.: Comparability of eu-silc survey and register data: the relationship among employment, earnings and poverty. J. Eur. Social Policy 21(1), 1–18 (2011)

    Article  Google Scholar 

  18. OECD: Divided We Stand - Why Inequality Keeps Rising (2011)

  19. Rosenbaum, P.R., Rubin, D.B.: The central role of the propensity in observational studies for causal effects. Biometrika 70(1), 41–55 (1983)

    Article  Google Scholar 

  20. Schouten, B., van den Brakel, J., Buelens, B., van der Laan, J., Klausch, T.: Disentangling mode-specific selection and measurement bias in social surveys. Soc. Sci. Res. 42, 1555–1570 (2013)

    Article  Google Scholar 

  21. Statistik Austria: Standard Dokumentation Metainformationen - Definitionen, Erlaeuterungen, Methoden, Qualitaet (2010)

  22. Vannieuwenhuyze, J.T., Loosveldt, G.: Evaluating relative mode effects in mixed-mode surveys: three methods to disentangle selection and measurement effects. Sociol. Methods Res. 42(1), 82–104 (2013)

    Article  Google Scholar 

  23. Vannieuwenhuyze, J.T., Loosveldt, G., Molenberghs, G.: Evaluating mode effects in mixed-mode survey data using covariate adjustment models. J. Off. Stat. 30 (1), 1–21 (2014)

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank Markus Knell and Alyssa Schneebaum for valuable comments and discussion. Additional to the usual disclaimer, the opinions expressed in this work are those of the authors and do not necessarily reflect the ones of the OeNB or the Eurosystem.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Pirmin Fessler.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(PDF 300 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Fessler, P., Kasy, M. & Lindner, P. Survey mode effects on measured income inequality. J Econ Inequal 16, 487–505 (2018). https://doi.org/10.1007/s10888-018-9378-x

Download citation

Keywords

  • Income inequality
  • Survey methodology
  • Survey modes
  • Distributional decompositions