Quality & Quantity

, Volume 53, Issue 1, pp 471–491 | Cite as

Comparing survey data and administrative records on gross earnings: nonreporting, misreporting, interviewer presence and earnings inequality

  • Peter ValetEmail author
  • Jule Adriaans
  • Stefan Liebig


Research on earnings inequality mostly relies on survey data, but these data may not be accurate. Survey data on earnings might be biased as research indicates that some respondents are likely to avoid reporting their gross earnings and others are likely to misreport them. In addition, the mode of data collection might affect responses to sensitive questions such as those on earnings. Given these three possibilities for bias, researchers’ conclusions on the degree of earnings inequality might be systematically biased as well. By comparing survey and linked administrative data, we looked for the nonreporting and misreporting biases suggested by the literature, investigated the presence of an interviewer as another source of non- and misreporting, and compared how nonreporting, misreporting, and the mode of data collection affected conclusions on earnings inequality. The analyses drew on a German employee survey and linked administrative data from the Federal Employment Agency. Using the administrative data as a benchmark, we found that respondents at the lower and upper end of the earnings distribution were more likely to not report and to misreport their earnings. Interviewer presence led to higher nonreporting but had no effect on misreporting. All these processes and especially nonreporting and interviewer presence led to an underestimation of earnings inequality based on survey data. We relate the relevance of these results to research on inequality and survey methodology and conclude that linking survey data to administrative records could be an avenue for safeguarding conclusions on earnings inequality.


Gross earnings Nonreporting Misreporting Survey mode Earnings inequality Validation study 


  1. AAPOR: Standard definitions: final dispositions of case codes and outcome rates for surveys. The American Association for Public Opinion Research (2016)Google Scholar
  2. Angel, S., Heuberger, R., Lamei, N.: Differences between household income from surveys and registers and how these affect the poverty headcount: evidence from the Austrian SILC. Soc. Indic. Res. (2017). Google Scholar
  3. Arulampalam, W., Booth, A.L., Bryan, M.L.: Is there a glass ceiling over Europe? Exploring the gender pay gap across the wage distribution. Ind. Labor Relat. Rev. 60, 163–186 (2007). CrossRefGoogle Scholar
  4. Atkinson, A.B., Piketty, T., Saez, E.: Top incomes in the long run of history. J. Econ. Lit. 49, 3–71 (2011). CrossRefGoogle Scholar
  5. Bartels, C., Metzing, M.: An integrated approach for top-corrected ginis. IZA Discuss. Pap. 10573 (2017)Google Scholar
  6. Bollinger, C.R., Hirsch, B.T.: Is earnings nonresponse ignorable? Rev. Econ. Stat. 95, 407–416 (2013). CrossRefGoogle Scholar
  7. Bollinger, C.R., Hirsch, B.T., Hokayem, C.M., Ziliak, J.P.: Trouble in the tails? What we know about earnings nonresponse thirty years after. Univ. Kentucky Cent. Poverty Res. Discuss. Pap. Ser. 2 (2015)Google Scholar
  8. Bound, J., Brown, C., Mathiowetz, N.: Measurement error in survey data. In: Heckman, J.J., Leamer, E. (eds.) Handbook of Econometrics, pp. 3705–3843. North-Holland, Amsterdam (2001)CrossRefGoogle Scholar
  9. Bound, J., Krueger, A.B.: The extent of measurement error in longitudinal earnings data: Do two wrongs make a right? J. Labor Econ. 9, 1–24 (1991)CrossRefGoogle Scholar
  10. Bricker, J., Engelhardt, G.V.: Measurement error in earnings data in the health and retirement study. J. Econ. Soc. Meas. 33, 39–61 (2008)Google Scholar
  11. Burauel, P., Caliendo, M., Fedorets, A., Grabka, M.M., Schröder, C., Schupp, J., Wittbrodt, L.: Minimum wage not yet for everyone: on the compensation of eligible workers before and after the minimum wage reform from the perspective of employees. DIW Econ. Bull. 49, 509–522 (2017)Google Scholar
  12. Connelly, R., Playford, C.J., Gayle, V., Dibben, C.: The role of administrative data in the big data revolution in social science research. Soc. Sci. Res. 59, 1–12 (2016). CrossRefGoogle Scholar
  13. De Leeuw, E.: To mix or not to mix data collection modes in surveys. J. Off. Stat. 21, 233–255 (2005). Google Scholar
  14. De Maio, F.G.: Income inequality measures. J. Epidemiol. Community Health 61, 849–852 (2007). CrossRefGoogle Scholar
  15. Ferrer-i-Carbonell, A.: Income and well-being: an empirical analysis of the comparison income effect. J. Public Econ. 89, 997–1019 (2005). CrossRefGoogle Scholar
  16. Frick, J.R., Grabka, M.M.: Missing income data in panel surveys: incidence, imputation and its impact on the income distribution. SOEP Surv. Pap. 225, (2004)Google Scholar
  17. Frick, J.R., Grabka, M.M.: Item non-response and imputation of annual labor income in panel surveys from a cross-national perspective. In: Harkness, J.A., Edwards, B., Braun, M., Johnson, T.P., Lyberg, L.E., Mohler, P.P., Pennell, B.-E., Smith, T. (eds.) Survey methods in multicultural, multinational, and multiregional contexts, pp. 355–374. Wiley & Sons, New York (2010)CrossRefGoogle Scholar
  18. Goerge, R.M., Lee, B.J.: Matching and cleaning administrative data. In: Ver Ploeg, M., Moffitt, R.A., Citro, C.F. (eds.) Studies of Welfare Populations: Data Collection and Research Issues, pp. 197–219. National Research Council, Washington (2001)Google Scholar
  19. Gottschalk, P., Huynh, M.: Validation study of earnings data in the SIPP—Do older workers have larger measurement error? Cent. Retire. Res. Work. Pap. 2005–7 (2005)Google Scholar
  20. Gottschalk, P., Huynh, M.: Are earnings inequality and mobility overstated? The impact of nonclassical measurement error. Rev. Econ. Stat. 92, 302–315 (2010). CrossRefGoogle Scholar
  21. Groves, R.M., Dillman, D.A., Eltinge, J.L., Little, J.A.L.: Survey nonresponse. Wiley & Sons, New York (2002)Google Scholar
  22. Groves, R.M., Peytcheva, E.: The impact of nonresponse rates on nonresponse bias: a meta-analysis. Public Opin. Q. 72, 167–189 (2008). CrossRefGoogle Scholar
  23. Hawkes, D., Plewis, I.: Modelling non-response in the National Child Development Study. J. R. Stat. Soc. Ser. A Stat. Soc. 169, 479–491 (2006). CrossRefGoogle Scholar
  24. Heerwegh, D.: Mode differences between face-to-face and web surveys: an experimental investigation of data quality and social desirability effects. Int. J. Public Opin. Res. 21, 111–121 (2009). CrossRefGoogle Scholar
  25. Hokayem, C., Bollinger, C., Ziliak, J.P.: The role of CPS nonresponse in the measurement of poverty. J. Am. Stat. Assoc. 110, 935–945 (2015). CrossRefGoogle Scholar
  26. Holbrook, A.L., et al.: Telephone versus Face-to-Face interviewing of national probability samples with long questionaires. Comparison of respondent satisfacing and social desirability response bias. Public Opin. Q. 67, 79–125 (2003). CrossRefGoogle Scholar
  27. Jacobebbinghaus, P., Seth, S.: The german integrated employment biographies sample IEBS. Schmollers Jahrb. 127, 335–342 (2007)Google Scholar
  28. Jenkins, S.P.: Pareto models, top incomes and recent trends in UK income inequality. Economica 84, 261–289 (2017). CrossRefGoogle Scholar
  29. Kapteyn, A., Ypma, J.Y.: Measurement error and misclassification: a comparison of survey and administrative data. J. Labor Econ. 25, 513–551 (2007). CrossRefGoogle Scholar
  30. Kim, C., Tamborini, C.R.: Response error in earnings: an analysis of the survey of income and program participation matched with administrative data. Sociol. Methods Res. 43, 39–72 (2014). CrossRefGoogle Scholar
  31. Krumpal, I.: Determinants of social desirability bias in sensitive surveys: a literature review. Qual. Quant. 47, 2025–2047 (2013). CrossRefGoogle Scholar
  32. Laurie, H.: Minimizing panel attrition. In: Menard, S. (ed.) Handbook of Longitudinal Research: Design, Measurement, and Analysis, pp. 167–184. Elsevier, Amsterdam (2008)Google Scholar
  33. Liebig, S., May, M., Sauer, C., Schneider, S., Valet, P.: How much inequality of earnings do people perceive as just? The effect of interviewer presence and monetary incentives on inequality preferences. Methods Data Anal. 9, 57–86 (2015). Google Scholar
  34. Lillard, L., Smith, J.P., Welch, F.: What do we really know about wages? The importance of nonreporting and census imputation. J. Polit. Econ. 94, 489–506 (1986). CrossRefGoogle Scholar
  35. Mandel, H., Semyonov, M.: Gender pay gap and employment sector: sources of earnings disparities in the United States, 1970–2010. Demography 51, 1597–1618 (2014). CrossRefGoogle Scholar
  36. Moore, J.C., Stinson, L.L., Welniak, E.J.J.: Income measurement error in surveys: a review. J. Off. Stat. 16, 31–361 (2000)Google Scholar
  37. Mouw, T., Kalleberg, A.L.: Occupations and the structure of wage inequality in the United States, 1980s to 2000s. Am. Sociol. Rev. 75, 402–431 (2010). CrossRefGoogle Scholar
  38. Pedace, R., Bates, N.: Using administrative records to assess earnings reporting error in the survey of income and program participation. J. Econ. Soc. Meas. 26, 173–192 (2000)Google Scholar
  39. Reinecke, J.: Interviewer- und Befragtenverhalten. Theoretische Ansätze und methodische Konzepte. Westdeutscher Verlag, Opladen (1991)Google Scholar
  40. Riphahn, R.T., Serfling, O.: Item non-response on income and wealth questions. Empir. Econ. 30, 521–538 (2005). CrossRefGoogle Scholar
  41. Rubin, D.B.: Multiple imputation for nonresponse in surveys. Wiley & Sons, New York (1987)CrossRefGoogle Scholar
  42. Sakshaug, J.W., Hülle, S., Schmucker, A., Liebig, S.: Exploring the effects of interviewer- and self-administered survey modes on record linkage consent rates and bias. Surv. Res. Methods 11, 171–188 (2017). Google Scholar
  43. Sauer, C., Valet, P., Meyer, L.: Expectations towards economy and society. Codebook of the employee survey. SFB Tech. Rep. Ser. DFG Res. Cent. 882 From Heterog. to Inequalities. 11 (2014)Google Scholar
  44. Schneider, S.M., Valet, P.: Relative standards and distributive justice: how social comparison orientations moderate the link between relative earnings and justice perceptions. Soc. Psychol. Q. 80, 276–287 (2017). CrossRefGoogle Scholar
  45. Schräpler, J.-P.: Respondent behavior in panel studies: a case study for income nonresponse by means of the German Socio-Economic Panel (SOEP). Sociol. Methods Res. 33, 118–156 (2004). CrossRefGoogle Scholar
  46. Spiess, M., Goebel, J.: On the effect of item nonresponse on the estimation of a two-panel-waves wage equation. All. Stat. Arch. 89, 63–74 (2005). Google Scholar
  47. Stocké, V.: Entstehungsbedingungen von Antwortverzerrungen durch soziale Erwünschtheit Ein Vergleich der Prognosen der Rational-Choice Theorie und des Modells der Frame-Selektion. Z. Soziol. 33, 303–320 (2004)Google Scholar
  48. Stocké, V.: Attitudes toward surveys, attitude accessibility and the effect on respondents’ susceptibility to nonresponse. Qual. Quant. 40, 259–288 (2006). CrossRefGoogle Scholar
  49. Sudman, S., Bradburn, N., Schwarz, N.: Thinking About Answers. The Application of Cognitive Processes to Survey Methodology. Jossey Bass, San Francisco (1996)Google Scholar
  50. Tourangeau, R., Rips, L.J., Rasinski, K.: The Psychology of Survey Response. Cambridge University Press, Cambridge (2000)CrossRefGoogle Scholar
  51. Tourangeau, R., Yan, T.: Sensitive questions in surveys. Psychol. Bull. 133, 859–883 (2007). CrossRefGoogle Scholar
  52. Valet, P., May, M., Sauer, C., Liebig, S.: LINOS-1: Legitimation of Inequality Over the Life-Span. SFB Tech. Rep. Ser. DFG Res. Cent. 882 From Heterog. to Inequalities. 13 (2014)Google Scholar
  53. White, I.R., Royston, P., Wood, A.M.: Multiple imputation using chained equations: issues and guidance for practice. Stat. Med. 30, 377–399 (2011). CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V., part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of SociologyUniversity of BambergBambergGermany
  2. 2.Socio-economic Panel Study (SOEP) at German Institute for Economic Research (DIW Berlin)BerlinGermany

Personalised recommendations