A reply to “Do surveys accurately report voters over 80 years old?”: testing for bias in probability-based surveys of private households

  • Jan-Lucas SchanzeEmail author
Original Article


This article is a reply and extension to a paper published by Jean-Yves Dormagen and Laura Michel in this journal in early 2018. In their paper “Do surveys accurately report voters over 80 years old?”, the authors examined the deviations of aggregated self-reported turnout in three French electoral surveys from the factual turnout in the French presidential elections of 2002, 2007 and 2012. The tested election studies used a quota sampling method, which suffers most likely from selection bias. Indeed, the authors found a clear underrepresentation of the oldest population older than 85 years in all three studies and a strong overestimation of voting in this age cohort. In this article, I replicate their analysis with probability-based surveys and test whether the bias in turnout of the elderly also occurs in randomly drawn samples. For this purpose, I analysed data of three cross sections of the European Social Survey (ESS-1, ESS-4 and ESS-6). My results show that the probability-based surveys are indeed closer to true values derived from large-scale administrative data. The bias in turnout in the ESS data is nevertheless still stronger in the oldest age cohorts compared to all the other age cohorts. This could be the cause of a higher nonresponse among the elderly. Moreover, the ESS does not cover the institutionalised population. In France, more than 13% of the population older than 80 years are institutionalised and live in retirement and nursing homes. I assume that this part of the population has a lower probability to vote than the elderly living in private households. Their exclusion from the ESS therefore increases the distortion of self-reported voting behaviour compared to the factual turnout of old-aged residents.


Replication Sampling methods Old respondents Institutionalised population French presidential elections 



  1. Angelini, V., and A. Laferrère. 2012. Residential mobility of the European elderly. CESifo Economic Studies 58(3): 544–569.CrossRefGoogle Scholar
  2. Bernstein, R., A. Chadha, and R. Montjoy. 2001. Overreporting voting: Why it happens and why it matters. Public opinion quarterly 65(1): 22–44.CrossRefGoogle Scholar
  3. Biemer, P.P. 2010. Total survey error: Design, implementation, and evaluation. Public Opinion Quarterly 74(5): 817–848.CrossRefGoogle Scholar
  4. Cautrès, B., N. Mayer, and A.C. Salomon. 2003. The French electoral panel 2002. French Politics 1(2): 243–245.CrossRefGoogle Scholar
  5. CEVIPOF-Ministère de l’Intérieure. 2007. Enquête post électorale présidentielle 2007: Résultats d’ensemble.Google Scholar
  6. Chambers, R., and C. Skinner. 2003. Analysis of survey data. New York: Wiley.CrossRefGoogle Scholar
  7. DeBell, M. et al. (2018). The turnout gap in surveys. Sociological Methods & Research. Scholar
  8. Désesquelles, A., and N. Brouard. 2003. The family networks of people aged 60 and over living at home or in an institution. Population 58(2): 201–227.CrossRefGoogle Scholar
  9. Dormagen, J.-Y., and L. Michel. 2018. Do surveys accurately report voters over 80 years old? French Politics. Scholar
  10. ESS. 2002. European social survey round 1: Data file edition 6.5, NSD—Norwegian Centre for Research Data, Norway—Data Archive and distributor of ESS data for ESS ERIC.Google Scholar
  11. ESS. 2008. European social survey round 4: Data file edition 4.4, NSD—Norwegian Centre for Research Data, Norway—Data Archive and distributor of ESS data for ESS ERIC.Google Scholar
  12. ESS. 2012. European social survey round 6: Data file edition 2.3., NSD—Norwegian Centre for Research Data, Norway—Data Archive and distributor of ESS data for ESS ERIC.Google Scholar
  13. ESS. 2013. ESS6—2012 documentation report.Google Scholar
  14. ESS. 2014. Documentation of ESS post-stratification weights. Accessed 14 Jan 2019.
  15. Eurostat. 2016. European statistical system—Census Hub. Accessed 14 Jan 2019.
  16. Groves, R., et al. 2009. Survey methodology, vol. 2. New York: Wiley.Google Scholar
  17. Groves, R.M., and L. Lyberg. 2010. Total survey error: Past, present, and future. Public Opinion Quarterly 74(5): 849–879.CrossRefGoogle Scholar
  18. INSEE. 2012. L’inscription et la participation électorales en 2012: Qui est inscrit et qui vote. Accessed 14 Jan 2019.
  19. Jowell, R., and CCT. 2003. European social survey 2002/2003: Technical Report, London.Google Scholar
  20. Laferrère, A., A. Van Den Heede, and K. Van Den Bosch. 2012. Entry into institutional care: Predictors and alternatives. In Active ageing and solidarity between generations in Europe, ed. A. Börsch-Supan et al., 253–264. Berlin: de Gruyter.Google Scholar
  21. Lessler, J.T., and W.D. Kalsbeek. 1992. Nonsampling error in surveys. New York: Wiley.Google Scholar
  22. Levy, P., and S. Lemeshow. 2013. Sampling of populations: Methods and applications. New York: Wiley.Google Scholar
  23. Lohr, S. 2008. Coverage and Sampling. In International handbook of survey methodology, ed. E. de Leeuw, J. Hox, and D. Dillman, 97–112. New York: European Association of Methodology.Google Scholar
  24. Luppa, M., et al. 2010. Prediction of institutionalization in the elderly. A systematic review. Age and Ageing 39(1): 31–38.CrossRefGoogle Scholar
  25. Lynn, P. 2008. The problem of nonresponse. In International handbook of survey methodology, ed. E. de Leeuw, J. Hox, and D. Dillman, 35–55. New York: European Association of Methodology.Google Scholar
  26. McCann, M., E. Grundy, and D. O’Reilly. 2012. Why is housing tenure associated with a lower risk of admission to a nursing or residential home? Wealth, health and the incentive to keep “my home”. Journal of Epidemiology and Community Health 66(2): 166–169.CrossRefGoogle Scholar
  27. Peytchev, A., S. Presser, and M. Zhang. 2018. Improving traditional nonresponse bias adjustments: Combining statistical properties with social theory. Journal of Survey Statistics and Methodology 6(4): 1–25.CrossRefGoogle Scholar
  28. Pirou, D., N. Poullain, and S. Rochelle. 2013. La vie en communauté.Google Scholar
  29. Rodríguez-Sánchez, B., et al. 2017. Diabetes-associated factors as predictors of nursing home admission and costs in the elderly across Europe. Journal of the American Medical Directors Association 18(1): 74–82.CrossRefGoogle Scholar
  30. Schanze, J.-L. 2017. Report on sampling practices for the institutionalized population in social. Accessed 14 Jan 2019.
  31. Sciarini, P., and A.C. Goldberg. 2016. Turnout bias in postelection surveys: political involvement, survey participation, and vote overreporting. Journal of Survey Statistics and Methodology 4(1): 110–137.CrossRefGoogle Scholar
  32. Toot, S., et al. 2017. Causes of nursing home placement for older people with dementia: a systematic review and meta-analysis. International Psychogeriatrics 29(02): 195–208.CrossRefGoogle Scholar
  33. Voogt, R. 2004. I’m not interested—Nonresponse bias, response bias and stimulus effects in election research. Universiteit van Amsterdam.Google Scholar
  34. Wolff, F.-C. 2013. Well-being of elderly people living in nursing homes: the benefits of making friends. Kyklos 66(1): 153–171.CrossRefGoogle Scholar

Copyright information

© Springer Nature Limited 2019

Authors and Affiliations

  1. 1.GESIS - Leibniz Institute for the Social SciencesMannheimGermany

Personalised recommendations