Skip to main content
SpringerLink
Log in

A reply to “Do surveys accurately report voters over 80 years old?”: testing for bias in probability-based surveys of private households

  • Original Article
  • Published:
French Politics Aims and scope

Abstract

This article is a reply and extension to a paper published by Jean-Yves Dormagen and Laura Michel in this journal in early 2018. In their paper “Do surveys accurately report voters over 80 years old?”, the authors examined the deviations of aggregated self-reported turnout in three French electoral surveys from the factual turnout in the French presidential elections of 2002, 2007 and 2012. The tested election studies used a quota sampling method, which suffers most likely from selection bias. Indeed, the authors found a clear underrepresentation of the oldest population older than 85 years in all three studies and a strong overestimation of voting in this age cohort. In this article, I replicate their analysis with probability-based surveys and test whether the bias in turnout of the elderly also occurs in randomly drawn samples. For this purpose, I analysed data of three cross sections of the European Social Survey (ESS-1, ESS-4 and ESS-6). My results show that the probability-based surveys are indeed closer to true values derived from large-scale administrative data. The bias in turnout in the ESS data is nevertheless still stronger in the oldest age cohorts compared to all the other age cohorts. This could be the cause of a higher nonresponse among the elderly. Moreover, the ESS does not cover the institutionalised population. In France, more than 13% of the population older than 80 years are institutionalised and live in retirement and nursing homes. I assume that this part of the population has a lower probability to vote than the elderly living in private households. Their exclusion from the ESS therefore increases the distortion of self-reported voting behaviour compared to the factual turnout of old-aged residents.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. In 2002 and 2012, this question was asked in the same year after the election took place. In 2008, the survey took place 1 year after the election. This might increase memory effects for the self-reported turnout (Voogt 2004; DeBell et al. 2018).

  2. The latter only affects a small number of 77 respondents in the first round of the ESS.

  3. See “Appendix”.

  4. The unweighted ESS results are not shown in this article.

  5. A value of 1 would indicate equivalence of shares in the survey data with the INSEE benchmark data.

  6. In France, the Enquête Handicaps-Incapacités-Dépendance (HID) and the Enquête auprès des établissements d’hébergement pour personnes âgées (EHPA) allow analyses of the institutionalised population (Désesquelles and Brouard 2003; Wolff 2013).

  7. Please note, in the simulation I assume that all French institutionalised residents are registered to vote. This assumption does probably not hold for all institutionalised residents, especially if they suffer from dementia. According to INSEE data, the oldest age cohorts show the highest rate of registration on the voting lists with 95% for the population older than 50 years (INSEE 2012).

  8. A similar turnout would make no difference in the results compared to the original ESS samples with community-dwelling respondents only.

  9. The tables for ESS-1 and ESS-4 can be found in “Appendix”.

  10. See “Appendix”.

  11. See “Appendix”.

References

  • Angelini, V., and A. Laferrère. 2012. Residential mobility of the European elderly. CESifo Economic Studies 58(3): 544–569.

    Article  Google Scholar 

  • Bernstein, R., A. Chadha, and R. Montjoy. 2001. Overreporting voting: Why it happens and why it matters. Public opinion quarterly 65(1): 22–44.

    Article  Google Scholar 

  • Biemer, P.P. 2010. Total survey error: Design, implementation, and evaluation. Public Opinion Quarterly 74(5): 817–848.

    Article  Google Scholar 

  • Cautrès, B., N. Mayer, and A.C. Salomon. 2003. The French electoral panel 2002. French Politics 1(2): 243–245.

    Article  Google Scholar 

  • CEVIPOF-Ministère de l’Intérieure. 2007. Enquête post électorale présidentielle 2007: Résultats d’ensemble.

  • Chambers, R., and C. Skinner. 2003. Analysis of survey data. New York: Wiley.

    Book  Google Scholar 

  • DeBell, M. et al. (2018). The turnout gap in surveys. Sociological Methods & Research. https://doi.org/10.1177/0049124118769085.

    Google Scholar 

  • Désesquelles, A., and N. Brouard. 2003. The family networks of people aged 60 and over living at home or in an institution. Population 58(2): 201–227.

    Article  Google Scholar 

  • Dormagen, J.-Y., and L. Michel. 2018. Do surveys accurately report voters over 80 years old? French Politics. https://doi.org/10.1057/s41253-018-0056-y.

    Google Scholar 

  • ESS. 2002. European social survey round 1: Data file edition 6.5, NSD—Norwegian Centre for Research Data, Norway—Data Archive and distributor of ESS data for ESS ERIC.

  • ESS. 2008. European social survey round 4: Data file edition 4.4, NSD—Norwegian Centre for Research Data, Norway—Data Archive and distributor of ESS data for ESS ERIC.

  • ESS. 2012. European social survey round 6: Data file edition 2.3., NSD—Norwegian Centre for Research Data, Norway—Data Archive and distributor of ESS data for ESS ERIC.

  • ESS. 2013. ESS6—2012 documentation report.

  • ESS. 2014. Documentation of ESS post-stratification weights. https://www.europeansocialsurvey.org/docs/methodology/ESS1-5_post_stratification_weights_documentation.pdf. Accessed 14 Jan 2019.

  • Eurostat. 2016. European statistical system—Census Hub. https://ec.europa.eu/CensusHub2. Accessed 14 Jan 2019.

  • Groves, R., et al. 2009. Survey methodology, vol. 2. New York: Wiley.

    Google Scholar 

  • Groves, R.M., and L. Lyberg. 2010. Total survey error: Past, present, and future. Public Opinion Quarterly 74(5): 849–879.

    Article  Google Scholar 

  • INSEE. 2012. L’inscription et la participation électorales en 2012: Qui est inscrit et qui vote. https://www.insee.fr/fr/statistiques/1281060. Accessed 14 Jan 2019.

  • Jowell, R., and CCT. 2003. European social survey 2002/2003: Technical Report, London.

  • Laferrère, A., A. Van Den Heede, and K. Van Den Bosch. 2012. Entry into institutional care: Predictors and alternatives. In Active ageing and solidarity between generations in Europe, ed. A. Börsch-Supan et al., 253–264. Berlin: de Gruyter.

    Google Scholar 

  • Lessler, J.T., and W.D. Kalsbeek. 1992. Nonsampling error in surveys. New York: Wiley.

    Google Scholar 

  • Levy, P., and S. Lemeshow. 2013. Sampling of populations: Methods and applications. New York: Wiley.

    Google Scholar 

  • Lohr, S. 2008. Coverage and Sampling. In International handbook of survey methodology, ed. E. de Leeuw, J. Hox, and D. Dillman, 97–112. New York: European Association of Methodology.

    Google Scholar 

  • Luppa, M., et al. 2010. Prediction of institutionalization in the elderly. A systematic review. Age and Ageing 39(1): 31–38.

    Article  Google Scholar 

  • Lynn, P. 2008. The problem of nonresponse. In International handbook of survey methodology, ed. E. de Leeuw, J. Hox, and D. Dillman, 35–55. New York: European Association of Methodology.

    Google Scholar 

  • McCann, M., E. Grundy, and D. O’Reilly. 2012. Why is housing tenure associated with a lower risk of admission to a nursing or residential home? Wealth, health and the incentive to keep “my home”. Journal of Epidemiology and Community Health 66(2): 166–169.

    Article  Google Scholar 

  • Peytchev, A., S. Presser, and M. Zhang. 2018. Improving traditional nonresponse bias adjustments: Combining statistical properties with social theory. Journal of Survey Statistics and Methodology 6(4): 1–25.

    Article  Google Scholar 

  • Pirou, D., N. Poullain, and S. Rochelle. 2013. La vie en communauté.

  • Rodríguez-Sánchez, B., et al. 2017. Diabetes-associated factors as predictors of nursing home admission and costs in the elderly across Europe. Journal of the American Medical Directors Association 18(1): 74–82.

    Article  Google Scholar 

  • Schanze, J.-L. 2017. Report on sampling practices for the institutionalized population in social. https://seriss.eu/resources/deliverables/. Accessed 14 Jan 2019.

  • Sciarini, P., and A.C. Goldberg. 2016. Turnout bias in postelection surveys: political involvement, survey participation, and vote overreporting. Journal of Survey Statistics and Methodology 4(1): 110–137.

    Article  Google Scholar 

  • Toot, S., et al. 2017. Causes of nursing home placement for older people with dementia: a systematic review and meta-analysis. International Psychogeriatrics 29(02): 195–208.

    Article  Google Scholar 

  • Voogt, R. 2004. I’m not interested—Nonresponse bias, response bias and stimulus effects in election research. Universiteit van Amsterdam.

  • Wolff, F.-C. 2013. Well-being of elderly people living in nursing homes: the benefits of making friends. Kyklos 66(1): 153–171.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. GESIS - Leibniz Institute for the Social Sciences, P.O. Box 12 21 55, 68072, Mannheim, Germany

    Jan-Lucas Schanze

Authors
  1. Jan-Lucas Schanze
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jan-Lucas Schanze.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

See Figs. 6, 7, 8, 9, 10 and Tables 4, 5.

Fig. 6
figure 6

Share of residents living in collective households within respective age and gender cohorts in France (2011; Eurostat 2016)

Full size image
Fig. 7
figure 7

Demographic representation of age groups in the population eligible to vote in the PEF and ESS (2008)

Full size image
Fig. 8
figure 8

Estimated turnout in the PEF and in the ESS in different age cohorts compared to true turnout (2008, weighted)

Full size image
Fig. 9
figure 9

Reduction in bias in turnout when synthetic institutionalised residents with given turnout are added to the ESS-1 sample

Full size image
Fig. 10
figure 10

Reduction in bias in turnout when synthetic institutionalised residents with given turnout are added to the ESS-4 sample

Full size image
Table 4 Changes in turnout estimates and deviation (in brackets) when synthetic institutionalised respondents with given turnout are added to the ESS samples (2002)
Full size table
Table 5 Changes in turnout estimates and deviation (in brackets) when synthetic institutionalised respondents with given turnout are added to the ESS samples (2008)
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Schanze, JL. A reply to “Do surveys accurately report voters over 80 years old?”: testing for bias in probability-based surveys of private households. Fr Polit 17, 26–44 (2019). https://doi.org/10.1057/s41253-019-00080-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1057/s41253-019-00080-y

Keywords

Search

Navigation