Quality & Quantity

, Volume 50, Issue 5, pp 1969–1991 | Cite as

Extreme nonresponse and response bias

A “worst case” analysis
Article
First Online:

Abstract

The article analyzes response bias in the Norwegian Monitor, a series of surveys carried out every second year since 1985, with a response rate of only 4 % in the last wave. One third of the respondents in a telephone interview completed the follow-up mail questionnaire. Their answers in the telephone interview are compared with those of the total telephone sample. Furthermore, results from the mail questionnaire are compared with population statistics and high-response surveys. Finally, the plausibility of nonresponse bias as an explanation regarding trends and correlations in the data is discussed. The conclusion is that even in this extreme case of nonresponse most results are not biased, suggesting that also survey data with very low response rates may have scientific value.

Keywords

Response rate Nonresponse bias Total survey error Survey research 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgments

I have received useful comments/suggestions from anonymous referees, from Erik Dalen, Kristin Rogge Pran, Jan-Paul Brekke, Karen Lillebøe and Arild Sæle at Ipsos MMI, and from Johannes Bergh, Gunnar Sæbø, Tale Hellevik and Erik Neslein Mønness.

References

  1. Berglund, F., Reymert I.S., Aardal, B.: Valgundersøkelse 2009. Dokumentasjonsrapport. SSB notater, Oslo 29/2011 (2011)Google Scholar
  2. Blanchflower, D.G., Oswald, A.J.: Is well-being U-shaped over the life cycle? Soc. Sci. Med. 66–8, 1733–1749 (2008)CrossRefGoogle Scholar
  3. Blanchflower, D.G., Oswald, A.J.: The U-shape without controls: a response to Glenn. Soc. Sci. Med. 69–4, 486–488 (2009)CrossRefGoogle Scholar
  4. Breivik, G., Hellevik, O.: More active and less fit: changes in physical activity in the adult Norwegian population from 1985 to 2011. Sport Soc. Cult. Commer. Media Polit. 17–2, 157–175 (2014)Google Scholar
  5. de Leeuwe, E., de Heer, W.: Trends in household survey nonresponse: a longitudinal and international comparison. In: Groves, R., Dillman, D., Eltinge, J., Little, R.J.A. (eds.) Survey Nonresponse, pp. 41–54. Wiley, New York (2002)Google Scholar
  6. Flanagan, S.C.: Changing values in advanced industrial society. Comp. Polit. Stud. 14, 403–444 (1982)CrossRefGoogle Scholar
  7. Glenn, N.: Is the apparent U-shape of well-being over the life course a result of inappropriate use of control variables? A commentary on Blanchflower and Oswald (66:8, 2008, 1733–1749). Soc. Sci. Med. 69(4) 481–485 (2009)Google Scholar
  8. Goyder, J.C.: The Silent Minority: Nonrespondents on Sample Surveys. Westview Press, Boulder (1987)Google Scholar
  9. Groves, R.M.: Nonresponse rates and nonresponse bias in household surveys. Public Opin. Q. 70–5, 646–675 (2006)CrossRefGoogle Scholar
  10. Groves, R.M., Peytcheva, E.: The impact of nonresponse rates on nonresponse bias. A Meta-Anal. Public Opin. Q. 72–2, 167–189 (2008)CrossRefGoogle Scholar
  11. Hellevik, O.: Postmaterialism as a dimension of cultural change. Intern. J. Public Opin. Res. 5–3, 211–233 (1993)CrossRefGoogle Scholar
  12. Hellevik, O.: Age differences in value orientation—life cycle or cohort effect? Intern. J. Public Opin. Res. 14–3, 286–302 (2002a)CrossRefGoogle Scholar
  13. Hellevik, O.: Beliefs, attitudes and behavior towards the environment. In: Lafferty, W.M., Nordskog, M., Aakre, H.A. (eds.) Realizing Rio in Norway, pp. 7–19. Prosus, Oslo (2002b)Google Scholar
  14. Hellevik, O.: Economy, values and happiness in Norway. J. Happiness Stud. 4–3, 243–283 (2003)CrossRefGoogle Scholar
  15. Hellevik, O.: Assessing long-term value changes in societies. In: Donsbach, W., Traugot, M. (eds.) Handbook of public opinion research, pp. 556–569. Sage, London (2008a)CrossRefGoogle Scholar
  16. Hellevik, O.: Jakten på den norske lykken. Norsk Monitor 1985–2007 (The Pursuit of Happiness in Norway. The Norwegian Monitor 1985–2007). Universitetsforlaget, Oslo (2008b)Google Scholar
  17. Hellevik, O.: Mål og mening. Om feiltolking av meningsmålinger. (Measures and Meaning. Misinterpretations of Opinion Polls). Universitetsforlaget, Oslo (2011)Google Scholar
  18. Hellevik, O.: Is the good life sustainable? A three decade study of values, happiness and sustainability in Norway. In: Mueller, M.L., Syse, K.V.L. (eds.) Sustainable Consumption and the Good Life, pp. 55–79. Routledge, London & New York (2015)Google Scholar
  19. Hellevik, O.: Is there a U-shaped relationship between age and happiness? (forthcoming)Google Scholar
  20. Inglehart, R.: The Silent Revolution—Changing Values and Political Styles Among Western Publics. Princeton University Press, Princeton (1977)Google Scholar
  21. Inglehart, R.: Culture Shift in Advanced Industrial Society. Princeton University Press, Princeton (1990)Google Scholar
  22. Keeter, S., Kennedy, C., Dimock, M., Best, J., Craighill, P.: Gauging the impact of growing nonresponse on estimates from a National RRD telephone survey. Public Opin. Quart. 70–5, 759–779 (2006)CrossRefGoogle Scholar
  23. HL-senteret: Antisemittisme i Norge. Den norske befolkningens holdninger til jøder og andre minoriteter. Oslo, HL-senteret (2012)Google Scholar
  24. PEW Research Center (2012). Assessing the representativeness of public opinion surveys. http://www.people-press.org/2012/05/15/assessing-the-representativeness-of-public-opinion-surveys/
  25. Singer, E.: Introduction. Nonresponse bias in household surveys. Public Opinion Quarterly. 70–5, 637–645 (2006)CrossRefGoogle Scholar
  26. Singleton, R., Straits, B.: Approaches to Social Research, 4th edn. Oxford University Press, New York (2005)Google Scholar
  27. Smith, T.W.: Developing nonresponse standards. In: Groves, R., Dillman, D., Eltinge, J., Little, R.J.A. (eds.) Survey Nonresponse, pp. 27–40. Wiley, New York (2002)Google Scholar
  28. Vedøy, T.F, Skretting, A.: Ungdom og rusmidler. Resultater fra spørreskjemaundersøkelser 1968–2008. SIRUS-Rapport nr. 5/2009. Statens institutt for rusmiddelforskning, Oslo (2009)Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2015

Authors and Affiliations

  1. 1.Department of Political ScienceUniversity of OsloOsloNorway

Personalised recommendations