Limited evidence of non-response bias despite modest response rate in a nationwide survey of long-term cancer survivors—results from the NOR-CAYACS study

  • Hanne C. LieEmail author
  • Corina S. Rueegg
  • Sophie D. Fosså
  • Jon H. Loge
  • Ellen Ruud
  • Cecilie E. Kiserud



Declining response rates threaten the generalizability of health surveys. We investigate (1) the effect of item order on response rate; (2) characteristics of early , late and non-responders; and (3) potential non-response bias in a population-based health survey of childhood, adolescent and young adult cancer survivors (CAYACS).


We mailed a questionnaire survey to 5361 eligible CAYACS identified by the Cancer Registry of Norway (CRN), representing a range of cancer diagnoses. The 302-item questionnaire included a range of survivorship-related questions and validated patient-reported outcome measures. To investigate item-order effects on response rates, we constructed two versions of the questionnaire presenting cancer-related or socio-demographic items first. The CRN provided demographic and clinical information for the total population. Risk of non-response bias was estimated by (1) comparing outcomes between early and late responders (answered after a reminder), and (2) by applying inverse probability of participation weights to construct a total population (with 100% response) and then compare 21 a priori selected outcomes between early responders, all responders (early + late) and the total population (all eligible).


Survey item order did not affect response rates (cancer first 49.8% vs socio-demographic first 50.2%). Shorter time since diagnosis, male gender and a malignant melanoma diagnosis remained significant predictors of non-response in a multivariable multinomial regression model. There were no significant differences on 16/21 survey outcomes between early and late responders, and 18/21 survey outcomes between early responders, all responders and the total population.


Despite a modest response rate, we found little evidence for a response bias in our study.

Implications for Cancer Survivors

Surveys of survivor-reported outcomes with low response rates may still be valuable and generalizable to the total survivor population.


Non-response bias Childhood cancer survivors Health survey Response rate 



The project was funded by The Norwegian Cancer Society (45980) and The Norway Research Council (218312). CSR has received funding from the European Union Seventh Framework Programme (FP7-PEOPLE-2013-COFUND) under grant agreement no. 609020 - Scientia Fellows. SDF received funding from The Radiumhospital Fund (335007). ER and HCL were partially funded by the Regional health authorities of South-Eastern Norway (2015084).

Compliance with ethical standards

The study was granted concession by The Norwegian Data Protection Authority (15/00395-2/CGN) and approved by the Regional Committee for Medical Research Ethics (2015/232 REK sør-øst B), and the Data Protection Officer at Oslo University and the Norwegian Cancer Registry. Informed consent was collected for both participation in the survey and data linkage to information in the CRN

Conflict of interests

The authors declare that they have no conflicts of interest.

Supplementary material

11764_2019_757_MOESM1_ESM.docx (74 kb)
ESM 1 (DOCX 73 kb)


  1. 1.
    Kilsdonk E, Wendel E, van Dulmen-den Broeder E, van Leeuwen FE, van den Berg MH, Jaspers MW. Participation rates of childhood cancer survivors to self-administered questionnaires: a systematic review. Eur J Cancer Care. 2017; e12462.
  2. 2.
    Hudson MM, Ness KK, Gurney JG. Clinical ascertainment of health outcomes among adults treated for childhood cancer. J Am Med Assoc. 2013;309(22):2371–81.CrossRefGoogle Scholar
  3. 3.
    Oeffinger KC, Mertens AC, Sklar CA, Kawashima T, Hudson MM, Meadows AT, et al. Chronic health conditions in adult survivors of childhood cancer. N Engl J Med. 2006;355(15):1572–82.CrossRefGoogle Scholar
  4. 4.
    Reulen Rc WDLFC, et al. Long-term cause-specific mortality among survivors of childhood cancer. J Am Med Assoc. 2010;304(2):172–9. Scholar
  5. 5.
    Brinkman TM, Recklitis CJ, Michel G, Grootenhuis MA, Klosky JL. Psychological symptoms, social outcomes, socioeconomic attainment, and health behaviors among survivors of childhood cancer: current state of the literature. J Clin Oncol. 2018;36(21):2190–7. Scholar
  6. 6.
    Jefford M, Ward AC, Lisy K, Lacey K, Emery JD, Glaser AW, et al. Patient-reported outcomes in cancer survivors: a population-wide cross-sectional study. Support Care Cancer. 2017;25(10):3171–9. Scholar
  7. 7.
    Tai E, Buchanan N, Townsend J, Fairley T, Moore A, Richardson LC. Health status of adolescent and young adult cancer survivors. Cancer. 2012;118(19):4884–91. Scholar
  8. 8.
    Robison LL, Mertens AC, Boice JD, Breslow NE, Donaldson SS, Green DM, et al. Study design and cohort characteristics of the Childhood Cancer Survivor Study: a multi-institutional collaborative project. Ped Blood Cancer. 2002;38(4):229–39.Google Scholar
  9. 9.
    Shaw AK, Morrison HI, Speechley KN, Maunsell E, Barrera M, Schanzer D, et al. The late effects study: design and subject representativeness of a Canadian, multi-centre study of late effects of childhood cancer. Chronic Dis Inj Can. 2004;25(3/4):119.Google Scholar
  10. 10.
    Overbeek A, van den Berg MH, Kremer LC, Van den Heuvel-Eibrink MM, Tissing WJ, Loonen JJ, et al. A nationwide study on reproductive function, ovarian reserve, and risk of premature menopause in female survivors of childhood cancer: design and methodological challenges. BMC Cancer. 2012;12(1):363. CrossRefGoogle Scholar
  11. 11.
    Hawkins MM, Lancashire ER, Winter DL, Frobisher C, Reulen RC, Taylor AJ, et al. The British Childhood Cancer Survivor Study: objectives, methods, population structure, response rates and initial descriptive information. Ped Blood Cancer. 2008;50(5):1018–25. Scholar
  12. 12.
    Kuehni CE, Rueegg CS, Michel G, Rebholz CE, Strippoli M-PF, Niggli FK, et al. Cohort profile: the Swiss Childhood Cancer Survivor Study. Int J Epidemiol. 2011;41:1553–64. Scholar
  13. 13.
    De Heer W, De Leeuw E. Trends in household survey nonresponse: a longitudinal and international comparison. Survey nonresp. 2002:41.Google Scholar
  14. 14.
    Galea S, Tracy M. Participation rates in epidemiologic studies. Ann Epidemiol. 2007;17:643–53. Scholar
  15. 15.
    Johnson TP, Wislar JS. Response rates and nonresponse errors in surveys. J Am Med Assoc. 2012;307(17):1805–6.CrossRefGoogle Scholar
  16. 16.
    Cheung KL, ten Klooster PM, Smit C, de Vries H, Pieterse ME. The impact of non-response bias due to sampling in public health studies: a comparison of voluntary versus mandatory recruitment in a Dutch national survey on adolescent health. BMC Public Health. 2017;17(1):276. Scholar
  17. 17.
    Rueegg CS, Gianinazzi ME, Michel G, Zwahlen M, von der Weid NX, Kuehni CE. No evidence of response bias in a population-based childhood cancer survivor questionnaire survey - results from the Swiss Childhood Cancer Survivor Study. PLoS One. 2017;12(5):e0176442. Scholar
  18. 18.
    Ojha RP, Oancea SC, Ness KK, Lanctot JQ, Srivastava DK, Robison LL, et al. Assessment of potential bias from non-participation in a dynamic clinical cohort of long-term childhood cancer survivors: results from the St. Jude lifetime cohort study. Ped Blood Cancer. 2013;60(5):856–64. Scholar
  19. 19.
    Kotaniemi J-T, Hassi J, Kataja M, Jönsson E, Laitinen LA, Sovijärvi AR, et al. Does non-responder bias have a significant effect on the results in a postal questionnaire study? Eur J Epidemiol. 2001;17(9):809–17.CrossRefGoogle Scholar
  20. 20.
    Rönmark EP, Ekerljung L, Lötvall J, Torén K, Rönmark E, Lundbäck B. Large scale questionnaire survey on respiratory health in Sweden: effects of late- and non-response. Respir Med. 2009;103(12):1807–15. Scholar
  21. 21.
    Bliddal M, Liew Z, Pottegård A, Kirkegaard H, Olsen J, Nohr EA. Examining non-participation to the maternal follow-up within the Danish National Birth Cohort. Am J Epidemiol. 2018 Jul 1;187(7):1511-1519.
  22. 22.
    Guo Y, Kopec JA, Cibere J, Li LC, Goldsmith CH. Population survey features and response rates: a randomized experiment. Am J Public Health. 2016;106(8):1422–6. Scholar
  23. 23.
    Hellevik O. Extreme nonresponse and response bias. Qual Quant. 2016;50(5):1969–91. Scholar
  24. 24.
    Larsen IK, Smastuen M, Johannesen TB, Langmark F, Parkin DM, Bray F, et al. Data quality at the Cancer Registry of Norway: an overview of comparability, completeness, validity and timeliness. Eur J Cancer. 2009;45(7):1218–31. Scholar
  25. 25.
    Dillman DA, Smyth JD, Christian LM. Internet, phone, mail, and mixed-mode surveys: the tailored design method. 4th ed. Hoboken: John Wiley & Sons, Inc.; 2014.Google Scholar
  26. 26.
    Hernán MA, Hernández-Díaz S, Robins JM. A structural approach to selection bias. Epidemiol. 2004;15(5):615–25. Scholar
  27. 27.
    Ness KK, Li C, Mitby PA, Radloff GA, Mertens AC, Davies SM, et al. Characteristics of responders to a request for a buccal cell specimen among survivors of childhood cancer and their siblings. Ped Blood Cancer. 2010;55(1):165–70.Google Scholar
  28. 28.
    Gatta G, Zigon G, Capocaccia R. Survival of European children and young adults with cancer diagnosed 1995–2002. Eur J Cancer. 2009;45(6):992–1005.CrossRefGoogle Scholar
  29. 29.
    Armstrong GT, Kawashima T, Leisenring W, Stratton K, Stovall M, Hudson MM, et al. Aging and risk of severe, disabling, life-threatening, and fatal events in the childhood cancer survivor study. J Clin Oncol. 2014;32(12):1218–27. Scholar
  30. 30.
    Groves RM. Nonresponse rates and nonresponse bias in household surveys. Public Opin Quart. 2006;70:646–75. Scholar
  31. 31.
    Groves RM, Peytcheva E. The impact of nonresponse rates on nonresponse bias: a meta-analysis. Public Opin Quart. 2008;72:167–89. Scholar
  32. 32.
    Massey DS, Tourangeau R. Where do we go from here? Nonresponse and social measurement. Ann Am Acad Pol Soc Sci. 2013;645(1):222–36. Scholar
  33. 33.
    Keeter S, Kennedy C, Dimock M, Best J, Craighill P. Gauging the impact of growing nonresponse on estimates from a national RDD telephone survey. Public Opin Quart. 2006;70(5):759–79. Scholar
  34. 34.
    Clough-Gorr KM, Fink AK, Silliman RA. Challenges associated with longitudinal survivorship research: attrition and a novel approach of reenrollment in a 6-year follow-up study of older breast cancer survivors. J Cancer Surviv. 2008;2(2):95–103.CrossRefGoogle Scholar
  35. 35.
    Deeg DJ, van Tilburg T, Smit JH, de Leeuw ED. Attrition in the longitudinal aging study Amsterdam: the effect of differential inclusion in side studies. J Clin Epidemiol. 2002;55(4):319–28.CrossRefGoogle Scholar
  36. 36.
    Nohr EA, Frydenberg M, Henriksen TB, Olsen J. Does low participation in cohort studies induce bias? Epidemiol. 2006;17(4):413–8.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Paediatric MedicineOslo University Hospital, RikshospitaletOsloNorway
  2. 2.Department of Behavioural Sciences in Medicine, Institute of Basic Medical Sciences in Medicine, Faculty of MedicineUniversity of OsloOsloNorway
  3. 3.National Advisory Unit on Late Effects after Cancer TreatmentOslo University Hospital, RadiumhospitaletOsloNorway
  4. 4.Oslo Centre for Biostatistics and Epidemiology, Oslo University Hospital and Institute of Basic Medical SciencesUniversity of OsloOsloNorway
  5. 5.Regional Advisory Unit in Palliative Care, Department of OncologyOslo University HospitalOsloNorway
  6. 6.Institute of Clinical MedicineUniversity of OsloOsloNorway

Personalised recommendations