Skip to main content
SpringerLink
Log in

Limited evidence of non-response bias despite modest response rate in a nationwide survey of long-term cancer survivors—results from the NOR-CAYACS study

  • Published:
Journal of Cancer Survivorship Aims and scope Submit manuscript

Abstract

Purpose

Declining response rates threaten the generalizability of health surveys. We investigate (1) the effect of item order on response rate; (2) characteristics of early , late and non-responders; and (3) potential non-response bias in a population-based health survey of childhood, adolescent and young adult cancer survivors (CAYACS).

Methods

We mailed a questionnaire survey to 5361 eligible CAYACS identified by the Cancer Registry of Norway (CRN), representing a range of cancer diagnoses. The 302-item questionnaire included a range of survivorship-related questions and validated patient-reported outcome measures. To investigate item-order effects on response rates, we constructed two versions of the questionnaire presenting cancer-related or socio-demographic items first. The CRN provided demographic and clinical information for the total population. Risk of non-response bias was estimated by (1) comparing outcomes between early and late responders (answered after a reminder), and (2) by applying inverse probability of participation weights to construct a total population (with 100% response) and then compare 21 a priori selected outcomes between early responders, all responders (early + late) and the total population (all eligible).

Results

Survey item order did not affect response rates (cancer first 49.8% vs socio-demographic first 50.2%). Shorter time since diagnosis, male gender and a malignant melanoma diagnosis remained significant predictors of non-response in a multivariable multinomial regression model. There were no significant differences on 16/21 survey outcomes between early and late responders, and 18/21 survey outcomes between early responders, all responders and the total population.

Conclusion

Despite a modest response rate, we found little evidence for a response bias in our study.

Implications for Cancer Survivors

Surveys of survivor-reported outcomes with low response rates may still be valuable and generalizable to the total survivor population.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Kilsdonk E, Wendel E, van Dulmen-den Broeder E, van Leeuwen FE, van den Berg MH, Jaspers MW. Participation rates of childhood cancer survivors to self-administered questionnaires: a systematic review. Eur J Cancer Care. 2017; e12462. https://doi.org/10.1111/ecc.12462.

  2. Hudson MM, Ness KK, Gurney JG. Clinical ascertainment of health outcomes among adults treated for childhood cancer. J Am Med Assoc. 2013;309(22):2371–81.

    Article  CAS  Google Scholar 

  3. Oeffinger KC, Mertens AC, Sklar CA, Kawashima T, Hudson MM, Meadows AT, et al. Chronic health conditions in adult survivors of childhood cancer. N Engl J Med. 2006;355(15):1572–82.

    Article  CAS  PubMed  Google Scholar 

  4. Reulen Rc WDLFC, et al. Long-term cause-specific mortality among survivors of childhood cancer. J Am Med Assoc. 2010;304(2):172–9. https://doi.org/10.1001/jama.2010.923.

    Article  Google Scholar 

  5. Brinkman TM, Recklitis CJ, Michel G, Grootenhuis MA, Klosky JL. Psychological symptoms, social outcomes, socioeconomic attainment, and health behaviors among survivors of childhood cancer: current state of the literature. J Clin Oncol. 2018;36(21):2190–7. https://doi.org/10.1200/jco.2017.76.5552.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Jefford M, Ward AC, Lisy K, Lacey K, Emery JD, Glaser AW, et al. Patient-reported outcomes in cancer survivors: a population-wide cross-sectional study. Support Care Cancer. 2017;25(10):3171–9. https://doi.org/10.1007/s00520-017-3725-5.

    Article  PubMed  Google Scholar 

  7. Tai E, Buchanan N, Townsend J, Fairley T, Moore A, Richardson LC. Health status of adolescent and young adult cancer survivors. Cancer. 2012;118(19):4884–91. https://doi.org/10.1002/cncr.27445.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Robison LL, Mertens AC, Boice JD, Breslow NE, Donaldson SS, Green DM, et al. Study design and cohort characteristics of the Childhood Cancer Survivor Study: a multi-institutional collaborative project. Ped Blood Cancer. 2002;38(4):229–39.

    Google Scholar 

  9. Shaw AK, Morrison HI, Speechley KN, Maunsell E, Barrera M, Schanzer D, et al. The late effects study: design and subject representativeness of a Canadian, multi-centre study of late effects of childhood cancer. Chronic Dis Inj Can. 2004;25(3/4):119.

    Google Scholar 

  10. Overbeek A, van den Berg MH, Kremer LC, Van den Heuvel-Eibrink MM, Tissing WJ, Loonen JJ, et al. A nationwide study on reproductive function, ovarian reserve, and risk of premature menopause in female survivors of childhood cancer: design and methodological challenges. BMC Cancer. 2012;12(1):363. https://doi.org/10.1186/1471-2407-12-363.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Hawkins MM, Lancashire ER, Winter DL, Frobisher C, Reulen RC, Taylor AJ, et al. The British Childhood Cancer Survivor Study: objectives, methods, population structure, response rates and initial descriptive information. Ped Blood Cancer. 2008;50(5):1018–25. https://doi.org/10.1002/pbc.21335.

    Article  CAS  Google Scholar 

  12. Kuehni CE, Rueegg CS, Michel G, Rebholz CE, Strippoli M-PF, Niggli FK, et al. Cohort profile: the Swiss Childhood Cancer Survivor Study. Int J Epidemiol. 2011;41:1553–64. https://doi.org/10.1093/ije/dyr142.

    Article  PubMed  Google Scholar 

  13. De Heer W, De Leeuw E. Trends in household survey nonresponse: a longitudinal and international comparison. Survey nonresp. 2002:41.

  14. Galea S, Tracy M. Participation rates in epidemiologic studies. Ann Epidemiol. 2007;17:643–53. https://doi.org/10.1016/j.annepidem.2007.03.013.

    Article  PubMed  Google Scholar 

  15. Johnson TP, Wislar JS. Response rates and nonresponse errors in surveys. J Am Med Assoc. 2012;307(17):1805–6.

    Article  CAS  Google Scholar 

  16. Cheung KL, ten Klooster PM, Smit C, de Vries H, Pieterse ME. The impact of non-response bias due to sampling in public health studies: a comparison of voluntary versus mandatory recruitment in a Dutch national survey on adolescent health. BMC Public Health. 2017;17(1):276. https://doi.org/10.1186/s12889-017-4189-8.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Rueegg CS, Gianinazzi ME, Michel G, Zwahlen M, von der Weid NX, Kuehni CE. No evidence of response bias in a population-based childhood cancer survivor questionnaire survey - results from the Swiss Childhood Cancer Survivor Study. PLoS One. 2017;12(5):e0176442. https://doi.org/10.1371/journal.pone.0176442.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Ojha RP, Oancea SC, Ness KK, Lanctot JQ, Srivastava DK, Robison LL, et al. Assessment of potential bias from non-participation in a dynamic clinical cohort of long-term childhood cancer survivors: results from the St. Jude lifetime cohort study. Ped Blood Cancer. 2013;60(5):856–64. https://doi.org/10.1002/pbc.24348.

    Article  Google Scholar 

  19. Kotaniemi J-T, Hassi J, Kataja M, Jönsson E, Laitinen LA, Sovijärvi AR, et al. Does non-responder bias have a significant effect on the results in a postal questionnaire study? Eur J Epidemiol. 2001;17(9):809–17.

    Article  CAS  PubMed  Google Scholar 

  20. Rönmark EP, Ekerljung L, Lötvall J, Torén K, Rönmark E, Lundbäck B. Large scale questionnaire survey on respiratory health in Sweden: effects of late- and non-response. Respir Med. 2009;103(12):1807–15. https://doi.org/10.1016/j.rmed.2009.07.014.

    Article  PubMed  Google Scholar 

  21. Bliddal M, Liew Z, Pottegård A, Kirkegaard H, Olsen J, Nohr EA. Examining non-participation to the maternal follow-up within the Danish National Birth Cohort. Am J Epidemiol. 2018 Jul 1;187(7):1511-1519. https://doi.org/10.1093/aje/kwy002.

  22. Guo Y, Kopec JA, Cibere J, Li LC, Goldsmith CH. Population survey features and response rates: a randomized experiment. Am J Public Health. 2016;106(8):1422–6. https://doi.org/10.2105/AJPH.2016.303198.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Hellevik O. Extreme nonresponse and response bias. Qual Quant. 2016;50(5):1969–91. https://doi.org/10.1007/s11135-015-0246-5.

    Article  Google Scholar 

  24. Larsen IK, Smastuen M, Johannesen TB, Langmark F, Parkin DM, Bray F, et al. Data quality at the Cancer Registry of Norway: an overview of comparability, completeness, validity and timeliness. Eur J Cancer. 2009;45(7):1218–31. https://doi.org/10.1016/j.ejca.2008.10.037.

    Article  PubMed  Google Scholar 

  25. Dillman DA, Smyth JD, Christian LM. Internet, phone, mail, and mixed-mode surveys: the tailored design method. 4th ed. Hoboken: John Wiley & Sons, Inc.; 2014.

    Google Scholar 

  26. Hernán MA, Hernández-Díaz S, Robins JM. A structural approach to selection bias. Epidemiol. 2004;15(5):615–25. https://doi.org/10.1097/01.ede.0000135174.63482.43.

    Article  Google Scholar 

  27. Ness KK, Li C, Mitby PA, Radloff GA, Mertens AC, Davies SM, et al. Characteristics of responders to a request for a buccal cell specimen among survivors of childhood cancer and their siblings. Ped Blood Cancer. 2010;55(1):165–70.

    Google Scholar 

  28. Gatta G, Zigon G, Capocaccia R. Survival of European children and young adults with cancer diagnosed 1995–2002. Eur J Cancer. 2009;45(6):992–1005.

    Article  PubMed  Google Scholar 

  29. Armstrong GT, Kawashima T, Leisenring W, Stratton K, Stovall M, Hudson MM, et al. Aging and risk of severe, disabling, life-threatening, and fatal events in the childhood cancer survivor study. J Clin Oncol. 2014;32(12):1218–27. https://doi.org/10.1200/jco.2013.51.1055.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Groves RM. Nonresponse rates and nonresponse bias in household surveys. Public Opin Quart. 2006;70:646–75. https://doi.org/10.1093/poq/nfl033.

    Article  Google Scholar 

  31. Groves RM, Peytcheva E. The impact of nonresponse rates on nonresponse bias: a meta-analysis. Public Opin Quart. 2008;72:167–89. https://doi.org/10.1093/poq/nfn011.

    Article  Google Scholar 

  32. Massey DS, Tourangeau R. Where do we go from here? Nonresponse and social measurement. Ann Am Acad Pol Soc Sci. 2013;645(1):222–36. https://doi.org/10.1177/0002716212464191.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Keeter S, Kennedy C, Dimock M, Best J, Craighill P. Gauging the impact of growing nonresponse on estimates from a national RDD telephone survey. Public Opin Quart. 2006;70(5):759–79. https://doi.org/10.1093/poq/nfl035.

    Article  Google Scholar 

  34. Clough-Gorr KM, Fink AK, Silliman RA. Challenges associated with longitudinal survivorship research: attrition and a novel approach of reenrollment in a 6-year follow-up study of older breast cancer survivors. J Cancer Surviv. 2008;2(2):95–103.

    Article  PubMed  Google Scholar 

  35. Deeg DJ, van Tilburg T, Smit JH, de Leeuw ED. Attrition in the longitudinal aging study Amsterdam: the effect of differential inclusion in side studies. J Clin Epidemiol. 2002;55(4):319–28.

    Article  PubMed  Google Scholar 

  36. Nohr EA, Frydenberg M, Henriksen TB, Olsen J. Does low participation in cohort studies induce bias? Epidemiol. 2006;17(4):413–8.

    Article  Google Scholar 

Download references

Funding

The project was funded by The Norwegian Cancer Society (45980) and The Norway Research Council (218312). CSR has received funding from the European Union Seventh Framework Programme (FP7-PEOPLE-2013-COFUND) under grant agreement no. 609020 - Scientia Fellows. SDF received funding from The Radiumhospital Fund (335007). ER and HCL were partially funded by the Regional health authorities of South-Eastern Norway (2015084).

Author information

Authors and Affiliations

  1. Department of Paediatric Medicine, Oslo University Hospital, Rikshospitalet, Oslo, Norway

    Hanne C. Lie & Ellen Ruud

  2. Department of Behavioural Sciences in Medicine, Institute of Basic Medical Sciences in Medicine, Faculty of Medicine, University of Oslo, Oslo, Norway

    Hanne C. Lie & Jon H. Loge

  3. National Advisory Unit on Late Effects after Cancer Treatment, Oslo University Hospital, Radiumhospitalet, PO 4953, Nydalen, NO-0424, Oslo, Norway

    Hanne C. Lie, Sophie D. Fosså & Cecilie E. Kiserud

  4. Oslo Centre for Biostatistics and Epidemiology, Oslo University Hospital and Institute of Basic Medical Sciences, University of Oslo, Oslo, Norway

    Corina S. Rueegg

  5. Regional Advisory Unit in Palliative Care, Department of Oncology, Oslo University Hospital, Oslo, Norway

    Jon H. Loge

  6. Institute of Clinical Medicine, University of Oslo, Oslo, Norway

    Ellen Ruud

Authors
  1. Hanne C. Lie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Corina S. Rueegg
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sophie D. Fosså
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jon H. Loge
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ellen Ruud
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Cecilie E. Kiserud
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hanne C. Lie.

Ethics declarations

The study was granted concession by The Norwegian Data Protection Authority (15/00395-2/CGN) and approved by the Regional Committee for Medical Research Ethics (2015/232 REK sør-øst B), and the Data Protection Officer at Oslo University and the Norwegian Cancer Registry. Informed consent was collected for both participation in the survey and data linkage to information in the CRN

Conflict of interests

The authors declare that they have no conflicts of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(DOCX 73 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lie, H.C., Rueegg, C.S., Fosså, S.D. et al. Limited evidence of non-response bias despite modest response rate in a nationwide survey of long-term cancer survivors—results from the NOR-CAYACS study. J Cancer Surviv 13, 353–363 (2019). https://doi.org/10.1007/s11764-019-00757-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11764-019-00757-x

Keywords

Search

Navigation