Journal of Cancer Survivorship

, Volume 11, Issue 3, pp 393–400 | Cite as

Optimizing patient-reported outcome and risk factor reporting from cancer survivors: a randomized trial of four different survey methods among colorectal cancer survivors

  • Heather Spencer FeigelsonEmail author
  • Carmit K McMullen
  • Sarah Madrid
  • Andrew T Sterrett
  • J David Powers
  • Erica Blum-Barnett
  • Pamala A Pawloski
  • Jeanette Y Ziegenfuss
  • Virginia P. Quinn
  • David E Arterburn
  • Douglas A Corley



The goal of this study was to determine response rates and associated costs of different survey methods among colorectal cancer (CRC) survivors.


We assembled a cohort of 16,212 individuals diagnosed with CRC (2010–2014) from six health plans, and randomly selected 4000 survivors to test survey response rates across four mixed-mode survey administration protocols (in English and Spanish): arm 1, mailed survey with phone follow-up; arm 2, interactive voice response (IVR) followed by mail; arm 3; email linked to web-based survey with mail follow-up; and arm 4, email linked to web-based survey followed by IVR.


Our overall response rate was 50.2%. Arm 1 had the highest response rate (59.9%), followed by arm 3 (51.9%), arm 2 (51.2%), and arm 4 (37.9%). Response rates were higher among non-Hispanic whites in all arms than other racial/ethnic groups (p < 0.001), among English (51.5%) than Spanish speakers (36.4%) (p < 0.001), and among higher (53.7%) than lower (41.4%) socioeconomic status (p < 0.001). Survey arms were roughly comparable in cost, with a difference of only 8% of total costs between the most (arm 2) and least (arm 3) expensive arms.


Mailed surveys followed by phone calls achieved the highest response rate; email invitations and online surveys cost less per response. Electronic methods, even among those with email availability, may miss important populations including Hispanics, non-English speakers, and those of lower socioeconomic status.

Implications for cancer survivors

Our results demonstrate effective methods for capturing patient-reported outcomes, inform the relative benefits/disadvantages of the different methods, and identify future research directions.


Colorectal cancer Mixed-mode survey Patient-reported outcomes Survivors 



The authors gratefully acknowledge the assistance of the following people in conduct of this research: Michelle Henton, MA (Kaiser Permanente, Colorado); Jane Anau and Doug Kane (Group Health Research Institute); the members of the PORTAL Patient Engagement Core (PEC): Rose Hesselbrock, Florence Kurtilla, and Charles Anderson; and our colleagues at Fight Colorectal Cancer, including Anjelica Davis, MPPA. We thank the developers of the web-based and IVR systems at Kaiser Permanente, Colorado: Jonah N. Langer, David A. Steffen, MPH, Michael R. Shainline, MS, MBA, and Andew Hamblin. We also sincerely thank the study participants for their contributions to this project.

Compliance with ethical standards


This work was support by Contract No.CDRN-1306-04681 from the Patient Outcomes Research Institute (Awarded to Drs. Elizabeth M. McGlynn and Tracy Lieu). The infrastructure builds upon data structures that receive ongoing support from the National Cancer Institute Cancer (NCI) Research Network (Grant No. U24 CA171524, awarded to Dr. Lawrence H Kushi, PI) and the Kaiser Permanente Center for Effectiveness and Safety Research.

Conflict of interest

All authors declare that they have no conflicts of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This study was approved by the Institutional Review Board of KPCO; all other participating health plans ceded oversight to KPCO.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Supplementary material

11764_2017_596_MOESM1_ESM.pdf (172 kb)
ESM 1 (PDF 172 kb.)


  1. 1.
    Beebe TJ, McAlpine DD, Ziegenfuss JY, Jenkins S, Haas L, Davern ME. Deployment of a mixed-mode data collection strategy does not reduce nonresponse bias in a general population health survey. Health Serv Res. 2012;47:1739–54.CrossRefPubMedPubMedCentralGoogle Scholar
  2. 2.
    Atrostic BK, Bates N, Burt G, Silberstein A. Nonresponse in US government household surveys: consistent measures, recent trends, and new insights. J Off Stat. 2001;17:209–26.Google Scholar
  3. 3.
    Sackett DL. Bias in analytic research. J Chronic Dis. 1979;32:51–63.CrossRefPubMedGoogle Scholar
  4. 4.
    Groves RM. Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly. 2006;70:646–75.CrossRefGoogle Scholar
  5. 5.
    Armes J, Crowe M, Colbourne L, et al. Patients’ supportive care needs beyond the end of cancer treatment: a prospective, longitudinal survey. J Clin Oncol. 2009;27:6172–9.CrossRefPubMedGoogle Scholar
  6. 6.
    van Ryn M, Phelan SM, Arora NK, et al. Patient-reported quality of supportive care among patients with colorectal cancer in the Veterans Affairs Health Care System. J Clin Oncol. 2014;32:809–15.CrossRefPubMedPubMedCentralGoogle Scholar
  7. 7.
    Patient-Centered Outcomes Research Institute: The National Patient-Centered Clinical Research Network. at 2016.
  8. 8.
    Corley DA, Feigelson HS, Lieu TA, McGlynn EA. Building data infrastructure to evaluate and improve quality: PCORnet. Journal of Oncology Practice. 2015;11:204–6.CrossRefPubMedPubMedCentralGoogle Scholar
  9. 9.
    Wagner EH, Greene SM, Hart G, et al. Building a research consortium of large health systems: the cancer research network. J Natl Cancer Inst Monogr. 2005;35:3–11.CrossRefGoogle Scholar
  10. 10.
    Ritzwoller DP, Carroll N, Delate T, et al. Validation of electronic data on chemotherapy and hormone therapy use in HMOs. Med Care. 2013;51:e67–73.CrossRefPubMedPubMedCentralGoogle Scholar
  11. 11.
    Ross TR, Ng D, Brown JS, et al. The HMO Research Network Virtual Data Warehouse: a public data model to support collaboration. EGEMS (Washington, DC) 2014; 2:1049.Google Scholar
  12. 12.
    Hornbrook MC, Hart G, Ellis JL, et al. Building a virtual cancer research organization. J Natl Cancer Inst Monogr. 2005;35:12–25.CrossRefGoogle Scholar
  13. 13.
    North American Association of Central Cancer Registries: NAACCR Strategic 460 Management Plan. at 2014.
  14. 14.
    Dillman DA, Phelps G, Tortora R, Swift K, Kohrell J, Berck J, et al. Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the Internet. Soc Sci Res. 2009;38(1):1–18.CrossRefGoogle Scholar
  15. 15.
    Dillman DA, Smyth JD, Christian LM. Internet, phone, mail, and mixed-mode surveys: the tailored design method: John Wiley & Sons; 2014.Google Scholar
  16. 16.
    Davern M. Nonresponse rates are a problematic indicator of nonresponse bias in survey research. Health Serv Res. 2013;48:905–12.CrossRefPubMedPubMedCentralGoogle Scholar
  17. 17.
    Downing A, Morris EJ, Richards M, et al. Health-related quality of life after colorectal cancer in England: a patient-reported outcomes study of individuals 12 to 36 Months after diagnosis. J Clin Oncol. 2015;33:616–24.CrossRefPubMedGoogle Scholar
  18. 18.
    McGowan EL, Speed-Andrews AE, Blanchard CM, et al. Physical activity preferences among a population-based sample of colorectal cancer survivors. Oncol Nurs Forum. 2013;40:44–52.CrossRefPubMedGoogle Scholar
  19. 19.
    Kelly BJ, Fraze TK, Hornik RC. Response rates to a mailed survey of a representative sample of cancer patients randomly drawn from the Pennsylvania Cancer Registry: a randomized trial of incentive and length effects. BMC Med Res Methodol. 2010;10:65.CrossRefPubMedPubMedCentralGoogle Scholar
  20. 20.
    Maxwell AE, Bastani R, Glenn BA, Mojica CM, Chang LC. An experimental test of the effect of incentives on recruitment of ethnically diverse colorectal cancer cases and their first-degree relatives into a research study. Cancer Epidemiol Biomark Prev. 2009;18:2620–5.CrossRefGoogle Scholar
  21. 21.
    Wendel CS, Grant M, Herrinton L, et al. Reliability and validity of a survey to measure bowel function and quality of life in long-term rectal cancer survivors. Qual Life Res. 2014;23:2831–40.CrossRefPubMedPubMedCentralGoogle Scholar
  22. 22.
    Mohler MJ, Coons SJ, Hornbrook MC, et al. The health-related quality of life in long-term colorectal cancer survivors study: objectives, methods, and patient sample. Curr Med Res Opin. 2008;24:2059–70.CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  • Heather Spencer Feigelson
    • 1
    Email author
  • Carmit K McMullen
    • 2
  • Sarah Madrid
    • 1
  • Andrew T Sterrett
    • 1
  • J David Powers
    • 1
  • Erica Blum-Barnett
    • 1
  • Pamala A Pawloski
    • 3
  • Jeanette Y Ziegenfuss
    • 3
  • Virginia P. Quinn
    • 4
  • David E Arterburn
    • 5
  • Douglas A Corley
    • 6
  1. 1.Institute for Health Research, Kaiser Permanente ColoradoDenverUSA
  2. 2.Center for Health Research, Kaiser Permanente NorthwestPortlandUSA
  3. 3.HealthPartners InstituteBloomingtonUSA
  4. 4.Department of Research & Evaluation, Kaiser Permanente Southern CaliforniaPasadenaUSA
  5. 5.Group Health Research InstituteSeattleUSA
  6. 6.Division of Research, Kaiser Permanente Northern CaliforniaOaklandUSA

Personalised recommendations