Quality of Life Research

, Volume 25, Issue 5, pp 1123–1130 | Cite as

Evaluation of mode equivalence of the MSKCC Bowel Function Instrument, LASA Quality of Life, and Subjective Significance Questionnaire items administered by Web, interactive voice response system (IVRS), and paper

  • Antonia V. Bennett
  • Kathleen Keenoy
  • Marwan Shouery
  • Ethan Basch
  • Larissa K. TempleEmail author



To assess the equivalence of patient-reported outcome (PRO) survey responses across Web, interactive voice response system (IVRS), and paper modes of administration.


Postoperative colorectal cancer patients with home Web/e-mail and phone were randomly assigned to one of the eight study groups: Groups 1–6 completed the survey via Web, IVRS, and paper, in one of the six possible orders; Groups 7–8 completed the survey twice, either by Web or by IVRS. The 20-item survey, including the MSKCC Bowel Function Instrument (BFI), the LASA Quality of Life (QOL) scale, and the Subjective Significance Questionnaire (SSQ) adapted to bowel function, was completed from home on consecutive days. Mode equivalence was assessed by comparison of mean scores across modes and intraclass correlation coefficients (ICCs) and was compared to the test–retest reliability of Web and IVRS.


Of 170 patients, 157 completed at least one survey and were included in analysis. Patients had mean age 56 (SD = 11), 53 % were male, 81 % white, 53 % colon, and 47 % rectal cancer; 78 % completed all assigned surveys. Mean scores for BFI total score, BFI subscale scores, LASA QOL, and adapted SSQ varied by mode by less than one-third of a score point. ICCs across mode were: BFI total score (Web–paper = 0.96, Web–IVRS = 0.97, paper–IVRS = 0.97); BFI subscales (range = 0.88–0.98); LASA QOL (Web–paper = 0.98, Web–IVRS = 0.78, paper–IVRS = 0.80); and SSQ (Web–paper = 0.92, Web–IVRS = 0.86, paper–IVRS = 0.79).


Mode equivalence was demonstrated for the BFI total score, BFI subscales, LASA QOL, and adapted SSQ, supporting the use of multiple modes of PRO data capture in clinical trials.


Mode equivalence Electronic PRO capture Rectal cancer Quality of life 



This study was funded by Cancer and Leukemia Group B Foundation (US) (Grant Number: 2U10CA037447-26). This study was also funded in part by the Memorial Sloan Kettering Cancer Center Core Grant P30 CA008748. The core grant provides support to institutional cores, such as Biostatistics and Pathology, which were used in this study.

Compliance with ethical standards

Conflict of interest

All authors declared that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.


  1. 1.
    Gwaltney, C. J., Shields, A. L., & Shiffman, S. (2008). Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: A meta-analytic review. Value in Health, 11(2), 322–333.CrossRefPubMedGoogle Scholar
  2. 2.
    Coons, S. J., Gwaltney, C. J., Hays, R. D., et al. (2009). Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force report. Value in Health, 12(4), 419–429.CrossRefPubMedGoogle Scholar
  3. 3.
    Temple, L. K., Bacik, J., Savatta, S. G., et al. (2005). The development of a validated instrument to evaluate bowel function after sphincter-preserving surgery for rectal cancer. Diseases of the Colon and Rectum, 48(7), 1353–1365.CrossRefPubMedGoogle Scholar
  4. 4.
    Chen, T. Y., Emertsen, K. J., & Laurberg, S. (2015). What are the best questionnaires to capture anorectal function after surgery in rectal cancer? Current Colorectal Cancer Reports, 11, 37–43.CrossRefPubMedPubMedCentralGoogle Scholar
  5. 5.
    Wong, C., Chen, J., Yu, C. L., Sham, M., & Lam, C. L. (2015). Systemic review recommends the European Organization for Research and Treatment of Cancer colorectal cancer-specific module for measuring quality of life in colorectal cancer patients. Journal of Clinical Epidemiology, 68(3), 266–278.CrossRefPubMedGoogle Scholar
  6. 6.
    Locke, D. E., Decker, P. A., Sloan, J. A., et al. (2007). Validation of single-item linear analog scale assessment of quality of life in neuro-oncology patients. Journal of Pain and Symptom Management, 34(6), 628–638.CrossRefPubMedPubMedCentralGoogle Scholar
  7. 7.
    Osoba, D., Rodrigues, G., Myles, J., Zee, B., & Pater, J. (1998). Interpreting the significance of changes in health-related quality-of-life scores. Journal of Clinical Oncology, 16(1), 139–144.PubMedGoogle Scholar
  8. 8.
    Bennett, A. V., Jensen, R. E., & Basch, E. (2012). Electronic patient-reported outcome systems in oncology clinical practice. CA: Cancer Journal for Clinicians, 62(5), 337–347.Google Scholar
  9. 9.
    Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334.CrossRefGoogle Scholar
  10. 10.
    Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). London: Routledge.Google Scholar
  11. 11.
    Yarandi, H. (2004). Crossover designs and proc mixed in SAS. Paper SD04. In The proceedings of the SouthEast SAS Users Group, Nashville, TN, 2004.
  12. 12.
    Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.Google Scholar
  13. 13.
    Walters, S. J., Campbell, M. J., & Paisley, S. (2001). Methods for determining sample sizes for studies involving health-related quality of life measures: A tutorial. Health Services and Outcomes Research Methodology, 2, 83–99.CrossRefGoogle Scholar
  14. 14.
    Stratford, P. W., & Binkley, J. M. (1999). Applying the results of self-report measures to individual patients: An example using the Roland–Morris Questionnaire. Journal of Orthopaedic and Sports Physical Therapy, 29(4), 232–239.CrossRefPubMedGoogle Scholar
  15. 15.
    Lundy, J. J., & Coons, S. J. (2011). Measurement equivalence of interactive voice response and paper versions of the EQ-5D in a cancer patient sample. Value in Health, 14(6), 867–871.CrossRefPubMedGoogle Scholar
  16. 16.
    Lundy, J. J., Coons, S. J., & Aaronson, N. K. (2014). Testing the measurement equivalence of paper and interactive voice response system versions of the EORTC QLQ-C30. Quality of Life Research, 23(1), 229–237.CrossRefPubMedGoogle Scholar
  17. 17.
    Rush, A. J., Bernstein, I. H., Trivedi, M. H., et al. (2006). An evaluation of the quick inventory of depressive symptomatology and the Hamilton rating scale for depression: A sequenced treatment alternatives to relieve depression trial report. Biological Psychiatry, 59(6), 493–501.CrossRefPubMedPubMedCentralGoogle Scholar
  18. 18.
    Dunn, J. A., Arakawa, R., Greist, J. H., & Clayton, A. H. (2007). Assessing the onset of antidepressant-induced sexual dysfunction using interactive voice response technology. Journal of Clinical Psychiatry, 68(4), 525–532.CrossRefPubMedGoogle Scholar
  19. 19.
    Agel, J., Rockwood, T., Mundt, J. C., Greist, J. H., & Swiontkowski, M. (2001). Comparison of interactive voice response and written self-administered patient surveys for clinical research. Orthopedics, 24(12), 1155–1157.PubMedGoogle Scholar
  20. 20.
    Bjorner, J. B., Rose, M., Gandek, B., Stone, A. A., Junghaenel, D. U., & Ware, J. E, Jr. (2014). Method of administration of PROMIS scales did not significantly impact score level, reliability, or validity. Journal of Clinical Epidemiology, 67(1), 108–113.CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Antonia V. Bennett
    • 1
  • Kathleen Keenoy
    • 2
  • Marwan Shouery
    • 2
  • Ethan Basch
    • 1
    • 2
  • Larissa K. Temple
    • 2
    Email author
  1. 1.UNC Lineberger Comprehensive Cancer CenterUniversity of North CarolinaChapel HillUSA
  2. 2.Memorial Sloan Kettering Cancer CenterNew YorkUSA

Personalised recommendations