Advertisement

Quality of Life Research

, Volume 23, Issue 1, pp 257–269 | Cite as

Cognitive interviewing of the US National Cancer Institute’s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE)

  • Jennifer L. Hay
  • Thomas M. Atkinson
  • Bryce B. Reeve
  • Sandra A. Mitchell
  • Tito R. Mendoza
  • Gordon Willis
  • Lori M. Minasian
  • Steven B. Clauser
  • Andrea Denicoff
  • Ann O’Mara
  • Alice Chen
  • Antonia V. Bennett
  • Diane B. Paul
  • Joshua Gagne
  • Lauren Rogak
  • Laura Sit
  • Vish Viswanath
  • Deborah Schrag
  • Ethan Basch
  • the NCI PRO-CTCAE Study Group
Article

Abstract

Purpose

The National Cancer Institute’s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) is a library of question items that enables patient reporting of adverse events (AEs) in clinical trials. This study contributes content validity evidence of the PRO-CTCAE by incorporating cancer patient input of the relevance and comprehensiveness of the item library.

Methods

Cognitive interviews were conducted among patients undergoing chemotherapy or radiation therapy at multiple sites to evaluate comprehension, memory retrieval, judgment, and response mapping related to AE terms (e.g., nausea), attribute terms (regarding frequency, severity, or interference), response options, and recall period. Three interview rounds were conducted with ≥20 patients completing each item per round. Items were modified and retested if ≥3 patients exhibited cognitive difficulties or if experienced by ≤25 % patients.

Results

One hundred and twenty-seven patients participated (35 % ≤high school, 28 % non-white, and 59 % female). Most AE terms (63/80) generated no cognitive difficulties. The remaining 17 were modified without further difficulties by Round 3. Terms were comprehended regardless of education level. Attribute terms and response options required no modifications. Patient adherence to recall period (7 days) was improved when the reference period was incorporated.

Conclusions

This study provides evidence confirming comprehension of the US English language versions of items in the PRO-CTCAE library for measuring symptomatic AEs from the patient perspective within the context of cancer treatment. Several minor changes were made to the items to improve item clarity, comprehension, and ease of response judgment. This study helps to establish the content validity of PRO-CTCAE items for patient reporting of AEs during cancer treatment.

Keywords

Drug toxicity Neoplasms Psychometrics Reproducibility of results Qualitative research 

Notes

Acknowledgments

Work described in this report was supported by contracts from the United States National Cancer Institute, HHSN261201000043C and HHSN261201000063C. Each author met all International Committee of Medical Journal Editors (ICMJE) requirements (i.e., conception/design/acquisition/analysis, drafting/revising article, and final approval) for authorship.

Supplementary material

11136_2013_470_MOESM1_ESM.docx (23 kb)
Supplementary material 1 (DOCX 23 kb)

References

  1. 1.
    Trotti, A., Colevas, A. D., Setser, A., Rusch, V., Jaques, D., Budach, V., et al. (2003). CTCAE v3.0: Development of a comprehensive grading system for the adverse events of cancer treatment. Seminars in Radiation Oncology, 13, 176–181.PubMedCrossRefGoogle Scholar
  2. 2.
    National Cancer Institute, National Institutes of Health, US Department of Health and Human Services. Common Terminology Criteria for Adverse Events (CTCAE) Version 4.0. Published May 28, 2009; Revised version 4.03 June 14, 2010 (Vol. Available from: http://evs.nci.nih.gov/ftp1/CTCAE/CTCAE_4.03_2010-06-14_QuickReference_5x7.pdf. Accessed 20 May 2013).
  3. 3.
    Basch, E. (2010). The missing voice of patients in drug-safety reporting. New England Journal of Medicine, 362(10), 865–869. doi: 10.1056/NEJMp0911494.PubMedCentralPubMedCrossRefGoogle Scholar
  4. 4.
    Basch, E., Iasonos, A., McDonough, T., Barz, A., Culkin, A., Kris, M. G., et al. (2006). Patient versus clinician symptom reporting using the National Cancer Institute Common Terminology Criteria for Adverse Events: Results of a questionnaire-based study. Lancet Oncology, 7, 903–909. doi: 10.1016/S1470-2045(06)70910-X.PubMedCrossRefGoogle Scholar
  5. 5.
    Basch, E., Jia, X., Heller, G., Barz, A., Sit, L., Fruscione, M., et al. (2009). Adverse symptom event reporting by patients versus clinicians: Relationships with clinical outcomes. Journal of the National Cancer Institute, 101(23), 1624–1632. doi: 10.1093/jnci/djp386.PubMedCrossRefGoogle Scholar
  6. 6.
    Atkinson, T. M., Li, Y., Coffey, C. W., Sit, L., Shaw, M., Lavene, D., et al. (2012). Reliability of adverse symptom event reporting by clinicians. Quality of Life Research, 21(7), 1159–1164. doi: 10.1007/s11136-011-0031-4.PubMedCentralPubMedCrossRefGoogle Scholar
  7. 7.
    Xiao, C., Polomano, R., & Bruner, D. W. (2012). Comparison between patient-reported and clinician-observed symptoms in oncology. Cancer Nursing,. doi: 10.1097/NCC.0b013e318269040f.Google Scholar
  8. 8.
    Greimel, E. R., Bjelic-Radisic, V., Pfisterer, J., Hilpert, F., Daghofer, F., Pujade-Lauraine, E., et al. (2011). Toxicity and quality of life outcomes in ovarian cancer patients participating in randomized controlled trials. Supportive Care in Cancer, 19(9), 1421–1427. doi: 10.1007/s00520-010-0969-8.PubMedCrossRefGoogle Scholar
  9. 9.
    Flores, L. T., Bennett, A. V., Law, E. B., Hajj, C., Griffith, M. P., & Goodman, K. A. (2012). Patient-reported outcomes versus clinician symptom reporting during chemoradiation for rectal cancer. Gastrointestinal Cancer Research, 5(4), 119–124.PubMedGoogle Scholar
  10. 10.
    Basch, E., Iasonos, A., Barz, A., Culkin, A., Kris, M. G., Artz, D., et al. (2007). Long-term toxicity monitoring via electronic patient-reported outcomes in patients receiving chemotherapy. Journal of Clinical Oncology, 25(34), 5374–5380. doi: 10.1200/JCO.2007.11.2243.PubMedCrossRefGoogle Scholar
  11. 11.
    Tang, B., Giuliani, M., Le, L. W., Higgins, J., Bezjak, A., Brade, A., et al. (2013). Capturing acute toxicity data during lung radiotherapy by using a patient-reported assessment tool. Clinical Lung Cancer, 14(2), 108–112. doi: 10.1016/j.cllc.2012.06.003.PubMedCrossRefGoogle Scholar
  12. 12.
    US Department of Health and Human Services. Guidance for industry. patient-reported outcome measures: Use in medical development to support labeling claims (December 2009. Available from: http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf. Accessed 20 May 2013).
  13. 13.
    European Medicines Agency. Committee for medicinal products for human use (CHMP). Pre-authorisation evaluation of medicines for human use: Reflection paper on the regulatory guidance for the use of health-related quality of life (HRQL) measures in the evaluation of medicinal products (January 2006, Available from: http://www.ispor.org/workpaper/emea-hrql-guidance.pdf. Accessed 20 May 2013).
  14. 14.
    Ashley, L., Jones, H., Thomas, J., Forman, D., Newsham, A., Morris, E., et al. (2011). Integrating cancer survivors’ experiences into UK cancer registries: Design and development of the ePOCS system (electronic Patient-reported Outcomes from Cancer Survivors). British Journal of Cancer, 105, S74–S81. doi: 10.1038/Bjc.2011.424.PubMedCentralPubMedCrossRefGoogle Scholar
  15. 15.
    Banerjee, A. K., & Ingate, S. (2012). Web-based patient-reported outcomes in drug safety and risk management: Challenges and opportunities? Drug Safety, 35(6), 437–446. doi: 10.2165/11632390-000000000-00000.PubMedCrossRefGoogle Scholar
  16. 16.
    Boers-Doets, C. B., Gelderblom, H., Lacouture, M. E., Epstein, J. B., Nortier, J. W., & Kaptein, A. A. (2013). Experiences with the FACT-EGFRI-18 instrument in EGFRI-associated mucocutaneous adverse events. Supportive Care in Cancer,. doi: 10.1007/s00520-013-1752-4.Google Scholar
  17. 17.
    Chan, A., & Tan, E. H. (2011). How well does the MESTT correlate with CTCAE scale for the grading of dermatological toxicities associated with oral tyrosine kinase inhibitors? Supportive Care in Cancer, 19(10), 1667–1674. doi: 10.1007/s00520-010-0999-2.PubMedCrossRefGoogle Scholar
  18. 18.
    Farnell, D. J., Mandall, P., Anandadas, C., Routledge, J., Burns, M. P., Logue, J. P., et al. (2010). Development of a patient-reported questionnaire for collecting toxicity data following prostate brachytherapy. Radiotherapy and Oncology, 97(1), 136–142. doi: 10.1016/j.radonc.2010.05.011.PubMedCrossRefGoogle Scholar
  19. 19.
    Quinten, C., Maringwa, J., Gotay, C. C., Martinelli, F., Coens, C., Reeve, B. B., et al. (2011). Patient self-reports of symptoms and clinician ratings as predictors of overall cancer survival. Journal of the National Cancer Institute, 103(24), 1851–1858. doi: 10.1093/Jnci/Djr485.PubMedCrossRefGoogle Scholar
  20. 20.
    Stacey, D., Bakker, D., Ballantyne, B., Chapman, K., Cumminger, J., Green, E., et al. (2012). Managing symptoms during cancer treatments: evaluating the implementation of evidence-informed remote support protocols. Implementation Science, 7, 110. doi: 10.1186/1748-5908-7-110.PubMedCrossRefGoogle Scholar
  21. 21.
    Johansen, M. A., Henriksen, E., Horsch, A., Schuster, T., & Berntsen, G. K. R. (2012). Electronic symptom reporting between patient and provider for improved health care service quality: A systematic review of randomized controlled trials. Part 1: State of the Art. Journal of Medical Internet Research, 14(5). doi: 10.2196/jmir.2214.
  22. 22.
    Johansen, M. A., Henriksen, E., Berntsen, G., & Horsch, A. (2011). Electronic symptom reporting by patients: A literature review. Studies in Health Technology and Informatics, 169, 13–17.PubMedGoogle Scholar
  23. 23.
    Seow, H., King, S., Green, E., Pereira, J., & Sawka, C. (2011). Perspectives of patients on the utility of electronic patient-reported outcomes on cancer care. Journal of Clinical Oncology, 29(31), 4213–4214. doi: 10.1200/JCO.2011.37.9750.PubMedCrossRefGoogle Scholar
  24. 24.
    King, A., Daniels, J., Lim, J., Cochrane, D. D., Taylor, A., & Ansermino, J. M. (2010). Time to listen: A review of methods to solicit patient reports of adverse events. Quality and Safety in Health Care, 19(2), 148–157. doi: 10.1136/qshc.2008.030114.PubMedCrossRefGoogle Scholar
  25. 25.
    Zhu, J., Stuver, S. O., Epstein, A. M., Schneider, E. C., Weissman, J. S., & Weingart, S. N. (2011). Can we rely on patients’ reports of adverse events? Medical Care, 49(10), 948–955. doi: 10.1097/MLR.0b013e31822047a8.PubMedCrossRefGoogle Scholar
  26. 26.
    Detmar, S. B., Muller, M. J., Schornagel, J. H., Wever, L. D., & Aaronson, N. K. (2002). Health-related quality-of-life assessments and patient-physician communication: A randomized controlled trial. JAMA, 288(23), 3027–3034.PubMedCrossRefGoogle Scholar
  27. 27.
    Basch, E. M., Reeve, B. B., Mitchell, S. A., Clauser, S. B., Minasian, L., Sit, L., et al. (2011). Electronic toxicity monitoring and patient-reported outcomes. Cancer Journal, 17(4), 231–234. doi: 10.1097/PPO.0b013e31822c28b3.CrossRefGoogle Scholar
  28. 28.
    Stull, D. E., Leidy, N. K., Parasuraman, B., & Chassany, O. (2009). Optimal recall periods for patient-reported outcomes: Challenges and potential solutions. Current Medical Research and Opinion, 25(4), 929–942. doi: 10.1185/03007990902774765.PubMedCrossRefGoogle Scholar
  29. 29.
    Kirkova, J., Davis, M. P., Walsh, D., Tiernan, E., O’Leary, N., LeGrand, S. B., et al. (2006). Cancer symptom assessment instruments: A systematic review. Journal of Clinical Oncology, 24(9), 1459–1473. doi: 10.1200/JCO.2005.02.8332.PubMedCrossRefGoogle Scholar
  30. 30.
    Wild, D., Eremenco, S., Mear, I., Martin, M., Houchin, C., Gawlicki, M., et al. (2009). Multinational trials-recommendations on the translations required, approaches to using the same language in different countries, and the approaches to support pooling the data: The ISPOR patient-reported outcomes translation and linguistic validation good research practices task force report. Value in Health, 12(4), 430–440. doi: 10.1111/j.1524-4733.2008.00471.x.PubMedCrossRefGoogle Scholar
  31. 31.
    Basch, E., Reeve, B. B., Mitchell, S. A., Clauser, S. B., Minasian, L. M., Atkinson, T. M., et al. (Under Review). Development of the National Cancer Institute’s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). Journal of National Cancer Institute.Google Scholar
  32. 32.
    Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks, CA: SAGE Publications, Inc.Google Scholar
  33. 33.
    Patrick, D. L., Burke, L. B., Gwaltney, C. J., Leidy, N. K., Martin, M. L., Molsen, E., et al. (2011). Content validity-establishing and reporting the evidence in newly Developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: Part 2-assessing respondent understanding. Value in Health, 14(8), 978–988. doi: 10.1016/j.jval.2011.06.013.PubMedCrossRefGoogle Scholar
  34. 34.
    Bowen, G. A. (2008). Naturalistic inquiry and the saturation concept: A research note. Qualitative Research, 8(1), 137–152. doi: 10.1177/1468794107085301.CrossRefGoogle Scholar
  35. 35.
    Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? An experiment with data saturation and variability. Field Methods, 18, 59–82. doi: 10.1177/1525822X05279903.CrossRefGoogle Scholar
  36. 36.
    Jobe, J. B., & Mingay, D. J. (1989). Cognitive research improves questionnaires. American Journal of Public Health, 79(8), 1053–1055.PubMedCrossRefGoogle Scholar
  37. 37.
    Jobe, J. B. (2003). Cognitive psychology and self-reports: Models and methods. Quality of Life Research, 12(3), 219–227.PubMedCrossRefGoogle Scholar
  38. 38.
    Miller, K. (2003). Conducting cognitive interviews to understand question-response limitations. American Journal of Health Behavior, 27(Suppl 3), S264–S272.PubMedCrossRefGoogle Scholar
  39. 39.
    Schwarz, N., Oyserman, D., & Petycheva, E. (2010). Cognition, communication, and culture: Implications for the survey response process. In J. A. Harkness, M. Braun, B. Edwards, T. P. Johnson, L. E. Lyberg, P. P. Mohler, et al. (Eds.), Survey methods in multinational, multicultural, and multiregional contexts (pp. 177–190). Hoboken, NJ: John Wiley & Sons.Google Scholar
  40. 40.
    Tourangeau, R. (1984). Cognitive science and survey methods. In T. Jabine, M. Straf, J. Tanur, & R. Tourangeau (Eds.), Cognitive aspects of survey design: Building a bridge between disciplines (pp. 73–100). Washington, DC: National Academy Press.Google Scholar
  41. 41.
    Namey, E., Guest, G., Thairy, L., & Johnson, L. (2008). Data reduction techniques for large qualitative data sets. In G. Guest & K. M. MacQueen (Eds.), Handbook for team-based qualitative research (pp. 137–161). Lanham, MD: AltaMira Press.Google Scholar
  42. 42.
    Saldana, J. (2009). The coding manual for qualitative researchers. Los Angeles, CA: Sage Publications.Google Scholar
  43. 43.
    Basch, E. (2012). Beyond the FDA PRO guidance: Steps toward integrating meaningful patient-reported outcomes into regulatory trials and US drug labels. Value in Health, 15(3), 401–403. doi: 10.1016/j.jval.2012.03.1385.PubMedCrossRefGoogle Scholar
  44. 44.
    Symptom Management and Quality of Life Steering Committee. Clinical trials planning meeting—building bridges: The identification of core symptoms and health-related quality of life domains for use in cancer research (September 22–23, 2011. Available from: http://transformingtrials.cancer.gov/files/SxQOLSCPROCTPMExecutive%20Summary_FINAL.pdf Accessed 7 June 2012).
  45. 45.
    Mitchell, S. A., Lang, K., Nichols, C., Clauser, S. B., Federico, V., Lalla, D., et al. (2012). Validation of the NCI Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) in women receiving treatment for metastatic breast cancer. Chicago, IL: Annual Meeting of the American Society of Clinical Oncology.Google Scholar
  46. 46.
    Dueck, A. C., Mendoza, T., Mitchell, S. A., Reeve, B. B., Castro, K., Denicoff, A., et al. (2012). Validity and reliability of the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). Chicago, IL: Annual Meeting of the American Society of Clinical Oncology.Google Scholar
  47. 47.
    Coons, S. J., Gwaltney, C. J., Hays, R. D., Lundy, J. J., Sloan, J. A., Revicki, D. A., et al. (2009). Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO good research practices task force report. Value in Health, 12(4), 419–429. doi: 10.1111/j.1524-4733.2008.00470.x.PubMedCrossRefGoogle Scholar
  48. 48.
    (2012) Methodological standards and patient-centeredness in comparative effectiveness research—The PCORI perspective. JAMA, 307(15), 1636–1640. doi: 10.1001/jama.2012.466.

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  • Jennifer L. Hay
    • 1
  • Thomas M. Atkinson
    • 1
  • Bryce B. Reeve
    • 2
  • Sandra A. Mitchell
    • 3
  • Tito R. Mendoza
    • 4
  • Gordon Willis
    • 3
  • Lori M. Minasian
    • 3
  • Steven B. Clauser
    • 3
  • Andrea Denicoff
    • 3
  • Ann O’Mara
    • 3
  • Alice Chen
    • 3
  • Antonia V. Bennett
    • 2
  • Diane B. Paul
    • 6
  • Joshua Gagne
    • 5
  • Lauren Rogak
    • 1
  • Laura Sit
    • 1
  • Vish Viswanath
    • 5
  • Deborah Schrag
    • 5
  • Ethan Basch
    • 1
    • 2
  • the NCI PRO-CTCAE Study Group
  1. 1.Department of Psychiatry and Behavioral SciencesMemorial Sloan-Kettering Cancer CenterNew YorkUSA
  2. 2.University of North Carolina at Chapel HillChapel HillUSA
  3. 3.United States National Cancer InstituteRockvilleUSA
  4. 4.University of Texas M.D. Anderson Cancer CenterHoustonUSA
  5. 5.Dana-Farber Cancer InstituteBostonUSA
  6. 6.New YorkUSA

Personalised recommendations