Setting standards for severity of common symptoms in oncology using the PROMIS item banks and expert judgment
- 839 Downloads
Although the use of patient-reported outcome measures (PROs) has increased markedly, clinical interpretation of scores remains lacking. We developed a method to identify clinical severity thresholds for pain, fatigue, depression, and anxiety in people with cancer.
Using available Patient-Reported Outcomes Measurement Information System (PROMIS) item bank response data collected on 840 cancer patients, symptom vignettes across a range of symptom severity were developed and placed on index cards. Cards represented symptom severity at five-point intervals differences on the T score metric [mean = 50; standard deviation (SD) = 10]. Symptom vignettes for each symptom were anchored on these standardized scores at 0.5 SD increments across the full range of severity. Clinical experts, blind to the PROMIS score associated with each vignette, rank-ordered the vignettes by severity, then arrived at consensus regarding which two vignettes were at the upper and lower boundaries of normal and mildly symptomatic for each symptom. The procedure was repeated to identify cut scores separating mildly from moderately symptomatic, and moderately from severely symptomatic scores. Clinician severity rankings were then compared to the T scores upon which the vignettes were based.
For each of the targeted PROs, the severity rankings reached by clinician consensus perfectly matched the numerical rankings of their associated T scores. Across all symptoms, the thresholds (cut scores) identified to differentiate normal from mildly symptomatic were near a T score of 50. Cut scores differentiating mildly from moderately symptomatic were at or near 60, and those separating moderately from severely symptomatic were at or near 70.
The study results provide empirically generated PROMIS T score thresholds that differentiate levels of symptom severity for pain interference, fatigue, anxiety, and depression. The convergence of clinical judgment with self-reported patient severity scores supports the validity of this methodology to derive clinically relevant symptom severity levels for PROMIS symptom measures in other settings.
KeywordsPROMIS Patient-reported outcomes Symptom severity levels Standard setting Cancer
Several colleagues contributed time and expertise to this effort, and we wish to acknowledge their effort. Clinical expert break-out group facilitators included some of the coauthors and also Nan Rothrock, PhD; and Zeeshan Butt, PhD. Expert raters included some of the coauthors and also Amy Peterman, PhD; Janine Gauthier, PhD; Lynne Wagner, PhD; Kimberly Davis, PhD; Margaret Whalen, RN; Gershon Locker, MD; Carmen Griza, MD; Jin-Shei Lai, PhD, OTR/L; Allen Heinemann, PhD; Lauren Mermel Welles, PT; Robin Mieli, MA, OTR/L; Kimberly Brennan, PT; Nan Rothrock, PhD; Zeeshan Butt, PhD; David Victorson, PhD; Judith Paice, PhD, RN, FAAN; Rose Catchatourian, MD; George Carro, PharmD; Mousami Shah, MD. Study coordination was done by Jacquelyn George and Rachel Hanrahan. Supported by a grant from the National Cancer Institute (R01 CA60068; Cella, PI).
- 3.Bennett, A. V., Jensen, R. E., & Basch, E. (2012). Electronic patient-reported outcome systems in oncology clinical practice. CA: A Cancer Journal for Clinicians, 62(5), 336–347.Google Scholar
- 4.Abernethy, A. P., Herndon Ii, J. E., Wheeler, J. L., Day, J. M., Hood, L., Patwardhan, M., et al. (2009). Feasibility and acceptability to patients of a longitudinal system for evaluating cancer-related symptoms and quality of life: Pilot study of an e/tablet data-collection system in academic oncology. Journal of Pain and Symptom Management, 37(6), 1027–1038.PubMedCrossRefGoogle Scholar
- 5.Berry, D. L., Blumenstein, B. A., Halpenny, B., Wolpin, S., Fann, J. R., Austin-Seymour, M., et al. (2011). Enhancing patient-provider communication with the electronic self-report assessment for cancer: A randomized trial. Journal of Clinical Oncology, 29(8), 1029–1035.PubMedCentralPubMedCrossRefGoogle Scholar
- 8.Cella, D., Yount, S., Rothrock, N., Gershon, R., Cook, K., Reeve, B., et al. (2007). The Patient-Reported Outcomes Measurement Information System (PROMIS): Progress of an NIH Roadmap Cooperative Group during its first two years. Medical Care, 45(5 Suppl 1), S3–S11.PubMedCentralPubMedCrossRefGoogle Scholar
- 9.Riley, W. T., Rothrock, N., Bruce, B., Christodolou, C., Cook, K., Hahn, E. A., et al. (2010). Patient-Reported Outcomes Measurement Information System (PROMIS) domain names and definitions revisions: further evaluation of content validity in IRT-derived item banks. Quality of Life Research, 19(9), 1311–1321.PubMedCentralPubMedCrossRefGoogle Scholar
- 10.Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve, B., Yount, S., et al. (2010). The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. Journal of Clinical Epidemiology, 63(11), 1179–1194.PubMedCentralPubMedCrossRefGoogle Scholar
- 15.Zelman, D., Hoffman, D., Reed, P., Edwards, L., & Dukes, E. (2003). Quantifying a “day of manageable pain control:” The relationship between pain severity and patient daily functional expectations. Journal of Pain, 4(Suppl 1), 1.Google Scholar
- 18.Given, B., Sikorskii, A., Given, C. W., Jeon, S., McCorkle, R., Champion, V., et al. (2008). Establishing mild, moderate, and severe scores for cancer-related symptoms: How consistent and clinically meaningful are interference-based severity cut-points? Journal of Pain and Symptom Management, 35(2), 126–135.PubMedCentralPubMedCrossRefGoogle Scholar
- 20.Angoff, W. H., & Thorndike, R. L. (1971). Scales, norms and equivalent scores. In Educational measurement. Washington, D.C.: American Council on Education.Google Scholar
- 21.Lewis, D., Mitzel, H. C., & Green, D. R. (1996). Standard setting: A bookmark approach. Paper presented at the Council of Chief State School Officers National Conference on Large Scale Assessment Phoenix, AZ.Google Scholar
- 22.Impara, J. C., & Plake, B. S. (1977). Standard setting: An alternative approach. Journal of Educational Measurement, 34, 355–368.Google Scholar
- 23.Baghaei, P. (2007). Applying The Rasch Rating-Scale model to set multiple cut-offs. Rasch Measurement Transactions, 20(4), 1075–1076.Google Scholar
- 24.Lai, J. S., Cella, D., Choi, S. W., Junghaenel, D. U., Christodolou, C., Gershon, R., et al. (2011). How item banks and their application can influence measurement practice in rehabilitation medicine: A PROMIS fatigue item bank example. Archives of Physical Medicine and Rehabilitation, 92(10 Supplement), S20–S27.PubMedCentralPubMedCrossRefGoogle Scholar
- 26.Pilkonis, P. A., Choi, S. W., Reise, S. P., Stover, A. M., Riley, W. T., & Cella, D. (2011). Item banks for measuring emotional distress from the Patient-Reported Outcomes Measurement Information System (PROMIS):depression, anxiety, and anger. Assessment, 18(3), 263–283.PubMedCentralPubMedCrossRefGoogle Scholar
- 27.Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., Teresi, J. A., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical Care, 45(5 Suppl 1), S22–S31.PubMedCrossRefGoogle Scholar
- 28.King, M. T., Cella, D., Osoba, D., Stockler, M., Eton, D., Thompson, J., et al. (2010). Meta-analysis provides evidence-based interpretation guidelines for the clinical significance of mean differences for the FACT-G, a cancer-specific quality of life questionnaire. Patient Related Outcome Measures, 1(1), 119–126.PubMedCentralPubMedCrossRefGoogle Scholar