Skip to main content

Advertisement

SpringerLink
Log in

Establishing clinically-relevant terms and severity thresholds for Patient-Reported Outcomes Measurement Information System® (PROMIS®) measures of physical function, cognitive function, and sleep disturbance in people with cancer using standard setting

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

Patient-Reported Outcomes Measurement Information System® (PROMIS®) physical function, cognitive function, and sleep disturbance measures are increasingly used in cancer care. However, there is limited guidance for interpreting the clinical meaning of scores. This study aimed to apply bookmarking, a standard setting methodology, to identify PROMIS score thresholds in the context of cancer care.

Methods

Using item parameters, we constructed vignettes of five items covering the range of possible scores. Focus groups were held with cancer care providers and people with cancer. Terminology for categorizing levels of severity was explored. Participants rank ordered vignettes by severity and then placed bookmarks between vignettes representing different levels of severity. Group discussion was held until consensus on bookmark placement was reached.

Results

Clinicians selected “within normal limits,” “mild,” “moderate,” and “severe” to describe levels of severity. Both patients and clinicians were able to apply these labels, but there was not unanimous support for any set of descriptors. Clinicians and patients agreed on all severity thresholds for sleep disturbance. For cognitive and physical function, clinicians and patients agreed on the threshold between “within normal limits” and “mild.” However, patients required greater dysfunction than clinicians before applying “moderate” and “severe” labels.

Conclusions

Bookmarking can be applied to develop provisional score interpretation for PROMIS measures. Patients and clinicians were frequently consistent in their bookmark placement. When there was variance, patients required more dysfunction before assigning more severity. Additional research with other cancer samples is needed to evaluate the replicability and generalizability of our findings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Chen, J., Ou, L., & Hollis, S. J. (2013). A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Services Research,13, 211–211.

    Article  Google Scholar 

  2. Howell, D., Molloy, S., Wilkinson, K., Green, E., Orchard, K., Wang, K., et al. (2015). Patient-reported outcomes in routine cancer clinical practice: A scoping review of use, impact on health outcomes, and implementation factors. Annals of Oncology,26(9), 1846–1858.

    Article  CAS  Google Scholar 

  3. Wagner, L. I., Schink, J., Bass, M., Patel, S., Diaz, M. V., Rothrock, N., et al. (2015). Bringing PROMIS to practice: Brief and precise symptom screening in ambulatory cancer care. Cancer,121(6), 927–934.

    Article  Google Scholar 

  4. Seneviratne, M. G., Bozkurt, S., Patel, M. I., Seto, T., Brooks, J. D., Blayney, D. W., et al. (2019). Distribution of global health measures from routinely collected PROMIS surveys in patients with breast cancer or prostate cancer. Cancer,125(6), 943–951.

    Article  Google Scholar 

  5. Lohr, K. N., Aaronson, N. K., Alonso, J., Audrey Burnam, M., Patrick, D. L., Perrin, E. B., et al. (1996). Evaluating quality-of-life and health status instruments: Development of scientific review criteria. Clinical Therapeutics,18(5), 979–992.

    Article  CAS  Google Scholar 

  6. Snyder, C. F., Smith, K. C., Bantug, E. T., Tolbert, E. E., Blackford, A. L., Brundage, M. D., et al. (2017). What do these scores mean? Presenting patient-reported outcomes data to patients and clinicians to improve interpretability. Cancer,123(10), 1848–1859.

    Article  Google Scholar 

  7. Cappelleri, J. C., & Bushmakin, A. G. (2014). Interpretation of patient-reported outcomes. Statistical Methods in Medical Research,23(5), 460–483.

    Article  Google Scholar 

  8. Given, B., Given, C. W., Sikorskii, A., Jeon, S., McCorkle, R., Champion, V., et al. (2008). Establishing mild, moderate, and severe scores for cancer-related symptoms: How consistent and clinically meaningful are interference-based severity cut-points? Journal of Pain and Symptom Management,35(2), 126–135.

    Article  Google Scholar 

  9. Palos, G. R., Mendoza, T. R., Mobley, G. M., Cantor, S. B., & Cleeland, C. S. (2006). Asking the community about cutpoints used to describe mild, moderate, and severe pain. The Journal of Pain,7(1), 49–56.

    Article  Google Scholar 

  10. Coon, C. D., & Cook, K. F. (2018). Moving from significance to real-world meaning: Methods for interpreting change in clinical outcome assessment scores. Quality of Life Research,27(1), 33–40.

    Article  Google Scholar 

  11. Basch, E., Reeve, B. B., Mitchell, S. A., Clauser, S. B., Minasian, L. M., Dueck, A. C., et al. (2014). Development of the National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). JNCI: Journal of the National Cancer Institute,106(9), 244.

    Article  Google Scholar 

  12. Cizek, G. J. (1993). Reconsidering standards and criteria. Journal of Educational Measurement,30(2), 93–106.

    Article  Google Scholar 

  13. Perie, M. (2005). Angoff and Bookmark methods. Workshop presented at the Annual Meeting of the National Council on Measurement in Education, Montreal, Canada.

  14. Cella, D., Choi, S., Garcia, S., Cook, K. F., Rosenbloom, S., Lai, J.-S., et al. (2014). Setting standards for severity of common symptoms in oncology using the PROMIS item banks and expert judgment. Quality of Life Research,23(10), 2651–2661.

    Article  Google Scholar 

  15. Cook, K. F., Victorson, D. E., Cella, D., Schalet, B. D., & Miller, D. (2015). Creating meaningful cut-scores for Neuro-QOL measures of fatigue, physical functioning, and sleep disturbance using standard setting with patients and providers. Quality of Life Research,24(3), 575–589.

    Article  Google Scholar 

  16. Nagaraja, V., Mara, C., Khanna, P. P., Namas, R., Young, A., Fox, D. A., et al. (2018). Establishing clinical severity for PROMIS® measures in adult patients with rheumatic diseases. Quality of Life Research,27(3), 755–764.

    Article  Google Scholar 

  17. Morgan, E. M., Mara, C. A., Huang, B., Barnett, K., Carle, A. C., Farrell, J. E., et al. (2017). Establishing clinical meaning and defining important differences for Patient-Reported Outcomes Measurement Information System (PROMIS®) measures in juvenile idiopathic arthritis using standard setting with patients, parents, and providers. Quality of Life Research,26(3), 565–586.

    Article  Google Scholar 

  18. Cook, K., Cella, D., & Reeve, B. (2019). PRO-bookmarking to estimate clinical thresholds for patient-reported symptoms and function. Medical Care,57, S13–S17.

    Article  Google Scholar 

  19. Rose, M., Bjorner, J. B., Becker, J., Fries, J., & Ware, J. (2008). Evaluation of a preliminary physical function item bank supported the expected advantages of the Patient-Reported Outcomes Measurement Information System (PROMIS). Journal of Clinical Epidemiology,61(1), 17–33.

    Article  CAS  Google Scholar 

  20. Rose, M., Bjorner, J. B., Gandek, B., Bruce, B., Fries, J. F., & Ware, J. E. (2014). The PROMIS Physical Function item bank was calibrated to a standardized metric and shown to improve measurement efficiency. Journal of Clinical Epidemiology,67(5), 516–526.

    Article  Google Scholar 

  21. Lai, J. S., Wagner, L. I., Jacobsen, P. B., & Cella, D. (2014). Self-reported cognitive concerns and abilities: two sides of one coin? Psycho-Oncology,23(10), 1133–1141.

    Article  Google Scholar 

  22. Buysse, D. J., Yu, L., Moul, D. E., Germain, A., Stover, A., Dodds, N. E., et al. (2010). Development and validation of patient-reported outcome measures for sleep disturbance and sleep-related impairments. Sleep,33(6), 781–792.

    Article  Google Scholar 

  23. Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve, B., Yount, S., et al. (2010). The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. Journal of Clinical Epidemiology,63(11), 1179–1194.

    Article  Google Scholar 

  24. Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., Teresi, J. A., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical Care,45(5), S22–S31.

    Article  Google Scholar 

  25. R Development Core Team. (2008). R: A Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing.

    Google Scholar 

Download references

Acknowledgements

This study was funded by the National Institutes of Health Grant U2C CA186878.

Author information

Authors and Affiliations

  1. Department of Medical Social Sciences, Feinberg School of Medicine of Northwestern University, Chicago, IL, USA

    Nan E. Rothrock, Karon F. Cook, Mary O’Connor, David Cella & Susan E. Yount

  2. Outcomes Research Branch, National Cancer Institute, Bethesda, MD, USA

    Ashley Wilder Smith

Authors
  1. Nan E. Rothrock
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Karon F. Cook
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mary O’Connor
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David Cella
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ashley Wilder Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Susan E. Yount
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nan E. Rothrock.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 28 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rothrock, N.E., Cook, K.F., O’Connor, M. et al. Establishing clinically-relevant terms and severity thresholds for Patient-Reported Outcomes Measurement Information System® (PROMIS®) measures of physical function, cognitive function, and sleep disturbance in people with cancer using standard setting. Qual Life Res 28, 3355–3362 (2019). https://doi.org/10.1007/s11136-019-02261-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-019-02261-2

Keywords

Search

Navigation