Quality of Life Research

, Volume 26, Issue 3, pp 565–586 | Cite as

Establishing clinical meaning and defining important differences for Patient-Reported Outcomes Measurement Information System (PROMIS®) measures in juvenile idiopathic arthritis using standard setting with patients, parents, and providers

  • Esi M. Morgan
  • Constance A. Mara
  • Bin Huang
  • Kimberly Barnett
  • Adam C. Carle
  • Jennifer E. Farrell
  • Karon F. Cook
Article

Abstract

Background

Patient-Reported Outcomes Measurement Information System (PROMIS) measures are used increasingly in clinical care. However, for juvenile idiopathic arthritis (JIA), scores lack a framework for interpretation of clinical severity, and minimally important differences (MID) have not been established, which are necessary to evaluate the importance of change.

Methods

We identified clinical severity thresholds for pediatric PROMIS measures of mobility, upper extremity function (UE), fatigue, and pain interference working with adolescents with JIA, parents of JIA patients, and clinicians, using a standard setting methodology modified from educational testing. Item parameters were used to develop clinical vignettes across a range of symptom severity. Vignettes were ordered by severity, and panelists identified adjacent vignettes considered to represent upper and lower boundaries separating category cut-points (i.e., from none/mild problems to moderate/severe). To define MIDs, panelists reviewed a full score report for the vignettes and indicated which items would need to change and by how much to represent “just enough improvement to make a difference.”

Results

For fatigue and UE, cut-points among panels were within 0.5 SD of each other. For mobility and pain interference, cut-scores among panels were more divergent, with parents setting the lowest cut-scores for increasing severity. The size of MIDs varied by stakeholders (parents estimated largest, followed by patients, then clinicians). MIDs also varied by severity classification of the symptom.

Conclusions

We estimated clinically relevant severity cut-points and MIDs for PROMIS measures for JIA from the perspectives of multiple stakeholders and found notable differences in perspectives.

Keywords

PROMIS Patient-reported outcomes Item response theory (IRT) Psychometric methods Juvenile idiopathic arthritis 

Notes

Acknowledgements

We are grateful for the contributions of the study participants, patients, parents, and clinicians who shared their time and insights to advance this work. We acknowledge Dr. Ryoungsun Park http://coe.wayne.edu/profile/fy3504 who developed the R-code for this work as part of a consultancy agreement with Northwestern University.

Funding

The Patient-Reported Outcomes Measurement Information System (PROMIS) is an NIH Roadmap initiative to develop a computerized system measuring PROs in respondents with a wide range of chronic diseases and demographic characteristics. PROMIS II was funded by cooperative agreements with a Statistical Center (Northwestern University, PI: David Cella, Ph.D., 1U54AR057951), a Technology Center (Northwestern University, PI: Richard C. Gershon, Ph.D., 1U54AR057943), a Network Center (American Institutes for Research, PI: Susan (San) D. Keller, Ph.D., 1U54AR057926) and thirteen Primary Research Sites which may include more than one institution (State University of New York, Stony Brook, PIs: Joan E. Broderick, Ph.D. and Arthur A. Stone, Ph.D., 1U01AR057948; University of Washington, Seattle, PIs: Heidi M. Crane, MD, MPH, Paul K. Crane, MD, MPH, and Donald L. Patrick, Ph.D., 1U01AR057954; University of Washington, Seattle, PIs: Dagmar Amtmann, Ph.D. and Karon Cook, Ph.D., 1U01AR052171; University of North Carolina, Chapel Hill, PI: Darren A. DeWalt, MD, MPH, 2U01AR052181; Children’s Hospital of Philadelphia, PI: Christopher B. Forrest, MD, Ph.D., 1U01AR057956; Stanford University, PI: James F. Fries, MD, 2U01AR052158; Boston University, PIs: Stephen M. Haley, Ph.D. and David Scott Tulsky, Ph.D. (University of Michigan, Ann Arbor), 1U01AR057929; University of California, Los Angeles, PIs: Dinesh Khanna, MD and Brennan Spiegel, MD, MSHS, 1U01AR057936; University of Pittsburgh, PI: Paul A. Pilkonis, Ph.D., 2U01AR052155; Georgetown University, PIs: Carol. M. Moinpour, Ph.D. (Fred Hutchinson Cancer Research Center, Seattle) and Arnold L. Potosky, Ph.D., U01AR057971; Children’s Hospital Medical Center, Cincinnati, PI: Esi M. Morgan DeWitt, MD, MSCE, 17 1U01AR057940; University of Maryland, Baltimore, PI: Lisa M. Shulman, MD, 1U01AR057967; and Duke University, PI: Kevin P. Weinfurt, Ph.D., 2U01AR052186). NIH Science Officers on this project have included Deborah Ader, Ph.D., Vanessa Ameen, MD, Susan Czajkowski, Ph.D., Basil Eldadah, MD, Ph.D., Lawrence Fine, MD, DrPH, Lawrence Fox, MD, Ph.D., Lynne Haverkos, MD, MPH, Thomas Hilton, Ph.D., Laura Lee Johnson, Ph.D., Michael Kozak, Ph.D., Peter Lyster, Ph.D., Donald Mattison, MD, Claudia Moy, Ph.D., Louis Quatrano, Ph.D., Bryce Reeve, Ph.D., William Riley, Ph.D., Ashley Wilder Smith, Ph.D., MPH, Susana Serrate-Sztein, MD, Ellen Werner, Ph.D. and James Witter, MD, Ph.D.

References

  1. 1.
    Barth, S., et al. (2016). Long-term health-related quality of life in German patients with juvenile idiopathic arthritis in comparison to German general population. PLoS ONE, 11(4), e0153267.CrossRefPubMedPubMedCentralGoogle Scholar
  2. 2.
    Armbrust, W., et al. (2016). Fatigue in patients with juvenile idiopathic arthritis: A systematic review of the literature. Seminars in Arthritis and Rheumatism, 45(5), 587–595.CrossRefPubMedGoogle Scholar
  3. 3.
    Hoeksma, A. F., et al. (2014). High prevalence of hand- and wrist-related symptoms, impairments, activity limitations and participation restrictions in children with juvenile idiopathic arthritis. Journal of Rehabilitation Medicine, 46(10), 991–996.CrossRefPubMedGoogle Scholar
  4. 4.
    Moorthy, L. N., et al. (2010). Burden of childhood-onset arthritis. Pediatric Rheumatology, 8, 20.CrossRefPubMedPubMedCentralGoogle Scholar
  5. 5.
    Broderick, J., et al. (2013). Advances in patient reported outcomes: The NIH PROMIS measures. eGEMs. doi: 10.13063/2327-9214.1015.PubMedPubMedCentralGoogle Scholar
  6. 6.
    Jensen, R. E., et al. (2015). The role of technical advances in the adoption and integration of patient-reported outcomes in clinical care. Medical Care, 53(2), 153–159.CrossRefPubMedPubMedCentralGoogle Scholar
  7. 7.
    Reeve, B. B., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical Care, 45(5 Suppl 1), S22–S31.CrossRefPubMedGoogle Scholar
  8. 8.
    Witter, J. P. (2016). The promise of patient-reported outcomes measurement information system-turning theory into reality: A uniform approach to patient-reported outcomes across rheumatic diseases. Rheumatic Diseases Clinics of North America, 42(2), 377–394.CrossRefPubMedGoogle Scholar
  9. 9.
    Irwin, D. E., et al. (2010). An item response analysis of the pediatric PROMIS anxiety and depressive symptoms scales. Quality of Life Research, 19(4), 595–607.CrossRefPubMedPubMedCentralGoogle Scholar
  10. 10.
    Varni, J. W., et al. (2010). PROMIS Pediatric Pain Interference Scale: An item response theory analysis of the pediatric pain item bank. J Pain, 11(11), 1109–1119.CrossRefPubMedPubMedCentralGoogle Scholar
  11. 11.
    Irwin, D. E., et al. (2012). PROMIS Pediatric Anger Scale: An item response theory analysis. Quality of Life Research, 21(4), 697–706.CrossRefPubMedGoogle Scholar
  12. 12.
    DeWitt, E. M., et al. (2011). Construction of the eight-item patient-reported outcomes measurement information system pediatric physical function scales: Built using item response theory. Journal of Clinical Epidemiology, 64(7), 794–804.CrossRefPubMedPubMedCentralGoogle Scholar
  13. 13.
    Dewalt, D. A., et al. (2013). PROMIS Pediatric Peer Relationships Scale: Development of a peer relationships item bank as part of social health measurement. Health Psychology, 32(10), 1093–1103.CrossRefPubMedGoogle Scholar
  14. 14.
    Varni, J. W., et al. (2014). PROMIS® Parent Proxy Report Scales for children ages 5–7 years: An item response theory analysis of differential item functioning across age groups. Quality of Life Research, 23(1), 349–361.CrossRefPubMedGoogle Scholar
  15. 15.
    Lai, J. S., et al. (2013). Development and psychometric properties of the PROMIS® pediatric fatigue item banks. Quality of Life Research, 22(9), 2417–2427.CrossRefPubMedGoogle Scholar
  16. 16.
    Guyatt, G. H., et al. (2002). Methods to explain the clinical significance of health status measures. Mayo Clinic Proceedings, 77(4), 371–383.CrossRefPubMedGoogle Scholar
  17. 17.
    Guyatt, G., Walter, S., & Norman, G. (1987). Measuring change over time: Assessing the usefulness of evaluative instruments. Journal of Chronic Diseases, 40(2), 171–178.CrossRefPubMedGoogle Scholar
  18. 18.
    Cook, K. F., et al. (2015). Creating meaningful cut-scores for Neuro-QOL measures of fatigue, physical functioning, and sleep disturbance using standard setting with patients and providers. Quality of Life Research, 24(3), 575–589.CrossRefPubMedGoogle Scholar
  19. 19.
    Zieky, M. J., Perie, M., & Livingston, S. A. (2008). Cutscores: A manual for setting standards of performance on educational and occupational tests. Princeton, NJ.: Educational Testing Service.Google Scholar
  20. 20.
    Cella, D., et al. (2014). Setting standards for severity of common symptoms in oncology using the PROMIS item banks and expert judgment. Quality of Life Research, 23(10), 2651–2661.CrossRefPubMedPubMedCentralGoogle Scholar
  21. 21.
    Huang, B., et al. (2012). ACR criteria, providers’ global rating of change and role of patient self-report in evaluating change in disease over time: A patient reported outcomes measurement information system study. In Arthritis & Rheumatism (Vol. 64, No. 10 (Supplement)).Google Scholar
  22. 22.
    Jacobson, C. J., Jr., et al. (2015). Qualitative evaluation of pediatric pain behavior, quality, and intensity item candidates and the PROMIS pain domain framework in children with chronic pain. The Journal of Pain, 16(12), 1243–1255.CrossRefPubMedPubMedCentralGoogle Scholar
  23. 23.
    Choi, S. W. (2009). Firestar: Computerized adaptive testing simulation program for polytomous item response theory models. Applied Psychological Measurement, 33(8), 644–645.CrossRefGoogle Scholar
  24. 24.
    Radowsky, J. S., et al. (2012). Pain ratings by patients and their providers of radionucleotide injection for breast cancer lymphatic mapping. Pain Medicine, 13(5), 670–676.CrossRefPubMedGoogle Scholar
  25. 25.
    Basch, E., et al. (2006). Patient versus clinician symptom reporting using the National Cancer Institute Common Terminology Criteria for Adverse Events: Results of a questionnaire-based study. The Lancet Oncology, 7(11), 903–909.CrossRefPubMedGoogle Scholar
  26. 26.
    Brossart, D. F., Clay, D. L., & Willson, V. L. (2002). Methodological and statistical considerations for threats to internal validity in pediatric outcome data: Response shift in self-report outcomes. Journal of Pediatric Psychology, 27(1), 97–107.CrossRefPubMedGoogle Scholar
  27. 27.
    Varni, J. W., et al. (2015). Item-level informant discrepancies between children and their parents on the PROMIS® pediatric scales. Quality of Life Research, 24(8), 1921–1937.CrossRefPubMedPubMedCentralGoogle Scholar
  28. 28.
    Lal, S. D., et al. (2011). Agreement between proxy and adolescent assessment of disability, pain, and well-being in juvenile idiopathic arthritis. Journal of Pediatrics, 158(2), 307–312.CrossRefPubMedPubMedCentralGoogle Scholar
  29. 29.
    Lipstein, E. A., et al. (2013). “I’m the one taking it”: Adolescent participation in chronic disease treatment decisions. Journal of Adolescent Health, 53(2), 253–259.CrossRefPubMedGoogle Scholar
  30. 30.
    Guzman, J., et al. (2014). What matters most for patients, parents, and clinicians in the course of juvenile idiopathic arthritis? A qualitative study. The Journal of Rheumatology, 41(11), 2260–2269.CrossRefPubMedGoogle Scholar
  31. 31.
    Wolfe, F., Michaud, K., & Strand, V. (2005). Expanding the definition of clinical differences: From minimally clinically important differences to really important differences. Analyses in 8931 patients with rheumatoid arthritis. Journal of Rheumatology, 32(4), 583–589.PubMedGoogle Scholar
  32. 32.
    Batalden, M., et al. (2016). Coproduction of healthcare service. BMJ Quality and Safety, 25(7), 509–517.CrossRefPubMedGoogle Scholar
  33. 33.
    Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59(1), 12–19.CrossRefPubMedGoogle Scholar
  34. 34.
    Thissen, D., et al. (2016). Estimating minimally important difference (MID) in PROMIS pediatric measures using the scale-judgment method. Quality of Life Research, 25(1), 13–23.CrossRefPubMedGoogle Scholar
  35. 35.
    Brunner, H. I., et al. (2005). Minimal clinically important differences of the childhood health assessment questionnaire. Journal of Rheumatology, 32(1), 150–161.PubMedGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Esi M. Morgan
    • 1
    • 2
    • 4
  • Constance A. Mara
    • 1
    • 3
  • Bin Huang
    • 1
    • 5
  • Kimberly Barnett
    • 2
  • Adam C. Carle
    • 1
    • 7
  • Jennifer E. Farrell
    • 2
  • Karon F. Cook
    • 6
  1. 1.Department of PediatricsUniversity of Cincinnati College of MedicineCincinnatiUSA
  2. 2.Division of RheumatologyCincinnati Children’s Hospital Medical CenterCincinnatiUSA
  3. 3.Behavioral Medicine and Clinical PsychologyCincinnati Children’s Hospital Medical CenterCincinnatiUSA
  4. 4.James M. Anderson Center for Health Systems ExcellenceCincinnati Children’s Hospital Medical CenterCincinnatiUSA
  5. 5.Division of Biostatistics and EpidemiologyCincinnati Children’s Hospital Medical CenterCincinnatiUSA
  6. 6.Department of Medical Social SciencesNorthwestern UniversityChicagoUSA
  7. 7.Department of PsychologyUniversity of Cincinnati College of Arts and SciencesCincinnatiUSA

Personalised recommendations