Skip to main content
SpringerLink
Log in

Idio Scale Judgment: evaluation of a new method for estimating responder thresholds

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

To pilot the newly developed Idio Scale Judgment method for estimating the amount of score change that matters to patients (i.e., change thresholds).

Methods

An online panel of 500 participants diagnosed with multiple sclerosis (MS) responded to the Neuro-QoL fatigue scale and to demographic and clinical questions. Participants compared their own fatigue to that represented by seven short form summaries (SFSs) that were located relatively close to their own fatigue levels. They judged these as representing the same, greater, or less fatigue than their own. We calculated the distances between patients’ own levels of fatigue and the location of SFSs they endorsed as a change that would make a difference in daily life. These distances were used as estimates of change thresholds. Logically inconsistent judgments were tabulated and associations with clinical and demographic variables were estimated.

Results

Change thresholds based on mean individual thresholds for consequential change were calculated for improvement (−3.5) and worsening (3.2). The majority of participants had no logically inconsistent judgments (69%). Having one or more reversals in judgment was not significantly associated with education or fatigue score, but was weakly associated with age, gender, and MS type and moderately associated with ratings of confidence in SFS comparisons.

Conclusions

As piloted, Idio Scale Judgment had a number of design strengths. Participants made comparisons to levels of fatigue that were within range of their own, and their judgments were contextualized in personally relevant consequences. The design lends itself to collection of data in large samples allowing evaluation of the range of judgments. Some study limitations could be mitigated with modifications. We concluded that the Idio Scale Judgment has substantial promise as a new tool for estimating change thresholds.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. What is comparative effectiveness research. https://effectivehealthcare.ahrq.gov/index.cfm/what-is-comparative-effectiveness-research1/

  2. Broderick, J. E., DeWitt, E. M., Rothrock, N., Crane, P. K., & Forrest, C. B. (2013). Advances in patient-reported outcomes: The NIH PROMIS((R)) measures. EGEMS, 1, 1015.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Khullar, O. V., Rajaei, M. H., Force, S. D., Binongo, J. N., Lasanajak, Y., Robertson, S., Pickens, A., Sancheti, M. S., Lipscomb, J., Gillespie, T. W., Fernandez, F. G. (2017). Pilot study to integrate patient reported outcomes after lung cancer operations into the Society of Thoracic Surgeons Database. The Annals of Thoracic Surgery.

  4. Betts, K. A., Griffith, J., Friedman, A., Zhou, Z. Y., Signorovitch, J. E., & Ganguli, A. (2016). An indirect comparison and cost per responder analysis of adalimumab, methotrexate and apremilast in the treatment of methotrexate-naive patients with psoriatic arthritis. Current Medical Research and Opinion, 32, 721–729.

    Article  CAS  PubMed  Google Scholar 

  5. Brasure, M., Fuchs, E., MacDonald, R., Nelson, V. A., Koffel, E., Olson, C. M., et al. (2016). Psychological and behavioral interventions for managing insomnia disorder: An evidence report for a Clinical Practice Guideline by the American College of Physicians. Annals of Internal Medicine, 165, 113–124.

    Article  PubMed  Google Scholar 

  6. Fehnel, S. E., Ervin, C. M., Carson, R. T., Rigoni, G., Lackner, J. M., Coons, S. J., et al. (2017). Development of the diary for irritable bowel syndrome symptoms to assess treatment benefit in clinical trials: Foundational Qualitative Research. Value Health, 20, 618–626.

    Article  PubMed  Google Scholar 

  7. Nunley, P. D., Patel, V. V., Orndorff, D. G., Lavelle, W. F., Block, J. E., & Geisler, F. H. (2017). Superion(R) interspinous spacer treatment of moderate spinal stenosis: 4-year results. World Neurosurgery, 104, 279–289.

    Article  PubMed  Google Scholar 

  8. US Department of Health and Human Services FDA Center for Drug Evaluation and Research, US Department of Health and Human Services FDA Center for Biologics Evaluation and Research, & US Department of Health and Human Services FDA Center for Devices and Radiological Health. (2006). Guidance for industry: Patient-reported outcome measures: use in medical product development to support labeling claims: Draft guidance. Health Qual Life Outcomes 4, 79.

  9. Uryniak, T. C. I., Fedorov, V. V., Jiang, Q., Oppenheimer, L., Stapinn, S., Teng, C.-H., et al. (2011). Responder analyses: A PhRMA position paper. Statistics in Biopharmaceutical Research, 3, 476–487.

    Article  Google Scholar 

  10. Wyrwich, K. W., Bullinger, M., Aaronson, N., Hays, R. D., Patrick, D. L., & Symonds, T. (2005). Clinical Significance Consensus Meeting Group: Estimating clinically significant differences in quality of life outcomes. Quality of Life Research, 14, 285–295.

    Article  PubMed  Google Scholar 

  11. Coon, C. D., Cook, K. F. (2016). Telling the interpretation story: the case for strong anchors and multiple methods. In Plenary presentation at the 23rd Annual Conference of the International Society of Quality for Life Research.

  12. Ross, M. (1989). Relation of implicit theories to the construction of personal histories. Psychological Review, 96, 341–357.

    Article  Google Scholar 

  13. Schmitt, J., & Di Fabio, R. P. (2005). The validity of prospective and retrospective global change criterion measures. Archives of Physical Medicine and Rehabilitation, 86, 2270–2276.

    Article  PubMed  Google Scholar 

  14. Kamper, S. J., Ostelo, R. W., Knol, D. L., Maher, C. G., de Vet, H. C., & Hancock, M. J. (2010). Global perceived effect scales provided reliable assessments of health transition in people with musculoskeletal disorders, but ratings are strongly influenced by current status. Journal of Clinical Epidemiology, 63, 760–766.

    Article  PubMed  Google Scholar 

  15. Metz, S. M., Wyrwich, K. W., Babu, A. N., Kroenke, K., Tierney, W. M., & Wolinsky, F. D. (2007). Validity of patient-reported health-related quality of life global ratings of change using structural equation modeling. Quality of Life Research, 16, 1193–1202.

    Article  PubMed  Google Scholar 

  16. Schwartz, C. E., Powell, V. E., & Rapkin, B. D. (2016). When global rating of change contradicts observed change: Examining appraisal processes underlying paradoxical responses over time. Quality of Life Research, 26, 847–857.

    Article  PubMed  Google Scholar 

  17. Cella, D., Lai, J. S., Nowinski, C. J., Victorson, D., Peterman, A., Miller, D., et al. (2012). Neuro-QOL: Brief measures of health-related quality of life for clinical research in neurology. Neurology, 78, 1860–1867.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Cella, D., Nowinski, C., Peterman, A., Victorson, D., Miller, D., Lai, J. S., et al. (2011). The neurology quality-of-life measurement initiative. Archives of Physical Medicine and Rehabilitation, 92, S28–S36.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Gershon, R. C., Lai, J. S., Bode, R., Choi, S., Moy, C., Bleck, T., et al. (2012). Neuro-QOL: Quality of life item banks for adults with neurological disorders: Item development and calibrations based upon clinical and general population testing. Quality of Life Research, 21, 475–486.

    Article  PubMed  Google Scholar 

  20. Neuro-QoL. www.healthmeasures.net/explore-measurement-systems/neuro-qol

  21. HealthMeasures.net. (2016). Quality of life in neurological disorders: Scoring manual.

  22. Cella, D., Choi, S., Garcia, S., Cook, K. F., Rosenbloom, S., Lai, J. S., et al. (2014). Setting standards for severity of common symptoms in oncology using the PROMIS item banks and expert judgment. Quality of Life Research, 23, 2651–2661.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Cook, K. F., Victorson, D. E., Cella, D., Schalet, B. D., & Miller, D. (2015). Creating meaningful cut-scores for Neuro-QOL measures of fatigue, physical functioning, and sleep disturbance using standard setting with patients and providers. Quality of Life Research, 24, 575–589.

    Article  PubMed  Google Scholar 

  24. Cella, D., Hahn, E. A., & Dineen, K. (2002). Meaningful change in cancer-specific quality of life scores: Differences between improvement and worsening. Quality of Life Research, 11, 207–221.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Northwestern University, Chicago, IL, USA

    Karon F. Cook, Michael A. Kallen & David Victorson

  2. Outcometrix, Tucson, AZ, USA

    Cheryl D. Coon

  3. Mellen Center, Cleveland Clinic, Cleveland, OH, USA

    Deborah M. Miller

  4. 801 Cortlandt St., Houston, TX, 77007, USA

    Karon F. Cook

Authors
  1. Karon F. Cook
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael A. Kallen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cheryl D. Coon
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David Victorson
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Deborah M. Miller
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Karon F. Cook.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cook, K.F., Kallen, M.A., Coon, C.D. et al. Idio Scale Judgment: evaluation of a new method for estimating responder thresholds. Qual Life Res 26, 2961–2971 (2017). https://doi.org/10.1007/s11136-017-1625-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-017-1625-2

Keywords

Search

Navigation