Skip to main content

A Review of the Psychometric Properties of Generic Utility Measures in Multiple Sclerosis

PharmacoEconomics volume 32, pages 759–773 (2014)Cite this article

Abstract

Objective

The reliability and validity of generic utility measures have not yet been summarized in people with multiple sclerosis (MS). It is important to assess the psychometric properties of these measures, to ensure that the values obtained by the scoring system are valid for interpretation and utilization by clinicians, researchers and policy makers. Therefore, the objective of this review was to summarize the evidence from published literature on the psychometric properties of generic utility measures in MS.

Methods

A structured literature search was conducted by using multiple electronic databases. All potentially relevant abstracts and full-text articles were read to identify publications that may be eligible for inclusion in the review. A meta-analysis was conducted to combine correlation coefficient values for convergent validity. The Schmidt–Hunter method, a weighted mean of the correlation coefficient values, was used. Heterogeneity, the percentage of total variation across studies that is due to between-study differences rather than chance, was assessed using the I 2 statistic.

Results

The following generic utility measures were identified: the EQ-5D (n = 9)/EQ-5D-5 Level (EQ-5D-5L) (n = 1), followed by the Health Utilities Index Mark 3/2 (HUI2/HUI3) (n = 3), the SF-6D (n = 2), the Assessment of Quality of Life (AQOL) (n = 2), and the Quality of Well-Being (QWB) scale (n = 1). Ceiling and floor effects were present for the EQ-5D and the SF-6D, but not for the HUI3. The EQ-5D, the SF-6D and the HUI3 demonstrated excellent reliability. In terms of discriminative ability, the SF-6D and the QWB scale were not able to differentiate between moderately and severely disabled MS patients, and the EQ-5D was not able to differentiate between those who were mildly and moderately disabled. The AQOL and the HUI3, on the other hand, demonstrated good discriminative ability, as both measures were able to differentiate between all levels of disability. As for convergent validity, the HUI2/HUI3 were highly correlated (r = 0.7) against measurement instruments that evaluated impairments such as disease severity, ambulation and manual dexterity. The EQ-5D, SF-6D and the QWB scale demonstrated small to moderate correlations (r = 0.4) against instruments evaluating impairments, and slightly stronger correlations against measures of activity limitations/participation restrictions and health-related quality of life (HRQL) (r = 0.6).

Conclusion

To our knowledge this is the first study to review the validity and reliability of generic utility measures in MS. The HUI3 demonstrated the strongest psychometric properties when compared with other utility measures. However, the HUI3 only measures impairment and excludes important components of HRQL such as participation restrictions. The EQ-5D, the SF-6D and the QWB scale, on the other hand, do include items on participation. However, these measures demonstrated a lack of content validity in MS by missing certain domains that were important to the disease, as well as difficulty in differentiating between different levels of disability. The addition of MS-specific ‘bolt-ons’ to generic utility measures and the development of an MS specific utility measure are possible areas of exploration for future research.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

49,54 €

Price includes VAT (Finland)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  1. Dolan P. Modeling valuations for EuroQol health states. Med Care. 1997;35:1095–108.

    CAS  PubMed  Article  Google Scholar 

  2. Shaw JW, Johnson JA, Coons SJ. US valuation of the EQ-5D health states: development and testing of the D1 valuation model. Med Care. 2005;43:203–20.

    PubMed  Article  Google Scholar 

  3. Brazier J, Roberts J, Deverill M. The estimation of a preference-based measure of health from the SF-36. J Health Econ. 2002;21:271–92.

    PubMed  Article  Google Scholar 

  4. Dyer MT, Goldsmith KA, Sharples LS, Buxton MJ. A review of health utilities using the EQ-5D in studies of cardiovascular disease. Health Qual Life Outcomes. 2010;8:13.

    PubMed Central  PubMed  Article  Google Scholar 

  5. Janssen MF, Lubetkin EI, Sekhobo JP, Pickard AS. The use of the EQ-5D preference-based health status measure in adults with type 2 diabetes mellitus. Diabet Med. 2011;28:395–413.

    CAS  PubMed  Article  Google Scholar 

  6. Richardson J, Iezzi A, Khan MA, Maxwell A. Validity and reliability of the Assessment of Quality of Life (AQoL)-8D multi-attribute utility instrument. Patient Patient Cent Outcomes Res 2013;1–12.

  7. Tran BX, Ohinmaa A, Nguyen LT. Quality of life profile and psychometric properties of the EQ-5D-5L in HIV/AIDS patients. Health Qual Life Outcomes. 2012;10:132.

    PubMed Central  PubMed  Article  Google Scholar 

  8. Garau M, Shah KK, Mason AR, Wang Q, Towse A, Drummond MF. Using QALYs in cancer. Pharmacoeconomics. 2011;29:673–85.

    PubMed  Article  Google Scholar 

  9. Espallargues M, Czoski-Murray CJ, Bansback NJ, Carlton J, Lewis GM, Hughes LA, Brand CS, Brazier JE. The impact of age-related macular degeneration on health status utility values. Invest Ophthalmol Vis Sci. 2005;46:4016–23.

    PubMed  Article  Google Scholar 

  10. Barton GR, Bankart J, Davis AC, Summerfield QA. Comparing utility scores before and after hearing-aid provision: results according to the EQ-5D, HUI3 and SF-6D. Appl Health Econ Health Policy. 2004;3:103–5.

    PubMed  Article  Google Scholar 

  11. Papaioannou D, Brazier J, Parry G. How valid and responsive are generic health status measures, such as EQ-5D and SF-36, in schizophrenia? A systematic review. Value Health. 2011;14:907–20.

    PubMed Central  PubMed  Article  Google Scholar 

  12. Myers JA, McPherson KM, Taylor WJ, Weatherall M, McNaughton HK. Duration of condition is unrelated to health-state valuation on the EuroQoL. Clin Rehabil. 2003;17:209–15.

    PubMed  Article  Google Scholar 

  13. Fisk JD, Brown MG, Sketris IS, Metz LM, Murray TJ, Stadnyk KJ. A comparison of health utility measures for the evaluation of multiple sclerosis treatments. J Neurol Neurosurg Psychiatry. 2005;76:58–63.

    CAS  PubMed Central  PubMed  Article  Google Scholar 

  14. Feeny D. Preference-based measures: utility and quality-adjusted life years. In: Fayers PM, Hays RD, editors. Assessing quality of life in clinical trials. 2nd ed. Oxford University Press; 2005. p. 405–29.

  15. Brazier J, Roberts J, Tsuchiya A, Busschbach J. A comparison of the EQ-5D and SF-6D across seven patient groups. Health Econ. 2004;13:873–84.

    PubMed  Article  Google Scholar 

  16. Noseworthy JH, Lucchinetti C, Rodriguez M, Weinshenker BG. Multiple sclerosis. N Engl J Med. 2000;343:938–52.

    CAS  PubMed  Article  Google Scholar 

  17. Terwee CB, Mokkink LB. Measurement in medicine: a practical guide. London: Cambridge University Press; 2011.

    Google Scholar 

  18. Kaplan RM, Anderson JP, Wu AW, Mathews WC, Kozin F, Orenstein D. The Quality of Well-Being Scale. Applications in AIDS, cystic fibrosis, and arthritis. Med Care. 1989;27:S27–43.

    CAS  PubMed  Article  Google Scholar 

  19. Kaplan RM, Ganiats TG, Sieber WJ, Anderson JP. The Quality of Well-Being Scale: critical similarities and differences with SF-36. Int J Qual Health Care. 1998;10:509–20.

    CAS  PubMed  Article  Google Scholar 

  20. Feeny D, Furlong W, Barr RD, Torrance GW, Rosenbaum P, Weitzman S. A comprehensive multiattribute system for classifying the health status of survivors of childhood cancer. J Clin Oncol. 1992;10:923–8.

    CAS  PubMed  Google Scholar 

  21. Feeny D, Furlong W, Boyle M, Torrance GW. Multi-attribute health status classification systems. Health Utilities Index. Pharmacoeconomics. 1995;7:490–502.

    CAS  PubMed  Article  Google Scholar 

  22. Feeny D, Furlong W, Torrance GW, Goldsmith CH, Zhu Z, DePauw S, Denton M, Boyle M. Multiattribute and single-attribute utility functions for the Health Utilities Index Mark 3 system. Med Care. 2002;40:113–28.

    PubMed  Article  Google Scholar 

  23. Sintonen H, Pekurinen M. A fifteen-dimensional measure of health-related quality of life (15D) and its applications. In: Walker SR, Rosser R, editors. Quality of life assessment: key issues in the 1990s. Springer, Netherlands; 1993. p. 185–95.

  24. Sintonen H. The 15D instrument of health-related quality of life: properties and applications. Ann Med. 2001;33:328–36.

    CAS  PubMed  Article  Google Scholar 

  25. Hawthorne G, Richardson J, Osborne R. The Assessment of Quality of Life (AQoL) instrument: a psychometric measure of health-related quality of life. Qual Life Res. 1999;8:209–24.

    CAS  PubMed  Article  Google Scholar 

  26. Richardson J, Atherton Day N, Peacock S, Iezzi A. Measurement of the quality of life for economic evaluation and the Assessment of Quality of Life (AQoL) Mark 2 instrument. Aust Econ Rev. 2004;37:62–88.

    Article  Google Scholar 

  27. Brink Y, Louw QA. Clinical instruments: reliability and validity critical appraisal. J Eval Clin Pract. 2012;18:1126–32.

    PubMed  Article  Google Scholar 

  28. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, Bouter LM, de Vet HC. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. 2010;63:737–45.

    PubMed  Article  Google Scholar 

  29. Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. New York: Oxford university press; 2008.

    Book  Google Scholar 

  30. McHorney CA, Tarlov AR. Individual-patient monitoring in clinical practice: are available health status surveys adequate? Qual Life Res. 1995;4:293–307.

    CAS  PubMed  Article  Google Scholar 

  31. Polgar S, Thomas SA. Introduction to research in the health sciences. Elsevier Health Sciences; 2013.

  32. StatsDirect L. StatsDirect statistical software. StatsDirect, UK; 2005.

  33. Kuspinar A, Mayo NE. Do generic utility measures capture what is important to the quality of life of people with multiple sclerosis? Health Qual Life Outcomes. 2013;11:71.

    PubMed Central  PubMed  Article  Google Scholar 

  34. Kikuchi H, Mifune N, Niino M, Ohbu S, Kira J, Kohriyama T, Ota K, Tanaka M, Ochi H, Nakane S, Maezawa M, Kikuchi S. Impact and characteristics of quality of life in Japanese patients with multiple sclerosis. Qual Life Res. 2011;20:119–31.

    PubMed  Article  Google Scholar 

  35. Twiss J, Doward LC, McKenna SP, Eckert B. Interpreting scores on multiple sclerosis-specific patient reported outcome measures (the PRIMUS and U-FIS). Health Qual Life Outcomes. 2010;8:117.

    PubMed Central  PubMed  Article  Google Scholar 

  36. Ploughman M, Austin M, Stefanelli M, Godwin M. Applying cognitive debriefing to pre-test patient-reported outcomes in older people with multiple sclerosis. Qual Life Res. 2010;19:483–7.

    PubMed  Article  Google Scholar 

  37. Orme M, Kerrigan J, Tyas D, Russell N, Nixon R. The effect of disease, functional status, and relapses on the utility of people with multiple sclerosis in the UK. Value Health. 2007;10:54–60.

    PubMed  Article  Google Scholar 

  38. Moore F, Wolfson C, Alexandrov L, Lapierre Y. Do general and multiple sclerosis-specific quality of life instruments differ? Can J Neurol Sci. 2004;31:64–71.

    PubMed  Google Scholar 

  39. Nicholl CR, Lincoln NB, Francis VM, Stephan TF. Assessing quality of life in people with multiple sclerosis. Disabil Rehabil. 2001;23:597–603.

    CAS  PubMed  Article  Google Scholar 

  40. Rothwell PM, McDowell Z, Wong CK, Dorman PJ. Doctors and patients don’t agree: cross sectional study of patients’ and doctors’ perceptions and assessments of disability in multiple sclerosis. BMJ. 1997;314:1580–3.

    CAS  PubMed Central  PubMed  Article  Google Scholar 

  41. Fogarty E, Walsh C, Adams R, McGuigan C, Barry M, Tubridy N. Relating health-related Quality of Life to disability progression in multiple sclerosis, using the 5-level EQ-5D. Mult Scler. 2013;19:1190–6.

    PubMed  Article  Google Scholar 

  42. Grima DT, Torrance GW, Francis G, Rice G, Rosner AJ, Lafortune L. Cost and health related quality of life consequences of multiple sclerosis. Mult Scler. 2000;6:91–8.

    CAS  PubMed  Article  Google Scholar 

  43. Jones CA, Pohar SL, Warren S, Turpin KV, Warren KG. The burden of multiple sclerosis: a community health survey. Health Qual Life Outcomes. 2008;6:1–7.

    PubMed Central  PubMed  Article  Google Scholar 

  44. Khan F, McPhail T, Brand C, Turner-Stokes L, Kilpatrick T. Multiple sclerosis: disability profile and quality of life in an Australian community cohort. Int J Rehabil Res. 2006;29:87–96.

    PubMed  Article  Google Scholar 

  45. Khan F, Pallant J. Chronic pain in multiple sclerosis: prevalence, characteristics, and impact on quality of life in an Australian community cohort. J Pain. 2007;8:614–23.

    PubMed  Article  Google Scholar 

  46. Schwartz CE, Vollmer T, Lee H. Reliability and validity of two self-report measures of impairment and disability for MS. Neurology. 1999;52:63.

    CAS  PubMed  Article  Google Scholar 

  47. Yorkston KM, Kuehn CM, Johnson KL, Ehde DM, Jensen MP, Amtmann D. Measuring participation in people living with multiple sclerosis: a comparison of self-reported frequency, importance and self-efficacy. Disabil Rehabil. 2008;30:88–97.

    PubMed Central  PubMed  Article  Google Scholar 

  48. Fougeyrollas LN. Long-term consequences of spinal cord injury on social participation: the occurrence of handicap situations. Disabil Rehabil. 2000;22:170–80.

    PubMed  Article  Google Scholar 

  49. Read JL, Quinn RJ, Hoefer MA. Measuring overall health: an evaluation of three important approaches. J Chronic Dis. 1987;40:7S–21S.

    PubMed  Article  Google Scholar 

  50. Kaplan RM, Sieber WJ, Ganiats TG. The Quality of Well-Being Scale: comparison of the interviewer-administered version with a self-administered questionnaire. Psychol Health. 1997;12:783–91.

    Article  Google Scholar 

  51. Andresen EM, Rothenberg BM, Kaplan RM. Performance of a self-administered mailed version of the Quality of Well-Being (QWB-SA) questionnaire among older adults. Med Care. 1998;36:1349–60.

    CAS  PubMed  Article  Google Scholar 

  52. Davis S, Wailoo A. A review of the psychometric performance of the EQ-5D in people with urinary incontinence. Health Qual Life Outcomes. 2013;11:20.

    PubMed Central  PubMed  Article  Google Scholar 

  53. Whitehurst DG, Noonan VK, Dvorak MF, Bryan S. A review of preference-based health-related quality of life questionnaires in spinal cord injury research. Spinal Cord. 2012;50:646–54.

    CAS  PubMed  Article  Google Scholar 

  54. Tosh J, Brazier J, Evans P, Longworth L. A review of generic preference-based measures of health-related quality of life in visual disorders. Value Health. 2012;15:118–27.

    PubMed Central  PubMed  Article  Google Scholar 

  55. Lin FJ, Longworth L, Pickard AS. Evaluation of content on EQ-5D as compared to disease-specific utility measures. Qual Life Res. 2013;22:853–74.

    PubMed  Article  Google Scholar 

Download references

Conflicts of interest

The authors have no conflicts of interest to declare.

Authors’ contribution

AK was primarily responsible for writing the manuscript in close cooperation with NM. Both authors read, edited, and approved the final manuscript. AK is the overall guarantor for the content.

Author information

Affiliations

  1. Faculty of Medicine, School of Physical and Occupational Therapy, McGill University, 3654 Prom Sir William Osler, Montreal, QC, H3G 1Y5, Canada

    Ayse Kuspinar & Nancy E. Mayo

  2. Division of Clinical Epidemiology, McGill University Health Center, Montreal, QC, Canada

    Nancy E. Mayo

Authors
  1. Ayse Kuspinar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nancy E. Mayo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ayse Kuspinar.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 58 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kuspinar, A., Mayo, N.E. A Review of the Psychometric Properties of Generic Utility Measures in Multiple Sclerosis. PharmacoEconomics 32, 759–773 (2014). https://doi.org/10.1007/s40273-014-0167-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40273-014-0167-5

Keywords

  • Multiple Sclerosis
  • Psychometric Property
  • Convergent Validity
  • Content Validity
  • Expand Disability Status Scale