Skip to main content

Advertisement

SpringerLink
Log in

Four of five frequently used orthopedic PROMs possess inadequate content validity: a COSMIN evaluation of the mHHS, HAGOS, IKDC-SKF, KOOS and KNEES-ACL

  • KNEE
  • Published:
Knee Surgery, Sports Traumatology, Arthroscopy Aims and scope

Abstract

Purpose

Content validity is the most important property of PROMs. The COSMIN initiative has published guidelines for evaluating the content validity of PROMs, but they have only sparsely been applied to relevant PROMs for musculoskeletal conditions. The aim of this study was to use the COSMIN Risk of Bias checklist to evaluate the content validity of five PROMs, that are highly relevant in musculoskeletal research and used by the arthroscopic surgery community: the modified Harris’ Hip Score (mHHS), the Copenhagen Hip and Groin Outcome Score (HAGOS), the International Knee Documentation Committee Subjective Knee evaluation Form (IKDC-SKF), the Knee injury and Osteoarthritis Outcome Score (KOOS) and the Knee Numeric-Entity Evaluation Score ACL (KNEES-ACL).

Methods

The development articles for the five PROMs were identified through searches in PubMed and SCOPUS. A literature search was performed to identify additional studies assessing content validity of the PROMs. Additional information, necessary for the assessments, was obtained from the PROM developers after direct request. To evaluate the quality of the development studies and rate the content validity, the COSMIN Risk of Bias checklist was applied to all studies.

Results

All five development studies were identified. Three subsequent content validity studies were identified, all evaluating KOOS and one also IKDC. One content validity study was of inadequate quality and excluded from further analysis. The development of mHHS, IKDC-SKF, and KOOS was rated inadequate and possess insufficient content validity for their target populations. Due to the irrelevance of multiple items, KOOS was in particular inappropriate to evaluate patients with an ACL injury. The development of HAGOS was also rated inadequate, although the insufficiency aspects can be regarded as minor. KNEES-ACL possessed sufficient content validity.

Conclusion

Out of five PROMs, only KNEES-ACL possessed sufficient content validity. Particularly, KOOS should not be used as an outcome for patients with an ACL injury. There is an urgent need for condition-specific PROMs for musculoskeletal conditions, developed with adequate methods.

Level of evidence

III.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Abbreviations

ACL:

Anterior cruciate ligament

ADL:

Activities of Daily Living

COSMIN:

COnsensus-based Standards for the selection of health Measurement Instruments

FAIS:

Femoroacetabular impingement syndrome

FAOS:

Foot and Ankle Outcome Score

HAGOS:

Copenhagen Hip and Groin Outcome Score

HOOS:

Hip disability and Osteoarthritis Outcome Score

IKDC-SKF:

International Knee Documentation Committee Subjective Knee Evaluation Form

KNEES-ACL:

Knee Numeric-Entity Evaluation Score ACL

KOOS:

Knee injury and Osteoarthritis Outcome Score

mHHS:

Modified Harris’ Hip Score

PROMs:

Patient reported outcome measures

SEFAS:

Self-reported Foot and Ankle Score

WOMAC:

Western Ontario and McMaster Universities Osteoarthritis Index

References

  1. Bellamy N, Buchanan WW (1986) A preliminary evaluation of the dimensionality and clinical importance of pain and disability in osteoarthritis of the hip and knee. Clin Rheumatol 5:231–241

    Article  CAS  Google Scholar 

  2. Brodersen JDL, Thorsen H, McKenna S (2013) Writing health-related items for Rasch models—patient-reported outcome scales for health sciences: from medical paternalism to patient autonomy. Rasch Models Health. https://doi.org/10.1002/9781118574454.ch15

    Article  Google Scholar 

  3. Byrd JW, Jones KS (2000) Prospective analysis of hip arthroscopy with 2-year follow-up. Arthroscopy 16:578–587

    Article  CAS  Google Scholar 

  4. Collins NJ, Prinsen CA, Christensen R, Bartels EM, Terwee CB, Roos EM (2016) Knee Injury and Osteoarthritis Outcome Score (KOOS): systematic review and meta-analysis of measurement properties. Osteoarthritis Cartilage 24:1317–1329

    Article  CAS  Google Scholar 

  5. Comins JD, Brodersen J, Siersma V, Jensen J, Hansen CF, Krogsgaard MR (2021) How to develop a condition-specific PROM. Scand J Med Sci Sports 31:1216–1224

    Article  Google Scholar 

  6. Comins JD, Krogsgaard MR, Brodersen J (2013) Development of the Knee Numeric-Entity Evaluation Score (KNEES-ACL): a condition-specific questionnaire. Scand J Med Sci Sports 23:e293-301

    CAS  PubMed  Google Scholar 

  7. Comins JD, Siersma VD, Lind M, Jakobsen BW, Krogsgaard MR (2018) KNEES-ACL has superior responsiveness compared to the most commonly used patient-reported outcome measures for anterior cruciate ligament injury. Knee Surg Sports Traumatol Arthrosc 26:2438–2446

    Article  Google Scholar 

  8. Engelhart L, Nelson L, Lewis S, Mordin M, Demuro-Mercon C, Uddin S et al (2012) Validation of the Knee Injury and Osteoarthritis Outcome Score subscales for patients with articular cartilage lesions of the knee. Am J Sports Med 40:2264–2272

    Article  Google Scholar 

  9. Gagnier JJ, Shen Y, Huang H (2018) Psychometric properties of patient-reported outcome measures for use in patients with anterior cruciate ligament injuries: a systematic review. JBJS Rev 6:e5

    Article  Google Scholar 

  10. Grevnerts HT, Terwee CB, Kvist J (2015) The measurement properties of the IKDC-subjective knee form. Knee Surg Sports Traumatol Arthrosc 23:3698–3706

    Article  Google Scholar 

  11. Hansen CF, Jensen J, Brodersen J, Siersma V, Comins JD, Krogsgaard MR (2021) Are adequate PROMs used as outcomes in randomized controlled trials? an analysis of 54 trials. Scand J Med Sci Sports 31:972–981

    Article  Google Scholar 

  12. Hansen CF, Jensen J, Siersma V, Brodersen J, Comins JD, Krogsgaard MR (2021) A catalogue of PROMs in sports science—Quality assessment of PROM development and validation. Article 7 in a series of 10. Scand J Med Sci Sports. https://doi.org/10.1111/sms.13923

    Article  PubMed  PubMed Central  Google Scholar 

  13. Huang H, Nagao M, Arita H, Shiozawa J, Nishio H, Kobayashi Y et al (2019) Reproducibility, responsiveness and validation of the Tampa Scale for Kinesiophobia in patients with ACL injuries. Health Qual Life Outcomes 17:150

    Article  CAS  Google Scholar 

  14. Hung M, Hon SD, Cheng C, Franklin JD, Aoki SK, Anderson MB et al (2014) Psychometric evaluation of the lower extremity computerized adaptive test, the modified harris hip score, and the hip outcome score. Orthop J Sports Med 2:2325967114562191

    Article  Google Scholar 

  15. Impellizzeri FM, Jones DM, Griffin D, Harris-Hayes M, Thorborg K, Crossley KM et al (2020) Patient-reported outcome measures for hip-related pain: a review of the available evidence and a consensus statement from the International Hip-related Pain Research Network, Zurich 2018. Br J Sports Med 54:848–857

    Article  Google Scholar 

  16. Irrgang JJ, Anderson AF, Boland AL, Harner CD, Kurosaka M, Neyret P et al (2001) Development and validation of the international knee documentation committee subjective knee form. Am J Sports Med 29:600–613

    Article  CAS  Google Scholar 

  17. Jacobs CA, Peabody MR, Lattermann C, Vega JF, Huston LJ, Spindler KP et al (2018) Development of the KOOSglobal platform to measure patient-reported outcomes after anterior cruciate ligament reconstruction. Am J Sports Med 46:2915–2921

    Article  Google Scholar 

  18. Kemp JL, Collins NJ, Roos EM, Crossley KM (2013) Psychometric properties of patient-reported outcome measures for hip arthroscopic surgery. Am J Sports Med 41:2065–2073

    Article  Google Scholar 

  19. Mokkink LB, de Vet HCW, Prinsen CAC, Patrick DL, Alonso J, Bouter LM et al (2018) COSMIN Risk of Bias checklist for systematic reviews of Patient-Reported Outcome Measures. Qual Life Res 27:1171–1179

    Article  CAS  Google Scholar 

  20. Mokkink LB, Terwee CB, Knol DL, Stratford PW, Alonso J, Patrick DL et al (2010) The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content. BMC Med Res Methodol 10:22

    Article  Google Scholar 

  21. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL et al (2010) The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol 63:737–745

    Article  Google Scholar 

  22. Moller A, Bissenbakker KH, Arreskov AB, Brodersen J (2020) Specific measures of quality of life in patients with multimorbidity in primary healthcare: a systematic review on patient-reported outcome measures’ adequacy of measurement. Patient Relat Outcome Meas 11:1–10

    Article  Google Scholar 

  23. Ohlin A, Ahlden M, Lindman I, Jonasson P, Desai N, Baranto A et al (2020) Good 5-year outcomes after arthroscopic treatment for femoroacetabular impingement syndrome. Knee Surg Sports Traumatol Arthrosc 28:1311–1316

    Article  Google Scholar 

  24. Patrick DL, Burke LB, Gwaltney CJ, Leidy NK, Martin ML, Molsen E et al (2011) Content validity–establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 1–eliciting concepts for a new PRO instrument. Value Health 14:967–977

    Article  Google Scholar 

  25. Patrick DL, Burke LB, Gwaltney CJ, Leidy NK, Martin ML, Molsen E et al (2011) Content validity–establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force report: part 2–assessing respondent understanding. Value Health 14:978–988

    Article  Google Scholar 

  26. Patrick DL, Burke LB, Powers JH, Scott JA, Rock EP, Dawisha S et al (2007) Patient-reported outcomes to support medical product labeling claims: FDA perspective. Value Health 10(Suppl 2):S125-137

    Article  Google Scholar 

  27. Pellekooren S, Ostelo R, Pool A, van Tulder M, Jansma E, Chiarotto A (2021) Content validity of patient-reported outcome measures of satisfaction with primary care for musculoskeletal complaints: a systematic review. J Orthop Sports Phys Ther 51:94–102

    Article  Google Scholar 

  28. Prinsen CAC, Mokkink LB, Bouter LM, Alonso J, Patrick DL, de Vet HCW et al (2018) COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res 27:1147–1157

    Article  CAS  Google Scholar 

  29. Roos EM, Roos HP, Lohmander LS, Ekdahl C, Beynnon BD (1998) Knee Injury and Osteoarthritis Outcome Score (KOOS)–development of a self-administered outcome measure. J Orthop Sports Phys Ther 28:88–96

    Article  CAS  Google Scholar 

  30. Roos EM, Toksvig-Larsen S (2003) Knee injury and Osteoarthritis Outcome Score (KOOS)—validation and comparison to the WOMAC in total knee replacement. Health Qual Life Outcomes 1:17

    Article  Google Scholar 

  31. Streiner DL, Norman GR, Cairney J (2015) Selecting the items. Health Measurement Scales—A practical guide to their development and use. Fifth edition. 5:74–99

  32. Terwee CB, Jansma EP, Riphagen II, de Vet HC (2009) Development of a methodological PubMed search filter for finding studies on measurement properties of measurement instruments. Qual Life Res 18:1115–1123

    Article  Google Scholar 

  33. Terwee CB, Prinsen CAC, Chiarotto A, Westerman MJ, Patrick DL, Alonso J et al (2018) COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Qual Life Res 27:1159–1170

    Article  CAS  Google Scholar 

  34. Thorborg K, Holmich P, Christensen R, Petersen J, Roos EM (2011) The Copenhagen Hip and Groin Outcome Score (HAGOS): development and validation according to the COSMIN checklist. Br J Sports Med 45:478–491

    Article  CAS  Google Scholar 

  35. van der Zwaard BC, Terwee CB, Roddy E, Terluin B, van der Horst HE, Elders PJ (2014) Evaluation of the measurement properties of the Manchester foot pain and disability index. BMC Musculoskelet Disord 15:276

    Article  Google Scholar 

  36. van Meer BL, Meuffels DE, Vissers MM, Bierma-Zeinstra SM, Verhaar JA, Terwee CB et al (2013) Knee injury and Osteoarthritis Outcome Score or International Knee Documentation Committee Subjective Knee Form: which questionnaire is most useful to monitor patients with an anterior cruciate ligament rupture in the short term? Arthroscopy 29:701–715

    Article  Google Scholar 

  37. Wiering B, de Boer D, Delnoij D (2017) Patient involvement in the development of patient-reported outcome measures: a scoping review. Health Expect 20:11–23

    Article  Google Scholar 

Download references

Funding

No particular funding was provided for this research.

Author information

Authors and Affiliations

  1. Section for Sports Traumatology M51, Bispebjerg and Frederiksberg Copenhagen University Hospitals, Copenhagen, Denmark

    Christian Fugl Hansen, Jonas Jensen, Jonathan David Comins & Michael Rindom Krogsgaard

  2. Department of Orthopaedic Surgery, Rigshospitalet Copenhagen University Hospital, Copenhagen, Denmark

    Anders Odgaard

  3. The Research Unit for General Practice and Section of General Practice, Department of Public Health, University of Copenhagen, Copenhagen, Denmark

    Volkert Siersma, Jonathan David Comins & John Brodersen

  4. Primary Health Care Research Unit, Region Zealand, Denmark

    John Brodersen

Authors
  1. Christian Fugl Hansen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jonas Jensen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anders Odgaard
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Volkert Siersma
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jonathan David Comins
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. John Brodersen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Michael Rindom Krogsgaard
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

CFH, JJ, VS, JB, JDC and MRK conceived and planned the project. AO was invited as an external assessor. CFH and JJ collected data for the project, and CFH, AO and VS completed data analyses. All authors participated in writing the manuscript. CFH and MRK directed the project.

Corresponding author

Correspondence to Christian Fugl Hansen.

Ethics declarations

Conflict of interest

One author (JDC) is the main developer of the KNEES-ACL, and two authors (MRK and JB) participated in the development of KNEES-ACL. These three authors did not participate in the assessment of the PROMs or the discussions related to consensus in the assessments. There are no financial conflicts.

Ethical approval

No approval required.

Informed consent

Not relevant.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

167_2021_6761_MOESM1_ESM.pdf

Supplementary file1 Additional file 1 Presents the correspondences with the original PROM developers regarding data that could not be obtained directly from the published development studies (PDF 110 KB)

167_2021_6761_MOESM2_ESM.docx

Supplementary file2 Additional file 2 Presents a more detailed description of the domains and scoring system of the COSMIN Risk of Bias checklist (DOCX 22 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hansen, C.F., Jensen, J., Odgaard, A. et al. Four of five frequently used orthopedic PROMs possess inadequate content validity: a COSMIN evaluation of the mHHS, HAGOS, IKDC-SKF, KOOS and KNEES-ACL. Knee Surg Sports Traumatol Arthrosc 30, 3602–3615 (2022). https://doi.org/10.1007/s00167-021-06761-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00167-021-06761-y

Keywords

Search

Navigation