Avoid common mistakes on your manuscript.
New methods for ACL reconstruction have been developed to improve the historical results of surgical treatment and it is important to critically evaluate these improvements in surgery. In addition, the value of non-operative treatment needs to be carefully investigated to determine the merits over surgical treatment. In the field of orthopaedic sports medicine, there are relatively few Level I randomized clinical trials that have investigated the methods for and benefits of reconstructing the anterior cruciate ligament (ACL) or to compare non-operative treatment to surgical reconstruction.
A well-done Level I randomized clinical trial requires substantial planning, collaboration, commitment and personnel and financial resources to successfully complete the study. In planning a clinical trial, selection of a patient-centred outcome measure is an important consideration. Patient-centred outcomes are those outcomes that are important to patients. Based on a Delphi study involving clinical experts conducted by the American Academy of Orthopaedic Surgeons (AAOS) Management of Anterior Cruciate Ligament Injuries Evidence-Based Clinical Practice Guidelines Work Group [3], critical and important outcomes for ACL injuries were found to include patient satisfaction, pain, function, ACL clinical failure, quality of life and return to play. Often these outcomes are assessed with the use of patient-reported outcome (PRO) measures, which are designed to assess the patient’s perception of their symptoms (pain, instability, etc.), activity limitations (difficulty walking, running, cutting, pivoting, etc.) and participation restrictions (restrictions participating in sports, recreation, work, etc.).
The most commonly used PRO measures for assessing the outcome of injury and treatment of ACL injuries are the International Knee Documentation Committee Subjective Knee Form (IKDC-SKF) and the Knee injury and Osteoarthritis Outcome Score (KOOS). In a systematic review of the 50 most frequently sited Level I clinical trials investigating the treatment of ACL injuries, the IKDC-SKF was used as an outcome measure in 28 studies, while the KOOS was used in 6 studies [1]. However, use of the KOOS (and pivot shift test) was associated with higher quality studies.
Because Level I studies constitute the highest level of evidence, it follows that the end-points used by these studies must also be selected with care and scrutiny based on their ability to provide relevant, reliable, valid and responsive measures of knee function in persons following ACL-R. More recently, to ensure that a PRO measure is truly patient-centred, there has been a greater emphasis placed on content of the PRO measure in the generation and selection of items. In considering content of a PRO, it is important to clearly specify the construct measured by the PRO measure. Once the construct is specified, it necessary to determine if the construct measured by the PRO is adequately represented by the items included on the PRO. This includes determination that collectively the items represent all important aspects of the construct (i.e. there is no under-representation of the construct) and that there are no items that are unrelated to construct (i.e. there no items that are irrelevant to the construct). Further, the comprehensibility and understandability of the items, including the stem (or question) and response options should also be evaluated. If a PRO measure lacks important content or includes content that is irrelevant to the construct, the resulting score may still demonstrate acceptable psychometric properties including reliability, validity and responsiveness; however, the meaning of the score may be uncertain. Further, under-representation of the construct or inclusion of irrelevant items may result in floor or ceiling effects and lack of responsiveness of the measure.
Historically, content of a PRO measure has been evaluated psychometrically by determining the presence of floor and ceiling effects and internal structure of the item responses using factor analytic methods and coefficient alpha to determine internal consistency. Today, cognitive interviews and focus groups with representative patient-stakeholders are recommended in the process to generate, review and select items during the development phase of the PRO measure. For legacy PRO measures, including the KOOS and IKDC-SKF, which were developed prior to the emphasis on the use of stakeholder engagement to establish content of the PRO measure, it is recommended that the items be reviewed by patient-stakeholders to determine if there is under-representation of the construct, inclusion of irrelevant items or difficulties with comprehensibility. If this process results in the creation of new items, removal of existing items, changes in the wording of the items, including the response options or recall period, then further evaluation of the content and psychometric properties of the revised PRO measure would be required before its adoption to replace the legacy measure.
Given the importance of the content of PRO measures, Hansen et al. [13] used the COSMIN Risk of Bias Checklist [20, 22, 27] to evaluate evidence for the content of five commonly used PRO measures by the orthopaedic sports medicine community, including the KOOS and IKDC-SKF. The authors concluded that development of the KOOS and IKDC-SKF was inadequate and both measures possess insufficient content for their target populations.
The KOOS is a comprehensive knee-specific PRO measure that was developed based on the Western and McMaster Universities Osteoarthritis Index (WOMAC), which was originally intended to evaluate symptoms and function of patients between the ages of 16 and 65 years with a focus on hip and knee osteoarthritis [23]. As evidenced by inclusion of the KOOS in the Scandinavian knee ligament registries, adoption of the KOOS has been widespread, in part due to support from influential orthopaedic surgeons and physical therapists despite limited evidence to support the content and psychometric properties of the KOOS.
Recent studies investigating the content validity of the KOOS to assess outcomes for individuals with an ACL injury have noted the issue of inclusion of items that are irrelevant to individuals with an ACL injury [5, 7, 13]. Further, review of the KOOS reveals under-representation of content important for individuals with an ACL injury, including the lack of items that represent the presence of instability or weakness and difficulty with activities such as starting and stopping quickly, sprinting, landing from a jump and changing directions quickly. Using a PRO measure with limited relevance and inadequate coverage of items specific to ACL injury, researchers risk the inability to detect important treatment effects, leading to false-negative results [5, 24, 25].
Of the five KOOS subscales, the sports/recreation and quality of life (QoL) subscales are the most relevant to individuals with ACL injury and are also the most important for achieving a patient acceptable symptom state (PASS) in terms of the KOOS [4, 21]. In contrast, the Pain and Activities of Daily Living (ADL) subscales have been found to be irrelevant for individuals with an ACL injury and demonstrate low levels of responsiveness for this patient population. [13, 21, 28]. Moreover, the Pain and ADL KOOS subscales demonstrate ceiling effects for patients with ACL injury, which leads to low discriminatory power in knee function following ACL-R in patients who attain scores at the higher end of the measurement spectrum [17]. Importantly, the Pain and ADL domains of the KOOS lack the ability to detect change over time in knee function 1 year after surgery or injury and fail to address factors that determine return to sport (RTS) after ACL injury [19, 24, 28].
Recognizing some of the shortcomings of the KOOS, efforts have been made to develop the 11-item KOOSglobal [17] and the composite KOOS4 [9]. While eliminating items and subscales that are irrelevant for individuals with an ACL injury may help to eliminate ceiling effects and the lack of responsiveness, these revisions have not been based on the input from individuals with an ACL injury. As such, these revised scales likely still suffer from under-representation of the construct and inclusion of irrelevant items. Further, the psychometric properties of these revised KOOS scores have not been validated for patients with ACL injuries and may, therefore, be unreliable study endpoints [18]. It is possible that compensating for the domains of the KOOS that are less than optimal for the assessment knee function following ACL injury may be a step in the wrong direction.
It is now clear that the PRO measures used in ACL clinical practice and research must be carefully considered to avoid inaccurate results and inadequate conclusions regarding the treatment of individuals with an ACL tear. While the ultimate goal should be the development and validation of PRO measures with strong evidence for content and psychometric properties specific to the ACL-injured patient, current alternatives to the KOOS should not be forgotten.
The International Knee Documentation Committee-Subjective Knee Form (IKDC-SKF) is the most common PRO measure used in studies investigating surgical reconstruction and/or non-operative treatment for ACL injuries [1, 14, 16]. The IKDC-SKF is a knee-specific PRO measure that was designed as an evaluative measure to detect improvement or deterioration in symptoms, function and sports activity for individuals with a variety of knee conditions, including ligament and meniscus injuries, articular cartilage lesions and patellofemoral pain [14].
Recently, development of the IKDC-SKF has been deemed to be inadequate with insufficient evidence for content validity, largely due to the lack of patient input in the generation of the initial pool of items [13]. However, item content for the IKDC-SKF was developed through review of existing knee-related PRO measures and with input from the International Knee Documentation Committee that consisted of highly recognized orthopaedic sports medicine surgeons from North America, Europe and the Pacific Rim. The initial pool of items consisted of 27 items related to symptoms, 8 items related to function during activities of daily living, 4 questions related to function during sports activities, 3 questions related to current function of the knee and 5 questions related to participation in sports and/or work. The initial pool of items was pilot tested (n = 144 patients) and field tested (n = 222 patients). By considering the content and statistical properties of the items, the data from the pilot and field testing were used to select the final set of 18 items for the IKDC-SKF.
Extensive evidence for internal consistency, test–retest reliability, construct validity, and responsiveness [8, 10, 14, 15] as well as normative data [2] and the Patient Acceptable Symptom Score [21] has been provided to support interpretation of the IKDC-SKF Scores. Further, evidence for content of the IKDC-SKF has been provided by ratings of the frequency of occurrence and importance of the item content by patients with ACL insufficiency or meniscus tears [26] and following ACL reconstruction [11, 28] or articular cartilage repair [12]. As a result of its widespread use and psychometric data supporting interpretation of the IKDC-SKF score, in 2019 the Panther ACL Consensus Conference endorsed the IKDC-SKF as the recommended PRO measure to assess outcome for individuals with an ACL tear [25].
To address the concerns related to the lack of evidence for content of the IKDC-SKF, consistent with the recommendations by Hansen et al., the American Orthopaedic Society for Sports Medicine and the American Board of Orthopaedic Surgery has funded a study that will conduct cognitive interviews with persons with a variety of knee complaints to determine item relevance and clarity and to identify gaps in item content coverage to measure the construct of symptoms, function and sports activity for individuals with a variety of knee conditions, including ligament and meniscus injuries, articular cartilage lesions and patellofemoral pain. Additionally, fit of an item response theory (IRT) model and a simulated computer adaptive test (CAT) will be performed to determine if use of IRT and CAT can enhance the efficiency of administering the IKDC-SKF without adversely affecting the resulting IKDC-SKF scores. If this work leads to the creation of new items, removal of existing items, or changes in the wording of the items, then further evaluation of the psychometric properties of the revised IKDC-SKF will be undertaken.
Recent reports have drawn attention to a relatively newly developed PRO measure, the Knee Numeric-Entity Evaluation Score ACL (KNEES-ACL), created for the specific assessment of ACL injury [6, 7, 13]. In patients before and up to 12 months after ACL-R, subscales of the KNEES-ACL have demonstrated superior responsiveness compared to the KOOS subscales and IKDC-SKF [7]. While use of the KNEES-ACL is currently limited, it is an ACL-specific, patient-generated and psychometrically validated PRO measure warranting further consideration in future clinical ACL studies.
Further improvements in the KOOS and other existing PRO measures that are used to measure ACL outcomes are necessary to enhance the evidence base for surgical and non-surgical treatment of ACL injuries. Despite its widespread use, there are some limitations related to the content of the KOOS as a measure of patient-reported outcome following surgical or non-surgical treatment of ACL injuries. The issues of inclusion of irrelevant content and omission of important content leads to problems with floor and ceiling effects and lack of responsiveness when using the KOOS to detect the effects of treatment of ACL injuries. Ultimately, identification and universal endorsement of an accepted PRO measure that has strong evidence for content as well as for reliability, validity and responsiveness needs to be adopted to serve as the primary outcome for all ACL clinical trials and knee ligament registries.
References
Ahmad SS, Meyer JC, Krismer AM, Ahmad SS, Evangelopoulos DS, Hoppe S et al (2017) Outcome measures in clinical ACL studies: an analysis of highly cited level I trials. Knee Surg Sports Traumatol Arthrosc 25:1517–1527
Anderson AF, Irrgang JJ, Kocher MS, Mann BJ, Harrast JJ, International Knee Documentation C (2006) The International Knee Documentation Committee Subjective Knee Evaluation Form: normative data. Am J Sports Med 34:128–135
Carey JL, Shea KG (2015) AAOS clinical practice guideline: management of anterior cruciate ligament injuries: evidence-based guideline. J Am Acad Orthop Surg 23:e6-8
Collins NJ, Prinsen CA, Christensen R, Bartels EM, Terwee CB, Roos EM (2016) Knee Injury and Osteoarthritis Outcome Score (KOOS): systematic review and meta-analysis of measurement properties. Osteoarthr Cartil 24:1317–1329
Comins J, Brodersen J, Krogsgaard M, Beyer N (2008) Rasch analysis of the Knee injury and Osteoarthritis Outcome Score (KOOS): a statistical re-evaluation. Scand J Med Sci Sports 18:336–345
Comins JD, Krogsgaard MR, Brodersen J (2013) Development of the Knee Numeric-Entity Evaluation Score (KNEES-ACL): a condition-specific questionnaire. Scand J Med Sci Sports 23:e293-301
Comins JD, Siersma VD, Lind M, Jakobsen BW, Krogsgaard MR (2018) KNEES-ACL has superior responsiveness compared to the most commonly used patient-reported outcome measures for anterior cruciate ligament injury. Knee Surg Sports Traumatol Arthrosc 26:2438–2446
Crawford K, Briggs KK, Rodkey WG, Steadman JR (2007) Reliability, validity, and responsiveness of the IKDC score for meniscus injuries of the knee. Arthroscopy 23:839–844
Frobell RB, Roos EM, Roos HP, Ranstam J, Lohmander LS (2010) A randomized trial of treatment for acute anterior cruciate ligament tears. N Engl J Med 363:331–342
Greco NJ, Anderson AF, Mann BJ, Cole BJ, Farr J, Nissen CW et al (2010) Responsiveness of the International Knee Documentation Committee Subjective Knee Form in comparison to the Western Ontario and McMaster Universities Osteoarthritis Index, modified Cincinnati Knee Rating System, and Short Form 36 in patients with focal articular cartilage defects. Am J Sports Med 38:891–902
Hambly K, Griva K (2010) IKDC or KOOS: which one captures symptoms and disabilities most important to patients who have undergone initial anterior cruciate ligament reconstruction? Am J Sports Med 38:1395–1404
Hambly K, Griva K (2008) IKDC or KOOS? Which measures symptoms and disabilities most important to postoperative articular cartilage repair patients? Am J Sports Med 36:1695–1704
Hansen CF, Jensen J, Odgaard A, Siersma V, Comins JD, Brodersen J et al (2021) Four of five frequently used orthopedic PROMs possess inadequate content validity: a COSMIN evaluation of the mHHS, HAGOS, IKDC-SKF, KOOS and KNEES-ACL. Knee Surg Sports Traumatol Arthrosc. https://doi.org/10.1007/s00167-021-06761-y
Irrgang JJ, Anderson AF, Boland AL, Harner CD, Kurosaka M, Neyret P et al (2001) Development and validation of the international knee documentation committee subjective knee form. Am J Sports Med 29:600–613
Irrgang JJ, Anderson AF, Boland AL, Harner CD, Neyret P, Richmond JC et al (2006) Responsiveness of the International Knee Documentation Committee Subjective Knee Form. Am J Sports Med 34:1567–1573
Irrgang JJ, Ho H, Harner CD, Fu FH (1998) Use of the International Knee Documentation Committee guidelines to assess outcome following anterior cruciate ligament reconstruction. Knee Surg Sports Traumatol Arthrosc 6:107–114
Jacobs CA, Peabody MR, Lattermann C, Vega JF, Huston LJ, Spindler KP et al (2018) Development of the KOOSglobal platform to measure patient-reported outcomes after anterior cruciate ligament reconstruction. Am J Sports Med 46:2915–2921
Krogsgaard MR, Brodersen J, Jensen J, Hansen CF, Comins JD (2021) Potential problems in the use of patient reported outcome measures (PROMs) and reporting of PROM data in sports science. Scand J Med Sci Sports 31:1249–1258
Meredith SJ, Rauer T, Chmielewski TL, Fink C, Diermeier T, Rothrauff BB et al (2021) Return to sport after anterior cruciate ligament injury: Panther Symposium ACL Injury Return to Sport Consensus Group. J ISAKOS 6:138–146
Mokkink LB, de Vet HCW, Prinsen CAC, Patrick DL, Alonso J, Bouter LM et al (2018) COSMIN Risk of Bias checklist for systematic reviews of patient-reported outcome measures. Qual Life Res 27:1171–1179
Muller B, Yabroudi MA, Lynch A, Lai CL, van Dijk CN, Fu FH et al (2016) Defining thresholds for the patient acceptable symptom state for the IKDC subjective knee form and KOOS for patients who underwent ACL reconstruction. Am J Sports Med 44:2820–2826
Prinsen CAC, Mokkink LB, Bouter LM, Alonso J, Patrick DL, de Vet HCW et al (2018) COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res 27:1147–1157
Roos EM, Roos HP, Lohmander LS, Ekdahl C, Beynnon BD (1998) Knee Injury and Osteoarthritis Outcome Score (KOOS)–development of a self-administered outcome measure. J Orthop Sports Phys Ther 28:88–96
Samuelsson K, Magnussen RA, Alentorn-Geli E, Krupic F, Spindler KP, Johansson C et al (2017) Equivalent knee injury and osteoarthritis outcome scores 12 and 24 months after anterior cruciate ligament reconstruction: results from the Swedish National Knee Ligament Register. Am J Sports Med 45:2085–2091
Svantesson E, Hamrin Senorski E, Webster KE, Karlsson J, Diermeier T, Rothrauff BB et al (2020) Clinical outcomes after anterior cruciate ligament injury: panther symposium ACL injury clinical outcomes consensus group. Knee Surg Sports Traumatol Arthrosc 28:2415–2434
Tanner SM, Dainty KN, Marx RG, Kirkley A (2007) Knee-specific quality-of-life instruments: which ones measure symptoms and disabilities most important to patients? Am J Sports Med 35:1450–1458
Terwee CB, Prinsen CAC, Chiarotto A, Westerman MJ, Patrick DL, Alonso J et al (2018) COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Qual Life Res 27:1159–1170
van Meer BL, Meuffels DE, Vissers MM, Bierma-Zeinstra SM, Verhaar JA, Terwee CB et al (2013) Knee injury and Osteoarthritis Outcome Score or International Knee Documentation Committee Subjective Knee Form: which questionnaire is most useful to monitor patients with an anterior cruciate ligament rupture in the short term? Arthroscopy 29:701–715
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zsidai, B., Narup, E., Senorski, E.H. et al. The Knee Injury and Osteoarthritis Outcome Score: shortcomings in evaluating knee function in persons undergoing ACL reconstruction. Knee Surg Sports Traumatol Arthrosc 30, 3594–3598 (2022). https://doi.org/10.1007/s00167-022-06990-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00167-022-06990-9