Introduction

Over the last few decades, the total number of hip (THA) and knee (TKA) arthroplasties performed annually in French hospitals has consistently increased reaching over 150,000 (French hospital discharge data 2017–2019, PMSI ATIH) [1,2,3]. This activity is rising due to the ageing population along with the increasing demands of patients in terms of functional capacities [4, 5]. Approximately 14.4% of patients report an adverse event during their surgical care, of which 5.2% are considered potentially preventable, especially after a joint replacement [6, 7]. The most frequent complications after THA and TKA (TH/KA) are infection with 1% of these surgeries subsequently resulting in surgical site infections (SSI) [8,9,10,11]. Due to the increasing frequency of arthroplasties and the potential impact of complications in terms of loss of quality of life and additional costs for the society [12, 13], SSI represent a key target for healthcare-associated infection surveillance [12,13,14,15,16].

In France, the SSI surveillance is one of the priority targets of the national program for the prevention of healthcare associated infections (PROgramme national de Prévention des Infections Associées aux Soins, PROPIAS [17]) as recommended by the High Council of Public Health. Whereas several countries are using national TH/KA registers for surveillance [18,19,20,21,22,23], no such dynamic and exhaustive national register exists in France. The French National Authority for Health (“Haute Autorité de Santé”, HAS) developed a national computerized SSI indicator in orthopaedic surgery for quality and safety improvement: ISO-ORTHO [27]. This indicator derived from an innovative French research project for SSI monitoring in orthopaedics, that validated an algorithm based on hospital discharge database (HDD) with an acceptable positive predictive value (PPV) of 87% [24,25,26]. In 2017, the HAS developed a revised version of this research SSI algorithm with a multidisciplinary working group composed of patients and experts of orthopaedics, anaesthesiology, epidemiology, infection control, medical information, and infectious diseases [27]. Subsequently, the HAS published the different methodological steps for its development and validation [19, 27,28,29,30,31,32,33]. Eventually, the definition of the ISO-ORTHO indicator was achieved through the optimization of SSI detection based on the coding practice guidelines of the Technical Agency for Information on Hospital Care (“Agence technique de l’information sur l’hospitalisation”, ATIH), along with redefining the exclusion criteria of the target population [34,35,36].

The aim of the present study was to validate the outcome indicator ISO-ORTHO by measuring its performance to detect SSI over the 90 days following a total TH/KA in France.

Methods

Study design and population

An evaluative study was performed to assess the performance of the ISO-ORTHO indicator via its PPV using a medical chart review. All adult patients with a hospital stay for TH/KA that occurred between January, 1st and September, 30th 2018 in the French national hospital discharge database were selected according to their surgical procedure for hip or knee replacement based on the French Common Classification of Medical Acts (THA: NEKA010, NEKA012, NEKA013, NEKA014, NEKA015, NEKA016, NEKA017, NEKA019, NEKA020, NEKA021; TKA: NFKA007, NFKA008, NFKA009). In case of multiple TH/KA stays, only the first was included. Exclusion criteria were patients at very high risk of SSI (history of complex SSI such as complex surgical procedures, another surgical act performed on hip or knee during the TH/KA stay, hip or knee surgery in the previous three months before hospital stay, hip fracture), with SSI not related to in-hospital care and/or data or linkage problems (error in the sex, birth date and/or social insurance number) (See Fig. 1, and arguments for inclusion and exclusion criteria detailed in AppendixA1).

Fig. 1
figure 1

Flow chart of the reviewed medical chart (n = 725)

SSI Surgical site procedure; TK/HA Total knee hip arthroplasty; HCO Healthcare Organisation

Full size image

Footnote of Fig. 1. Flowchart of the target population, SSI and HCOs

Inclusion criteria

2018

Stays with at least 1 TH/TKA stay

158 478

HCOs

791

Exclusion criteria

 

errors in the coding, misclassification, or linkage problems

730

age under 18

69

Stays of adult patients with at least 1 TH/KA coded and without data and/or linkage problems

157 682

admission for infection

117

infection not related to the THA or TKA procedure :

47

fracture as the reason for hip replacement

10 386

emergency admission

8 080

several acts of orthopaedic surgery in the same initial stay, or act of device change

677

mechanic complication coded during the stay

1 447

patients transferred from another HCO

1 424

patients with a hospitalisation between TH/KA and the readmission for SSI

106

hip or knee surgery in the previous three months before hospital stay

1 160

palliative care detected 1 year before, within or 3 months after the TH/KA stay

357

history of complex SSI detected in the previous year

87

patient coming from another country (non-French residents)séjours de patients résidant hors France (codes géographiques entre 99,101 et 99,517 + 99,999)

276

death during the surgical stay without any code of SSI

283

length of stay over 90 days

23

patients discharged against medical advice or escaped

46

Excluded stays : N (%)

13 635 (9%)

Target TH/TKA stays

143 227

Target THA

75 311

Target TKA

67 916

HCOs with at leat 1 THA/TKA target stay

777

SSI detected 3 months after TH/KA replacement

1,279

HCOs with at least 1 SSI detected by ISO-ORTHO

470

HCOs with at least 1 SSI detected in their HCO : concerned by the chart review

448

HCOs participating to the SSI chart review

250

SSI reviewed

725

Outcome

The occurrence of SSI was sought from the date of admission to 90 days after TH/KA surgery, using 15 combinations of ICD-10 codes of infection and/or complication and therapeutic or diagnostic procedure codes from the national HDD (Case definition of SSI in Table 1, and AppendixA.1). Any SSI occurring after 90 days has not been included.

Table 1 Case definition of Surgical site infections
Full size table

Assessment

The SSI detected with ISO-ORTHO were checked using a medical chart review performed in volunteer healthcare organisations (HCO) having at least one SSI detected in their patients during the 90 days following surgery, or during the TH/KA stay, or a readmission in the same HCO between January 1st and September 30th 2018. The manual review occurred between December 20th 2019 and March 24th 2020. The process of chart review was carried out on a secured platform, including access to the medical charts meeting the SSI case definition and the forms required for case analysis. The chart review was conducted under the responsibility of the medical information department. The participation of a clinician (notably a surgeon or an anaesthetist) was required. To contribute to coding quality analysis and further improvement, guidelines for SSI and TH/KA coding were also provided to the HCO [35, 36].

The assessment of PPV was conducted in two consecutive steps (i) validation of the target population to confirm the TH/KA procedure with respect to the inclusion and exclusion criteria, (ii) identification of the true positive and residual false positive SSI cases. True positive cases were defined as SSI detected by the algorithm and confirmed in the patient chart, and false positive cases as SSI detected by the algorithm and not confirmed in the patient chart. Eventually, the PPV was estimated as the percentage of true positive SSI cases divided by the total SSI cases detected by the algorithm.

Results

Over the study period, 143,227 TH/KA stays were detected in the national HDD, coded by 777 HCO with at least one target TH/KA stay. Among them, 1,279 SSI were detected within 90 days after surgery, mostly during a readmission (97.4%)(Fig. 1). SSI rate was 0.89% in the study population, 0.98% for THA and 0.80% for TKA. The sociodemographic characteristics showed a male predominance (sex ratio 1.77), and a median age of 70 years old, independent of the prosthesis site.

Among the 777 HCO, 470 (60.5%) had at least one SSI detected within 90 days after arthroplasty. HCO reviewed patient charts of patients readmitted for SSI in the hospital where TH/KA procedure was performed, corresponding to 448 (95.3%) HCO. Of the 448 HCO, 250 HCO (56% of the 448) volunteered to perform it. A majority of them were private hospitals (68%), 22.8% were public general hospitals, and 9.2% were teaching hospitals. Their distribution of the type of HCO was similar to the 470 HCO with at least one case of SSI detected (AppendixA.2).

Eventually, 725 medical charts of SSI were reviewed, representing 57% of the hospital stays with SSI. 86% of the 725 medical charts were reviewed by infection control practitioners, 36% by orthopaedist surgeons, 19% by doctors specialized in medical information, 13% by doctors specialized in infectious diseases and 8.4% by anaesthesiologists.

Of the 725 reviewed SSI, 655 were confirmed SSI and 63 were not. Among the 63 cases of false positive SSI, 29 were SSI suspected but not subsequently confirmed, 10 were uninfected hematomas, 10 had missing information about a potential SSI in the medical chart, four were infections not related to the surgical site, three were related to a history of SSI, three were a revision surgery for another reason. Seventy-three target stays (10%) had at least one exclusion factor of the target population (Table I). The absence of, or incorrect, coding by HCO were the main reasons for the absence of SSI confirmation.

Table 3 Table Ibis: Review the exclusion criteria of the target population
Full size table

The PPV of the SSI algorithm was 90.3% [88.2%; 92.5%]. Among THA, 384 of the 427 SSI (89.9% [87.1%; 92.8%]) were true positives. Among TKA, 271 of the 298 SSI (90.9% [87.7%; 94.2%]) were true positives (Table II).

Table 4 Table II: Performance of ISO-ORTHO Algorithm overall and by surgery site
Full size table

Discussion

We successfully validated the computerized indicator ISO-ORTHO, an outcome indicator in orthopaedics assessing SSI after a TH/KA in France. We demonstrated that automated SSI detection after total hip or knee arthroplasty using mandatory and available HDD is feasible and reliable regarding its performance to detect SSI. The PPV was estimated at 90% for both the combined indicator and the individual THA or TKA measures, confirming its potential use for different purposes (e.g. quality improvement and risk management, public disclosure, financial purpose) according to the HAS method [31]. Indeed, performance parameters can vary in quality and accuracy, as previously demonstrated [37,38,39,40]. According to the coding guidelines, SSI must be coded with a dedicated ICD-10 code: T84.5 associated with an infection code. This recommended combination of codes detected 66% of the SSI confirmed by the chart review. Variation in the coding practices required the use of 15 combinations of ICD-10 and procedure codes.

ISO-ORTHO has a very high PPV compared to the PPV found in literature, reported as 63.6% and 78% in other studies [11, 40, 41]. A PPV of at least 75% [31] is essential to ensure that it is consistent with the real practice and usable for quality and safety improvements [42]. The main reasons for false positives were suspicions of SSI that were not subsequently confirmed, the presence of an uninfected hematoma and a lack of SSI occurrence in the reviewed medical charts. They are mainly linked to the incorrect use of the coding instructions for SSI regarding the clinical information in medical charts [26, 30, 36, 43]. Validation of the target population showed 10% of remaining exclusion criteria linked to the absence of traceability of the information in the medical charts and/or the absence or incorrect coding by HCO.

Our study has, however, some limitations. The use of administrative hospital databases introduced an inherent bias that should be taken into consideration. The strengths and limitations of using healthcare databases for epidemiological purposes have already been extensively discussed [27, 42, 44, 45]. The coding practices could be heterogeneous and not accurate, especially concerning the type of SSI (deep or superficial) or the joint laterality. The built algorithm selects superficial and deep SSI and takes into account only the first primary arthroplasty. Moreover, SSI are rare events. In France, the incidence estimated in 2018 based on the reports of 258 voluntary HCO was 1.35% [1.16-1.54%] for THA including hip fractures, and 0.9% [1.72%-1.08%] for TKA [43]. The SSI rates detected by ISO-ORTHO are consistent with these previous results: 0.96% for THA and 0.80% for TKA and the difference could be notably explained by the population exclusion criteria. It was chosen to control only positive predictive value for reasons of relevancy when low event rates are concerned, as well as material and human feasibility. Given the low frequency of SSI, the number of subjects required to calculate a negative predictive value would be too high. For a purpose of quality management and improvement, a manual review of the medical charts on a voluntary basis was performed,to ensure consistency [46]. This has not led to a selection bias, as the volunteer HCO were representative of the entire target HCO (types of hospitals and regions) and more than half of the detected SSI were analysed. The PPV over 75% is reliable for quality assessment purposes [31, 42, 47]. Eventually, the SSI detection was focused on the 90 days following the arthroplasty whereas the SSI definition is up to 365 days; however, previous studies showed that a majority of SSI after a primary arthroplasty occurred in the first three months [19, 48].

Another limitation of indicators based on HDD is the lack of clinical and medical device data and the absence of a dynamic link between the occurrence of complications and electronic health records [49, 50]. In the absence of exhaustive clinical registries for THA and TKA, several things would be relevant to detect SSI, including patient follow up, automated complication monitoring from clinical, microbiological and imaging data warehouses. They could also be used to assess ISO-ORTHO PPV as well as its negative predictive value and sensitivity. Indeed, the algorithms and methods developed for this study have been refined and validated by matching HDD with clinical data from hospital repositories and data from biomedical laboratories available in the clinical data warehouses [49,50,51,52,53,54].

Conclusion

The ISO-ORTHO PPV over 90% confirms its validity to detect SSI over the 90 days following a TH/KA in France. According to the HAS method, it is suitable for healthcare quality improvement and in-hospital risk management, hospital accreditation as well as public disclosure and financial purposes. Following this validation study, 2018-19 results of the validated indicator (adjusted ratio of observed on expected SSI in the target TH/KA population) were released to the HCO in September 2020, enabling their use alongside other outcome indicators in orthopaedics using the French national HDD [31, 55,56,57,58] measuring thromboembolic events after TH/KA. They are used for in-hospital quality improvement and risk management, hospital accreditation and as decision making tools for regional and national policies; they also respond to users’ demand for more transparency and could be used for financial purposes.