FormalPara Take-home message

Extracorporeal membrane oxygenation (ECMO) has been used widely throughout the coronavirus disease 2019 (COVID-19) pandemic, however, conflicting outcomes have been reported. We found that in a well-matched cohort of ECMO-referred patients, retrieval on mobile ECMO to a specialist center conferred survival benefit.

Introduction

Extracorporeal membrane oxygenation (ECMO) has become a widespread therapy for severe respiratory failure (SRF) in coronavirus disease 2019 (COVID-19). While some reports indicate high mortality amongst COVID-19 patients receiving ECMO [1,2,3,4], other studies describe improved outcomes [5, 6], with a recent review reporting a 37.1% pooled in-hospital mortality [7]—comparable to pandemic influenza and non-COVID-19 acute respiratory distress syndromes [8]. Consensus guidelines have given cautious recommendation for ECMO in selected COVID-19 patients [9, 10]. Limited evidence from causal analyses in cohorts of COVID-19 ECMO patients [11, 12] suggests ECMO confers a survival benefit. However, given varied selection criteria, outcomes, and baseline characteristics [7, 13], the added benefit of ECMO in the COVID-19 SRF population remains uncertain.

A centralised national referral system was established at the start of the pandemic, providing a unified pathway for hospitals in the United Kingdom (UK) to refer patients for consideration of ECMO to specialist centres [14]. The high volume of referrals [15], and the detailed data recorded through the referral system, allow analysis of ECMO as an intervention in a large and well-characterized cohort of patients with COVID-19 SRF. In this study, we obtain a dataset of patients referred from 111 hospitals in the UK to two national ECMO centres, serving 24.5 million people in southern England and Wales. We perform a matching analysis to estimate the added survival benefit of receiving treatment via a specialist ECMO service, when compared to patients who received conventional therapy at their referring centre.

Methods

Study design and setting

A multi-centre retrospective cohort study was conducted in patients referred to two national ECMO centres [Guy’s and St Thomas’ Foundation Trust (GSTFT) and Royal Brompton and Harefield Trust (RBHT)] in the UK [16], between 3 March 2020 and 28 February 2021.

Following National Health Service commissioning in 2011, the UK patients with SRF may be referred to a designated ECMO centre, where specialists perform remote assessment, deliver advice, and consider patients against national eligibility criteria for retrieval on mobile ECMO. Criteria are outlined by NHS England (Supplementary Table A), establishing consistent indications for respiratory ECMO provision [17]. The primary indication is potentially reversible SRF (as indicated by severe hypoxaemia, Murray-score > 3, or uncompensated hypercapnia, in absence of significant frailty and organ failure severity).

Pathway changes took place at the start of the pandemic. Patient referrals were delivered through an online ‘Refer-a-patient’ portal (Bloomsbury Health, London, UK). Eligibility criteria were revised on 10 April 2020 to include consideration of respiratory ECMO survival prediction (RESP) score [14], and again in the latter half of the data collection period to include days of non-invasive ventilation (NIV) [14]. The pathway for COVID-19 ECMO provision has been previously described [6].

Patient selection

All referred patients with COVID-19 were eligible for inclusion. UK practice encourages active referral and discussion [6]. This approach includes patients when referring clinicians considered them to have COVID-19 SRF requiring specialist input. In a subset of patients referred twice (following closure of initial referral), the second referral instance was eligible for inclusion, as only the second could result in ECMO retrieval.

Patients referred with a non-COVID-19 diagnosis, or those referred for non-respiratory ECMO, were excluded. Patients with missing outcome data were excluded.

Study exposure

In the UK pandemic ECMO provision, referred and accepted patients are retrieved on mobile ECMO to a specialist centre (treatment group). Our exposure is therefore two-pronged, including provision of ECMO, and specialist SRF care in the centre itself. The control group contains patients whose referrals were not accepted, remaining in referring centres receiving conventional care.

This on-going practice of mobile ECMO retrieval has removed separation between transfer to specialist centres, and ECMO initiation, described in Peek et al. [18], where patients were transferred conventionally, with only a subset receiving ECMO.

Patient outcomes

Our primary outcome measure was in-hospital mortality during the same hospital episode as the referral. We treated inter-hospital transfers or repatriations as continuing the same hospital episode. There were no secondary outcome measures.

Data collection and curation

Demographic, clinical, and physiological parameters were submitted by the referring team at time of referral via the ‘Refer-a-patient’ system. Referral data were immediately reviewed by the designated ECMO centre, and following consideration, electronic responses were returned, documenting clinical decision (retrieve, manage at referring hospital, or perceived futility), and ECMO eligibility status by NHS criteria. A decision of ‘futility’ indicated an illness trajectory that was deemed to have limited reversibility with addition of ECMO. In the absence of validated scoring systems, these decisions were undertaken in discussion with specialists in referring and ECMO teams.

A bespoke data extraction pipeline was created in Python (Python Software Foundation, https://www.python.org/) to curate data into a structured database, ensure consistency and excluding erroneous values. Parallel manual cross-checking was employed to ensure data quality.

Outcome data were collected manually through direct contact with hospital audit, medical records, and clinical departments. Direct follow-up ensured mortality during the relevant hospital episode was identified.

Missing data handling

A subset of patient referrals was incomplete. We used random forest imputation [19] to handle missingness, subsequently testing this approach in sensitivity analysis.

Statistical analysis

We described the cohort across two pandemic waves, and separately examined referrals labelled ‘perceived futility’. The pandemic waves were defined as occurring before or after 1 August 2020. Continuous variables are presented with median and interquartile range and categorical with count and percentage. Between-groups comparisons are made using Mann–Whitney U for continuous and Pearson’s Chi-squared for categorical variables. A p value of < 0.05 was considered statistically significant.

Propensity score and matching

To measure the effect of our exposure on the primary outcome, we adopted propensity score (PS) matching. This enabled consideration of numerous covariates as contributors to confounding relative to number of events, and more granular exploration of matched and unmatched populations. Compared to multivariable analysis, matching accounts for changes in case characteristics and decision-making over time, by identifying comparable samples from observational real-world data [20].

To account for confounding from unmeasured discriminators of ‘too severe disease’ that would preclude ECMO, we excluded patients with a decision of ‘perceived futility’ from matching.

We additionally considered handling of patients labelled as ‘not meeting NHS criteria’. Changes to criteria occurred over the course of the pandemic (see “Study design”), and application of criteria required use of expert clinical judgement in deciding eligibility. Once patients labelled with ‘futility’ were excluded from the ‘not meeting NHS criteria’ group, the remainder contained many marginal cases, and the label was not felt to confer clear properties that might confound matching. We included this group for the main analysis, and tested assumptions in sensitivity analysis.

PS for treatment with ECMO was constructed using multivariable logistic regression. Covariates for PS generation (Supplementary Table F) were chosen based on their consideration in decision-making, and representation of severity.

Patients retrieved on mobile ECMO were matched with referred patients managed conventionally at referring centres. Three different matching techniques were evaluated: nearest neighbour calliper matching, genetic matching (GenMatch) [21], and optimal pair matching. A calliper width of 0.1 standard deviations of the logit of the PS was used for nearest neighbour and GenMatch. All matching was conducted in a 1:1 ratio without replacement. Balance between groups was assessed using standardised mean difference (SMD), with a SMD of < 0.1 indicating good balance [22].

The marginal odds ratio (OR) for hospital mortality with ECMO in the matched cohorts was estimated using univariable logistic regression. Bias-corrected confidence intervals were calculated using bootstrapping (4999 replications). The E value for each OR estimate of treatment effect was calculated to estimate unmeasured confounding [23, 24].

Statistical analysis and matching procedures were performed using R 4.0.3 (R Core Team, 2018) and the Matchit package [25].

Sensitivity analyses

Sensitivity analysis of PS matching was performed by (1) restricting eligible patients to those documented as ‘meeting NHS criteria’; (2) restricting population to patients with < 10% missing data. The matching procedure which generated the best combination of matches and covariate balance was used.

Ethics and approval

This study qualified as a service evaluation, as defined by the UK Health Research Authority, with waiver of individual informed consent. Data collection and analysis were carried out under evaluation for national ECMO commissioning.

Results

Patient characteristics

During the study period, 1580 patients with COVID-19 SRF were referred from 111 hospitals. Following exclusions, final analysis included 1363 patients, of which 243 were transferred to an SRF centre on mobile ECMO for continued treatment. Figure 1 shows a flow chart of patient inclusion. All patients in the treatment group received veno-venous (vv) ECMO (vv-ECMO). Referrals and outcomes over time are shown in Fig. 2. Prior to imputation, the dataset exhibited 11.1% missingness (Supplementary Table B, Supplementary Figure A).

Fig. 1
figure 1

Flow chart of patient inclusion, exclusion and sensitivity analyses

Full size image
Fig. 2
figure 2

Histogram of referrals for consideration of ECMO grouped by 7-day period with treatment and outcome shown

Full size image

Characteristics of the overall population and comparison based on referral decision are presented in Table 1. 430 (31.5%) patients had ‘perceived futility’ documented and were excluded from matching analysis. Compared to included patients, this group was older, with more severe organ failure, was hospitalised and ventilated for longer, and more likely to die (73% vs 43.2%, p < 0.001).

Table 1 Characteristics, decision and outcomes of COVID-19 patients referred for ECMO
Full size table

Comparisons between two COVID-19 waves are presented in Supplementary Tables C–E. Mortality for conventionally managed referrals increased in wave 2 (51.9% vs 62.4%, p = 0.001), but remained stable in ECMO-treated patients (22.9% vs 26.1%, p = 0.672). Wave 2 patients experienced shorter durations of hospitalisation and IMV prior to referral. They received more immunomodulation, less renal replacement therapy, more NIV, and developed greater barotrauma (with more chest drains). Overall, a greater proportion of patients with documented ‘perceived futility’ outcomes and a lower proportion of ECMO-treated patients were seen in wave 2.

Matching and treatment effect of ECMO

933 patients were available for matching. Nearest neighbour matching generated matches for 208 ECMO-treated patients, with a marginal OR for mortality of 0.47 (95% CI 0.32–0.73, p < 0.001, E value 2.28, E value lower CI 1.62) and absolute risk reduction (ARR) of 16.3% (42.3% vs 26%, p = 0.001) given ECMO treatment. GenMatch generated matches for 209 ECMO patients, with a marginal OR for mortality of 0.44 (95% CI 0.29–0.68, p < 0.001, E value 2.38, E value lower CI 1.72) and ARR of 18.2% (44% vs 25.8%, p < 0.001) given ECMO treatment. Optimal pair matching generated matches for 237 ECMO patients, with a marginal OR for mortality of 0.47 (95% CI 0.31–0.70, p < 0.001, E value 2.28, E value lower CI 1.68) and ARR of 16.5% (41.4% vs 24.9%, p < 0.001) given treatment with ECMO.

GenMatch matching resulted in the best combination of matches and covariate balance. Characteristics, outcomes, and standardised mean difference (SMD) of patients matched via this method are presented in Table 2. All covariates had a SMD ≤ 0.1 using GenMatch. The 34 ECMO-treated patients who were unable to be matched using this method were younger, had a shorter duration of IMV, were more hypoxic, had higher RESP scores, and received less corticosteroids (Table 3).

Table 2 Characteristics and outcomes of conventionally and ECMO-treated patients before and after GenMatch matching, showing all co-variates used in matching procedure
Full size table
Table 3 Comparison of matched (by GenMatch) and unmatched ECMO-treated patients
Full size table

Characteristics, outcomes, and balance metrics of patients matched via nearest neighbour and optimal pair matching are presented in Supplementary Tables G–I.

Sensitivity analysis

In sensitivity analysis of selection criteria, a subgroup with 463 patients were documented as ‘meeting NHS criteria’ for respiratory ECMO (220 conventional, 243 received ECMO). GenMatch generated 138 matches in this restricted cohort. The marginal OR for mortality was 0.52 (95% CI 0.32–0.89, p = 0.012, E value 2.12, E value lower CI 1.31) with an ARR of 14.5% (42% vs 27.5%, p = 0.016). Characteristics are shown in Supplementary Table J. Good balance was achieved with an SMD ≤ 0.1 for all covariates except for pH, which was slightly higher in the ECMO-treated cohort (SMD − 0.11).

In sensitivity analysis for missingness, a subgroup containing 550 patients remained (418 conventional, 132 ECMO) after exclusion for > 10% of missing data. Matches for 116 patients were generated using GenMatch, with marginal OR for mortality of 0.41 (95% CI 0.24–0.71, p = 0.002, E value 2.5, lower CI 1.66). Good balance was achieved with an SMD of ≤ 0.1 for all covariates. Characteristics shown in Supplementary Table K.

Discussion

The use of ECMO for COVID-19 SRF is supported by reports of safety and acceptable outcomes in retrospective cohorts [26], and more limited causal analysis in observational data. Our findings suggest that ECMO delivery at specialist SRF centres confers significant survival benefit, compared to matched patients who received conventional therapy in referring centres.

The considerable treatment effect (ARR 18.2%) produced in our study warrants closer examination. This effect was robust through different matching techniques and sensitivity analyses. For comparison, EOLIA [27] found an ARR of 11% between two arms, but was underpowered with substantial, late cross-over in the control group. Previous commentaries have noted that ARR would be greater if survival of cross-over patients was considered [28, 29]. The most comparable study from COVID-19 ECMO is Shaefi et al. [12], who adopted a multivariable adjustment approach, finding adjusted hazard ratio of 0.55 for death given ECMO (ARR 12.8%). In contrast, our results may represent added effect of treatment in a specialist SRF centre. Peek et al. [18] previously demonstrated ARR of 16% following transfer to a UK SRF centre, but with ECMO provided to only 76% of patients in the treatment group (as discussed, this separation is not present in standard UK ECMO pandemic pathways, where retrieval on mobile ECMO is usual practice).

Reported outcomes for COVID-19 ECMO remain varied. Recent examination of a large COVID-19 ECMO cohort [4] reported in-hospital mortality of 68%. Mortality in our study is considerably lower than Karagiannidis et al., and is also lower than that reported via international registries [26, 30] and previous non-COVID-19 trials [27]. The increase in ECMO mortality over time observed in other studies [30, 31] was also not seen in our cohort. Our population differs significantly from other published cohorts. First, while severity of respiratory failure was similar, it is likely that strict criteria enacted during the pandemic resulted in a more “well” population. Compared to published data, this is evidenced by younger age, shorter pre-ECMO ventilation, less renal support, lower SOFA, and higher RESP scores. Application of these criteria appeared consistent over time in our study, with clinical characteristics of ECMO patients remaining similar across waves (barring pre-ECMO therapeutic interventions). Second, all patients were retrieved on ECMO and treated in high-volume ECMO centres, which may associate with improved survival [26, 32]. These differences may highlight the impact of selective inclusion criteria, and centralised ECMO provisioning structures.

Neither our nor Shaefi et al.’s approach can negate effects of residual confounding. In considering unmeasured factors, it is notable that close matches were found from patients referred and considered for ECMO, but subsequently treated conventionally. Unmeasured features that may have influenced contemporaneous decision-making include views expressed over a telephone consult that are not reflected in available data. We note that characteristics and outcomes of referred patients changed over the course of our study period, and it is likely that understanding of patients thought to benefit from ECMO evolved over time. To challenge this, we restricted patient population in sensitivity analysis using human assigned labels (“met NHS criteria”), finding a reduced, but still considerable, ARR of 14.5%. The use of case-based matching also served to minimise temporal bias, with each case closely matched across clinical and physiological parameters. E values across our analyses suggest that substantial unmeasured confounding would be required to negate the observed treatment effect [23].

It is also important to acknowledge that realities of unprecedented pandemic demands may account for some of these close matches. It is reasonable to assume that some patients who could have benefited from ECMO may not have received it. The UK doubled its maximum ECMO capacity, indicative of significant system load, and it is conceivable that decision-making for cases with marginal or unclear benefit could have been influenced by factors related to resource availability at referring or receiving centre.

This study builds on previous studies of ECMO in H1N1 pandemic influenza [33, 34]. Our pool of eligible non-ECMO patients for matching was larger: n = 690 vs n = 195 (Noah et al.) and n = 157 (Pham et al.), resulting in more matches (n = 209 out of 243 eligible). Additionally, referrals recorded detailed pre-ECMO data; thus, matching was performed using variables that might represent heterogenous phenotypes [35, 36]. In designing this study, we address potential limitations discussed by Pham et al.: incorporation of rich covariates, reporting of matching procedures, use of stringent calliper width, and matching without replacement.

Our study presents several strengths, including use of rich covariate data to control most patient-centric factors. Our results remained robust through sensitivity analyses. ECMO patients were selected through the same nationally defined pathway and, outside of the concerns discussed, present a well-defined treatment group (previous study populations were treated in numerous centres with heterogeneous ECMO criteria). The use of data on which real-world decision-making took place adds pragmatism, and interpretability.

We also note important limitations. First, the possibility of residual confounding, which has been discussed. Second, pre-selection of patients via national referral criteria, and mixed exposure of both ECMO and SRF centre treatment, may limit generalisability to international cohorts where pathways differ. Third, we did not introduce measures of hospital-level strain or differences in conventional management, potentially resulting in exposure heterogeneity (i.e. transfer of patients from a ‘high strain’ hospital to a lower strain’ hospital or varying application of conventional management). However, analysis of UK hospital-level variation provides little evidence of differing strain between sites [37, 38] in context of nation-wide measures to ‘offload’ hospitals via patient transfer services [39]. Fourth, analysis is limited to in-hospital mortality, without examination of delayed out-of-hospital mortality, functional status of survivors or secondary complications on ECMO that may contribute to patient morbidity. Fifth, despite consistent results in sensitivity analysis for case-based data missingness, imputation of high missing for some variables (with assumption of missing-at-random) may reduce robustness. In summary, while similar outcomes were achieved across multiple analyses, results should be interpreted with caution.

Conclusion

Our study suggests that ECMO provided at specialist centres offers a survival benefit in selected patients with COVID-19 SRF, compared to similar patients treated with conventional therapy in their local hospital. Where resources and specialisation allow, ECMO should be offered in this critically unwell patient group. The subset of patients with severe COVID-19 who are likely to benefit most from ECMO remains unclear, and further work is required to determine factors which can assist in the decision-making process.