Background

The novel coronavirus 2019 (COVID-19) infection has spread worldwide causing thousands of cases of acute respiratory failure with an associated high mortality rate [1, 2]. Critically-ill patients with COVID-19 often have profound hypoxemia which may partially explain the extremely high use of invasive ventilatory support for long periods of time shown in these subjects [3, 4]. This issue, combined with the sharp rise in the incidence of this disease, has led to an unprecedented pressure on many healthcare systems and hospitals worldwide [4,5,6,7].

High-flow nasal oxygen (HFNO) reduces the need for endotracheal intubation in patients with acute respiratory failure [8,9,10]. In the last few months, several studies have reported experiences with HFNO therapy in patients with COVID-19 [11, 12]. Also, a recent publication suggested that HFNO compared to oxygen therapy could decrease the requirements for invasive mechanical ventilation in these patients [13]. If validated, the use of HFNO would not only be beneficial for individual patients treated noninvasively but also to those planned for invasive mechanical ventilation through the rational allocation of resources. Conversely, delaying intubation by choosing a non-invasive approach may be associated with worse outcomes in patients with the acute respiratory distress syndrome (ARDS) [3, 14,15,16]. Therefore, identifying those at higher risk of failure could be highly valuable for avoiding delays in choosing the best management approach.

In this study, we sought to describe the use of HFNO in adult patients with COVID-19 acute respiratory failure and to identify factors associated with a greater risk of intubation. We also aimed to derive a parsimonious predictive score for intubation as an aid in daily clinical decision-making.

Material and methods

Study design and setting

We conducted a prospective, multicenter, cohort study of consecutive patients with COVID-19 related acute respiratory failure admitted to 36 hospitals from Spain and Andorra (see Supplementary file) [17]. The study was approved by the referral Ethics Committee of Hospital Clínic, Barcelona, Spain (code #HCB/2020/0399) and was conducted according to the amended Declaration of Helsinki. This report follows the “Strengthening the Reporting of Observational Studies in Epidemiology (STROBE)” guidelines for observational cohort studies [18]. Gathering of data is ongoing and as of August 13, a total of 1129 patients were included.

Study population

For the present study, all consecutive patients included in the database from March 12 to August 13, 2020 that fulfilled the following inclusion criteria were analyzed: age ≥18 years, ICU admission with a diagnosis of COVID-19 related acute respiratory failure, positive confirmatory nasopharyngeal or pulmonary tract sample, and HFNO initiated on ICU admission day. Exclusion criteria were the use of oxygen therapy and non-invasive or invasive mechanical ventilation prior to HFNO or the absence of data regarding respiratory management on day 1 after ICU admission.

Data collection

Patients’ characteristics were collected prospectively from electronic medical records by physicians trained in critical care according to a previously standardized consensus protocol. Each investigator had a personal username/password, and entered data into a specifically pre-designed online data acquisition system (CoVid19.ubikare.io) endorsed and validated by the Spanish Society of Anesthesiology and Critical Care (SEDAR) [19]. Patient confidentiality was protected by assigning a de-identified code. Recorded data included demographics [age, gender, body mass index (BMI)], comorbidities and disease chronology [time from onset of symptoms and from hospital admission to initiation of respiratory support, ICU length of stay], vital signs [temperature, mean arterial pressure, heart rate], laboratory parameters (blood test, coagulation, biochemical), ratio of oxygen saturation to inspired oxygen fraction, divided by respiratory rate (ROX) index, and severity scores such as the Sequential Organ Failure Assessment (SOFA) and Acute Physiology and Chronic Health Evaluation II (APACHE II) scores. Data regarding physiological parameters was collected once daily. Site investigators collected what they considered to be the most representative data of each day from ICU admission to ICU discharge. After ICU discharge, patients were followed-up until hospital discharge.

Study outcomes

The primary outcome was the assessment of factors at ICU admission (ICU day 1) associated with the need for endotracheal intubation up to 28 days after HFNO initiation. The decision to intubate was made at the discretion of the attending physician at each participating site. Secondary goals were the development of a predictive model to estimate the probability of endotracheal intubation after HFNO and the assessment of between-center variability in the likelihood of receiving intubation after HFNO had been started.

Statistical analysis

We used descriptive statistics to summarize patients’ baseline characteristics. We compared the baseline characteristics of patients who required intubation with those who did not require intubation. Specifically, continuous variables were compared with the T test with unequal variances or the Mann-Whitney U test, as appropriate. Categorical variables were compared using the chi-square tests or Fisher’s exact test as appropriate. In order to identify factors associated with the likelihood of intubation, we fit a multivariable logistic regression model with endotracheal intubation as the dependent variable. A priori selected variables were those considered of clinical relevance as well as variables that were significantly associated with the outcome in the bivariate analysis (at a p value threshold of 0.2 or less). We report odds ratios (OR) with their associated 95% confidence intervals (CI).

Then, we sought to derive a parsimonious predictive model for intubation among patients treated with HFNO on the first day of ICU admission. Thus, we randomly split the full dataset in two parts: (1) a training dataset including 70% of the patients, and (2) a validation dataset including the remaining 30% of subjects. In the derivation step, all variables showing statistical significance with the outcome were chosen, and a final model based on the best accuracy was selected after performing tenfold cross-validation. The final model calibration was tested in the split validation cohort with the use of the Brier score. A receiver operating characteristic (ROC) curve was constructed to display the area under the curve (AUC) for the predictive model. The optimal cutoff was considered as the one showing the best accuracy. At this cutoff, the performance of the model is presented as sensitivity, specificity, positive and negative predictive values, and positive and negative likelihood ratios and their accompanying 95% CI. An online calculator is shown to estimate the likelihood of HFNO failure for each individual patient. Since validation datasets with few observations can provide imprecise estimates of performance, a sensitivity analysis to assess final model performance using enhanced bootstrapping was also carried out [20].

Additionally, since one of the goals of the present study was to assess center-related variability regarding the clinical decision to intubate, a mixed-effects multivariable logistic regression was fit as a secondary analysis. We fit a logistic model with a random intercept (for each center that recruited more than 10 patients), to account for possible correlation and differences in the baseline risk of intubation based on practice variation between sites. The proportion of variance explained by all fixed factors is presented as the marginal R2 and the proportion of variance explained by the whole model is presented as the conditional R [2, 21].

To account for missing data, which occurred in 6% of the observations of interest, we performed multiple imputation based on Markov chain Monte Carlo methods [22]. Specifically, for regression analysis, we removed subjects with extensive missing data (>50%). Briefly, for every missing value, we created 5 matrices, each one with 1000 imputations. Final imputed values for each missing observation were calculated as the median of all imputations. Imputation of the dependent variable (intubation) was not performed. We used a threshold of 0.05 for statistical significance and all reported tests are two-sided. For statistical analysis, we used the R software (R Foundation for Statistical Computing, Vienna, Austria) and included mice, lme4, caret, OptimalCutpoints, performance, and pROC packages.

Results

From March 12 to August 13, 2020, 259 critically ill patients with COVID-19 related acute respiratory failure were initially treated with HFNO and were included in the present study (Fig. 1). From those, 140 (54.0%) patients were intubated and mechanically ventilated after ICU admission, of whom 74 patients (52.9%) were intubated on the ICU admission day. SOFA score and APACHE II were higher in patients requiring intubation while respiratory rate, PaO2/FiO2 ratio, and ROX index were lower (Table 1).

Fig. 1
figure 1

Patient flowchart. Two hundred fifty-nine patients were included and followed up until ICU discharge or death. NIV, non-invasive ventilation; IMV, mechanical ventilation

Full size image
Table 1 Baseline characteristics of 259 patients with COVID-19 acute respiratory failure
Full size table

Associated factors and predictive model for intubation

After excluding 3 subjects for extensive missing data, 256 patients were included in the multivariable logistic regression analysis. Baseline non-respiratory SOFA score (OR 1.78; 95% CI 1.41-2.35), ROX index (OR 0.53; 95% CI 0.38-0.72), and pH (OR 0.47; 95% CI: 0.24-0.86) were associated with the need for intubation (Table 2). A model including the non-respiratory SOFA, the ROX index and cancer showed the best accuracy in the training dataset (see Additional file 1, Table S1). However, given that cancer was a protective factor for intubation, which probably meant treatment escalation limitation, a simpler model including non-respiratory SOFA and the ROX index was selected. In the validation subset, this model had excellent calibration (Brier score of 0.14) and discrimination (AUC of 0.88, 95% CI 0.80-0.96) (see Table 3 and Fig. 2).

Table 2 Associated factors with intubation in 256 patients with COVID-19 treated with HFNO
Full size table
Table 3 Discrimination ability of the model in the test dataset using a 50% probability-of-intubation cut-off
Full size table
Fig. 2
figure 2

ROC curve in the validation dataset. AUC, area under the curve. The black dot on the ROC curve depicts the optimal threshold, as defined as the probability cut-off with the best accuracy

Full size image

Additionally, 216 patients, enrolled in 7 centers with 10 or more cases, were included in a mixed-effect analysis (see Additional file 1, Table S2). Baseline non-respiratory SOFA score and ROX index remained as independent predictors of intubation (see Additional file 1, Table S2). Overall, fixed effects explained 63% of the variability of the outcome while individual centers explained an additional 1% (see Additional file 1, Table S3 and Figure S1). An online calculator to predict the likelihood of intubation given baseline non-respiratory SOFA score and ROX index was developed (see https://desbancar.shinyapps.io/DESBANCAR/).

Out-of-sample model performance using enhanced bootstrapping is shown in the supplementary file (“Further details on statistical analysis,” “Results,” and “Figure S2”).

Discussion

In this multicenter cohort study of 259 critically ill adult patients with COVID-19 initially treated with HFNO, the need for intubation and invasive mechanical ventilation was frequent and occurred in more than 50% of patients. Non-respiratory SOFA and the ROX index were the main predictors of endotracheal intubation.

Unlike previous studies in non-COVID patients [9, 23], poor oxygenation at baseline, as measured by PaO2/FiO2, was not a reliable predictor of intubation. While hypoxemia seems often homogenously noticeable in this population, its mechanisms may be multifactorial and might change over time as the disease progresses [24]. Cressoni et al. described the distinction between anatomic to functional shunt in ARDS, and Gattinoni et al. have recently reported that the ratio of the shunt fraction to the gasless compartment in COVID-19 subjects is often higher than the values found in ARDS [25, 26]. Recently, Chiumello et al. highlighted the differential radiologic pattern of COVID-19 patients as compared to non-COVID-19 ARDS [27]. Similar to previous studies in both non-COVID and COVID patients, our study supported how ROX index, which encompasses information from both oxygenation and respiratory rate, was useful to predict intubation [12, 28]. In the absence of non-pulmonary involvement, a ROX index of 3.5 at admission conferred a 50% chance of intubation, which was 83% sensitive and 89% specific for HFNO failure. Of note, the present study differs from previous reports in the percentage of patients receiving HFNO from the total population of patients with COVID-19 related acute respiratory failure [5, 6]. Specifically, the patient population in the present study comprised 24% of the whole database, potentially showing that clinicians seemed to be keener (compared to previously published reports) on using this non-invasive oxygenation strategy in this patient population. This in turn may also explain the lower PaO2/FiO2 ratios that were often observed [5, 6] and potentially, the lack of impact on the initial decision to switch from HFNO to invasive mechanical ventilation. Although high-quality evidence is needed to assess the effect of HFNO in COVID-19 patients, its use has increased since the start of the pandemic [29]. Moreover, recently published observational data suggests HFNO might increase ventilator-free days and decrease ICU length of stay without incurring in excessive mortality [10].

Our parsimonious model, which included non-respiratory SOFA and the ROX index, to predict intubation among patients with COVID-19 treated with HFNO showed excellent discrimination and may be helpful in the decision-making process at the bedside. The model also shows strong clinical rationale. It is plausible that as lung mechanics deteriorated in some patients, respiratory drive increased, making the ROX index a valuable tool to predict HFNO failure. Likewise, pH was often lower and PaCO2 higher in subjects who later became intubated, suggesting fatigue or increased lung injury in failing subjects. Non-respiratory SOFA score was higher in intubated patients and this was mostly related to hemodynamic impairment. Finally, our mixed-effects analysis showed that most of the variability for the need of invasive mechanical ventilation can be explained by baseline factors at admission, while differential “ICU culture” does not appear to play a major role in this decision. This needs to be analyzed in comparison to previous research showing fairly strong center effects, both in the care of patients with septic shock and mechanically ventilated critically ill adults [30, 31].

Our study has several strengths. First, data were collected prospectively in a nationwide project and one of its main goals was to specifically study the relationship between respiratory treatment and outcome. Second, we were able to derive a parsimonious, potentially easy-to-use model that could aid in the identification of patients who may need intubation while being treated with HFNO. However, we acknowledge some limitations of our findings. First, observational studies, especially those multicenter in nature, as our study, are prone to misclassification of relevant covariates and potential predictors. Specifically, physiological parameters were collected once daily, and researchers were instructed to collect the most representative data over the study day. Although unlikely that researchers disregarded the values obtained during HFNO (since they were likely more abnormal than during mechanical ventilation), we cannot ensure completely that some patients, who became intubated on day 1, had their data collected after mechanical ventilation had been started, thus, representing a potential source of bias in the estimation of the predictive model for HFNO failure. Second, missing data on candidate predictors was present in the final sample, rendering our reported associations subject to information bias, and potentially decreasing the precision of our estimates. However, our results were robust while using multiple imputation.

Conclusions

In conclusion, in this observational study of 259 adult critically ill patients with COVID-19 related acute respiratory failure receiving HFNO, approximately 1 out of 2 patients were intubated during the subsequent ICU stay. Oxygenation at baseline was not a good predictor of HFNO failure, while non-respiratory SOFA, pH, and ROX index were independently associated with intubation. Little variation on the decision to intubate was observed across included centers. Future studies should confirm our findings and evaluate the performance of our model in external cohorts.