Background

In immunocompromised patients with acute respiratory failure (ARF), invasive mechanical ventilation remains associated with a high mortality rate [1]. Several oxygenation strategies to avoid intubation have been tested in this setting. More recently, high-flow nasal cannula (HFNC) has been evaluated in non-immunocompromised patients, resulting in an improved oxygenation ratio and reduced intubation rate, as shown in several recent meta-analyses [2, 3]. In immunocompromised patients, the role of HFNC seems less clear, and a higher oxygenation rate does not translate into improved outcomes [4, 5]. Moreover, all these studies highlight the risk of delayed intubation, which is associated with a higher mortality rate [6]. Therefore, particular attention should be paid to the time of intubation to determine which patient would benefit from a non-invasive strategy of oxygenation [7]. So far, there is a lack of objective criteria to decide intubation, particularly within the first hours after the onset of the oxygenation strategy. Some recent studies have assessed several indices, including clinical and respiratory parameters [8, 9], such as the ROX index—including oxygen saturation (SpO2), fraction of inspired oxygen (FiO2), and respiratory rate (RR)—which is easy to use [10, 11]. A ROX index (SpO2/FiO2/RR) over 4.88 within 2–12 h of starting HFNC was associated with a lower risk of intubation. The area under the curve (AUC) of the receiver operating characteristic (ROC) curve in the validation cohort was 0.703 (0.616–0.790) at 6 h and 0.752 (0.664–0.840) at 12 h. However, the studies were carried out in non-immunocompromised patients with pneumonia-related ARF. For immunocompromised patients, ARF aetiologies are numerous and response to HFNC could vary [7].

The aim of this study was to validate the ROX index in an external dataset, including only immunocompromised patients with ARF.

Methods

This is an ancillary study including two previous trials, IVNICTUS and HIGH [5, 12]. The IVNICTUS study, described elsewhere, evaluated an oxygenation strategy using non-invasive ventilation (NIV) versus oxygen in immunocompromised patients with ARF [12]. Oxygen could be maintained by HFNC in both groups. The HIGH study, also described elsewhere, compared oxygen versus high-flow nasal oxygen in immunocompromised patients with ARF [5]. In both studies, HFNC was used according to recommendations (gas flow of 50–60 L/min, FiO2 adjusted for SpO2 over 92%, temperature 37 °C). Institutional review board agreement and written informed consent were obtained from each patient or surrogate decision-maker (CPP Ile de France IV 2012/11SC and IDRCB: 2016-A00220-51). The inclusion criteria in both studies were ARF, defined by tachypnea over 30/min, respiratory distress (laboured breathing) and SpO2 < 90% on room air at ICU admission. The non-inclusion criteria were ARF being related to isolated cardiogenic pulmonary oedema and invasive mechanical ventilation at ICU admission. Persistent hypoxemia or dyspnoea were among the intubation criteria.

The secondary analysis included all patients who received HFNC at least 6 h after randomization. The exclusion criteria for this secondary analysis were: do not intubate order, intubation before 6 h, and missed data on the ROX index at 6 h.

The ROX index (SpO2/FiO2)/RR) was calculated at 6 h and also at 12 h for patients included from the IVNICTUS study (the variable was not available in the HIGH study). Patient characteristics at ICU admission and outcomes were also recorded.

The ARF aetiology was determined using the appropriate diagnostic strategy and was classified as bacterial or viral pneumonia, opportunistic infection, lung involvement by the underlying disease, drug-related pulmonary toxicity, other identified cause, or undetermined aetiology. The diagnostic strategy and criteria for each diagnosis are described elsewhere [13]. Patients with ARF related to isolated cardiogenic oedema were not included in the trials [5, 12]. Radiographic patterns at ICU admission were described as 1–2 or 3–4 involved quadrants. The primary endpoint was the need for intubation throughout the ICU stay. The primary objective was to assess the ability to discriminate between patients who would need intubation and patients who would not require intubation.

The secondary objective was to evaluate the ability of the ROX index to stratify intubation risk.

Statistical analyses

Results were expressed as median and 25th and 75th quartiles (Q1–Q3) for the quantitative data, and numbers and percentages for the categorical data. The quantitative variables were compared using the Student’s t test, or the Wilcoxon test in case of non-normal distribution. The qualitative variables were compared using the Chi-square test or Fisher’s exact test, as appropriate. Missing data were not imputed, and the number of analysed patients is described in each table in the results section.

Several analyses were carried out. The first was a univariate analysis to describe the characteristics and outcomes of intubated and non-intubated patients, while the second analysis assessed the predicting value of the ROX index for intubation using a ROC curve to discriminate between patients who needed intubation and those who had HFNC success. The thresholds found in a previous study were then evaluated for sensibility, specificity, likelihood positive ratio and likelihood negative ratio [10, 11].

In a third analysis, the ROX index was adjusted with other variables associated with a high risk of intubation [14] that could be assessed during the first 24 h of ICU. A multivariate logistic model was performed including characteristics at randomization (oxygen flow over 9 L/min, study), ARF related to pneumonia and SOFA score at day 1. All clinically relevant variables decided a priori and those with a low level of missing data were included in the multivariate analysis, even in cases of non-significant difference in the univariate analysis. The odds ratios (ORs) of variables present in the final model are given with their 95% confidence intervals (CI).

Several sensitivity analyses were performed

Firstly, a ROC curve analysis was performed to assess the accuracy of the multivariate model to discriminate between patients who would need intubation and those who would not. Secondly, in order to assess the impact of the ROX index, we assessed the predicting value of a modified multivariate model of intubation risk, excluding the ROX index, using a ROC curve to discriminate between patients who needed intubation and those who had HFNC success.

Thirdly, the risk stratification of the ROX index was then explored with the probability of intubation for each quartile of the ROX index.

Fourthly, one sensibility analysis was carried out excluding all patients who received NIV. In the INVICTUS study [12], half of the patients were randomized to receive a NIV session immediately after randomization. Although the ROX index was calculated outside a NIV session, the NIV session may modified the risk of intubation, the RR and the SpO2 at hour 6. To assess the impact of this device, we performed the same analysis excluding patients who received NIV.

The ROX index at 12 h was also analysed, where available (in the IVNICTUS cohort only).

All analyses were carried out using the R 3.3.3 statistical software, and the statistical significance level was fixed at 0.05.

Results

This study included 302 patients admitted with ARF (Fig. 1). Patients were mostly male (n = 208, 68.9%), with a median age of 63 (55–70). The underlying disease was mostly haematological malignancy (n = 225, 74.5%), while ARF was related to viral or bacterial pneumonia (n = 150, 49.7%), opportunistic pneumonia (n = 54, 17.9%), toxicity or underlying disease (n = 26, 8.6%) and other miscellaneous reasons (n = 52, 17.2%) including pulmonary embolism, extrapulmonary acute respiratory distress syndrome (ARDS), pleural effusion and atelectasis.

Fig. 1
figure 1

Flowchart

Full size image

The oxygen flow needed at randomization was 10 (7–15) L/min, and 166 (55%) patients required an oxygen flow over 9 L/min. Forty-five (14.9%) patients received NIV and HFNC. The RR was 27 (22–33)/min, 30 min after randomization. In the HIGH study, there were guidelines for HFNC use with at least 50 L/min [5]. In the IVNICTUS study, HFNC flow was 50 (35–50) L/min at initiation. For 122 (49.6%) patients, 3–4 quadrants were involved on the chest X-ray. The median ROX index at 6 h was 5.62 (4.17–8.29).

Throughout the ICU stay, invasive mechanical ventilation was required by 115 (38.1%) patients within the 2 (1–3) days. The length of hospital stay was 24 (14–29) days, while the ICU mortality rate was 27.4% (n = 83). Table 1 summarizes patient characteristics at ICU admission and outcomes based on the need for invasive mechanical ventilation. The ROX index at 6 h was significantly different between patients who required intubation and those who did not. The probability of intubation increased when the ROX index decreased (Fig. 2). Table 2 describes patient characteristics according to the ROX index value (over or below 4.88).

Table 1 Characteristics and outcomes based on the need for intubation
Full size table
Fig. 2
figure 2

Probability of intubation according the ROX index

Full size image
Table 2 Characteristics and outcomes based on ROX index over or under 4.88
Full size table

Prediction of intubation need using the ROX index at 6 h

Additional file 1: Figure S1 describes the ROC curve for the probability of intubation based on the ROX index in our cohort. The AUC was 0.623 (0.557–0.689). Using the threshold previously found (ROX index over 4.88 at 6 h, was associated with lower intubation rate) [11], the prediction of intubation was very low (sensibility of 52.1%, specificity of 68.9%, positive likelihood ratio of 1.69 and negative likelihood ratio of 0.69). Also, using an ROX index below 3.47 [11], the prediction of HFNC failure had higher sensibility (90.9%) but a specificity of 21%, a positive predictive value of 65%, a negative predictive value of 59%, a positive likelihood ratio of 2.3 and a negative likelihood ratio of 0.86.

Adjusted ROX index at 6 h based on other risks of intubation

In a multivariate analysis that included patients’ characteristics at admission (Table 3), a higher ROX index was still independently associated with a lower intubation rate (OR = 0.89 [0.82–0.96], p = 0.04). The SOFA score at day 1 was associated with a higher risk of intubation. There was no interaction between SOFA J1 and ROX index. A modified model without SOFA day 1 did not modify the result on the ROX index [OR = 0.89 (0.83–0.96)].

Table 3 Adjusted ROX index 6 h after HFNC onset according other risk factors of intubation in all cohorts (multivariate analysis)
Full size table

Additional file 1: Figure S2 describes the probability of intubation according to the multivariable model including the ROX index.

Sensibility analyses

Additional file 1: Figure S3 describes the ROC curve of the probability of intubation with an adjusted ROX index at 6 h. The AUC of this curve was 0.662 (0.596–0.729). This ROC curve was compared to the one from a model including all variables a priori associated with intubation but without the ROX index at 6 h (Additional file 1: Figure S4). The AUC of this second ROC curve was 0.632 (0.563–0.702). The model including ROX index showed a better overall fit (p = 0.002).

Additional file 1: Figure S5 describes the risk stratification of intubation according to the quartile of the ROX index. The probability of intubation decreased with a high ROX index, without overlap between each quartile, leading to a good performance of risk stratification.

Statistical analysis of the impact of NIV was explored by excluding the patients who eventually received NIV within the first 6 h. The AUC of the ROC curve was 0.630 (0.558–0.703). In the multivariate analysis, the ROX index at 6 h was still independently associated with a lower intubation rate [OR = 0.90 (0.82–0.97), p = 0.01]. In this secondary analysis, the SOFA score at day 1 was also associated with intubation [OR = 1.16 (1.06–1.27), p = 0.001].

Finally, in the IVNICTUS study, the ROX index was available for 89 out of 95 (93.6%) patients, among whom 39 (43.8%) were intubated during the ICU stay after 12 h. The AUC of the ROC curve was 0.655 (0.538–0.772).

Discussion

In this study, the ROX index 6 h after HFNC onset had a poor performance in discriminating between patients who would be intubated or not, but this index could be a tool for risk stratification. Also, the ROX index was still associated with the risk of intubation, even when other significant predictors were included, such as the SOFA score [14].

Nevertheless, its accuracy to discriminate between patients who would need intubation and those who would receive only a non-invasive oxygenation strategy was very low, with low specificity and sensibility. In addition, the performance of a lower threshold (3.47) to predict HFNC failure was poor. Despite the low accuracy of this index to discriminate between patients who would ultimately need intubation or not, the ROX index can be easily recorded at the bedside. However, although RR, saturation and oxygen levels could be helpful for extreme values, the performance of these parameters to determine the intubation risk remains poor for medium values. Moreover, the ROX index at 6 h is a static measurement of clinical condition in ARF patients. Predicting intubation remains difficult and may depend on several conditions. Apart from respiratory parameters, the decision to intubate would also be determined by other organ dysfunctions, particularly haemodynamic and neurological dysfunctions. Repeated measures of oxygenation may also be of importance [11]. Few studies included the neurological and haemodynamic status in their predictive score [9]. Our study included ARF aetiology, oxygen needs at randomization and organ failure in the multivariate analysis, as these parameters have been associated with a risk of intubation [14]. Regarding the duration of oxygen needs and the assumed evolution of the disease, intubation would be necessary in the most severe patients and should not be delayed [6]. The physician’s experience could also influence the decision to intubate. As not all these characteristics can be included in a sole oxygenation index, this contributes to the poor accuracy of the ROX index to discriminate between patients who would need intubation and those who would not.

Moreover, the ROX index is a single measure and may not reflect the clinical evolution of a patient. With the sensitivity analysis at 12 h, we explored a second ROX index value, but this was also only a static measurement. Mauri et al. assessed the response to different levels of gas flow [15]; interestingly, they found a subgroup of patients where the ROX index of the most severe one rose within 20 min of increasing the gas flow on the HFNC. It is possible that a higher gas flow could increase lung recruitment. These changes highlighted the difficulty of determining a sole oxygenation index to distinguish between patients who would need intubation and those who would not. Roca et al. suggested a repeated measurement of the ROX index to increase its accuracy [11]. Also, this result identified the need for a scalable variable to detect a worsening outcome. Probably only the analysis of a large dataset could determine the profile of a patient who would need intubation.

According to our study, more than accuracy to discriminate between patients who need intubation or not, the ROX index may have a good performance for risk stratification [16]. Contrary to the limited clinical application of the ROX index itself, the stratification in low, intermediate and high-risk patients may be helpful for the early identification of patients who would fail the HFNC trial in further studies. Such results should be confirmed in further studies.

This study has several limitations. Firstly, it was a retrospective analysis of data, and the reason for intubation was not described. Although intubation occurred within the first days of ICU admission, a later intubation may not be related to the ROX index value at 6 h. Moreover, 108 patients could not be included in this analysis because of missing data. Those patients might have modified the conclusion. However, the study included a high number of patients, as in previous studies on the ROX index [11].

Secondly, this external validation cohort included only immunocompromised patients. Indeed, different aetiology of ARF [13], different oxygenation needs and different comfort related to the device [17] in immunocompromised patients may have modified the response to HFNC and, consequently, the ROX index accuracy.

Thirdly, we could not analyse the need for a vasopressor or other organ failure supply before intubation. Only the SOFA score at day 1 was analysed, even in cases of intubation. This may be a major limitation, because some patients would display haemodynamic instability after intubation. However, we did not detect any interaction between the ROX index and the SOFA day 1 score. Only prospective data could provide a remedy to this point.

Fourthly, some patients received NIV within the first 6 h. Those NIV sessions could have increased lung recruitment and modified the ROX index. However, the sensibility analysis excluding those patients did not show any difference, and the ROX index remained associated with the risk of intubation.

Fifth, unlike previous studies [11], several ARF aetiologies were included in our research. In the setting of immunocompromised patients, the length of oxygen needs and the severity of ARF may vary according to the ARF aetiology. However, half of patients had bacterial pneumonia.

Conclusion

Although the ROX index appears to have a low performance in predicting intubation among immunocompromised patients with acute respiratory failure, it still represents one of the strongest associated factors with intubation. More data including non-respiratory parameters may be included in further prospective cohorts to confirm these results. However, the index seems to have a good performance for risk stratification and may be useful for further studies.