Introduction

There is an increasing use of high flow nasal cannula oxygen therapy (HFNC) for acute hypoxemic respiratory failure, encouraged by evidence suggesting reduced intubation rates and possibly lower mortality [1, 2]. It is an attractive alternative to non-invasive ventilation (NIV) or conventional oxygen therapy because of its reported advantages in patient comfort, improved oxygenation, and decreased work of breathing in respiratory failure [3,4,5]. HFNC may also be used to reduce the rate of respiratory failure following a planned extubation [6,7,8,9]. Along with the growing use of HFNC, there is a need to improve early prediction of HFNC failure as delayed intubation is associated with increased mortality [10,11,12]. Patients identified as having a high risk of HFNC failure should be closely monitored or considered for early intubation, which may potentially improve patient outcomes.

The ROX (respiratory rate oxygenation) index, a ratio of pulse oximetry/fraction of inspired oxygen (P/F ratio) to respiratory rate per minute, has been validated to predict HFNC success in patients with pneumonia and acute respiratory failure [13, 14]. It is easily derived from commonly recorded variables measured in a non-invasive manner. However, it remains to be seen if the ROX index will perform as well in patients with respiratory failure from other causes than pneumonia, or in patients with HFNC initiated after a planned extubation. In addition, it is also unclear if the ROX index can be further improved by incorporating other vital sign parameters. Tachycardia recorded as early as 1 h into HFNC therapy has been found to be associated with HFNC failure [15]. Heart rate is a commonly measured vital sign, and incorporation into the ROX index may improve the diagnostic accuracy of the index.

In this study, we aim to evaluate the ROX index and a modified ROX index incorporating HR, in patients initiated on HFNC for hypoxemic respiratory failure and as a preventative treatment following extubation. As heart rate has an inverse relation to HFNC success, we defined the ROX-HR (respiratory rate oxygenation-heart rate) index as the ratio of ROX index over HR (beats/min) and multiplying by a factor of 100 (Figure S1).

Methods

Study design

We performed a prospective observational cohort study of patients initiated on HFNC (Optiflow device—MR850 heated humidified, RT202 delivery tubing and nasal cannula; Fisher & Paykel Healthcare, Auckland, New Zealand)TM at a medical intensive care unit (ICU) and intermediate care area of a tertiary-care medical centre. All consecutive patients initiated on HFNC from February 2017 to September 2019 were recruited for the study. Exclusion criteria of the study were patients initiated on HFNC for bronchoscopic procedures or as a rescue therapy post-extubation, patients who were started on non-invasive ventilation after HFNC failure, and patients with a ‘do not resuscitate or intubate’ order. We obtained approval from our institutional review board for this study (CIRB Ref 2016/2988). No written consent was required in view of the purely observational nature of the study.

HFNC protocol and management

Patients eligible for HFNC in our centre’s protocol include patients with acute hypoxemic respiratory failure, defined as having a respiratory rate > 25 breaths/min and a P/F ratio of < 300 mmHg on an oxygen device delivering 10 ≥ litres/min (LPM), in the absence of chronic respiratory failure. In our centre, HFNC may also be initiated as a preventative treatment, initiated immediately following a planned extubation. Prior to extubation, all patients had to fulfil clinical weaning criteria with a successful spontaneous breathing trial. Protocol exclusion criteria for all patients are the presence of hypercapnia (PaCO2 > 45 mmHg), acute respiratory failure secondary to asthma, chronic obstructive pulmonary disease (COPD) exacerbation or cardiogenic pulmonary edema, hemodynamic instability requiring vasopressor support, Glasgow coma scale (GCS) < 12 and epistaxis or recent facial or nasal surgery. HFNC was initiated at a minimum initial flow of 40 LPM. Flows were increased up to 60 LPM if required, or FIO2 adjusted as appropriate, with a target SpO2 of ≥ 92%. Discontinuation of HFNC and initiation of intubation and mechanical ventilation were based on the clinical judgement of the primary physician, guided by a protocol recommendation to consider mechanical ventilation in the presence of persistent/worsening respiratory distress, respiratory rate > 40 breaths/min, SpO2 < 90% for more than 5 min despite maximum flow and FIO2, acidemia with pH < 7.35, significant hemodynamic instability (defined as systolic blood pressure < 90 mmHg, mean arterial pressure < 65 mmHg or vasopressor requirement), deterioration in neurological status (GCS < 12) or inability to clear secretions.

Data collection

Patient demographics, Charlson comorbidity index (CCI), clinical severity scores and arterial blood gas sampling before initiation of HFNC were recorded upon inclusion into the study. Chest radiographs (CXR) were evaluated at the beginning of HFNC therapy. The acute physiologic assessment and chronic health evaluation II (APACHE II) score and sequential organ failure assessment score (SOFA) were recorded based on the highest scores in the 24 h preceding HFNC initiation [16, 17]. The presence of chronic kidney disease (CKD) was defined as having a baseline serum creatinine of > 265 μmol/L or requiring long-term dialysis. Chronic pulmonary disease was defined as symptomatic dyspnea from a chronic respiratory condition such as COPD or interstitial lung disease. Patients were considered as immunocompromised if they had one or more of the following: haematological or solid organ malignancy, prior haematological or solid organ transplantation, human immunodeficiency (HIV) infection, liver cirrhosis with portal hypertension or receiving long-term immunosuppressive therapy. Patients were followed up until in-hospital death or discharge from hospital.

ROX and ROX-HR index

We recorded the HR and ROX index before initiation of HFNC and at 1, 2, 4, 6, 8, 10, 12, 18, 24 and 48 h after HFNC initiation. HFNC success was defined by liberation of HFNC, and failure was defined by intubation and mechanical ventilation. Duration of HFNC was recorded as the time (h) from initiation of HFNC to successful liberation or failure. At the time of termination of HFNC, the ROX and ROX-HR index based on the latest available parameters from 1 h before termination were also recorded.

Statistical analysis

Data are presented as number (%) for categorical variables and median (interquartile range [IQR]) for continuous variables. Data and analyses are separated based on the indication for HFNC: acute respiratory failure vs post-extubation. Patients with and without successful HFNC were compared with respect to clinical and demographic characteristics by using the Mann-Whitney U test for continuous variables and the chi-square test or Fisher exact test as appropriate for categorical variables. The ROX and ROX-HR index at different time points were evaluated with the area under the receiver operating characteristic curve (AUROC) for the ability to correctly classify patients as HFNC success or failure. Cutoffs for the ROX and ROX-HR index, rounded off to the nearest 0.1, were chosen to maximise the sum of sensitivity and specificity based on the receiving operating characteristic curves. From these cutoffs, Kaplan-Meier (KM) plots for HFNC failure were determined and compared using the log-rank test. Univariate and multivariate Cox proportional regression analysis was performed to evaluate the hazard ratio for cumulative probability of HFNC failure based on the ROX and ROX-HR index at different time points. Covariates that were associated with HFNC failure (p value of < 0.10) on univariate Cox proportional regression analysis were included in the multivariate analysis. Statistical difference was considered significant at p ≤ 0.05. All statistical analyses were performed using the SPSS software (IBM SPSS Statistics ver. 22 Chicago, IL, USA).

Results

Patient population and HFNC outcomes

One hundred and forty-five patients were included in the study analysis. Nineteen patients were excluded: six patients had HFNC support initiated for bronchoscopy, five patients were switched from HFNC to NIV therapy, one patient had HFNC terminated due to epistaxis, one patient had HFNC terminated for transfer to the operating theatre for surgery, and six patients had a ‘do not resuscitate or intubate’ order. Immunocompromised patients made up 56.6% (n = 82) of the study population. Twenty patients had recently received chemotherapy for solid organ malignancies, 28 patients had an underlying haematological malignancy or a bone marrow transplant, and 29 patients were receiving chronic immunosuppressive therapy.

Ninety-nine patients (68.3%) were initiated on HFNC for acute hypoxemic respiratory failure (Table 1). Pneumonia was the most common primary diagnosis (87/99, 87.9%). The median P/F ratio was 94 (IQR 74–138), and SOFA score was 4 (IQR 3–6) at the time of HFNC initiation. Forty-five (45.5%) patients required intubation (HFNC failure) at a median of 16 (IQR 7–36) h after HFNC initiation. HFNC failure was associated with a higher SOFA and APACHE II score (recorded as the highest score in the preceding 24 h before initiation of HFNC) and an increased proportion of CXR quadrants affected at the time of HFNC initiation (Table 1). There were no significant differences found in the proportion of immunocompromised patients or pre-HFNC arterial blood gas analysis (pH, P/F ratio, PaCO2 and serum bicarbonate).

Table 1 Comparing baseline characteristics and outcomes of patients with acute hypoxemic respiratory failure (n = 99)
Full size table

Forty-six patients (31.7%) were initiated on HFNC post-extubation (Table 2). The median duration of mechanical ventilation prior to extubation was 114 (IQR 61–194) h. Prior to extubation, the median pH was 7.45 (IQR 7.42–7.48), with a PaCO2 of 41 (IQR 36–44) mmHg and P/F ratio of 164 (IQR 137–184). Sixteen (16/46, 34.8%) patients required re-intubation, at a median duration of 46 (10–87) h after HFNC initiation. Patients with HFNC failure were more likely to be immunocompromised (75% vs 37%, p = 0.029).

Table 2 Comparing baseline characteristics and outcomes of patients initiated on HFNC post extubation (n = 46)
Full size table

In both groups of patients (acute respiratory failure and post-extubation), HFNC failure was associated with a higher hospital and ICU mortality rate (Tables 1 and 2). Among all patients with HFNC failure, 22 (22/61, 36.1%) and 38 (38/61, 62.3%) patients were initiated on mechanical ventilation within 12 and 24 h, respectively. Initiation of mechanical ventilation after 24 h of HFNC was associated with a higher in-hospital (78.3% vs 50.0%, p = 0.029) and ICU mortality rate (69.6% vs 35.1%, p = 0.009). For the 61 patients with HFNC failure, the Kaplan-Meier plot for the probability of being free of mechanical ventilation is shown in Fig. 1a and b, which illustrates that patients who did not survive hospital admission had a longer duration of HFNC before intubation.

Fig. 1
figure 1

a Kaplan-Meier plot for the time free of mechanical ventilation in patients with HNFC failure (n = 61). b Among patients with HFNC failure, patients who did not survive had a longer duration of HFNC before intubation and mechanical ventilation

Full size image

Performance of the ROX and ROX-HR index

In patients initiated on HFNC for acute respiratory failure, after 2, 6, 10 and 24 h, 98 (99.0%), 90 (90.9%), 83 (83.9%) and 67 (67.7%) patients remained free of mechanical ventilation, respectively (Table 3). Patients with HFNC failure had a significantly lower ROX and ROX-HR index recorded at all time points, and a significantly higher heart rate was observed at 1, 2, 4, 10 and 12 h of HFNC (Table 3). Within the first 12 h, both indices appeared to have the highest diagnostic accuracy at 10 h with an AUROC of 0.723 (95% confidence interval (CI) 0.605–0.862) and 0.739 (95% CI 0.626–0.853) for the ROX index and ROX-HR index, respectively. Figure 2 illustrates the proportion of patient with successful HFNC for acute respiratory failure, based on ROX-HR index scores at 2 and 10 h.

Table 3 Variables and diagnostic accuracy (for HFNC outcomes) at different time points during HFNC therapy initiated for acute respiratory failure
Full size table
Fig. 2
figure 2

Proportion of patients with successful HFNC for acute respiratory failure, based ROX-HR index at 2 h (top graph) and 10 h (bottom graph)

Full size image

In patients initiated on HFNC post-extubation, 45 (97.8%), 43 (93.5%), 42 (91.3%) and 40 (87.0%) patients remained free of mechanical ventilation at 2, 6, 10 and 24 h, respectively (Table 4). Apart from the 48-h time point, patients with HFNC failure had a consistently lower ROX-HR, while no significant difference was seen with the ROX index measured at 2, 4, 8 and 18 h (Table 4). Heart rate alone predicted HFNC outcomes with an AUROC of 0.693 (95% CI 0.529–0.856) and 0.699 (95% CI 0.518–0.881) at 2 and 4 h, respectively. Within the first 12 h, the highest AUROC was found with the ROX index (0.773, 95% CI 0.617–0.928) and the ROX-HR index (0.804, 95% CI 0.660–0.948) at 10 h of HFNC therapy. Figure S2 illustrates the proportion of patient with successful HFNC after a planned extubation, based on ROX-HR index scores at 2 and 10 h.

Table 4 Variables and diagnostic accuracy (for HFNC outcomes) at different time points during HFNC therapy initiated after a planned extubation
Full size table

In all patients, HFNC success was associated with a significantly higher increase in ROX-HR index from the 2 to 10 h and 6 to 10 h time points—this was not observed with the ROX index (Table S1).

Evaluating cutoffs of the ROX-HR and ROX index for patients with acute respiratory failure

Using the ROC curve at 10 h into HFNC therapy, cutoffs for the ROX and ROX-HR were determined to be 5.80 and 6.80, respectively, for the prediction of HFNC success. The sensitivity, specificity, positive predictive value (PPV) and negative predictive values (NPV) of each index at 2, 6, 10, 18 and 24 h are summarised in Table 5. With Cox proportional regression analysis, a ROX-HR index of > 6.80 was associated with a lower risk of HFNC failure at all time points in the first 24 h, even after correcting for possible confounders (Gender, APACHE II score, CCI and the number of CXR quadrants involved) (Table 6). Kaplan-Meier plots illustrating significant differences in probability of HFNC success with a cutoff of 6.80 for the ROX-HR index are illustrated in Fig. 3a–c. A second cutoff of ROX < 5.00 and ROX-HR < 5.00 was determined from the ROC curves at 10 h, and their performance for the predictirised in Table S2. Compared to ROX < 5.00, a ROX-HR index < 5.00 appeared to perform with higher positive and lower negative likelihood ratios at 6, 10, 18 and 24 h.

Table 5 Prediction of HFNC success based on ROX and ROX-HR cut offs at different time points
Full size table
Table 6 Cox proportional regression analysis evaluating ROX > 5.80 and ROX-HR > 6.80 for the likelihood of HFNC failure in patients initiated on HFNC for acute respiratory failure
Full size table
Fig. 3
figure 3

a–c Kaplan-Meier plots of HFNC success probability based on ROX-HR index at 2, 6 and 10 h for patients initiated on HFNC for acute respiratory failure. d, e Kaplan-Meier plots of HFNC success probability based on ROX-HR index at 2, 6 and 10 h for patients initiated on HFNC after a planned extubation

Full size image

We evaluated the ROX index based on previously established cutoffs of 4.88 (by Roca et al.) at 2, 6 and 12 h (Table S3 and S4) [14]. Similarly, as with a ROX index > 5.80, there was no significant association of a lower risk of HFNC failure with a ROX index ≥ 4.88 at 12 h (Table S3). Roca et al. also reported cutoffs of 2.85, 3.47 and 3.85 at 2, 6 and 12 h for the ROX index for the prediction of HFNC failure [14]. We evaluated these cutoffs in our study population, while these cutoffs performed with good specificity (> 95%), the sensitivity remained poor (< 15%). In contrast, a ROX-HR index of < 4.50 at 2 h and < 5.00 at 6 and 12 h performed with reasonable sensitivity (> 34%) and specificity (> 88%).

Evaluating cutoffs of the ROX-HR and ROX index for patients extubated to HFNC

A cutoff of 7.00 and 8.00 was determined for the ROX and ROX-HR index, respectively, based on the ROC curves at 10 h into HFNC therapy. Table 5 summarises the sensitivity, specificity, PPV and NPV of the cutoffs at various time points within 24 h of HFNC initiation. The ROX-HR index performed with equal or higher sensitivity and specificity at 2, 6 and 10 h (Table 5). On univariate Cox proportional regression analysis, a ROX-HR index of > 8.00 was significantly associated with a lower risk of HFNC failure at 6 and 10 h, which was not seen with the ROX index >7.00 (Table 7). Kaplan-Meier plots illustrating the probability of HFNC success with a cutoff of 8.00 for the ROX-HR index at 2, 6 and 10 h are shown in Fig. 3d, e. A second cutoff of 6.00 was determined from ROC curves at 10 h, and the performance of a cutoff of ROX < 6.00 and ROX-HR < 6.00 for the prediction of HFNC failure is also summarised in Table S2.

Table 7 Cox proportional regression analysis evaluating ROX > 7.00 and ROX-HR > 8.00 for the prediction of HFNC failure in patients initiated on HFNC after a planned extubation
Full size table

Discussion

The results of this study suggest that the ROX-HR index may be a useful tool for early prediction of HFNC outcomes. This applies to patients with acute hypoxemic respiratory failure as well as patients initiated on HFNC as a preventative treatment following a planned extubation. It is easily derived from commonly recorded variables measured in a non-invasive manner and is a practical assessment tool that can be readily applied by the bedside.

For patients initiated on HFNC for acute hypoxemic respiratory failure, the ROX-HR index appears to perform consistently (AUROC > 0.65) in discriminating between HFNC success and failure at all time points. Using selected cutoffs, the ROX-HR index continues to perform well in categorising patients into low and high risk for HFNC failure, as early as 2 h into treatment. With a cutoff of ROX > 5.80 and ROX-HR > 6.80, only the ROX-HR index remained significantly associated with a lower risk of HFNC failure at all time points, after correction for possible confounders. Similarly, for patients initiated on HFNC post-extubation, the ROX-HR index remained consistently lower for patients with HFNC failure. Unlike the ROX-HR index, however, the ROX index did not appear to discriminate well between patients with HFNC success vs failure at 2, 4, 8 and 18 h.

The incorporation of the heart rate therefore appears to add value to the prediction accuracy of the ROX index. In our study, all patients with HFNC failure had a significantly higher heart rate recorded at 2 and 4 h. In patients with HFNC initiated post-extubation, HR alone recorded at 2 and 4 h achieved reasonable AUROCs (0.69 and 0.70, respectively) for the prediction of HFNC outcomes, suggesting that tachycardia, especially soon after initiation of HFNC, is associated with treatment failure. A multicentre analysis performed by Frat et al. also observed an association of HFNC failure with tachycardia, as early as 1 h into HFNC therapy [15]. An elevated heart rate may reflect an increased sympathetic drive or a decompensation of the cardiopulmonary system, and therefore be a marker for worse outcomes. Close to one third (31.7%) of our study population were patients who had HFNC initiated post-extubation. In these patients, the presence of tachycardia may also reflect an impaired cardiac reserve, which is a risk factor for the development of post-extubation respiratory failure.

The ROX-HR index also provides a means of early assessment of patients on HFNC. Early prediction of HFNC failure is crucial as most patients are intubated within 24 h of HFNC initiation (62.3% in our study) [14]. Furthermore, delayed intubation with HFNC has been shown to be associated with increased mortality [10, 14]. In our study, increased mortality was also seen in patients who were intubated after 24 h. While validation in a multicentre study is needed, the ROX-HR index appears to be a promising tool for the early identification of patients at high risk of HFNC failure.

To our knowledge, our study is also the first evaluating the use of the ROX index in patients initiated on HFNC after a planned extubation. It appears that the ROX-HR performs equally well, if not better than the ROX index in these patients. Of note, the re-intubation rate seen in our cohort (35%) is relatively higher than previous reported studies (22–23%) [7, 8]. However, there are significant differences in the study populations. Our study population was relatively more ill, with a median P/F ratio of 164, APACHE II of 15 (24 h preceding extubation) and a high proportion of immunocompromised patients (50%). This contrasts with other studies where the reported median P/F ratios (191–240) and APACHE II (median of 11) on extubation day were relatively lower. Early recognition of the need for reintubation, which is associated with worse outcomes including mortality, is an important clinical need. Indices like the ROX-HR may therefore be a useful for early assessment during the post-extubation period.

Immunocompromised patients made up more than half our study population (56.6%). There is a large interest in HFNC therapy for immunocompromised patients, with several studies suggesting that HFNC may be associated with reduced intubation rates [11, 18,19,20]. Previous studies have identified lower oxygenation and a higher organ dysfunction (SOFA scores) as predictive factors for HFNC failure in immunocompromised patients [12, 21]. However, there is still a paucity of evidence to guide the use of HFNC in immunocompromised patients. The substantial proportion of immunocompromised patients in our study adds strength to the applicability of the ROX-HR index to these patients.

One limitation is that this was a single centre study and conducted in a medical unit, therefore excluding surgical or post-operative patients. We also did not evaluate for the presence of atrial fibrillation or the use of beta blockers in our study. It is possible that the presence of atrial fibrillation with rapid ventricular response may by itself be a poor prognostic marker for the success of HFNC. The effects of beta-blockers on the performance of the ROX-HR index are also unclear. Furthermore, bradycardia will elevate the ROX-HR index, and if associated with hemodynamic instability, will provide physicians with a false sense of assurance—this is an important consideration when applying the ROX-HR index. In our study, 2 patients had significant bradycardia (< 50 beats/min) recorded during HFNC; both patients did not require intubation. Thirdly, evaluating a cutoff of 5.80 and 6.80, for the ROX and ROX-HR, respectively, was determined based on examination of the ROC curves of this study. Roca et al. examined the use of the ROX index for patients with acute respiratory failure from pneumonia and documented a best cutoff of 4.88 at 2, 6 and 12 h [13, 14]. Applying these cutoffs for the ROX index in our study population appeared to perform with better sensitivity compared to a ROX-HR index of > 6.80, but had poor specificity (29–51%) for HFNC success. The ROX index ≥ 4.88, when subjected to multivariate analysis, also did not appear to be consistently associated with HFNC outcomes.

Clearly, determining an ideal cutoff is challenging. Firstly, differences in study populations may lead to variation in findings. Compared to the report by Roca et al., a higher proportion of our patients on HFNC for acute respiratory failure was immunocompromised (60% vs 34%), with a higher median age (63 vs 53 years) observed in patients who failed HFNC—which may also explain the higher rate of HFNC failure seen in our study. Secondly, depending on specific clinical needs, physicians may have different priorities over the sensitivity versus specificity of the ROX or ROX-HR index, and it is likely that a “best” cutoff may also vary with differences in medical practices and ICU protocols. In our study, a lesser increase in ROX-HR index between 2 to 10 h and 6 to 10 h was also observed in patients with HFNC failure—this was not seen with the ROX index. This dynamic perspective suggests that trends in the ROX-HR index may also provide physicians with useful information. Furthermore, the absolute quantity of the index (where a ROX-HR index of < 4 or ≥ 14 is associated with a very high and low risk of HFNC failure, respectively) may also assist in clinical decision-making (Fig. 2 and S2). More studies are needed to evaluate these hypotheses carefully. Nevertheless, the ROX-HR index was consistently able to identify patients at high risk or low risk of HFNC failure based on identified cutoffs during the first 24 h into treatment. This will help to provide assurance and guidance to physicians, even in the early stages of HFNC therapy.

Conclusion

The ROX-HR index appears to be a promising tool in the early identification of patients who are at high risk of HFNC failure and for patients initiated on HFNC for acute respiratory failure as well as a preventative strategy after a planned extubation. Larger multicentre validation studies are needed to establish the role of the ROX-HR index in patients on HFNC.