Introduction

From its first report in December 2019 [1], coronavirus disease (COVID-19) has been an endemic for > 2 years. In Japan, the endemic is still ongoing with approximately 1,720,000 patients and 18,000 deaths during five major epidemics since October 31, 2021. Hypoxemia is one of the most important symptoms and causes of death in moderate-to-severe COVID-19 pneumonia [2], and it requires appropriate respiratory support including invasive or non-invasive ventilatory support.

High-flow nasal cannula (HFNC) therapy is an important non-invasive respiratory support device in acute respiratory failure. It delivers heated and humidified oxygen via the nose at flows as high as 40–60 L/min and at oxygen concentrations up to 100% [3], and it is increasingly being used to support COVID-19 patients with hypoxemic respiratory failure in Japan [4] because of its efficacy in patients and safety for health care providers [5, 6]. Compared with conventional oxygen therapy, HFNC therapy significantly decreases the need for invasive mechanical ventilation and time to clinical recovery in the case of hypoxemic respiratory failure [7]. However, there have been concerns about increased mortality due to the late failure of patients after application of HFNC therapy [8, 9]. Therefore, useful indicators for respiratory failure are important in the management of HFNC therapy.

The respiratory rate and oxygenation (ROX) index, i.e., the ratio of oxygen saturation as measured by pulse oximetry/fraction of inspired oxygen (FiO2) to the respiratory rate, is a simple, useful, and well validated tool for predicting the outcome of HFNC therapy (the need for intubation or not and death). Since the ROX index was first reported by Roca et al., the ROX index for HFNC therapy has been discussed from 2 to 24 h after initiation, and its cut-off value is 4.88 ≤ (area under the curve [AUC] 0.74 [0.64–0.84], p < 0.002) [10]. However, the S/F ratio (saturation of inspired oxygen [SpO2]/fraction of inspired oxygen [FiO2] ratio) by itself correlates with the P/F ratio (PaO2/FiO2 ratio) and is an important indicator of respiratory failure [11]. In COVID-19, the respiratory rate may not indicate severity because the disease is often present with silent hypoxemia, which is associated with an increased respiratory rate [12]; therefore, the S/F ratio is a possible comparable predictive parameter of the ROX index. On the other hands, the presence of silent hypoxemia does not immediately undermine the usefulness of the ROX index, as an observational study showed that even silent hypoxemia is associated with an increased respiratory rate [11]. Several reports have shown the relationship between the ROX index, S/F ratio, and HFNC failure or mortality; yet, the real-world data for HFNC therapy, the ROX index, and S/F ratio in COVID-19 pneumonia are still limited. We hypothesized the reliability of ROX index for HFNC success and failure is still useful, be equal or more than S/F ratio.

Herein, we describe the use of HFNC therapy for patients with moderate to severe COVID-19 pneumonia. The purpose of this study was to investigate the use of HFNC therapy in patients with moderate to severe COVID-19 pneumonia and the factors associated with HFNC failure. We also evaluated 28-day mortality and the factors associated with 28-day mortality.

Patients and methods

Study design and population

This single-center, retrospective, cohort study included patients with moderate-to-severe COVID-19 pneumonia from July 20, 2020 to October 31, 2021. Patients at least 18 years or older who required supplemental oxygen were eligible. The following patients were excluded from the study: those who did not require supplemental oxygen and invasive mechanical ventilation, those admitted with other diseases or wounds that did not have a course or clinical findings of COVID-19 pneumonia and were judged clinically not to require oxygen administration due to COVID-19 pneumonia, those who received HFNC therapy for acute respiratory failure with hypercapnia, those who received HFNC therapy as post-extubation support, and those for whom data for the ROX index were not collected within the first 4 h using HFNC therapy. We also excluded patients with do-not-intubate orders on hospital admission.

Data collection

Patients’ characteristics, medical history, laboratory examination data, imaging findings, treatment, outcome, and complications were collected from the medical records. All patients were diagnosed with COVID-19 based on a severe acute respiratory syndrome coronavirus 2-specific polymerase chain reaction test result obtained from a nasal or oropharyngeal swab. A chest X-ray or chest computed tomography scan was evaluated to detect bilateral consolidation or ground-glass opacity, which was indicative of pneumonia. All treatments followed the Japanese guideline for COVID-19 or Infectious Diseases Society of American guideline for COVID-19 at the time and included a systemic glucocorticoid (e.g., dexamethasone, methylprednisolone, and hydrocortisone), remdesivir, tocilizumab, baricitinib, and unfractionated heparin.

Respiratory support and outcomes

We used Optiflow™ (Fisher and Paykel Healthcare) as the HFNC device for COVID-19 pneumonia. HFNC therapy was considered for the patients who needed > 5 L/min of supplemental oxygen and a respiratory ratio of > 25 /min, and those who expected to avoid intubation management with the introduction of HFNC. HFNC therapy was started and fixed at a 40-L/min flow and had an appropriate FiO2. Patients discontinued HFNC therapy at the discretion of the attending physician or intensive care unit doctor based on an FiO2 of 0.35–0.40 and clinical improvement. All patients except those who refused intubation were considered for intubation and mechanically ventilation at a P/F ratio < 70–100 with clinical worsening. Intubation and mechanical ventilation were initiated at the discretion of the attending physicians and intensive care specialists when patients or their family members refused the do-not-intubate order based on the physicians' opinion of the patient's condition. Specialists followed the mechanical ventilation protocol at our institution with reference to the criteria used in previous studies. For example, the following criteria were used for initiating invasive ventilation: hemodynamic instability, deterioration of neurologic status, or signs of persisting or worsening respiratory failure. This was defined by at least two of the following criteria: respiratory rate of > 40 breaths per minute, lack of improvement in signs of high respiratory muscle workload, and copious tracheal secretions. Other criteria included acidosis with a pH of < 7.35 and a percutaneous arterial oxygen saturation of < 90% for > 5 min without technical dysfunction or poor response to oxygenation techniques. Non-invasive positive-pressure ventilation was not used for acute respiratory failure to ensure infection control for the medical staff.

The ROX index [10] and S/F ratio were measured based on the patient’s vital signs (saturation of inspired oxygen [SpO2] and respiratory rate) and FiO2. Because these vital signs were not routinely measured immediately after HFNC initiation, we adopted the ROX index as early as possible after HFNC initiation; the acceptable range was within 4 h.

We defined the primary outcome for this study as HFNC failure, which means intubation or death with HFNC therapy. We defined the secondary outcome as 28-day hospital mortality, which is the death within 28 days in our hospital, with HFNC use and with or without intubation. The cause of death was not considered in this study. We also compared the reliability of the ROX index and S/F ratio as the predictor of HFNC success or failure and 28-days mortality in our cohort.

Cut-off values and statistical analysis

Based on the risk factors for severity and death on hospital admission for Japanese patients with COVID-19 pneumonia [12], we evaluated the followed factors: older age (> 65 years), higher body mass index (BMI, > 30 kg/m2), cardiovascular disease, cerebrovascular disease, chronic obstructive pulmonary disease (COPD), diabetes mellitus, solid tumor, and hypertension. We also considered past or current smoking status [13], C-reactive protein (CRP) level > 10.0 mg/dL (cut-off value, 9.7–11 mg/dL [14, 15]), and lactate dehydrogenase (LDH) level > 450 IU/L (cut-off value, 245–450 U/L [15, 16]) as the risk factors for respiratory failure or death. We defined the cut-off value for the ROX index as < 4.88, according to the original study [10].

Continuous variables are reported as median and interquartile range (IQR), and categorical variables are expressed as number and percentage. Comparisons between the HFNC failure and success groups were performed using the Wilcoxon rank-sum test for continuous variables and the Fisher exact test for categorical variables. To analyze the primary and secondary outcome, we used a Cox regression model adjusted for older age (> 65 years), higher BMI (> 30 kg/m2), cardiovascular disease, cerebrovascular disease, COPD, diabetes mellitus, solid tumor, hypertension, past or current smoking status, CRP level > 10.0 mg/dL, LDH level > 450 IU/L, and ROX index < 4.88. HFNC failure were compared between the ROX ≥ 4.88 and < 4.88. For the HFNC failure group, 28-day mortality was compared between these groups. We used the log-rank test and the Kaplan–Meier Method to perform survival analysis. Based on our cohort, we created an ROC curve and examined cut-off values by Youden index. Missing values were found at a level of < 10% of the total, and we did not perform imputation. We used JMP, version 16.0 (JMP Statistical Discovery LLC) to perform the statistical analyses. A 2-tailed p-value of < 0.05 was considered statistically significant.

Results

Among the 683 patients, 435 were excluded for the following reasons; transferred to another hospital before starting treatment, improved and did not need treatment, received HFNC therapy for chronic hypercapnia, missing vital signs and scores, and had do-not-intubate orders. Therefore, 248 patients were included in this study (Fig. 1). All patients received HFNC therapy and mechanical ventilation without facility restrictions.

Fig. 1
figure 1

Trial profile

Full size image

Table 1 shows the patient characteristics. In the HFNC success group, 6 patients had missing data for the BMI and 1 patient had missing data for the serum albumin level. In the failure group, 2 patients had missing data for the BMI, 6 for the lymphocyte count, and 2 for the LDH level. The HFNC failure group included more patients with older age, COPD, and diabetes mellitus than the HFNC success group. Overall, 133 patients failed HFNC therapy. HFNC failure patients had a higher respiratory rate, lower lymphocyte count, and higher LDH and creatinine levels. There were no significant differences between the two groups; the smoking history, frequency of cardiovascular disease or hypertension, albumin level, and CRP level.

Table 1 Patient characteristics
Full size table

Table 2 shows the intervention including respiratory support by HFNC and complications. All patients received systemic corticosteroids (6 mg of dexamethasone or an equivalent dose of methylprednisolone or hydrocortisone, for at least 10 days). The HFNC flow was fixed at 40 L/min. There was no significant difference in adjunctive treatment between the HFNC success and failure groups. Major complications were gastroenteric bleeding, hospital-acquired pneumonia, and bacteremia. All cases of gastroenteric bleeding were not severe or fatal. Compared to the HFNC success group, the HFNC failure group had a shorter time from hospital admission to the start of HFNC therapy, higher respiratory rate, lower S/F ratio, and lower ROX index. The numbers of patients with an ROX index ≥ 4.88 were 97 (84.4%) in the HFNC success group and 84 (63.2%) in the failure group. Because of delayed implementation, 17.7% of patients were treated in the awake prone position (HFNC success group: 20%; HFNC failure group: 15.8%), but there was no statistically significant difference between the groups.

Table 2 Intervention, respiratory support, parameters, and complications
Full size table

Table 3 shows the result of multivariate analysis of the predictors for HFNC success versus HFNC failure. Figure 2 shows log-rank test and Kaplan-Mayer curve for HFNC failure between ROX index < 4.88. The predictors of HFNC failure were as follows: age ≥ 65 years, hypertension, LDH level > 450 U/L, and ROX index < 4.88. The time to event is shown in Fig. 2.

Table 3 Predictors of HFNC failure
Full size table
Fig. 2
figure 2

log-rank test and Kaplan-Mayer curve for HFNC failure

Full size image

Table 4 shows results of the analysis of 28-day mortality for HFNC failure, adjusted by intubated patients. Figure 3 shows the log-rank test and Kaplan-Mayer curve for 28-day mortality in HFNC failure patients. In univariate analysis, hypertension and ROX index are significantly different (Table 4). In multivariate analysis, ROX index < 4.88 and low CRP level were independent predictors of 28-day mortality (Table 5). The time to event is shown in Fig. 3.

Table 4 Predictors of 28-day mortality adjusted for patients with HFNC failure, univariate analysis
Full size table
Fig. 3
figure 3

log-rank test and Kaplan-Mayer curve for 28-day mortality in HFNC failure patients

Full size image
Table 5 Predictors of 28-day mortality adjusted for patients with HFNC failure, multivariate analysis
Full size table

Supplementary table shows the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and positive/negative likelihood ratio (LR ±) for HFNC success or failure and 28-day mortality among patients with an ROX index < 4.88 in this study. Supplementary figure shows the receiver operating characteristic curves for the S/F ratio and ROX index in this cohort. The cut-off value for HFNC failure in this cohort was an ROX index of 6.57 and S/F ratio of 134.29, with AUCs of 0.67 (p ≤ 0.0001) and 0.67 (p ≤ 0.0001), respectively. The cut-off value for mortality was an ROX index of 5.28 and S/F ratio of 95.0, with AUCs of 0.72 (p = 0.0009) and 0.68 (p = 0.005), respectively. (These table and figure are in supplemental material).

Discussion

Main findings

This was a single-center, large retrospective study of HFNC therapy for COVID-19 in Japan. Patients with HFNC failure were significantly older, had more COPD or diabetes mellitus, had a worse respiratory condition, and had a higher LDH level than those with HFNC success. A ROX index of < 4.88 within 4 h of starting HFNC therapy was significant indicator of HFNC failure and 28-day mortality. For instance, the predictive value of this cut-off index was poor, especially in sensitivity. HFNC failure was correlated not only with the ROX index but also with various background diseases and blood test results at hospital admission. Twenty-eight-day mortality was significantly associated with an ROX index of < 4.88 and the CRP level.

As shown in Table 3, it is difficult to predict HFNC failure based on the ROX index; the cut-off value of 4.88 for the ROX index is a predictive value for avoidance of intubation, and the cut-off value leading to intubation is lower. Therefore, the present results are consistent with those of a previous report [10]. The association of low CRP levels with mortality is paradoxical. This study included patients who were transported while being treated at another facility, and prior treatment may have influenced the results.

ROX index and HFNC failure or mortality

In our study, the ROX index predicted HFNC failure well. The ROX index has been shown to be associated with HFNC failure in several reports on COVID-19 pneumonia [17,18,19,20,21]. The variation between our findings and those from these previous reports is attributed to the optimal results for each cohort, but we believe that the first assessment of the ROX index within 4 h can be an indicator of HFNC failure. A large validation study is needed to evaluate this point in more detail. In the latest meta-analysis of HFNC use and the ROX index, the sensitivity, specificity, and AUC of the ROX index were 0.70, 0.79, and 0.81, respectively [22]. In the current study, the ROX index thresholds was variable compared to the original ROX index study [10] or this meta-analysis, and its sensitivity and specificity were not high. These variabilities can be attributed to the patients’ severity, background disease, or characteristics. The current study included very severe patients compared to the cohort of the original study, and the patient characteristics were different between the studies. The selection and use of antiviral drugs, immunosuppressants, and steroids, as well as the response to these drugs, may also be attributable to this variability.

Based on our results, 28-day mortality did not show many associated factors compared to HFNC failure; only the CRP level and ROX index were significant factors for 28-day mortality. Few reports have studied the ROX index and mortality [23, 24]. Interestingly, Kljakovic Gaspic et al. examined in-hospital mortality scoring for patients with HFNC therapy [24], with the following requirements: age ≥ 65 years, ROX index ≤ 4.11, LDH-to-white blood cell count ratio, and Charlson Comorbidity Index as the index for past medical history. In this scoring, the ROX index is a major factor compared to the other requirements for scoring. The ROX index itself plays a greater role in in-hospital mortality than underlying disease, age, or laboratory data. This may be reflected in the results of the present study. However, the mortality rate is affected by the treatment choice and quality of intensive care.

A previous study showed an association with the severity of acute respiratory distress syndrome (ARDS) and mortality in COVID-19 pneumonia [25]. A lower P/F ratio, which means poor oxygenation, is a poor prognostic factor in patients with COVID-19 [26, 27]. The respiratory rate is also associated with mortality in ARDS [28,29,30]. In the early phases of ARDS, before a patient has fatigued or been sedated, the high transpulmonary pressures associated with spontaneous vigorous inspiratory effort may contribute to damage (so-called patient self-induced lung injury [P-SILI]) [31]. The ROX index is based on SpO2/FiO2 and the respiratory rate, each of which reflects very poor oxygenation and higher respiratory drive. In particular, a worsening respiratory rate may not only be affected by P-SILI but also by respiratory muscle fatigue [32], which may affect respiratory failure and prognosis. Thus, these composite outcomes may more clearly define HFNC failure or mortality.

Several reports have examined various factors in the study of hospital mortality due to COVID-19 pneumonia [21,22,23], and other reports have shown the usefulness of existing composite outcomes such as the Acute Physiology and Chronic Health Evaluation (APACHE) II and Sequential Organ Failure Assessment (SOFA) scores [16, 19]. The data collection criteria such as the hospital admission date, inclusion criteria, or intubation also differ. Considering these findings, a large number of factors can be associated with mortality in terms of items and time. The present study only examined past medical history, some laboratory data, and the ROX index, so the selection and use of antiviral drugs, immunosuppressants, and steroids, as well as the response to these drugs, may be attributable to this variability. Further examination with larger sample sizes and adequate data on hospital admission and intubation may yield more accurate results.

ROX index and S/F ratio

In the present study, both the S/F ratio and ROX index were shown to be useful as predictors of HFNC failure and in-hospital mortality. Kim et al. already showed that the S/F ratio at the time of HFNC initiation is an acceptable predictor of HFNC failure [33], and the results of this study were similar, indicating that the S/F ratio is also an acceptable predictor for in-hospital mortality.

Our study did not compare the SF ratio and ROX index in terms of their usefulness. The S/F ratio is a simple index, it can be more easily measured, and it may reduce contact between patients and health care workers because it does not require a direct contact to patients. Therefore, it is necessary to evaluate each of these parameters and to use them appropriately depending on the situation. The external validity of the ROX index has been studied more than that of the S/F ratio, and we proposed a cut-off value of 4.88 for the ROX index; therefore, it can be used for risk assessment. Although the S/F ratio has not been well validated in COVID-19, it can be considered as an useful index, as its easy measurement characteristics are one of its advantages. In present study, cut-off values for both parameters were retrieved and their usefulness was presented because both of them have usefulness that should be utilized.

Strengths

Although the ROX index can be related to the risk of 28-day mortality, there have been few reports about the risk. Herein, we showed that the ROX index at the start of HFNC therapy is a predictive factor for 28-day mortality. This result may be due to the disease severity and proportion of HFNC failures in this cohort (more than half of patients were considered a failure) compared to cohorts in previous reports.

Limitations

There are several limitations to this study. This study was a single-center, retrospective study with a small number of missing data. Not all tests were performed on hospital admission, and factors such as the partial pressure of oxygen level, procalcitonin level, D-dimer level, SOFA score, and APACHE II score were not considered. This study did not fully examine the background of treatment changes and epidemic variants during the study period. However, the omicron variant, which is the mainstay of the current infection epidemic but is not as severe as the delta variant, was not reported in Japan during the study period. The ROX index could not be measured immediately after HFNC initiation in some cases, which may bias the data.

Conclusion

This study showed that the ROX index is an important predictive factor for both HFNC success and 28-day mortality, making it useful information for clinical management.