Introduction

Coronavirus disease 2019 (COVID-19) is a newly recognized infectious disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) which has rapidly spread around the world reaching pandemic proportions [1]. Up to 20% of patients develop acute respiratory distress syndrome (ARDS), which may require different grades of oxygen therapy up to ventilatory support and intensive care unit (ICU) admission [2,3,4]. During the COVID-19 outbreak in Italy, patients not immediately requiring ICU were hospitalized either in medical wards, where oxygen could be primarily delivered with Venturi masks and masks with reservoir bags, or in sub-intensive wards, where high-flux nasal oxygen (HFNC), as well as continuous positive airway pressure (cPAP) and non-invasive ventilation (NIV) were available. These patients often needed to be transferred urgently to the ICU for mechanical ventilation because of respiratory failure refractory to oxygen therapy or NIV [5,6,7,8].

Arterial blood gas (ABG) test is considered the gold standard for the diagnosis of respiratory failure; however, especially in medical wards, such test is not always feasible and available. The ABG test allows the assessment of the ratio between arterial partial pressure of oxygen (PaO2, expressed in mmHg) and fractional inspired oxygen concentration (FiO2) (PaO2/FiO2 ratio), which is a validated index of hypoxaemia and is associated with adverse outcomes in patients with respiratory failure [9, 10]. The PaO2/FiO2 ratio is widely used in clinical practice especially for the diagnosis of ARDS according to the Berlin definition [11]. Furthermore, the PaO2/FiO2 ratio helps in patients risk stratification identifying three different categories of ARDS severity (mild, moderate, and severe) [11].

The peripheral blood oxygen saturation (SatO2), obtained through pulse-oximetry, is an easy, non-invasive, and universally used clinical parameter that can provide real-time information about tissue oxygenation. However, since the oxygen–haemoglobin dissociation curve presents a S-shape, the SatO2 has a low sensitivity for the detection of hypoxaemia and might not closely correlate with the PaO2 [12]. Despite this, the SatO2/FiO2 ratio, which is calculated dividing the SatO2 by the FiO2, was shown to be closely correlated with the PaO2/FiO2 ratio, suggesting that the SatO2/FiO2 ratio could be used for the identification and follow-up of patients with ARDS avoiding arterial puncture and blood gas analysis [13]. The respiratory rate-OXygenation (ROX) index is another interesting and non-invasive index that has been recently proposed and validated to assess the effectiveness of HFNC treatment in patients with acute hypoxemic respiratory failure [14, 15]. The ROX index is calculated as the ratio of the SatO2/FiO2 ratio and respiratory rate (RR) and it might be a useful predictor of important outcomes, such as mortality or intubation.

Thus, the aim of the present study was to evaluate the clinical performance of non-invasive respiratory function indices, such as the ROX index and the SatO2/FiO2 ratio, as compared to that of the PaO2/FiO2 ratio, for the prediction of clinically relevant outcomes (death or intubation, whichever comes first) in hospitalized patients for COVID-19 pneumonia.

Methods

Study design

This is a four-centre retrospective cohort study conducted in patients hospitalized for COVID-19 pneumonia in medical and sub-intensive wards. We classified as medical wards both the Internal medicine and Liver Unit at Verona and Padua University Hospitals, whereas both the San Luca Hospital, Istituto Auxologico Italiano, in Milan and the Respiratory Intensive Care Unit in Mantua Hospital were considered as sub-intensive wards, since they were properly equipped to perform intensive monitoring and non-invasive ventilation. All the patients admitted to these wards between March and May 2020 were included. The study was approved as patients’ registry by the Ethical committee of all recruiting centres. All patients provided oral informed consent to be included in the study.

Patients

Four hundred and fifty-six consecutive patients were finally included in the study. Inclusion criteria were: (1) hospital admission with a diagnosis of COVID-19 pneumonia based on the detection of SARS-CoV-2 RNA in nasopharyngeal swabs plus one between (a) either a chest X-ray showing pneumonia or (b) a chest computed tomography showing pneumonia, (c) or respiratory failure that required oxygen treatment [16]. Patients younger than 18 years, patients with an indication for immediate intubation, and those with a do-not-intubate order were excluded. Patients electively intubated for diagnostic or therapeutic procedures (e.g., bronchoscopy and surgery) were also excluded. A composite outcome of transfer to ICU or death (whichever comes first) within 28 days after hospital admission, was assessed. Patients transferred to ICU were those that required intubation and invasive mechanical ventilation.

Medical history, laboratory data, device description and management

For each participant, anthropometric and demographic data were extracted from medical records as well as data about medical history and current medications. In detail, a complete medical history (including hypertension, diabetes, ischemic heart disease, heart failure, chronic obstructive pulmonary disease, chronic kidney disease and chronic liver disease) and medication use in the previous 14 days (including angiotensin receptor enzyme inhibitors, angiotensin receptor blockers, beta-blockers, antiplatelet agents and oral anticoagulants) were recorded. Data about laboratory tests performed during hospital admission were extracted and recorded. These included: haemoglobin, white blood cells, lymphocytes, platelets, C-reactive protein, d-dimer, serum creatinine, alanine aminotransferase, lactate dehydrogenase, creatine kinase and serum ferritin. Data about the SatO2, measured by pulse oximetry; the PaO2, derived from ABG test; the FiO2, predetermined in patients treated with Venturi masks, extrapolated in patients treated with nasal cannulas or reservoir bag masks, set in patients under NIV; were recorded and used to calculate the PaO2/FiO2 ratio, the SatO2/FiO2 ratio, and the ROX index within the first 24 h of hospital admission; in particular, the first data obtained on arrival at medical or sub-intensive wards was registered. The ROX index was calculated as the ratio of the SatO2/FiO2 ratio and RR as previously described [14, 15]. The decision for a single patient’s intubation was agreed upon after discussion between medical doctors in the medical or sub-intensive wards and from the ICU (Intensivists). It was based upon clinical criteria, including oxygen desaturation, increased RR, and arterial blood gas evaluation (including the PaO2/FiO2 ratio) indicating actual or incipient respiratory failure.

Statistical analysis

Continuous variables are presented as mean ± standard deviation or median with interquartile range based on data distribution. Categorical variables are expressed as percentages. Either one-way ANOVA or Kruskal Wallis one-way ANOVA were used to compare continuous variables according to the data distribution pattern (normal or not normal). Categorical variables were compared using the Chi-square test. Logistic regression analyses were performed to determine if any anamnestic or clinical variables (age; sex; a history of either hypertension or diabetes; ALT; blood lymphocyte count; platelet count; levels of d-dimer, LDH or serum ferritin) could be independently associated with the study endpoint, a composite of admission to the ICU for intubation or death whichever came first. The variable selection was done through sequential replacement (a stepwise method) which consists of a combination of backward and forward techniques. If the p value was less than 0.05 or above 0.1 the covariates were, respectively, included and excluded from the regression model. No fixed variables were considered. The Area Under the Receiver Operator Characteristic Curves (AUROC) were performed to assess the capacity of PaO2/FiO2 ratio, SatO2/FiO2 ratio and ROX index for predicting the study endpoint. Furthermore, we tried to identify the optimal cutoff point of the PaO2/FiO2 ratio, SatO2/FiO2 ratio and ROX index for the prediction of the composite outcome. The main analysis was done according to the different intensity of care of the four centers: the medical wards (Verona and Padua) were compared to the sub-intensive wards (Milan and Mantua). Data about each centers considered separately are presented in the supplementary material. Statistical package for social science (SPSS) version 22 was used for all data analysis. All tests were two-sided, and p values < 0.05 were considered statistically significant.

Results

Four hundred and fifty-six consecutive patients were enrolled. In the medical wards (Verona and Padua) 30 patients required NIV after hospital admission, while 51 patients were transferred to the ICU for intubation and 45 died. In the sub-intensive wards (Milan and Mantua) 121 patients underwent NIV during hospitalization, while the numbers of patients transferred to the ICU for intubation or deceased were 16 and 33, respectively (Fig. 1). Clinical and laboratory characteristics, according to the different intensity of care of the hospitalization wards, are presented in Table 1. In brief, patients hospitalized in the medical wards were older, while those admitted to the sub-intensive wards suffered from more severe forms of acute respiratory failure. Regarding biochemical variables, patients hospitalized in the sub-intensive wards presented higher levels of serum ferritin, lactate dehydrogenase, alanine aminotransferase, C-reactive protein and d-dimer. Furthermore, in the sub-intensive wards, a greater number of patients underwent NIV after admission. On the other hand, in this setting, the incidence of the composite outcome of transfer to the ICU for intubation or death was significantly lower than in the medical wards.

Fig. 1
figure 1

Patient’s flow diagram

Full size image
Table 1 Clinical and laboratory characteristics of patients at admission in the overall population, medical wards (Verona and Padua) and sub-intensive wards (Milan and Mantua)
Full size table

Multiple logistic regression analysis showed that, both in the overall population and medical wards (Tables S1 and S2), the PaO2/FiO2 ratio, the ROX index and the SatO2/FiO2 ratio were all significantly associated with the composite outcome, whereas in the sub-intensive wards none of the three respiratory function indices was able to predict death or intubation (data not shown). In this setting, the only clinical variable that remained significantly associated with the composite outcome was age (data not shown).

In the overall population the PaO2/FiO2 ratio, the ROX index and the SatO2/FiO2 ratio predicted admission to the ICU for intubation or death with an accuracy of 67%, 69% and 66%, respectively (Table 2 and Fig. 2), not significantly different between each others. The diagnostic performance of the three respiratory function indices resulted higher in the medical wards, where the transfer to the ICU for intubation or death were predicted by the PaO2/FiO2 ratio, the ROX index and the SatO2/FiO2 ratio with an accuracy of 75%, 75% and 74%, respectively, while was not significant in the sub-intensive wards (Table 2 and Figs. 3, 4).

Table 2 Discrimination ability of the PaO2/FiO2 ratio, ROX index and SatO2/FiO2 ratio in predicting transfer to the ICU for intubation or death in the overall population, medical wards (Verona and Padua) and sub-intensive wards (Milan and Mantua)
Full size table
Fig. 2
figure 2

ROC curve: end point intubation or death in the overall population

Full size image
Fig. 3
figure 3

ROC curve: end point intubation or death in the medical wards

Full size image
Fig. 4
figure 4

ROC curve: end point intubation or death in the sub-intensive wards

Full size image

Traditionally the PaO2/FiO2 ratio cutoffs of 300 mmHg, 200 mmHg and 100 mmHg are used to identify mild, moderate, or severe ARDS, respectively [11]. In the medical wards, these cutoffs for the composite outcome resulted in a sensitivity of 85%, 65%, and 22% and a specificity of 35%, 70%, and 95%, respectively. The same levels of sensitivity and specificity, as for the three cutoffs of the PaO2/FiO2 ratio, could be obtained in the medical wards with ROX index values of 17.9, 13.7 and 4.0; and SatO2/FiO2 ratio values of 422.6, 295.4 and 114.6 (Table 3). The sensibility and specificity for the composite outcome in the overall population of the three PaO2/FiO2 ratio cutoffs and the corresponding values of ROX index and SatO2/FiO2 ratio are also presented in Table 3.

Table 3 Comparison between diagnostic performance of PaO2/FiO2 ratio, ROX index and SatO2/FiO2 ratio in predicting transfer to the ICU for intubation or death and relative cutoffs in the overall population, medical wards (Verona and Padua) and sub-intensive wards (Milan and Mantua)
Full size table

Apart from the respiratory indices, the other covariates that remained independently associated with the composite outcome in all logistic models, both in the overall population and medical wards, were age and blood lymphocyte count. In the overall analysis, also being hospitalized in Mantua and serum levels of lactate dehydrogenase were statistically associated with the composite outcome (Table S1), whereas in the medical wards another significant covariate for the outcome of intubation or death was serum levels of alanine aminotransferase (Table S2). The adjustment for these covariates, when significant, highly increased the ability of the PaO2/FiO2 ratio, the ROX index and the SatO2/FiO2 ratio to predict transfer to the ICU for intubation or death (Table 4).

Table 4 Discrimination ability of the PaO2/FiO2 ratio, ROX index and SatO2/FiO2 ratio in predicting transfer to the ICU for intubation or death in the overall population and medical wards (Verona and Padua) after adjustment for age, blood lymphocyte count, alanine aminotransferases and care departments, when significant
Full size table

The secondary analysis, comparing the different Units, showed results in line with the main one (Supplementary material, Tables S3–S5).

Discussion

The results of our multicentric study of 456 hospitalized patients for COVID-19 pneumonia showed an acceptable performance of two non-invasive and easily measurable respiratory indices in predicting transfer to ICU or death. Interestingly, in medical wards, the prediction accuracy of the ROX index and the SatO2/FiO2 ratio was comparable to that of the PaO2/FiO2 ratio, which is considered the gold standard for the diagnosis of respiratory failure and ARDS in clinical practice.

Unfortunately, frequent and repeated ABG tests are not always feasible in medical wards. This occurred, especially, during COVID-19 outbreak when hospitals withstood unrelenting pressure due to the continuous inflow of patients with SARS-CoV-2 infection and acute respiratory failure that required constant monitoring of the respiratory function. In this dramatic scenario, the availability of respiratory indices which allow a rapid and non-invasive evaluation of disease severity and treatments efficacy may help clinicians to promptly identify those patients at higher risk of intubation and death.

There is a growing interest in the validation of non-invasive respiratory indices, such as the ROX index, as easy-to-apply prognostic tools in COVID-19 pneumonia. In the last few months, a couple of studies were set to investigate whether the ROX index could predict the effectiveness of HFNC treatment for SARS-CoV-2 infection [17, 18]. Furthermore, Zaboli and colleagues applied the ROX index at triage assessment of 273 patients who presented to the emergency department for dyspnoea due to SARS-CoV-2 infection. The study aimed to explore the prognostic ability of the ROX index in predicting ARDS development or need for intubation within 72 h after triage evaluation. They found median ROX index values at the admission of 13.1 and 15.3 in those that either subsequently required intubation or were diagnosed with ARDS, respectively. The ROX index showed good accuracy in predicting ARDS development with an AUROC of 0.845 [19]. These results are in line with those by Suliman and colleagues, where the ROX index at admission was one of the independent predictors of intubation [20]. In another Italian study, the ROX index measured at admission in the emergency department was a strong predictor not only of subsequent hospitalization but also of higher 30-day mortality. In particular, ROX index values < 22.3 were found to be significantly related to higher mortality with an AUROC of 0.764 [21].

In contrast with previous investigations, our multicentric study is the first that assessed the performance of the ROX index and the SatO2/FiO2 ratio compared to the gold standard PaO2/FiO2 ratio invasive index. We found that not only the traditional PaO2/FiO2 ratio but also the ROX index and the SatO2/FiO2 ratio were highly significantly associated with the composite outcome of transfer to ICU for intubation or death. Furthermore, the prognostic accuracy of the ROX index and the SatO2/FiO2 ratio was non-inferior to that of PaO2/FiO2 ratio and, especially in the medical wards, was associated with the outcome. Instead, in the sub-intensive wards of Milan and Mantua hospitals, the association between these indices and the outcome was not statistically significant. Thus, the performance of the ROX index and the other respiratory indices can partly depend on the kind of patients admitted in wards with different intensities of care. The differences regard both the disease severity in the admitted patients and the devices available to support respiratory function during the first wave of SARS-CoV-2 infection in Italy. In fact, the indices of respiratory function, the percentage of patients using NIV support as well as the inflammatory biomarkers, such as levels of serum ferritin, LDH, d-dimer and lymphocytes were significantly higher in the sub-intensive wards, as compared to the medical ones indicating a more severe form of respiratory failure and systemic inflammation in these patients.

Finally, our study identifies different values of the ROX index and the SatO2/FiO2 ratio that correspond to the PaO2/FiO2 ratio cutoffs conventionally used to identify higher severity of respiratory failure. These non-invasive indices might allow for early identification of those patients who could benefit from more aggressive interventions, such as an earlier transfer to the ICU and intubation.

The present study has several limitations. Its retrospective and observational design does not allow to draw any definitive conclusion about the actual interchangeability of different indices of respiratory distress. Moreover, some biochemical tests were lacking or were available in restricted sub-groups of patients, which did not allow us to consider them in our analyses. Even if to date this appears to be the largest database evaluating the ROX index as a prognostic tool in COVID-19 pneumonia, the sample size of our study is somehow limited, when data from each medical Unit are analysed separately. The inclusion of both low-intensity and medium-intensity care Units and the multicentre design of our study could be considered a strength, since they allowed for meaningful comparisons of different settings of care, but also a weakness, since other differences among Centres in the care of patients could have in some way influenced the results. Many other studies evaluated different clinical, laboratory, and imaging markers in their ability to predict outcomes. Indeed, many risk scores were proposed but most of them did not include the respiratory indices that we have considered [22,23,24]. Our study does not propose another prognostic model to predict the outcome of patients affected by COVID-19 pneumonia, but rather shows that both non-invasive and invasive indices of respiratory distress have similar accuracy in predicting the clinical outcomes of COVID-19 patients admitted to medical wards.

In conclusion, the ROX index and the SatO2/FiO2 ratio demonstrated, in medical wards, an acceptable accuracy in the identification of hospitalized patients for SARS-CoV-2 infection at higher risk of death or transfer to the ICU for intubation. The non-invasiveness of the ROX index and the SatO2/FiO2 ratio, their simplicity and good reproducibility, could make them excellent prognostic and monitoring tools, especially in medical wards, where repeated ABG tests are not always feasible.