Introduction

From December 2019, a worldwide pandemic with an kk coronavirus SARS-CoV-2 is responsible for Coronavirus disease (COVID-191. Up to two-thirds of hospitalized COVID-19 patients develop severe pneumonia leading to an acute respiratory distress syndrome (ARDS)2,3. ARDS is characterized by an impairment in arterial oxygenation, leading to profound hypoxemia. Hence, the ratio between arterial oxygen partial pressure (PaO2) and inspired fraction in oxygen (FiO2), delivered by the mechanical ventilator, is a crucial in ARDS management. Yet, PaO2 requires arterial blood gas analyses and cannot be continuously monitored. It may be approximated by arterial oxygen saturation (SaO2), which in turn, may be estimated by pulse oximetry (SpO2), a continuous non-invasive measurement4. The latter showed benefit in decreasing complications related to intraoperative hypoxemia incidents5, rate of unplanned intensive care unit (ICU) admissions of surgical patients6 and number arterial blood gas analysis7,8,9,10.

Managing patients with ARDS requires frequent changes in mechanical ventilator respiratory settings. While arterial oxygenation is paramount towards guiding the fine-tuning of mechanical ventilators in ARDS, SpO2 has been suggested as surrogate for arterial oxygenation11,12,13. Known caveats of SpO2 mostly lie in its accuracy in ICU due to confounding factors : hypoxemia14,15,16,17,18, vasoconstriction with microcirculatory disorders19, vasopressor treatments15,20 and racial bias21.

Since some COVID-19 patients are black2222/11/2021 13:53:00, severely hypoxemic23 and may also experience microcirculatory disorders24, the main goal of the following study was to investigate the impact of COVID-19 status on the association between SpO2 and arterial oxygenation (SaO2 and PaO2) in patients with ARDS. Furthermore, we assessed whether SpO2 changes could track changes in arterial oxygenation (SaO2 and PaO2) during positive end-expiratory pressure (PEEP) trial.

Methods

This prospective and observational study was conducted in a 24-bed ICU of a French university hospital between March 2019 and July 2020 for non COVID-19 patients and in the first pandemic wave (March–April 2020) for COVID-19 patients, in accordance with relevant guidelines and regulations.

Patients and data collection

We included all consecutive patients admitted in our ICU department for acute respiratory failure and presented ARDS criteria, according to Berlin definition26 at the initial phase of ventilatory management (within one hour after intubation). Exclusion criteria were those related to known causes of low-reliability of SpO2 measurements: (i) low-quality SpO2 signal assessed on the SpO2 curve aspect and (ii), patients with nail varnish, methaemoglobinemia or carbon monoxide poisoning4 and (iii), patients with marked movements leading to SpO2 signal artefacts27.

ARDS severity was categorized as mild for PaO2/FiO2 ratio between 200 and 300 mmHg, moderate between 100 and 200 mmHg, and severe, under 100 mmHg26.

Data were prospectively collected using a clinical software allowing data management and extraction, Centricity Critical Care (General Electric Healthcare, Massachusetts, United States of America).

SpO2 and SaO2 measurements

SpO2 was continuously measured using a finger or ear probe of a last generation pulse oximeter (Masimoset®, Masimo Corporation, Irvine, CA, USA). The location of SpO2 probe was left at the discretion of nurses to obtain the best possible SpO2 signal. The pulse oximeter was connected to patients’ monitor (GE Healthcare, Chicago, Il, USA) and SpO2 signal maximized on the monitor screen. SpO2 measurement was recorded at the time the arterial blood gas was performed by the patient’s nurse.

A blood sample was obtained from radial or femoral arterial catheter, not necessarily located on the same side as SpO2 probe. SaO2 was measured with the ICU blood gas analyser (ABL800, Radiometer®, Copenhagen, Denmark) and thus obtained within five minutes of blood sampling.

Occult hypoxemia was defined as SaO2 < 88%, while concomitant SpO2 measure was ≥ 92%21.

Ventilatory settings and respiratory measurements

All patients were placed in 45-degree semi-recumbent position and mechanically ventilated (CARESCAPE R860, GE Healthcare, Chicago, Il, USA) in pressure regulated volume control mode. Tidal volume was set at 6 mL/kg of predicted body weight. Respiratory rate and inspiratory/expiratory times ratio were adjusted to prevent hypercapnia (pH goal: 7.30–7.45) and to avoid dynamic hyperinflation, without exceeding a respiratory rate of 35 breaths/min12,28. The fraction of inspired oxygen (FiO2) was adjusted to obtain a SpO2 ≥ 90%29. An airway humidification system was used in all patients.

The plateau pressure was calculated during a 5-s inspiratory hold and total PEEP during a 5-s expiratory hold. The driving pressure was calculated as the difference between plateau pressure and total PEEP. The static respiratory system compliance was calculated as tidal volume/(plateau pressure–total PEEP).

PEEP trial description

Since only patients with moderate to severe ARDS with a high potential for lung recruitment benefit from high PEEP strategies and since the amount of potentially recruitable lung vary widely in the population30, we systematically performed a PEEP trial in all patients with ARDS to individualize at best the initial PEEP level29. According to local protocols, PEEP level was initially set at 5 cmH2O31 and a first set of measurements, including SpO2, SaO2 and PaO2 was performed. Then, PEEP was increased to 15 cmH2O and a second set of measurements was performed. All measurements were recorded after a 10-min period of stabilization32 and two couples of simultaneous SpO2 and SaO2 measurements were obtained per patient. Other ventilatory settings were unchanged during the study period.

COVID-19 diagnosis

All patients were confirmed with SARS-Cov-2 using routine RT-PCR methodology, with two sets: either Allplex® 2019-nCoV assay (Seegene, Seoul, South Korea) with Microlab NIMBUS® extractor (Hamilton Bonaduz AG, Rapperswil-Jona, Switzerland) and CFX96® thermocycler (Bio-Rad laboratories, Hercules, California, United States of America); or RealStar® SARS-CoV-2 RT-PCR Kit 1.0 assay (Altona Diagnostics, Hamburg, Germany) with QIAsymphony SP® extractor (QIAGEN, Hilden, Germany) and QuantStudio® thermocycler (Thermo Fisher Scientific, Waltham, Massachusetts, United States of America). RT-PCR was performed on nasopharyngeal swabs or on distal bronchial samples.

Statistical analysis

Normality distribution of continuous variables was tested using the Agostino-Pearson test. Continuous variables were expressed as mean (standard deviation) or median [interquartile] and categorical variables as counts (percentages). Continuous variables were compared using Wilcoxon or paired Student t-tests and Mann–Whitney U-test or Student t-tests. Categorical variables were compared using Chi-2 or Fisher-exact tests.

We used four different methods to assess the association between SaO2 and SpO2: correlations, agreement, multivariable regression models and concordance. Specifically, correlations were performed using Pearson or Spearman’s correlation coefficients, according to data distribution. Agreement between SpO2 and SaO2 measurements was assessed with Bland–Altman analysis33 and intraclass correlation coefficients (ICC), with ICC value > 0.7 and > 0.9 indicating satisfactory and excellent agreement respectively34. Accuracy of SpO2 measurement was estimated by the bias, calculated as the mean difference between SpO2 and SaO2 measurements. Precision of SpO2 measurement was estimated by the standard deviation of the bias and the 95% limits of agreement33. The concordance between relative changes in SpO2 and SaO2 was assessed (i) with a four-quadrant plot analysis35, (ii) the clinical concordance method36 and (iii) the inter-rater agreement kappa coefficient (κ), with κ-value < 0.20 and > 0.80 indicating poor and good strength of agreement respectively37.

Multiple linear regressions were performed to identify variables associated with the dependent variable SpO2. Separate models were built depending on the selected variable (SaO2 or PaO2) in regard to SpO2. Moreover, multivariable models accounted for confounding variables (in regard to SpO2), and their interactions: SaO2 or PaO2, COVID-19 status, ethnic group, PEEP level, FiO2 setting, temperature, SpO2 probe and arterial catheter side (ipsilateral or contralateral), ARDS severity (as defined above, with mild category used as reference) and norepinephrine administration (defined as a categorical binary variable)14,15,16,17,18,19,20,21.

Assuming a correlation coefficient of 0.69 between SpO2 and SaO238 and an ICC > 0.9 with a 95% confidence interval (CI) (0.84–0.96), we planned to include at least 50 patients with at least 25 COVID-19 patients. A p value < 0.05 was considered statistically significant. Statistical analyses were performed using MedCalc 11.6.0 software (MedCalc®, Mariakerke, Belgium), and SPSS version 25.0 (IBM®, Armonk, USA).

Ethics approval and consent to participate

This study was approved by the Comité de Protection des Personnes Ile-de-France X (IDCRB2018-A00050-55, protocol 59–2018) and by the Ethics Committee of the Société de Réanimation de Langue Française (CE SRLF 20–72)25, which waived the need for informed consent, following national regulation on standard-of-care data collection. Indeed, this study was performed on data collected in the course of a prospective data collection, without any additional blood sample, as compared to standard of care. As such, only refusal to participate was systematically sought.

Results

Patient characteristics

Among 55 included patients: 33(60%) were men, 17(31%) were black and no patient had sickle-cell anaemia history. The ICU mortality rate was 38%. All patients had pulmonary ARDS and 26(47%) had an ARDS related to COVID-19. COVID-19 patients were more frequently black, tended to have more diabetes mellitus and had a lower SAPS-3 than non-COVID-19 patients (Table 1). The other characteristics of patients are shown in Tables 1 and 2.

Table 1 Patient characteristics.
Full size table
Table 2 Respiratory, haemodynamic and oxygenation effects of the PEEP trial according to patients’ COVID-19 status.
Full size table

A SpO2 finger probe was used in all patients but three and a radial arterial catheter was used in 39(71%) patients. SpO2 probe and arterial catheter were located on the same side in 30(55%) patients (Table 1). SpO2 values ranged from 83 to 100% and SaO2 values from 80 to 100%. Carboxyhaemoglobin and methaemoglobin rates were < 2% in all patients.

Association between SpO2, SaO2 and PaO2

In the whole population, SpO2 and SaO2 measurements were significantly correlated (r = 0.70, p < 0.0001). Correlation between SpO2 and SaO2 was lower in COVID-19 than in non-COVID-19 patients (r = 0.55, p < 0.0001 vs. r = 0.84, p < 0.0001, p = 0.002 for intergroup comparison) (Fig. 1A). After adjusting for confounding covariables, variables independently associated with SpO2 were: SaO2 (unstandardized β = 0.46, 95%CI = 0.35–0.57, p < 0.0001), COVID-19 status (unstandardized β = 1.96, 95%CI = 1.03–2.88 l, p < 0.0001) and PEEP level (unstandardized β = 0.11, 95%CI = 0.01–0.21, p = 0.03).

Figure 1
figure 1

Panel A: correlation between pulse oximetry (SpO2) and arterial oxygen saturation (SaO2) in COVID-19 patients (red points, n = 52 measurements) and non-COVID-19 patients (blue points, n = 58 measurements). The solid line represents the correlation line. The dotted lines represent the 95% confidence interval of each correlation. Panel B: correlation between pulse oximetry (SpO2) and arterial oxygen partial pressure (PaO2) in COVID-19 patients (red points, n = 52 measurements) and non-COVID-19 patients (blue points, n = 58 measurements). The dotted lines represent the 95% confidence interval of each correlation.

Full size image

For sensitivity, correlation between SpO2 and PaO2 was also assessed and found significant (r = 0.62, p < 0.0001). Similarly, correlation between SpO2 and PaO2 was lower in COVID-19 than in non-COVID-19 patients (r = 0.47, p= 0.001 vs. r = 0.75, p <  0.0001, p = 0.002 for intergroup comparison) (Fig. 1B). After adjusting for confounding covariables, variables independently associated with SpO2 were: PaO2 (unstandardized β = 0.03, 95%CI = 0.02–0.05, p < 0.0001), COVID-19 status (unstandardized β = 2.04, 95%CI = 0.99–3.08, p < 0.0001) and PEEP level (unstandardized β = 0.17, 95%CI = 0.06–0.28, p = 0.002).

Agreement between SpO2 and SaO2

In the whole population, the bias between SpO2 and SaO2 measurements was 1.1%, the precision was 3.4% and the limits of agreement ranged from -5.6 to + 7.9% (Fig. 2A). Bias and SaO2 measurements were significantly correlated (r = -− 0.56, p < 0.0001). The bias was higher (2.0 vs. 0.3%; p = 0.02) and the precision lower (4.1 vs. 2.5%) in COVID-19 than in non-COVID-19 patients (Fig. 2B). ICC = 0.79 (95%CI = 0.69–0.86), 0.62 (95%CI = 0.34–0.78) and 0.90 (95%CI = 0.84–0.94) in the whole population, in COVID-19 and in non-COVID-19 patients respectively. After adjusting for confounding covariables, COVID-19 status (unstandardized β = 1.77, 95%CI = 0.38–3.15, p = 0.01), ethnic group (unstandardized β = − 1.58, 95%CI = − 2.99 to − 0.18, p = 0.03) and ARDS severity (unstandardized β = 1.09, 95%CI = 0.21–1.97, p = 0.02) were independently associated with the bias. Interaction analyses did not yield significant association between COVID-19 status, ethnic group and bias.

Figure 2
figure 2

Comparison of pulse oximetry (SpO2) and arterial oxygen saturation (SaO2) measurements using the Bland–Altman method. The solid line represents the bias and the dotted lines represent the 95% limits of agreement (mean ± 1.96 standard deviation). Panel A: in the whole population (n = 110 measurements). Panel B: in COVID-19 patients (n = 52 measurements) and non-COVID-19 patients (n = 58 measurements).

Full size image

Occult hypoxemia, occurred in 6 (5.5%) pairs of measurements. There was no significant difference between COVID-19 and non-COVID-19 patients (7.7% vs. 3.4%, respectively, p = 0.42).

Effects of PEEP trial on SpO2, SaO2 and PaO2

In the whole population, PEEP increase from 5 to 15 cmH2O significantly increased SpO2, SaO2 and PaO2 by 3 ± 4%, 4 ± 5% and 43 ± 57% respectively, (Table S1). SpO2 changes were significantly correlated to that of SaO2 (r = 0.65, p < 0.0001) and PaO2 (r = 0.47, p < 0.001). The concordance rate between PEEP trial-induced changes in SpO2 and SaO2 was 84% (Fig. 3A) and the κ-coefficient was 0.60. The concordance rate between PEEP trial-induced changes in SpO2 and PaO2 was 85% and the κ-coefficient was 0.55 (Figure S1).

Figure 3
figure 3

Trending ability of pulse oximetry (SpO2) against arterial oxygen saturation (SaO2) measurements during a positive end-expiratory pressure trial based on four-quadrant concordance analysis. The error grid reflects the therapeutic consequences in specific zones in the concordance plot. Orange zone: ΔSpO2 and ΔSaO2 change in the same direction and to the same extent. Pink zone: ΔSpO2 and ΔSaO2 change in the same direction but not to the same extent. Green zone: ΔSpO2 changes while ΔSaO2 is constant or vice versa. Blue zone: ΔSpO2 and ΔSaO2 change in opposite directions. Panel A: in the whole population (n = 55). Panel B: in COVID-19 (n = 26) and non-COVID-19 (n = 29) patients.

Full size image

Concordance rate between PEEP trial-induced changes in SpO2 and SaO2 was lower in COVID-19 than in non-COVID-19 patients (69 vs. 97%), as was the κ-coefficient (0.35 vs. 1.00) (Fig. 3B). Similar results were found for PaO2 (73 vs. 97% for concordance rate and 0.28 vs. 1.00 for the κ-coefficient) (Figure S1).

Discussion

A reliable SpO2 measurement would be interesting in patients with ARDS, given the medico-economic impact of the decrease in arterial blood gas analysis overuse10. In this cohort of patients with ARDS monitored with last generation pulse oximeter, we found that (i) SpO2 was overall well-correlated with arterial oxygenation (SaO2 and PaO2) and that SpO2 slightly overestimated SaO2, (ii) COVID-19 status significantly impacted the association between SpO2 and arterial oxygenation (SaO2 and PaO2) and (iii), changes in SpO2 could not reliably track changes in arterial oxygenation (SaO2 and PaO2) during PEEP trial in COVID-19 patients.

The accuracy of SpO2 measurement in ICU is still debated because of several confounding factors14,15,16,17,18,19,20,39,40 with an unconstant and unpredictable bias between SpO2 and SaO2 measurements14,15,18,19,20,39,41. Overall, we found that SpO2 and arterial oxygenation (SaO2 and PaO2) were well-correlated and that SpO2 only slightly overestimated SaO2 but with wide limits of agreement, suggesting the lack of precision of SpO2 measurement. Our results are comparable to those from previous ICU studies, which found similar correlation38 and a bias ranging from 0.2 to 3%14,15,38,40,42 with a mean bias of 1.5%43 but wide limits of agreement14,15,38,40,42, highlighting the lack of improvement in the last generation pulse oximeter reliability. We also confirmed that that SpO2 measurement was still less reliable in black21 and in most hypoxemic14,15,16,17,18 patients. It has been very recently shown that black patients had nearly three times the frequency of occult hypoxemia that was not detected by SpO2 measurement as Caucasian patients21. Conversely to previous studies14,39, we also found that changes in SpO2 could track changes in arterial oxygenation (SaO2 and PaO2) during PEEP trial with a moderate to good strength of agreement. To our knowledge, this is one of the first studies analyzing not only the magnitude (correlation) but also the direction (concordance rate and κ-coefficient) of SpO2 and SaO2 changes. The discrepancy of our results to those of previous studies may be explained by the fact that rather than considering multiple therapeutic manoeuvres15,40, we only considered a PEEP trial to ensure that the possible lack of agreement between changes in SpO2 and SaO2 could not be attributed to the multiple therapeutic manoeuvres used.

We found that COVID-19 significantly impacted the association between SpO2 and arterial oxygenation (SaO2 and PaO2). First, SpO2 measurement was less reliable, as illustrated be weaker correlation, higher bias, wider limits of agreement, lower precision and lower ICC with an ICC < 0.7. Second, changes in SpO2 could not reliably track changes in arterial oxygenation (SaO2 and PaO2) during PEEP trial, with a concordance rate of 69% and a κ-coefficient of 0.35 for changes in SaO2 and a concordance rate of 73% and a κ-coefficient of 0.28 for changes in PaO2 in COVID-19 patients, but a concordance rate of 97% and a κ-coefficient of 1.00 both for SaO2 and PaO2 in non-COVID-19 patients. To our knowledge, this is the first study investigating the reliability of SpO2 measurement in COVID-19 patients, despite it has been suggested to use SpO2 monitoring in these patients to detect “silent hypoxemia”44,45. Importantly, the impact of COVID-19 on the association between SpO2 and arterial oxygenation cannot be explained solely by the fact that COVID-19 patients were predominantly black22. Moreover, the proportion of occult hypoxemia that we observed was that expected of our population, although no thorough analysis could be performed on this criterion due to its low prevalence. Indeed, although lack of power may have been involved, interaction analyses did not yield significant association between COVID-19 status, ethnic group and SpO2. With caution, this may suggest that COVID-19 per se may alter the accuracy of SpO2 measurement and the agreement between SpO2 and arterial oxygenation. This could be explained by the fact that COVID-19 patients may experience systemic microvascular alterations that appear to be common and linked to coagulopathy24 and/or endothelial dysfunction and endotheliitis46. Indeed, it has been shown that SARS-CoV-2 infection may facilitate the induction of endotheliitis with viral elements within endothelial cells and accumulation of inflammatory cells leading to apoptosis found in in several organs46. Further studies are needed to address the potential impact of COVID-19 on SpO2 measurement.

The clinical implications of our results are two-fold. In non COVID-19 patients, our results suggest that SpO2 measurements with latest generation pulse oximeter may be sufficient to monitor and track changes in arterial oxygenation in the most critically-ill patients. Thus, arterial blood gazes may not be necessarily mandatory after every ventilatory setting modification during the weaning process in patients with ARDS. Moreover, our results confirm those published by Martínez-Balzano who showed that there was an overuse of arterial blood gazes in critically ill patients and that their number could be reduced without negatively impacting patient care but with clear medico-economic impact10. Meanwhile, in COVID-19 patients, SpO2 measurement may not be as reliable, and iterative arterial blood gazes’ analyses to measure SaO2 may still be mandatory.

Our study has some limitations. First, we assessed only one type of last generation pulse oximeter. Second, we only considered the same two PEEP levels in all patients and we cannot exclude that considering other PEEP levels might influence the association between SpO2 and SaO2 as well as the concordance between PEEP trial-induced SpO2 and SaO2 changes. Third, we did not study, as previous studies18,40,41, the influence of carboxyhaemoglobin and methaemoglobin on the accuracy of SpO2 measurement. Indeed, pulse oximeters cannot differentiate between these two forms of haemoglobin and oxyhaemoglobin, leading to SaO2 overestimation in patients with high carboxyhaemoglobin and/or methaemoglobin rates47,48. Yet, we systematically excluded patients with methaemoglobinemia or carbon monoxide poisoning (all had carboxyhaemoglobin and methaemoglobin rates < 2%). Interestingly, Kerget and colleagues very recently showed that endogenous carboxyhaemoglobin may be an easily accessible biomarker of clinical course and prognosis in COVID-19 patients49. Fourth, we did not perform multiple arterial blood gas analyses, which may have allowed the estimation of the metrological precision feature of the pulse oximeter device. Indeed, random error variability of replicate measurements, defined as precision, usually requires to perform multiple measurements for a given timepoint50. In our study, we did not deem ethical to sample arterial blood gases several times for each datapoint. We pragmatically assumed that precision remained constant and random error measurements were reproducible, following the device specifications51.

Conclusion

In patients with ARDS, SpO2 was associated with arterial oxygenation (SaO2 and PaO2) and could track changes in arterial oxygenation with good reliability. Nevertheless, COVID-19 status significantly impacted the association between SpO2 and arterial oxygenation.