Abstract
Background
In the first wave of the COVID-19 pandemic, the hypothesis that angiotensin receptor blockers (ARBs) and angiotensin-converting enzyme inhibitors (ACEIs) increased the risk and/or severity of the disease was widely spread. Consequently, in many hospitals, these drugs were discontinued as a “precautionary measure”. We aimed to assess whether the in-hospital discontinuation of ARBs or ACEIs, in real-life conditions, was associated with a reduced risk of death as compared to their continuation and also to compare head-to-head the continuation of ARBs with the continuation of ACEIs.
Methods
Adult patients with a PCR-confirmed diagnosis of COVID-19 requiring admission during March 2020 were consecutively selected from 7 hospitals in Madrid, Spain. Among them, we identified outpatient users of ACEIs/ARBs and divided them in two cohorts depending on treatment discontinuation/continuation at admission. Then, they were followed-up until discharge or in-hospital death. An intention-to-treat survival analysis was carried out and hazard ratios (HRs), and their 95%CIs were computed through a Cox regression model adjusted for propensity scores of discontinuation and controlled by potential mediators.
Results
Out of 625 ACEI/ARB users, 340 (54.4%) discontinued treatment. The in-hospital mortality rates were 27.6% and 27.7% in discontinuation and continuation cohorts, respectively (HR=1.01; 95%CI 0.70–1.46). No difference in mortality was observed between ARB and ACEI discontinuation (28.6% vs. 27.1%, respectively), while a significantly lower mortality rate was found among patients who continued with ARBs (20.8%, N=125) as compared to those who continued with ACEIs (33.1%, N=136; p=0.03). The head-to-head comparison (ARB vs. ACEI continuation) yielded an adjusted HR of 0.52 (95%CI 0.29–0.93), being especially notorious among males (HR=0.34; 95%CI 0.12–0.93), subjects older than 74 years (HR=0.46; 95%CI 0.25–0.85), and patients with obesity (HR=0.22; 95%CI 0.05–0.94), diabetes (HR=0.36; 95%CI 0.13–0.97), and heart failure (HR=0.12; 95%CI 0.03–0.97).
Conclusions
The discontinuation of ACEIs/ARBs at admission did not improve the in-hospital survival. On the contrary, the continuation with ARBs was associated with a trend to a reduced mortality as compared to their discontinuation and to a significantly lower mortality risk as compared to the continuation with ACEIs, particularly in high-risk patients.
Similar content being viewed by others
Background
In mid-March, at the start of the first wave of COVID-19 pandemic in Europe, the hypothesis that the renin-angiotensin system inhibitors (RASIs), including angiotensin receptor blockers (ARBs) and angiotensin-converting enzyme inhibitors (ACEIs), increased the risk, and/or severity of the disease [1,2,3], was widely spread. Consequently, many hospitals and clinicians adopted the “precautionary measure” to discontinue these drugs from patients who regularly used them. Promptly, in the first weeks of May, three large epidemiological studies were published supporting the lack of association between the outpatient use of RASIs and risk of COVID-19 [4,5,6]. Later on, a plethora of studies and meta-analyses were published [7, 8] reaching the same conclusion, which provides reassurance on the safety of these drugs. Yet, the extent of RASI discontinuation at hospital admission during the first wave of the pandemic and, importantly, its impact on health outcomes have been scarcely studied [9,10,11,12].
The downregulation of angiotensin-converting enzyme type 2 (ACE2), as resulted from the SARS-CoV-2 endocytosis, has been postulated to play a key role in the progression of COVID-19 to severe forms [13]. In physiological conditions, the ACE1-angiotensin II-AT1R axis (the classical RAS) is counter-regulated by the ACE2-Angiotensin (1-7)-MasR axis. Thus, when the latter weakens, angiotensin II is unopposed and its vasoconstrictor, pro-inflammatory, and pro-thrombotic actions may contribute to the pathophysiology of severe COVID-19 [13,14,15]. In this context, it is conceivable that treatment with RASIs in COVID-19 inpatients could compensate the ACE1/ACE2 imbalance provoked by the SARS-CoV-2 and produce a net beneficial effect. According to this, several observational studies have reported a protective effect of inpatient use of RASIs on mortality as compared to non-use (or non-RASI use) in COVID-19 patients [9,10,11,12]. However, such studies have been criticized for incurring in several types of bias [16, 17]. Recently, two randomized clinical trials have been published [18, 19] reporting no difference in mortality between discontinuation and continuation arms. However, these trials and most observational studies have pooled ACEIs and ARBs and analyzed in a unique group, overlooking that they have different pharmacological actions [20] that could lead to distinct clinical effects [20], particularly in COVID-19 patients [15]. In this sense, no study has carried out a head-to-head comparison of in-hospital use of these drugs in COVID-19 patients admitted to the hospital.
The present research was aimed (1) to quantify the magnitude of RASI discontinuation at admission in seven hospitals from the Autonomous Community of Madrid, Spain; (2) to compare in real-life conditions the in-hospital mortality in patients in whom ACEIs or ARBs were discontinued with those in whom RASIs were continued; and (3) to perform a head-to-head comparison between in-hospital use of ACEIs and ARBs regarding mortality in admitted patients for COVID-19.
Methods
Study design, subject selection, and follow-up
We collected information from patients aged 18 years or older admitted to the hospital from March 1, 2020, to March 31, 2020, with a diagnosis of COVID-19 confirmed by RT-PCR. Seven hospitals of the Autonomous Community of Madrid (Spain) took part. According to drug exposure in the month prior to admission, patients were classified in three study groups: (1) users of RASIs, (2) users of non-RASI antihypertensive drugs, and (3) non-users of antihypertensive drugs. For the present study, only RASI users were considered. Among them, we excluded those in whom the continuation or discontinuation of RASI treatment could not be properly assessed at admission, including patients transferred to another hospital from the emergency department (ED) and patients who presented the outcome (death or admission to the intensive care unit (ICU)) or were discharged within the first 3 days of hospital admission. Hence, eligible patients had to survive and be outcome-free in a hospital ward (excluding ICU) at least during the first 3 days since admission to the ED. Then, they were subdivided in two closed cohorts: (1) Continuation cohort: patients in whom RASI prescriptions were recorded in at least 2 of the first 3 days since ED admission (including switching from one RASI to another) and (2) Discontinuation cohort: patients in whom no prescription of RASI was recorded in the first 3 days since ED admission. When there was a sole prescription of RASIs in the first 3 days, the intention-to-discontinue was considered uncertain and these patients were not included in the main analysis; however, we carried out two sensitivity analyses in which these patients were re-classified (see “Sensitivity analyses”). Both cohorts were then followed-up until discharge or in-hospital death (any cause), recording any ICU admission. The date of admission to the ED was considered the index date for the follow-up, so the above definitions assume an immortal time of 3 days in both continuation and discontinuation cohorts (avoiding this way a bias).
Sources of information and data collection
The information on co-morbidities and drug exposure before admission was extracted from electronic primary healthcare records, which are accessible through the viewer HORUS from any hospital in Madrid for authorized healthcare workers. The information on disease severity at admission and its clinical evolution (death, discharge, ICU admission, and in-hospital treatment received) was retrieved from hospital medical records. All data extracted were anonymized and included in ad hoc case report forms in each participating hospital, then sent out to the coordinating center, where a data quality control was undertaken to assure that all hospitals collected the information in the same manner.
Baseline co-morbidities and outpatient treatments
The presence of the following baseline co-morbidities was recorded at index date: antecedents of hypertension, dyslipidemia (recorded as such or when there was at least one prescription of a lipid-lowering drug), diabetes (recorded as such or when there was at least one prescription of a glucose-lowering drug), ischemic heart disease, atrial fibrillation, heart failure, thromboembolic disease, cerebrovascular accident (including stroke and transient ischemic accident), asthma, chronic obstructive pulmonary disease (COPD), chronic renal failure, and cancer (past and active). We also collected information on obesity (defined as a body mass index—BMI—≥ 30 kg/m2), smoking (current smoker, past smoker, non-smoker, or not recorded), and the outpatient use of calcium channel blockers (CCBs), beta-blocking agents, alpha-adrenoceptor antagonists with cardiovascular (CV) indications, high-ceiling diuretics, low-ceiling diuretics, antagonists of mineralocorticoid receptor (AMRs), lipid-lowering drugs, glucose-lowering drugs, antiplatelet drugs, oral anticoagulants, nonsteroidal anti-inflammatory drugs (NSAIDS), systemic corticosteroids, and non-opioid analgesics (paracetamol and metamizole).
Disease severity
To characterize the severity of COVID-19 at admission, we collected information on the presence of pneumonia, hypoxemia (defined as oxygen saturation ≤90% at rest breathing ambient air, or a PaO2/FiO2 ratio ≤300 mm Hg), lymphopenia, and abnormal values of five inflammatory biomarkers (according to the reference values of each hospital laboratory), when available: C-protein reactive (CPR), procalcitonin, troponin, D-dimer, and N-terminal type B natriuretic propeptide (NT-pro-BNP) [13]. With these 5 biomarkers plus hypoxemia and lymphopenia (1: abnormal; 0: otherwise), we generated a “severity score” ranging from 0 to 7 (values 0 and 1, as well as 6 and 7, were collapsed to assure enough number of patients) which showed a positive linear trend with the hazard ratio of in-hospital mortality (p=0.01), after adjusting for age, sex, baseline characteristics, outpatient treatments, hospital, and date of admission (see Additional file 1: Figure S1).
In-hospital drug exposure
The main exposure of interest was the inpatient use of RASIs (ACEIs and ARBs), including combinations with other antihypertensive drugs. We also collected information of in-hospital use of the following drugs: calcium channel blockers (CCBs), beta-blocking agents, alpha-adrenoceptor antagonists with cardiovascular (CV) indications, high-ceiling diuretics, low-ceiling diuretics, AMRs, lipid-lowering drugs, glucose-lowering drugs (oral and insulin), antiplatelet drugs, anticoagulants (oral or parenteral), antiviral agents, chloroquine/hydroxychloroquine, azithromycin, and other macrolides, other antibiotic agents, systemic steroids, and other immunomodulators.
Outcomes
The main outcome variable was time to in-hospital death for any cause. As a secondary outcome, we also considered the time to a composite of in-hospital death and time to ICU admission, whichever occurred first.
Statistical analysis
We expressed quantitative variables as mean and standard deviation (SD), or median and interquartile range (IQR) for not normally distributed data, and qualitative variables as frequencies and percentages. Differences in quantitative variables were assessed using the Student’s t test or Mann-Whitney U test (for parametric or non-parametric evaluation between two groups, respectively). Differences in frequencies were assessed using the chi-squared test or Fisher’s exact test when assumptions for chi-square test were not met. The standardized difference was also calculated for means and proportions as a measure of the covariate balance between the exposure groups [21].
To estimate the effect of RASI discontinuation on the outcomes, we carried out an intention-to-treat (ITT) analysis, so that patients were analyzed in their assigned closed cohorts (discontinuation or continuation) defined in the first 3 days of hospitalization, whatever happened thereafter. Then, we proceeded as follows: (1) A binary logistic model was constructed to estimate the propensity score (PS) of RASI discontinuation conditioned on baseline co-morbidities, outpatient treatments, hospital of admission, date of admission (in three periods of equal length), severity score at admission, presence of pneumonia, and treatments prescribed in the first 3 days of hospitalization (including antihypertensive drugs, chloroquine/hydroxychloroquine, and antivirals, the latter two prescribed per protocol for most admitted COVID-19 patients) [22]; (2) Then, we built a Cox proportional hazards model which included the exposure and the estimated PS as a flexible function (restricted cubic splines with 5 knots accounting for 5th, 25th, 50th, 75th, and 95th percentiles) to compute the PS-adjusted hazard ratios (HRs) and their 95% confidence intervals (95%CI); we preferred to use a flexible function instead of simple PS adjustment due to the lack of a linear relationship between PS and the outcome) [23]; (3) We also estimated the controlled direct effect of RASI discontinuation on outcomes by including in the PS-adjusted Cox model the potential mediators (those associated with the exposure, as well as the outcome, controlling for the exposure [23]: systemic corticosteroids, anticoagulants, and immunomodulators when death was the outcome and immunomodulators and anticoagulants when the outcome was death plus ICU admission). To avoid a collider bias, we also included potential mediator-outcome confounders in the Cox model [24, 25] (antiplatelet drugs when the outcome was death and systemic steroids when the outcome was death plus ICU admission), according to our hypothesized causal graph (see Additional file 1: Figure S2). This way we computed the mediator-controlled HRs (MC-HR) and their 95% CIs.
We also built univariate Kaplan-Meier survival curves for the exposures and outcomes of interest, using log-rank test to evaluate the differences in survival curves across different levels of exposure. The proportional hazard assumption of COX models was checked using the Schoenfeld residuals test and confirmed graphically with a log-minus-log survival plot and by comparison of the Kaplan-Meier survival curves with the Cox predicted curves [23].
The possible effect modification (or interaction) by gender, age, diabetes, obesity, background CV risk, heart failure, severity score (in two categories, using the median as the cut-off point), and in-hospital use of corticosteroids and beta-blockers was assessed stratifying the Cox model by the categories of the potential interacting variables and then comparing the HRs across strata with the Altman and Bland test for interaction [26]. The background CV risk was built as a composite variable with two categories: (1) antecedents of CV disease which includes ischemic heart disease, cerebrovascular accident, heart failure, atrial fibrillation, and thromboembolic disease and (2) CV risk factors only which includes hypertension, dyslipidemia, diabetes, or chronic renal failure.
All the aforementioned analyses were performed for the following comparisons: (1) RASI discontinuation vs. RASI continuation, (2) ACEI discontinuation vs. ACEI continuation, (3) ARB discontinuation vs. ARB continuation, and (4) ARB continuation vs. ACEI continuation.
All analyses were performed with STATA/SE v.15 (StataCorp LLC, College Station, TX. USA. 2017) and Python (Python Software Foundation, 2001–2020).
Sensitivity analyses
Three sensitivity analyses were performed: (1) reclassifying patients in whom RASI discontinuation was uncertain, so that those with a sole prescription recorded in day 2 or day 3 were assigned to the continuation cohort, and patients with a sole prescription recorded in day 1 were assigned to the discontinuation cohort; (2) assigning all patients in whom discontinuation was uncertain to the discontinuation cohort; and (3) using a 2-day window, instead of a 3-day window, to define RASI (dis)continuation (see Additional file 1: Figure S3).
Results
Patient selection and discontinuation rates
A total of 2029 patients were consecutively admitted with a PCR-confirmed COVID-19, being 819 outpatient users of RASIs. In 141 of them, we were unable to assess the continuation of RASIs (59 patients were directly admitted to the ICU: 47 from the ED and 12 from other hospitals; 44 were transferred from the ED to another hospital; 38 had the event—death or ICU admission—or were discharged within the first 3 days of admission); and in 53, the intention-to-discontinue was uncertain (22 presented a sole prescription in days 1 and 31 in days 2 or 3). Overall, 625 patients were included in the main analysis; out of them, 285 (45.6%) continued and 340 (54.4%) discontinued RASI treatment (Fig. 1).
RASI discontinuation rates varied greatly across participating hospitals (ranging from 23.5 to 93.0%) and proved to be highly dependent on the date of admission (from 32.1% in the first 10 days of March to 74.2% in the last 10 days of March) (Table 1 and Additional file 1: Figure S4). Among patients who discontinued RASIs, 131 (38.5%) received treatment with CCBs (alone or combined with other antihypertensive drugs), 51 (15.0%) with other antihypertensive drugs (OADs) alone, and 158 (46.5%) had no recorded antihypertensive treatment within the first 3 days of admission (furosemide excluded) (Fig. 2). A similar pattern was observed when ACEIs and ARBs were considered separately (Additional file 1: Figure S5).
Switching from RASIs to CCBs, other antihypertensive drugs (OADs), or no antihypertensive treatment during the first 3 days since hospital admission (patients with uncertain discontinuation were excluded). Of all outpatient RASI users, 45.6% continued with RASIs (alone or combined with CCBs or OADs), 29.2% were switched to CCBs or OADs, and 25.3% were left without any antihypertensive treatment. Dynamic visualization available in: https://public.flourish.studio/visualisation/4808393/. Abbreviations: CCBs calcium-channel blockers, OADs other antihypertensive drugs (different from RASIs or CCBs), RASIs renin-angiotensin system inhibitors. RASIs+CCB combined use with OADs allowed, RASIs+OADs use of CCBs excluded, CCBs alone or combined with OADs and RASIs excluded, OADs use of RASIs and CCBs excluded
Patient characteristics
The baseline characteristics of patients who discontinued and continued treatment with RASIs are shown in Table 1. Baseline co-morbidities and co-medications appeared to be well-balanced, though patients who discontinued had a broadly lower prevalence of co-morbidities (statistically significant for obesity, history of heart failure, and history of a cerebrovascular accident). At admission, severity markers appeared to be well-balanced, though patients who discontinued presented a higher proportion of pneumonia (93.8% vs. 88.4%; p=0.02), and average severity score (3.1 vs. 2.9; p=0.03) (Table 1). The distribution of estimated PS for RASI discontinuation according to actual discontinuation or continuation of RASIs is shown in Additional file 1: Figures S8a and S8b.
During hospitalization, patients in whom RASIs were discontinued presented a higher proportion of treatment with parenteral anticoagulants, systemic corticosteroids, and CCBs, while patients who continued with RASIs presented a higher use of oral anticoagulants, statins, oral glucose-lowering drugs, other macrolides (different from azithromycin), tocilizumab or other immunomodulating agents, beta-blockers, and low-ceiling diuretics (Table 2). ICU admission was similar in both groups (5.6% vs. 6.0% for patients who discontinued and continued with RASIs, respectively), as well as the median hospital stay (11 vs. 10 days). Similar patterns were observed when RASIs were disaggregated by ACEIs and ARBs (Additional file 1: Tables S1 and S2).
Mortality rates associated with RASI discontinuation vs. RASI continuation
Among patients in whom RASIs were discontinued, 94 (27.6%) died in-hospital whereas 79 (27.7%) died among patients in whom RASIs were continued, which yielded a PS-adjusted HR of 1.01 (95%CI 0.71–1.46) that was not modified after controlling for potential mediators (MC-HR=1.01; 95%CI 0.70–1.46). Similar results were found when the outcome was the composite of death and ICU admission (Table 3). Of note, when RASIs were disaggregated by ACEIs and ARBs, we found a non-significant increased mortality risk among patients in whom ARBs were discontinued (PS-adjusted HR=1.58; 95%CI 0.87–2.87; and MC-HR=1.59; 95%CI 0.89–2.85), whereas among patients in whom ACEIs were discontinued we observed the opposite trend (PS-adjusted HR=0.73; 95%CI 0.44–1.19; and MC-HR=0.70; 95%CI 0.42–1.17) (Table 3).
Head-to-head comparison between ARB versus ACEI continuation
Among 285 patients who continued with RASIs, 136 did so with ACEIs and 125 with ARBs; 24 patients who used dual therapy or were crossed over to the other treatment were excluded from this analysis. The baseline characteristics and in-hospital treatment of patients who continued with ARBs and ACEIs appeared to be evenly distributed with some exceptions (i.e., use of corticosteroids, beta-blockers, and low-ceiling diuretics, all of them greater among ARB users) (Additional file 1: Table S3), but the mortality rates were remarkably different (20.8% vs. 33.1% for ARBs and ACEIs, respectively; p=0.03), yielding a head-to-head crude HR of 0.57 (95%CI 0.35–0.93), which barely changed after adjustment for baseline covariates (PS-HR=0.56; 95%CI 0.32–0.99) and after controlling for mediators (including systemic corticosteroids, immunomodulators, and anticoagulants) (MC-HR=0.52; 95%CI 0.29–0.93) (Table 4). The respective Kaplan-Meier survival curves are shown in Fig. 3, with the log-rank test resulting in a p value of 0.02. The median survival time was 25 days for patients who continued with ACEIs and was not reached for patients who continued with ARBs. For the composite outcome, the trend to a reduced mortality risk associated with ARBs as compared to ACEIs was still present, but did not reach statistical significance (MC-HR= 0.59; 95%CI 0.35–1.01) (Table 4).
Kaplan-Meier survival curves of in-hospital death among patients in whom treatment with ARBs was continued as compared to those in whom ACEIs was continued (defined in the first 3-day window). Abbreviations: ACEIs angiotensin-converting enzyme inhibitors, ARBs angiotensin receptor blockers. *Log-rank test
Analysis of potential interactions
No statistically significant interaction was observed by gender, age (<75; 75+years), obesity, diabetes, heart failure, background cardiovascular risk, severity score (0–3; 4–7), and in-hospital use of corticosteroids or beta-blockers (Additional file 1: Figure S6). The results disaggregated by ACEIs and ARBs are shown in Additional file 1: Figure S7. A trend to a higher risk associated with ARBs discontinuation was observed in all subgroups, being particularly relevant for obese people (MC-HR= 5.40; 95%CI 1.25–23.3; test for interaction, p=0.08)
For the comparison between continuation with ARBs vs continuation with ACEIs, we found a statistically significant interaction with a past history of heart failure (Fig. 4). It is interesting to note that the reduced risk of mortality associated with ARB continuation as compared to ACEI continuation was particularly relevant (and statistically significant) in high-risk subgroups: males, patients aged 75 years or older, obese, diabetics, and patients with antecedents of heart failure (Fig. 4). It is also important to highlight that the use of in-hospital systemic corticosteroids did not appear to mediate or modify the reduced risk associated with ARB continuation (MC-HR in patients who received corticosteroids=0.54, 95%CI 0.27–1.09 and MC-HR in patients who did not = 0.46 (95%CI 0.17–1.23) (Fig. 4).
Head-to-head comparison of continuation with angiotensin receptor blockers vs. continuation with angiotensin-converting enzyme inhibitors, by different subgroups. Abbreviations: ACEIs angiotensin-converting enzyme inhibitors, ARBs angiotensin receptor blockers, CV cardiovascular. *Mediator-controlled hazard ratio (controlled direct effect): including systemic corticosteroids (excepting stratification by corticosteroids), anticoagulants, and immunomodulators
Sensitivity analyses
Sensitivity analyses performed after reclassifying patients with uncertain (dis)continuation or using a 2-day window yielded similar results to the main analysis (Additional file 1: Table S4).
The proportional hazards assumption was fulfilled for all Cox regression analyses according to the Schoenfeld residuals test.
Discussion
The main findings of the present study are as follows: (1) RASIs were discontinued in around half of the patients admitted to hospital for COVID-19 during March 2020; (2) the discontinuation rate increased over time, being particularly notorious since March 11; (3) the discontinuation of RASIs as a group was not associated with an increased or decreased risk of in-hospital death or ICU admission, but the results disaggregated by ARBs and ACEIs were not homogeneous; and (4) the continuation of treatment with ARBs was associated with a significantly lower all-cause mortality than the continuation of treatment with ACEIs.
The RASI discontinuation rate was strongly influenced by the date of admission (doubling from mid-March), which seems to be a direct consequence of the hypothesis that quickly spread since March 11 on the possibility that these drugs could make COVID-19 more severe [3]. Notwithstanding, the rate varied considerably by the hospital (and possibly by the attending physician within each hospital). In other countries, researchers have reported discontinuation rates ranging from 12.4 to 67.7%, though using different definitions for discontinuation (Additional file 1: Table S5) [9,10,11,12, 17, 27,28,29,30,31,32,33,34] . Of note, in our study, as much as 46.5% of patients in whom treatment with RASIs were discontinued (25.3% of the total number of patients who used them prior to admission) were left without any antihypertensive drug (excluding furosemide), which suggests that in a relevant part of patients RASIs were discontinued for medical reasons, likely related to an unstable hemodynamic situation.
Our main finding is that the discontinuation of RASIs, as a group, did not have an impact on in-hospital mortality or in the composite of in-hospital mortality plus ICU admission. This result seems robust as it hardly varied in different sensitivity analyses in which we modified the definition of (dis)continuation. Contrary to the huge number of studies carried out to assess the impact of outpatient use of RASIs on different outcomes (COVID-19 infection, hospitalization, and mortality, among others) [7, 8]; fewer studies have been performed thus far to examine the association of inpatient use of RASIs with in-hospital mortality. One of the first studies was published by Zhang et al. [11] with data from 9 hospitals in Hubei province (China). They found an all-cause mortality among inpatients treated with RASIs much lower than non-treated patients, with an adjusted HR of 0.42 (95%CI 0.19–0.92). However, this study was criticized because the authors considered exposure to all patients who received RASIs at any time point during hospitalization, which implies that exposed patients had to survive long enough, or be clinically stable enough, to receive the treatment with RASIs [16]. Thus, such definition of the exposure could have introduced an immortal-time bias [16] and a confounding by severity (also graphically called “healthy user-sick stopper” bias” [17], that is, RASIs were more likely to be continued, initiated, or reinstated in less severe cases), both favoring an overestimation of the benefit of RASIs on mortality. Most researchers thereafter used similar definitions incurring in the same types of bias and most coinciding to show an important reduced mortality risk associated with RASIs [9, 10, 12, 27,28,29,30,31,32,33,34] (see Additional file 1: Table S5 for a detailed description of studies). To overcome these problems, we defined continuation or discontinuation during the first 3 days (or during the first 2 days in a sensitivity analysis) and then followed an ITT analysis (each patient analyzed in his/her assigned closed cohort), as it would have been done in a clinical trial. Also, to avoid a reverse causation, we excluded patients directly admitted to the ICU (from the ED or from another hospital), a situation in which RASIs are usually discontinued as a consequence of the disease severity. Interestingly, if we had defined continuation as “use of RASIs at any time point during hospitalization” and included patients directly admitted to the ICU in the discontinuation cohort, the mortality rates would have been 25.3% and 30.3% in the continuation and discontinuation cohorts, respectively, yielding a HR of 0.83 (95%CI 0.66–1.05) for in-hospital mortality. For the composite outcome (death plus ICU admission), the rates would have been 30.0% for patients in whom RASIs were continued and 43.6% in those who discontinued giving rise to a HR of 0.67 (95%CI 0.57–0.83). Therefore, the results would have been dramatically different than the ones we actually obtained, showing the extent of such biases.
Recently, the results from two randomized clinical trials in which regular users of RASIs who were admitted to hospital for COVID-19 were assigned to discontinuation or continuation arms, have been reported (BRACE-CORONA [18] and REPLACE COVID [19] trials) and both found no difference in the mortality rates, supporting our results. However, it is important to emphasize that in the BRACE-CORONA trial the mortality rates were very low (2.7% among patients assigned to discontinuation and 2.8% in those assigned to continuation), casting doubts on the generalizability of their results (the mean age of the study population was 55 years old, 20 years younger than our population). Also, the measure of association of mortality was too imprecise (odds ratio=0.97; 95% CI 0.38–2.52) to be informative. Interestingly, 80% of patients were prior users of ARBs, and the authors found quasi-significant results favoring continuation in older persons, obese patients, and in those clinically more severe, in line with our findings (see later). The REPLACE COVID trial had a more representative population and consistently found no difference in all-cause mortality (15% and 13% in the continuation and discontinuation arms, respectively). Unfortunately, the sample size was too small to make a meaningful separate analysis by ACEIs and ARBs.
The different mortality rates among patients who continued with ACEIs versus those who continued with ARBs is a novel finding that merits specific comments. Firstly, it is important to emphasize that this comparison is ideal for several reasons: (a) these drugs have overlapping indications, thereby the subjects who use them are highly comparable, seemingly reducing by design the possibility of confounding (due to either known and unknown factors); (b) the possibility of an immortal-time bias is inexistent, as the same definition of continuation was applied to both cohorts; (c) the possibility of a confounding by severity is unlikely, as it is not reasonable to think that physicians used different criteria for the continuation of ARBs or ACEIs, and additionally, we applied an ITT analysis once continuation was defined based on the records of the first 3 days of hospitalization; and, finally, (d) the few differences we found (such as the greater in-hospital use of systemic corticosteroids in the ARB continuation cohort) were controlled for by including this factor in the outcome regression model and by stratification, and none of these strategies changed the results, reinforcing the internal validity of the comparison.
Secondly, most previous studies have pooled ACEIs and ARBs (see Additional file 1: Table S5), as if they were the same type of drugs. However, our results show that this approach may be wrong; also, there are profound pharmacological reasons that make this grouping invalid, in particular for COVID-19 patients. ARBs block selectively the action of angiotensin II on AT1 receptor (AT1R), and free angiotensin II is then converted by the ACE2 into angiotensin (1–7) which acts on Mas1 receptor (Mas1R) to induce opposite actions to angiotensin II (anti-inflammatory, anti-oxidant, anti-fibrotic, anti-thrombotic, anti-hypertrophic, vasodilatation, and natriuresis) [13,14,15]. Also, angiotensin II not used in activating AT1R acts on AT2 receptor (AT2R) (for which ARBs have no affinity), whose activation is known to produce opposite actions to the ones derived from the activation of AT1R [15], thereby collaborating with the protective effect of angiotensin (1–7). Instead, ACEIs inhibit the formation of angiotensin II, which pre-empts the generation of angiotensin (1–7) from both angiotensin II via ACE2, but also from angiotensin (1–9) via ACE1 [13,14,15]; additionally, the beneficial actions derived from activation of AT2R do not take place. In sum, both ARBs and ACEIs effectively block RAS, whereas only ARBs appear to reinforce its counterregulatory system, via ACE2-angiotensin (1–7)-Mas1R axis and AT2R activation, a difference that could be critical in COVID-19 patients. Additionally, ACE1 is well-known to be the major vascular peptidase of bradykinin, an abundant peptide which promotes vasodilatation, vascular permeability, and liberation of inflammatory cytokines (IL-1, IL-2, IL-6, IL-8, and TNF-alpha) implicated in the cytokine storm associated with the severe forms of COVID-19 [15]. Therefore, ACEIs will reduce bradykinin degradation, thereby potentiating its effects, which ultimately could be detrimental for COVID-19 patients [15, 35, 36]. These negative collateral actions of ACEIs may offset the benefits derived from the inhibition of angiotensin II formation and, we postulate, that these could account for the important difference we found in the mortality rates among inpatients treated with ARBs and those treated with ACEIs (an absolute difference of 12.3%, corresponding to a number needed to treat as low as 8). Importantly, the benefit of ARBs seems to be particularly evident in high-risk subgroups: males, the very old, obese, diabetics, and patients with antecedents of heart failure (as the BRACE-CORONA trial [18] also has shown, as commented before). Nevertheless, our results need confirmation, in particular through randomized clinical trials, and until then, we should take these findings with caution. Some are in progress aiming to assess the benefits of using ARBs in COVID-19 patients with the acute respiratory syndrome as compared to placebo or standard care [NCT04394117, NCT04312009, and NTC04355936], but, as far as we know, no study has been designed to compare ARBs with ACEIs in this context. Rodilla et al. [30] compared survival of COVID-19 patients according to the use of ARBs and ACEIs prior to admission and found a significantly reduced mortality risk with the former (25.6% vs. 30.4%, respectively, p=0.0001); but, unfortunately, a head-to-head comparison of in-hospital use of ARBs vs. ACEIs was not reported. Finally, it is of interest to note that in the study by Zhang et al. [11], 83.5% of patients reported to be on RASIs were actually treated with ARBs.
Our study has some limitations that must be discussed: (1) as in all observational studies, the possibility exists that there is some residual confounding due to unknown or unmeasured factors; also, a residual confounding by indication cannot be ruled out. Notwithstanding, it is important to remark that all our patients were users of RASIs prior to admission and were highly comparable at baseline, as shown by the good balance of covariates and the fact that the mean and median of the propensity scores for RASI discontinuation was close to 0.5 (Additional file 1: Figure S8); indirectly, it is likely that unmeasured confounding variables are evenly distributed too, albeit this cannot be assured; as previously commented, this is specially applicable to the comparison of ACEI and ARB continuation cohorts; (2) the information on some severity biomarkers (i.e., interleukins 6 or 1β) were not routinely performed at that time and were not considered in the severity score built for this study; on this regard, we would like to emphasize that such score was created to reduce the number of covariates included in the PS models, and it is not proposed as a prognostic index (as we are quite aware that a specific and independent validation study would be necessary for that); (3) the study period selected (March, 2020) was the most critical of the first wave in Spain and, at that time, health professionals worked under an extraordinary pressure, which may have led to under-recording of some relevant clinical information; this limitation, however, does not apply to drugs as they were prescribed through an electronical tool, making unlikely the misclassification of drug exposure; and (4) the mortality rates recorded in our study were extraordinary high (partly accounted for the lack of preparedness of the health system to address this disease at the very beginning of the pandemic) and remarkably different from figures corresponding to other periods during the first and successive waves in Spain or in other countries, so the generalizability of our data on this regard cannot be assured; however, we do not think that this affects the internal validity of our results.
Conclusions
The discontinuation of RASIs at hospital admission was common place in the first wave of COVID-19 pandemic in Spain, influenced by the widely spread hypothesis that postulated a more severe disease in patients treated with these drugs. Our results show that the discontinuation of these drugs at admission did not improve the in-hospital survival. On the contrary, we found that the discontinuation of treatment with ARBs was associated with a trend to an increased mortality risk as compared to their continuation. Moreover, the continuation with ARBs was associated to a significantly lower mortality risk as compared to the continuation with ACEIs, particularly evident in high-risk subgroups.
Availability of data and materials
The anonymized dataset and the complete analysis will be publicly available at this site: https://github.com/antoniohupa/RASI_discontinuation_COVID19.
Abbreviations
- ACE1:
-
Angiotensin-converting enzyme 1
- ACE2:
-
Angiotensin-converting enzyme 2
- ACEIs:
-
Angiotensin-converting enzyme inhibitors
- AMRs:
-
Antagonists of mineralocorticoid receptor
- ARBs:
-
Angiotensin receptor blockers
- AT1R:
-
Receptor for angiotensin II type 1
- AT2R:
-
Receptor for angiotensin II type 2
- BMI:
-
Body mass index
- CCBs:
-
Calcium channel blockers
- CI:
-
Confidence interval
- COPD:
-
Chronic obstructive pulmonary disease
- COVID-19:
-
Coronavirus disease 19
- CPR:
-
C-protein reactive
- CV:
-
Cardiovascular
- ED:
-
Emergency department
- HR:
-
Hazard ratio
- ICU:
-
Intensive care unit
- IQR:
-
Interquartile range
- ITT:
-
Intention-to-treat
- MC-HR:
-
Mediator-controlled hazard ratio
- NSAIDs:
-
Nonsteroidal anti-inflammatory drugs
- NT-pro-BNP:
-
N-terminal pro-peptide of natriuretic factor type B
- OADs:
-
Other antihypertensive drugs different from RASIs or CCBs
- OR:
-
Odds ratio
- PS:
-
Propensity scores
- PS-adjusted HR:
-
Propensity score adjusted hazard ratio
- RASIs:
-
Renin-angiotensin system inhibitors
- SD:
-
Standard deviation
References
O´Mara G. Could ACE inhibitors and particularly ARBs increase susceptibility to COVID-19 infection? BMJ. 2020;368 Available in https://www.bmj.com/content/368/bmj.m406/rr-13.
Sommerstein R. Preventing a COVID-19 pandemic: ACE inhibitors as potential risk factor for fatal Covid-19. BMJ. 2020;368:m810. https://www.bmj.com/content/368/bmj.m810/rr-2.
Fang L, Karakiulakis G, Roth M. Are patients with hypertension and diabetes mellitus at increased risk for COVID-19 infection? Lancet Respir Med. 2020;8(4):e21. https://doi.org/10.1016/S2213-2600(20)30116-8.
Mancia G, Rea F, Ludegnani M, Apolone G, Corrao G. Renin-angiotensin-aldosterone system blockers and the risk of Covid-19. N Engl J Med. 2020;382(25):2431–40. https://doi.org/10.1056/NEJMoa2006923.
Reynolds H, Adhikari S, Pulgarin C, Troxel AB, Iturrate E, Johnson SB, et al. Renin-angiotensin-aldosterone system inhibitors and risk of COVID-19. N Engl J Med. 2020;382(25):2441–8. https://doi.org/10.1056/NEJMoa2008975.
De Abajo FJ, Rodríguez-Martin S, Lerma V, Mejía-Abril G, Aguilar M, García-Luque A, et al. Use of renin-angiotensin-aldosterone system inhibitors and risk of COVID-19 causing hospital admission: a case-population study. Lancet. 2020;395(10238):1705–14. https://doi.org/10.1016/S0140-6736(20)31030-8.
Flacco ME, Martellucci CA, Bravi F, Parruti G, Cappadona R, Mascitelli A, et al. Treatment with ACE inhibitors or ARBs and risk of severe/lethal COVID-19: a meta-analysis. Heart. 2020;106(19):1519–24. https://doi.org/10.1136/heartjnl-2020-317336.
Ren L, Yu S, Xu W, Oveton JL, Chiamvimonvat N, Thai PN. Lack of association of antihypertensive drugs with the risk and severity of COVID-19: a meta-analysis. J Cardiol. 2020;S0914-5087(20):30353–1. https://doi.org/10.1016/j.jjcc.2020.10.015.
Cannata F, Chiarito M, Reimers B, Azzolini E, Ferrante G, My I, et al. Continuation versus discontinuation of ACE inhibitors or angiotensin II receptor blockers in COVID-19: effects on blood pressure control and mortality. Eur Heart J Cardiovasc Pharmacother. 2020;6(6):412–4. https://doi.org/10.1093/ehjcvp/pvaa056.
Lam KW, Chow KW, Vo J, Hou W, Li H, Richman PS, et al. Continued in-hospital angiotensin converting enzyme inhibitor and angiotensin II receptor blocker use in hypertensive COVID-19 patients is associated with positive clinical outcome. J Infect Dis. 2020;222(8):1256–64. https://doi.org/10.1093/infdis/jiaa447.
Zhang P, Zhu L, Cai J, Lei F, Qin JJ, Xie J, et al. Association of inpatient use of angiotensin converting enzyme inhibitors and angiotensin ii receptor blockers with mortality among patients with hypertension hospitalised with COVID-19. Circ Res. 2020;126(12):1671–81. https://doi.org/10.1161/CIRCRESAHA.120.317134.
Chaudhri I, Koraishy FM, Bolotova O, Yoo J, Marcos LA, Taub E, et al. Outcomes associated with the use of renin-angiotensin-aldosterone system blockade in hospitalised patients with SARS-CoV-2 infection. Kidney 360. 2020;1:801–9. https://doi.org/10.34067/kid.0003792020.
Vaduganathan M, Vardeny O, Michel T, McMurray JJV, Pfeffer MA, Solomon SD. Renin-angiotensin-aldosterone system inhibitors in patients with COVID-19. N Engl J Med. 2020;382(17):1653–9. https://doi.org/10.1056/NEJMsr2005760.
Sriram K, Insel PA. A hypothesis for pathobiology and treatment of COVID-19: the centrality of ACE1/ACE2 imbalance. Br J Pharmacol. 2020;177(21):4825–44. https://doi.org/10.1111/bph.15082.
Chung M, Karnik S, Saef J, Bergman C, Barnard J, Lederman MM, et al. SARS-CoV-2 and ACE2: the biology and clinical data settling the ARB and ACEI controversy. EBioMedicine. 2020;58:102907. https://doi.org/10.1016/j.ebiom.2020.102907.
Cohen JB, Hanff TC, South AM, Sparks MA, Hiremath S, Bress AP, et al. Response by Cohen et al to letter regarding article, “association of inpatient use of angiotensin-converting enzyme inhibitors and angiotensin ii receptor blockers with mortality among patients with hypertension hospitalised with COVID-19”. Circ Res. 2020;126(12):e140–1. https://doi.org/10.1161/CIRCRESAHA.120.317205.
Lahens A, Mullaert J, Gressens S, Gault N, Flamant M, Deconinck L, et al. Association between renin-angiotensin-aldosterone system blockers and outcome in coronavirus disease 2019: analysing in-hopsital exposure generates a biased seemingly protective effect of treatment. J Hypertens. 2020;39(2):367–75. https://doi.org/10.1097/HJH.0000000000002658.
Lopes RD, Macedo AVS, de Barros E, Silva PGM, Moll-Bernardes RJ, Dos Santos TM, et al. Effect of discontinuing vs continuing angiotensin-converting enzyme inhibitors and angiotensin ii receptor blockers on days alive and out of the hospital in patients admitted with COVID-19: a randomised clinical trial. JAMA. 2021;325(3):254–64. 33464336. https://doi.org/10.1001/jama.2020.25864.
Cohen JB, Handff TC, William P, Sweitzer N, Rosado-Santander NR, Medina C, et al. Continuation versus discontinuation of renin-angiotensin system inhibitors in patients admitted to hospital with COVID-19: a prospective, randomised, open-label trial. Lancet Respir Med. 2021; https://doi.org/10.1016/S2213-2600(20)30558-0.
Messerli FH, Bangalore S, Bavishi C, Rimoldi SF. Angiontensin-converting enzyme inhibitors in hypertension – to use or not use? J Am Coll Cardiol. 2018;71(13):1474–82. https://doi.org/10.1016/j.jacc.2018.01.058.
Austin PS. Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Stat Med. 2009;28(25):3038–107. https://doi.org/10.1002/sim.3697.
Austin PS. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behav Res. 2011;46(3):399–424. https://doi.org/10.1080/00273171.2011.568786.
Vittinghof E, Gludden DV, Shiboski SC, McCulloch CE. Survival Analysis. In: Survival analysis, in regression methods in biostatistics. 2nd ed. New York: Springer; 2012. p. 203–59.
Glymour MM, Greenland S. Causal diagrams. In: Rothman K, Greeland S, Lash TL, editors. Modern Epidemiology. 3rd ed. Philadelphia: Wolters Kuwer/Lippincott Williams & Wilkins; 2008. p. 183–209.
Richiardi L, Bellocco R, Zugna D. Mediation analysis in epidemiology: methods, interpretation and bias. Int J Epidemiol. 2013;42(5):1511–9. https://doi.org/10.1093/ije/dyt127.
Altman DG, Bland JM. Interaction revisited: the difference between two estimates. BMJ. 2003;326(7382):219. https://doi.org/10.1136/bmj.326.7382.219.
Li J, Wang X, Chen J, Zhang H, Deng A. Association of renin-angiotensin system inhibitors with severity or risk of death in patients with hypertension hospitalised for coronavirus disease 2019 (COVID-19) infection in Wuhan, China. JAMA Cardiol. 2020;5(7):825–30. https://doi.org/10.1001/jamacardio.2020.1624.
COVID-19 RISk and Treatments (CORIST) Collaboration. RAAS inhibitors are not associated with mortality in COVID-19 patients: findings from an observational multicenter study in Italy and a meta-analysis of 19 studies. Vascul Pharmacol. 2020;135:106805. https://doi.org/10.1016/j.vph.2020.106805.
Wang Z, Zhang D, Wang S, Jin Y, Huan J, Wu Y, et al. A retrospective study from 2 centers in China on the effects of continued use of angiotensin-converting enzyme inhibitors and angiotensin II receptor blockers in patients with hypertension and COVID-19. Med Sci Monit. 2020;26:e926651. https://doi.org/10.12659/MSM.926651.
Rodilla E, Saura A, Jiménez I, Mendizábal A, Pineda-Cantero A, Lorenzo-Hernández E, et al. Association of hypertension with all-cause mortality among hospitalised patients with COVID-19. J Clin Med. 2020;9(10):3136. https://doi.org/10.3390/jcm9103136.
Soleimani A, Kazemian S, Karbalai Saleh S, Aminorroaya A, Shajari Z, Hadadi A, et al. Effects of angiotensin receptor blockers (ARBs) on in-hospital outcomes of patients with hypertension and confirmed or clinically suspected COVID-19. Am J Hypertens. 2020;33(12):1102–11. https://doi.org/10.1093/ajh/hpaa149.
Chen R, Yang J, Gao X, Ding X, Yang Y, Shen Y, et al. Influence of blood pressure control and application of renin-angiotensin-aldosterone system inhibitors on the outcomes in COVID-19 patients with hypertension. J Clin Hypertens (Greenwich). 2020;22(11):1974–83. https://doi.org/10.1111/jch.14038.
Zhou F, Liu YM, Xie J, Li H, Lei F, Yang H, et al. Comparative impacts of ACE (angiotensin-converting enzyme) inhibitors versus angiotensin II receptor blockers on the risk of COVID-19 mortality. Hypertension. 2020;76(2):e15–7. https://doi.org/10.1161/HYPERTENSIONAHA.120.15622.
Meng J, Xiao G, Zhang J, He X, Ou M, Bi J, et al. Renin-angiotensin system inhibitors improve the clinical outcomes of COVID-19 patients with hypertension. Emerg Microbes Infect. 2020;9(1):757–60. https://doi.org/10.1080/22221751.2020.1746200.
Garvin MR, Alvarez C, Miller JI, Prates ET, Walker AM, Amos BK, et al. A mechanistic model and therapeutic interventions for COVID-19 involving a RAS-mediated bradykinin storm. Elife. 2020;9:e59177. https://doi.org/10.7554/eLife.59177.
van de Veerdonk F, Netea MG, van Deuren M, van der Meer JW, de Mast Q, Bruggemann RJ, et al. Kinins and cytokines in COVID-19: a comprehensive pathophysiological approach. Preprints. 2020:2020040023. https://doi.org/10.20944/preprints202004.0023.v1.
Acknowledgements
The authors would like to thank health professionals, patients, and administrative staff from hospitals participating in this study. *MED-ACE2-COVID19 Study Group: Hospital Universitario Príncipe de Asturias: F J de Abajo, A Rodríguez-Miguel, S Rodríguez-Martín, V Lerma, A García-Lledó, D Barreira-Hernández; D Rodríguez-Puyol; Hospital Universitario de Getafe: O Laosa, L Pedraza, L Rodríguez-Mañas; Hospital Universitario Ramón y Cajal: M Aguilar, I de Pablo, MA Gálvez; Hospital Central de la Defensa Gómez Ulla: A García-Luque, M Puerro; RM Aparicio, V García-Rosado, C Gutiérrez-Ortega; Hospital Clínico San Carlos: L Laredo, E González-Rojano, C Pérez, A Ascaso, C Elvira; Hospital Universitario de La Princesa: G Mejía-Abril, P Zubiaur, E Santos-Molina, E Pintos-Sánchez, M Navares-Gómez; F Abad-Santos; Hospital Universitario Puerta de Hierro-Majadahonda: G A Centeno, A Sancho-Lopez, C Payares-Herrera, E Diago-Sempere.
Funding
This study is part of a larger project which received funding from the Instituto de Salud Carlos III (#COV20/00027). Additionally, the University of Alcalá (#COVID-19 UAH 2019/00003/016/001/028) and the Biomedical Research Foundation from the University Hospital Príncipe de Asturias granted to FdA complementary grants for this project. The funding sources had no role in the study design, data collection, analysis and interpretation of data, writing of the report, and in the decision to submit the paper for publication.
Author information
Authors and Affiliations
Consortia
Contributions
FdA was involved in the conception and design of this study. All members of the MED-ACE2-COVID19 Study Group collected the data. SRM did the quality control. ARM, SRM, and FdA did the statistical analysis. All authors interpreted the data. FdA obtained the funding. SRM and FdA supervised the work. FdA and ARM drafted the manuscript. FdA, ARM, SRM, VL, and AGL critically revised the manuscript. All members of the MED-ACE2-COVID19 Study Group read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
The present study was an extension of a previous study approved by the Ethics Research Committee of the University Hospital “Príncipe de Asturias” on March 18, 2020 (#SRAA-COVID19), including a waiver for the informed consent [6]. The Ethics Research Committee was informed of this extension, and no additional ethical assessment was required. Data extracted were fully anonymised and no attempt was made to interview patients or relatives. The study complied with the provisions of the European and Spanish legislation: Order SAS/3470/2009, December 16 and Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Additional file 1.
Figures S1-S8 and Tables S1-S5.
Additional file 2.
Graphical abstract.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
de Abajo, F.J., Rodríguez-Miguel, A., Rodríguez-Martín, S. et al. Impact of in-hospital discontinuation with angiotensin receptor blockers or converting enzyme inhibitors on mortality of COVID-19 patients: a retrospective cohort study. BMC Med 19, 118 (2021). https://doi.org/10.1186/s12916-021-01992-9
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12916-021-01992-9