FormalPara Key Summary Points

Head-to-head comparisons of immunomodulators for the treatment of COVID-19 are lacking.

We aim to rank immunomodulators in the treatment of COVID-19 according to their efficacy and safety.

Regarding both efficacy and safety, ruxolitinib was the best treatment followed by baricitinib. Meanwhile, methylprednisolone had the worst combined efficacy and safety among the examined treatments.

Further well-conducted randomized controlled trials should focus on JAK inhibitors. Methylprednisolone use should be discouraged because of its poor efficacy and high risk of superimposed infection.

Introduction

Approximately one-fourth of cases of symptomatic coronavirus disease 2019 (COVID-19) progress to severe or critical illness in part because of immune dysregulation [1]. This dysregulation leads to enormous inflammatory responses including excessive production of cytokines, such as interleukin (IL)-1, IL-6, interferon (IFN)-γ, and tumor necrosis factor-α [2]. These consequences result in severe complications and poor clinical outcomes [3]. The key treatments during this stage are immunomodulatory agents, most of which are experimental at present.

Several immunomodulators are recommended according to current guidelines. First, corticosteroids inhibit the production of several cytokines [4]. Although corticosteroids provided no benefit in previous outbreaks of novel coronaviruses (i.e., Middle East respiratory syndrome, severe acute respiratory syndrome) [5], these drugs were linked to lower mortality rates among hospitalized patients with COVID-19 who required supplementary oxygen or mechanical ventilation [6]. However, corticosteroids differ by formulation and treatment duration. Although the RECOVERY trial provided solid evidence of the benefit of dexamethasone, few clinical studies have directly compared different corticosteroids in the treatment of COVID-19. One problem in clinical practice is that physicians tend to increase the corticosteroid dosage or switch to a formulation with greater activity when the treatment response is poor. In addition, safety concerns, particularly the risk of superimposed infection, must be considered for different corticosteroid formulations.

Second, IL-6 inhibitors (anti-IL-6 receptor monoclonal antibodies, e.g., sarilumab, tocilizumab; and anti-IL-6 monoclonal antibodies, e.g., siltuximab) have been used to treat COVID-19 because excessive IL-6 levels caused by the hyperinflammatory response are associated with high disease severity [7]. In addition, elevation of IL-1 levels in COVID-19 has promoted the use of anakinra [8]. At present, tocilizumab tends to reduce mortality rates among patients with severe or critical disease, whereas there is limited evidence supporting for the clinical use of other IL-1 or IL-6 inhibitors [9, 10].

Next, the imbalance between the Janus kinase (JAK) and signal transducer and activator of transcription (STAT) pathways found in severe COVID-19 is linked to immune dysregulation [11]. Baricitinib is a JAK inhibitor that regulates cytokine levels to prevent disease progression [12]. However, additional data from ongoing studies of JAK inhibitors are awaited.

Lastly, despite the in vitro and in vivo antiviral properties of IFN [13], its use in clinical studies has been limited. Consequently, we excluded studies of IFN because its significant toxicities outweigh its benefit. The current guideline also recommends against the use of IFN [10].

To our knowledge, previous systematic reviews regarding immunomodulatory therapy for COVID-19 did not perform meta-analysis [14], or direct meta-analysis was applied for specific groups of immunomodulators [8, 15,16,17,18,19,20,21]. Moreover, in the absence of head-to-head comparisons across or within groups of immunomodulators, uncertainty regarding their relative efficacy and safety remains.

Therefore, we conducted a systematic review and network meta-analysis to estimate the comparative efficacy (i.e., mortality rate, incidence of invasive mechanical ventilation [IMV]) and adverse events (superimposed infection) of currently reported immunomodulators in the treatment of COVID-19 using peer-reviewed articles. Namely, we aimed to synthesize all possible pairwise comparisons across multiple treatments between or within groups of immunomodulators via direct comparisons with common comparators. The efficacy and safety of the treatments were estimated and ranked accordingly.

Methods

The systematic review and network meta-analysis protocol was developed following the guidelines in the PRISMA extension of network meta-analysis. Intention-to-treat analysis was performed for all quantitative syntheses. The review protocol was registered with PROSPERO (CRD 42021257421). In our network meta-analysis, no ethical approval or informed consent was needed as this article is based on previously published article and does not contain any new studies with human participants or animals performed by any of the authors.

Search Strategies

We identified potential studies from MEDLINE via PubMed, SCOPUS, and clinical trial registries as well as the reference lists of selected studies published up to June 2021. Two investigators (T.N., R.K.) developed search strategies that were accepted by the team. The search terms and strategies for each database were generated on the basis of the targeted population, types of immunomodulator, and outcome. The full search strategies are available in the supplementary material.

Selection of Studies

The identified studies were independently selected according to the title and abstract by two independent reviewers (T.N. and R.K.). Disagreement was resolved by discussion with a third reviewer (C.S.). Titles and abstracts were screened, and the full text was reviewed when a decision could not be made after reading the abstracts. Study selection and agreement measurement were performed using the Covidence® program.

Inclusion Criteria

We included randomized controlled trials (RCTs) in any language when they met the following criteria:

  1. 1.

    Participants were adults aged ≥ 18 years who were hospitalized with COVID-19 infection and hyperinflammatory responses.

  2. 2.

    The studies compared any pair of the follow interventions: corticosteroids, IL-6 inhibitors, IL-1 inhibitors, kinase inhibitors, immunomodulators, and any placebo or standard of care (SOC).

  3. 3.

    The study outcome was the mortality rate, incidence of IMV, or risk of superimposed infection.

  4. 4.

    The full text could be retrieved, and data were available for extraction.

  5. 5.

    The study is currently published in a peer-reviewed journal or clinical trial registry.

Studies were excluded if IFN was used as the immunomodulator because of its high risk of toxicities. Ongoing studies or study protocols from which we could not extract the results were also excluded.

Data Extraction

At least two of the three reviewers (T.N., R.K., and T.S.) independently extracted data using a standardized extraction form. The extracted data included the general characteristics of the articles, population, intervention, and outcome of interest for pooling. Discrepancies were resolved by discussion or by a third reviewer (C.S.). Missing data were obtained by contacting the corresponding authors.

Outcome of Interest

The primary outcomes were mortality rate and incidence of IMV. The adverse event was superimposed infection. Each outcome referred to the definition provided in the original studies. From each study, we extracted the severity of disease and types of immunomodulators that might be effect modifiers. The severity of diseases was categorized as mild, moderate, severe, or critically ill. Immunomodulators were classified by group as follows: IL antagonists, anakinra (ANA), sarilumab (SAR), and tocilizumab (TOC); corticosteroids, dexamethasone (DEX), hydrocortisone (HYD), and methylprednisolone (MET); and JAK inhibitors, baricitinib (BAR), ruxolitinib (RUX), and tofacitinib (TOF).

Transitivity Assessment Across Treatment Comparisons

The common treatment used to compare different immunomodulators indirectly is similar when it appears in different trials. All patients who exhibited excessive inflammatory responses required immunomodulators to control dysregulation. In addition, pairwise comparisons did not differ regarding the effect of severity and types of immunomodulators in subgroup analysis. Therefore, we anticipated that the transitivity assumption was valid.

Quality Assessment

At least two of the three reviewers (T.N., R.K., and V.L.) independently evaluated the risk of bias of each study using the Cochrane Risk of Bias 2.0 tool for RCTs. The Risk of Bias 2.0 tool evaluates five domains of bias: randomization process, deviations from intended interventions, missing outcome data, measurement of the outcome, and selection of the reported results. The risk of bias for each of the five domains and the overall risk of bias were described as low, intermediate, or high. Disagreement between two authors was resolved by consensus and discussion. Cohen’s kappa was used to determine the agreement for each domain and the overall risk of bias.

Statistical Analysis

We reported our systematic review according to the 2009 PRISMA guidelines. Direct meta-analysis of comparisons between immunomodulators and placebo or SOC was performed by pooling risk ratios (RRs) if there were at least three studies. The Q test and I2 statistic were used to assess heterogeneity. The DerSimonian and Laird method was used to pool RRs. Subgroup analysis was subsequently performed to assess the source(s) of heterogeneity. We assessed publication bias using funnel plots and the Egger test.

For indirect comparisons, network meta-analysis was applied to determine the treatment effect for all possible treatment groups. A two-stage network meta-analysis was applied to estimate the relative effects (RRs or risk difference [RD]) of all treatments using information from comparisons with common comparators to increase the power of the test by integrating direct and indirect comparisons. In the comparison, ANA, SAR, TOC, DEX, HYD, MET, BAR, RUX, TOF, and SOC were coded as 1, 2, 3, 4, 5, 6, 7, 8, 9, and 0, respectively. Regarding transitivity, we performed subgroup analysis to explore heterogeneity on the basis of the type of immunomodulator and baseline severity. Inconsistency was calculated as the difference between indirect and direct estimates for a common comparator. The surface under the cumulative ranking curve (SUCRA) was used to rank the regimens in terms of efficacy and safety. A clustered stacked bar was used to present the benefits of the treatments regarding the mortality rate, incidence of IMV, and superimposed infection risk according to the SUCRA. Publication bias was presented using a comparison-adjusted funnel plot.

All analyses were performed using STATA® version 17.0 (StataCorp LP, College Station, TX, USA). A two-sided p value of less than 0.05 was considered statistically significant, except for the heterogeneity test, in which a p value of less than 0.10 was used.

Results

Study Characteristics and Included Studies

The electronic search identified 10,158 potentially relevant studies, of which 7692 potentially eligible articles were analyzed. We excluded 7664 reports that did not meet eligibility criteria (Fig. 1). We identified six additional eligible trials from the Cochran library, registry database, and Clinicaltrials.gov. Agreement on study selection between the two reviewers was high (κ = 0.95).

Fig. 1
figure 1

PRISMA flow diagram of screening studies. ANA anakinra, SAR sarilumab, TOC tocilizumab, DEX dexamethasone, HYD hydrocortisone, MET methylprednisolone, BAR baricitinib, RUX ruxolitinib, TOF tofacitinib, SOC standard of care

Full size image

Overall, 28 trials were identified for the analysis, and two studies were excluded because of insufficient data for extraction. Therefore, 26 studies were included in the quantitative analysis. Ten treatments were analyzed: ANA, SAR, TOC, DEX, HYD, MET, BAR, RUX, TOF, and SOC. Most of these trials were conducted in multinational settings, and they recruited hospitalized patients with moderate-to-severe COVID-19 and compared immunomodulators with SOC (Table 1). Only one study compared two different treatments with SOC [22], and another study compared outcomes between two immunomodulators [23]. Corticosteroids were used as the SOC in some trials examining other treatments. The overall statistical heterogeneity was low according to the baseline severity (I2 = 0.0–10.0%; Figs. S1–3 in the supplementary material). We assessed the included studies according to outcome as follows: 26 studies reported mortality rates, 19 studies reported the incidence of IMV, and 21 studies examined the rates of superimposed infection.

Table 1 Characteristics of the included studiesa (n = 26)
Full size table

Quality Assessment Across Included RCTs

The overall quality of the studies was rated as intermediate. Nine (34.6%) of the studies showed good quality which was noted for the randomization process and missing outcome data domains (88.5%). The lowest quality was noted for the deviations from intended interventions domain (50.0%). The study by Jeronimo [24] had a high risk of bias owing to missing outcome data (see Fig. S4 for details).

Direct Meta-analysis

The mortality rate among hospitalized patients was reduced by approximately 10.0% by treatment with corticosteroids (pooled RR 0.90; 95% CI 0.83–0.97; p < 0.01) compared to SOC (Figs. S5–6). Corticosteroids decreased the incidence of IMV versus placebo, albeit without statistical significance (Fig. S8–9). MET non-statistically decreased mortality rates but tended to increase the rates of IMV and superimposed infection compared with the effects of SOC (Figs. S9, S12). Although IL antagonists did not greatly reduce mortality rates, patients who received these immunomodulators had a significantly lower incidence of IMV versus SOC (pooled RR 0.79; 95% CI 0.70–0.89; p < 0.01; Fig. S17). The risk of superimposed infection was not significantly reduced by IL antagonists. A nearly 40% reduction in the mortality rate was observed among patients treated with JAK inhibitors (pooled RR 0.61; 95% CI 0.38–0.95; Fig. S23). Patients treated with JAK inhibitors had a lower risk of superimposed infection than those treated with SOC without statistical significance (Fig. S25). There was no publication bias as assessed by the funnel plot and Egger’s test for all outcomes.

Network Meta-analysis

Figure 2 presents the network of eligible comparisons for mortality rate in the multiple-treatment meta-analysis (the networks for the incidence of IMV and superimposed infection are presented in Figs. S27B and S27C, respectively).

Fig. 2
figure 2

Network meta-analysis of eligible comparisons for efficacy (mortality rate). A network meta-analysis of eligible comparisons for the mortality rate following immunomodulator therapy for coronavirus disease 2019 was performed. The figure plots the network of direct comparisons (black bold lines) and indirect comparisons (dashed line). The width of the lines is proportional to the number of trials comparing every pair of treatments. The size of each circle is proportional to the number of randomly assigned participants (sample size). The networks of eligible comparisons for the incidence of invasive mechanical ventilation and superimposed infection are presented in Figs. S27B, C

Full size image

Mortality Rate and Incidence of IMV

Mortality rate data from 26 studies (N = 16,733) consisting of 11 direct comparisons among 10 treatments were pooled. Data from 18 studies (N = 15,130) using direct comparisons among nine treatments were pooled for the incidence of IMV. Overall, immunomodulators displayed better efficacy than SOC. Namely, DEX and TOC were linked to significantly lower mortality rates than SOC with pooled RRs of 0.91 (95% CI 0.84–0.99) and 0.88 (95% CI 0.82–0.96), respectively. Patients who received SAR, BAR, or TOC exhibited a lower incidence of IMV than those treated with SOC with pooled RRs of 0.38 (95% CI 0.18–0.79), 0.68 (95% CI 0.46–0.93), and 0.78 (95% CI 0.70–0.87), respectively (Table 2). The relative treatment efficacy among corticosteroids demonstrated that HYD most strongly reduced the mortality rate reduction and incidence of IMV. MET tended to increase the mortality rate and incidence of IMV compared with the findings for HYD (Table S5). SAR was the most effective IL antagonist in terms of mortality and the incidence of IMV reduction. ANA had relatively worse efficacy than other the IL antagonists (Table S6). Among JAK inhibitors, RUX had relatively better efficacy than the other treatments (Table S7). The highest probability of efficacy regarding the mortality rate as indicated by the highest SUCRA was identified for RUX, followed by TOF and BAR, whereas SAR had the greatest efficacy in terms of the incidence of IMV, followed by RUX and BAR (Table S8). The cumulative probability curves are presented in Figs. S28–S30. The curves for efficacy and superimposed infection were consistent (see Figs. S31–S34 for details).

Table 2 Head-to-head comparison of the efficacy of immunomodulators
Full size table

Efficacy Regarding Superimposed Infection

Data from 21 studies (N = 15,502) reporting 9 direct comparisons of 10 treatments were pooled to assess the risk of superimposed infection. The overall probability of efficacy and risk of superimposed infection was presented as a clustered stacked bar chart. SUCRAs were scored up to a maximum of 50 points for efficacy and 50 points for low risk of superimposed infection, and the cumulative percentages after normalization are displayed. In terms of efficacy and the low risk of superimposed infection, RUX was identified as the optimal treatment, followed by BAR. Meanwhile, MET was ranked last in this analysis (Fig. 3a, b).

Fig. 3
figure 3

Drugs ordered by probability of being the best treatment in terms of combined effects on the risks of mortality and IMV (a) and the risks of IMV and superimposed infection (b), revealing the separate contribution of effects on each variable to the total score. The cumulative percentages after normalization (0–100) are presented in the figure. Each drug was given a maximum score of 50 for efficacy and 50 for the low risk of superimposed infection (maximum score 100) using the surface under the cumulative ranking curve. IMV invasive mechanical ventilation

Full size image

Publication Bias of the Network

Comparison-adjusted funnel plots were applied for all outcomes. There was no evidence of publication bias based on the symmetry of the funnel plot (Figs. S35–S37).

Discussion

We found in our systematic review that immunomodulators played a major role in the treatment of inflammatory responses associated with COVID-19. In comparison to SOC, immunomodulators reduced the mortality rate and incidence of IMV in RCTs of patients with mostly moderate-to-severe COVID-19. This finding was consistent with the results of several published systematic reviews and meta-analyses [15,16,17,18,19,20,21, 25].

Previous studies reported the efficacy of corticosteroids in the treatment of COVID-19. These drugs have emerged as the SOC for severe or critical COVID-19 on the basis of the results of the RECOVERY trial. Studies of IL antagonists or JAK inhibitors used corticosteroids as the SOC in some participants in the controlled arm. We were able to explore this effect using the results of treatment ranking based on the SUCRA score. Therefore, the ranking of ANA and TOF regarding the incidence of IMV should be interpreted with caution.

HYD and DEX tended to reduce the risks of mortality and IMV. However, we recorded higher rates of IMV and superimposed infection in patients with COVID-19 who received MET. We did not observe this finding in a preceding meta-analysis [19, 21]. This discrepancy is attributable to the influence of additional studies of MET in our analysis. Nevertheless, when we weigh the benefits and risks of MET, we strongly discourage its use regardless of the dose or regimen in clinical practice and further clinical studies. Although HYD was more effective than DEX in our network meta-analysis, a large-scale RCT following the protocol of the RECOVERY trial should be performed. We anticipate that the timing and dose of treatment and severity of disease strongly contributed to the heterogeneity regarding the benefit of corticosteroids.

We observed little benefit of IL antagonists, contradicting the results of prior meta-analyses [15, 20] because we included a large-scale phase 3 RCT of SAR that reported a negative result [26]. IL antagonists significantly decreased the mortality rate and the incidence of IMV, primarily based on the effect of TOC, as reported previously [15, 16, 18]. Our network meta-analysis revealed that IL-6 antagonists (TOC or SAR) were superior to IL-1 antagonists in terms of mortality and IMV risk. We anticipate that IL-6 plays a greater role than IL-1 in the hyperinflammatory phase. In addition, IL-6 levels predict the possibility of IMV [27]. Whereas Kyriazopoulou et al. found that IL-1 antagonists provided a mortality benefit in a systematic review and meta-analysis, most included studies were observational studies [8]. Compared with the effects of corticosteroids, IL-6 antagonists had varied effects on the risks of mortality and IMV; however, most clinical studies of IL-6 antagonists used DEX as the SOC. This is because in addition to IL-6-mediated pathways, the observed dysfunction is attributable to a complex network of pathways [14, 28]. These findings were supported by the results of the RECOVERY trial, which reported a mortality benefit among corticosteroid subgroups [29].

The efficacy of JAK inhibitors in our study was notable, particularly in terms of mortality. JAK inhibitors mitigate STAT3 hyperactivity, thereby improving immune dysregulation in severe COVID-19 [11]. In terms of the incidence of IMV, JAK inhibitors were inferior to other immunomodulators. Our findings were consistent with those of a previous meta-analysis [25]. Among JAK inhibitors, RUX was more beneficial than BAR concerning both mortality and IMV, but large-scale clinical trials are needed, as well as data for TOF. Considering the risk of superimposed infection, JAK inhibitors were superior to other treatments in all aspects. The risk–benefit ratio should be balanced for all immunomodulators. We expect that the half-life and target of action of immunomodulators determine the risk of infection. A single dose of an IL-6 antagonist can significantly suppress temperature or body responses to infection for a week [30]. These effects may mask normal clinical signs, thereby delaying diagnosis. Corticosteroids can inhibit multiple inflammatory pathways, resulting in a poor immune response to infection. Superimposed infection had been reported as both a nosocomial infection and secondary infection such as invasive fungal infection, disseminated strongyloidiasis [31], and reactivation of latent infection including hepatitis B [32] or tuberculosis [33]. In clinical practice, some non-infectious complications must be considered, such as hyperglycemia, gastrointestinal bleeding, and bowel perforation [34].

Our study had several strengths. First, we performed an indirect comparison among current immunomodulators. Second, we only evaluated RCTs that were published in peer-reviewed journals to enhance the validity of the included studies. Lastly, we considered the risk–benefit ratios of immunomodulators using a numerical ranking and represented the findings using a comprehensive clustered stacked bar. However, some study limitations must be addressed. First, the severity criteria were heterogeneous across the studies. However, most included studies included patients with moderate-to-severe COVID-19 and inflammatory reactions that required immunomodulators. In addition, we assessed a heterogeneity stratified by severity, the minimal heterogeneity was observed (Figs. S1–S3). Second, although we did not explore the dominant COVID-19 variants at the time of each study, the effects of variants on the treatment response to immunomodulators were inconclusive. Third, countries or regions might be confounding factors. However, as included studies were multinational and mixed populations were recruited, these will minimize the threat to the validity of this study. Finally, heterogeneity concerning the dosage, timing, and duration of immunomodulator therapy should be further explored, particularly in large-scale clinical trials.

Conclusions

Immunomodulators were generally more effective than SOC in terms of efficacy and adverse events. Important differences exist among immunomodulators concerning both efficacy and safety in favor of RUX and BAR. Further well-conducted RCTs should focus on JAK inhibitors. MET use should be discouraged because of its poor efficacy and high risk of superimposed infection.