Background

Network meta-analysis (NMA) has become increasingly popular over recent years for estimating the relative effectiveness of several treatments in the absence of direct head-to-head evidence. When direct and indirect evidence is combined in a meta-analysis, there is a risk that patients in different trials differ in terms of demographics, disease, or other patient characteristics. There can also be differences in trial specific features, such as country of origin and trial design. If these differences are effect modifiers, they can result in between-study heterogeneity and create biased comparisons. In a NMA context, such biases and heterogeneity can also lead to inconsistency, i.e. conflict between direct and indirect evidence on the same comparison. It is therefore important to adjust for effect modifiers in a NMA; this can be done by restricting inclusion in the NMA to certain subgroups of patients with similar characteristics or by conducting meta-regression. Focusing the inclusion criteria on key participant or study characteristics to produce smaller, more homogenous networks can reduce the risk of both heterogeneity and inconsistency, and give more valid results [1]. Alternatively, meta-regression, on for example the average weight or proportion of included patients with certain characteristics, can also be conducted. When conducting network meta-regression, a sufficient number of studies is needed to estimate independent coefficients for each treatment comparison. Otherwise, additional assumptions of common regression coefficients must be made, which may not be clinically plausible. In addition, results are often uncertain and hard to interpret. Therefore, it is often more useful to identify clinically meaningful discrete participant and study characteristics which could be expected to lead to different decisions, and restrict inclusion in the NMA.

Previous work carried out for the National Institute for Health and Care Excellence (NICE) has highlighted that several NMAs undertaken for NICE single technology appraisals (STAs) of psoriasis treatments have included heterogeneous studies. However, the very short timeframe of a STA does not allow sufficient time to fully explore the impact of heterogeneity on the NMA results [2]. Therefore, this small methodological project aimed to explore the impact of heterogeneous patient characteristics on the results of a NMA, using data from NICE STAs of psoriasis treatments, since we identified this as an area where previous NMAs have included studies with heterogeneous patient characteristics.

There have been several NICE STAs of systemic therapies for the second-line treatment of moderate-to-severe plaque psoriasis. Psoriasis is a chronic, inflammatory immune-mediated skin disorder with a prevalence of around 3% in the UK [3]. Standard first-line treatment includes topical therapy, or systemic non-biologic therapies or phototherapy for patients with more severe disease. For adults with moderate-to-severe psoriasis who do not respond to, are intolerant of, or have a contraindication to standard systemic therapies and phototherapy, NICE recommends systemic biologic therapies, apremilast or dimethyl fumarate.

The severity of psoriasis is measured using the Psoriasis Area and Severity Index (PASI), which combines the assessment of severity of lesions and the area affected into a single score. PASI is also used to assess response to psoriasis treatment, presented as a percentage response rate; PASI 75 response is a 75% or greater improvement in PASI score, PASI 90 response is a 90% or greater improvement and PASI 100 response is 100% improvement in PASI score (total skin clearance).

The key objectives of this methodological project were:

  1. 1.

    To identify NMAs undertaken as part of a STA of a second-line therapy for moderate -to -severe plaque psoriasis.

  2. 2.

    To identify and tabulate all relevant studies included in the NMAs, recording patient and study characteristics that may influence relative treatment effects (PASI response).

  3. 3.

    To map a range of smaller, less heterogeneous networks.

  4. 4.

    To run the NMAs and compare results with the results of the overall network of evidence.

Methods

Two researchers (RW and SS) independently screened the NICE website for STAs of second-line therapies for moderate -to- severe plaque psoriasis that included a NMA. The researchers also identified any sensitivity analyses undertaken by the company who undertook the NMA, as an indication of the characteristics that may be considered to have an impact on relative treatment effectiveness.

All studies included in the NMAs were tabulated. Additional randomised controlled trials (RCTs) of second-line therapies for psoriasis were not sought since the search strategies used in the STAs were adequate and the aim of this methodological project was to compare results of NMA subgroups with the original network, rather than to update the previous NMAs. Details of important patient and study characteristics that may influence relative treatment effects were tabulated, such as timeframe at which treatment response was assessed, drug dose, concomitant psoriatic arthritis and prior treatments received (i.e. biologic naïve versus biologic experienced patients). Dermatologists who had acted as clinical advisors to the Centre for Reviews and Dissemination/Centre for Health Economics Technology Assessment Group in previous STAs of second-line therapies for psoriasis were emailed regarding their opinion on the characteristics considered most likely to have an impact on the relative effectiveness of psoriasis treatments on PASI response. The outcome used in the analysis was PASI 75 response, as it is the most widely reported response outcome in the included trials and is used as a measure of treatment response in clinical practice.

Study details were obtained from tables presented as part of the STA of brodalumab [4], supplemented with data presented in primary study reports, where necessary. The brodalumab appraisal was chosen as the primary source of data because it included comprehensive study characteristics tables. The tables were independently checked for accuracy and completeness by a second researcher using tables from two different STAs, supplemented with data presented in primary study reports. All missing data/discrepancies were added/corrected using the original study reports.

Study and patient characteristics considered most likely to have an impact on relative treatment effectiveness were compared for each of the primary studies. New networks, including only studies with similar study and patient characteristics, were defined and mapped using the netmeta package [5] in R [6]. This package uses contrast-level data to create plots of all the trials included in the NMA, highlighting the number of trials between each treatment. All networks were checked for connectivity, making sure that all interventions were directly connected to at least one other intervention, forming one linked network.

Binomial logit-link models were used for the NMAs [2]. Both fixed effect and random effects models were fitted for each network. The choice of prior distributions for the between-study variance was also explored. Model fit was assessed by comparing the total residual deviance to the number of data points in the model. Models were compared using the deviance information criterion (DIC) which accounts for model fit and complexity. The model with a lower DIC (a difference in value of 3 is seen as meaningful) was selected. Where the DIC were within 3 points of each other, the simplest model with fewer parameters was chosen.

Results

Review of NICE technology appraisals

There have been ten NICE STAs of systemic therapies for the second-line treatment of moderate-to-severe plaque psoriasis. The second-line systemic therapies that have been appraised are the anti-tumour necrosis factor (TNF) alpha therapies adalimumab, infliximab and certolizumab pegol; the anti-interleukin (IL)-12/23 ustekinumab; the anti-IL-17 therapies secukinumab, ixekizumab and brodalumab; the anti-IL-23 tildrakizumab; the anti-phosphodiesterase (PDE) 4 apremilast; and the nuclear factor (erythroid-derived 2)-like 2 (Nrf2) activator dimethyl fumarate. Other than infliximab, which is only recommended for patients with very severe disease, each of the company submissions included a NMA (see Table 1).

Table 1 NICE single technology appraisals of systemic therapies for psoriasis that include network meta-analyses
Full size table

Patient characteristics that may contribute to heterogeneity in relative treatment effects

Sensitivity analyses undertaken alongside the STA NMAs related to the following study/patient characteristics: size of the trial; licensed and NICE approved treatment doses; timing of primary outcome assessment; patients’ baseline PASI score; patients’ baseline DLQI score; duration of disease; and prior exposure to biologic therapy. Two dermatologists (Professor Catherine Smith and Dr Phil Hampton) provided advice on the study and patient characteristics considered most likely to have an impact on the relative effectiveness of psoriasis treatments on PASI response. Important characteristics for which adequate data were available in the studies of psoriasis treatments were patient weight, exposure to previous biologic therapy, white versus non-white ethnicity and baseline PASI score.

Network identification

We identified 72 studies from previous NMAs of STAs of second-line therapies for moderate to severe plaque psoriasis. We excluded any studies with unlicensed treatments or treatment doses, of which there were two. One study was excluded due to the results being unpublished. Therefore, we included 69 studies in our synthesis (34,924 participants). Characteristics of patients included in the 69 RCTs included in the networks are presented in Additional file 1.

The impact of four patient characteristics on relative treatment effectiveness was investigated by producing four smaller networks: ‘no previous biologic use’ (< 25% patients had prior exposure to a biologic therapy), ‘PASI ≤ 25’ (average PASI score was 25 or less), ‘weight ≤ 90 kg’ (average weight was 90 kg or less) and ‘white ethnicity’ (≥ 90% patients were white). Cut-off choice was informed by clinical opinion as well as being pragmatically chosen in order to ensure a sufficient number of studies was still included in each network. The studies included in each of the four networks and the original (all licensed doses) network are listed in Table 2. The network diagrams are shown in Figs. 1, 2, 3, 4 and 5. The width of the connecting lines is proportional to the number of trial level comparisons available and the size of the nodes is proportional to the number of patients who received the corresponding treatment.

Table 2 Studies included in each network meta-analysis
Full size table
Fig. 1
figure 1

Network of all studies with licensed treatment doses

Full size image
Fig. 2
figure 2

Network of studies of patients with no previous biologic exposure (< 25%)

Full size image
Fig. 3
figure 3

Network of studies of patients with PASI < 25%

Full size image
Fig. 4
figure 4

Network of studies of patients with weight ≤ 90 kg

Full size image
Fig. 5
figure 5

Network of studies including > 90% white patients

Full size image

Model fit

In all models, both a uniform (0, 3) prior distribution and an empirically based log normal (− 2.70, 1.522) informative prior distribution [15] were used. The random effects model with a uniform prior distribution was found to have a superior fit for the network of all studies with licensed doses (Table 3) as the residual deviance was closer to the number of unconstrained data points than the fixed effects model and the random effects model with log-normal prior distribution. The deviance information criterion (DIC) was also lower for the uniform prior random effects model than the other two models.

Table 3 Measures of goodness of fit of fixed and random effects models for each of the five network meta-analyses
Full size table

The random effects model with a log-normal prior distribution was chosen for the network of patients with no previous biologic use (< 25% patients had previous biologic use), the network of patients with PASI score ≤ 25, the network of patients with weight ≤ 90 kg and the network of ≥ 90% white patients (Table 3). The DIC and residual deviance was much lower for the random effects models than the fixed effects models. Although the DIC was very similar between the random effects models, the log-normal prior model was chosen as it had a much smaller number of parameters (pD) than the uniform prior model.

Heterogeneity

The network of all studies with licensed doses had the highest between-study heterogeneity (0.31, 95% CrI 0.17–0.45). The between-study heterogeneity was reduced for the four smaller networks, which all had similar values. However, the network of patients with no previous biologic use had the smallest heterogeneity (0.14, 95% CrI 0.09–0.23), alongside the network of patients with weight ≤ 90 kg (0.15, 95% CrI 0.09–0.24). The densities of the posterior between-study heterogeneity for each network meta-analysis are shown in Fig. 6.

Fig. 6
figure 6

Posterior between-study heterogeneity density for the five NMAs

Full size image

Effects of the interventions

Relative risk ratios for each treatment compared against placebo are shown in Table 4. Across the five NMAs, the relative risks for each treatment appear to be similar, with all credible intervals overlapping. However, there are some noticeable differences. Etanercept 50 mg (once-weekly) had a higher relative treatment effect of achieving PASI 75 in the licensed doses network (10.67, 95% CrI 7.96–13.53) compared to all other networks and methotrexate had a higher relative effect in the network of patients with no previous biologic use (< 25% had previous use) (10.47, 95% CrI 6.73–14.41) compared to the other networks. In the ≥ 90% white patients network, secukinumab had a higher relative treatment effect than in all other networks (18.67, 95% CrI 16.22–20.81) and guselkumab had a lower relative treatment effect compared to all the other networks (15.30, 95% CrI 10.89-18.39). However, their credible intervals were large.

Table 4 Median risk ratio for each treatment compared against placebo in all five network meta-analyses
Full size table

Log-odds ratios for each network and for each treatment compared to placebo are shown in Fig. 7. Absolute probabilities of achieving PASI 75 for each treatment across the five networks are shown in Additional file 2.

Fig. 7
figure 7

Relative treatment effects split by network group for each treatment

Full size image

The median rankings of treatments based on the relative risks are shown in Table 5. Ixekizumab ranks best in all networks, except the network with predominantly white patients, in which secukinumab ranks best. Dimethyl fumarate ranks worst in all five networks. The rankings are generally consistent across the networks. However, the NMA that included only studies in which less than 25% of patients had prior exposure to a biologic therapy had slightly different treatment rankings; the anti-TNF therapies certolizumab pegol (median rank of 8 [95% CrI 2–13] for the 200 mg dose and 6 [95% CrI 1–11] for the 400 mg dose) and infliximab (median rank of 3 [95% CrI 3–11]) ranked higher in this network group than any of the other networks, indicating that these two therapies may work better in patients who have not previously received biologic therapy, although we note the large uncertainty in these rankings. However, biologic experienced patients are more likely to have had prior exposure to an anti-TNF therapy (i.e. adalimumab or etanercept) which may explain why subsequent response to the anti-TNF therapies certolizumab pegol and infliximab was lower in the networks that did not include primarily biologic-naïve patients.

Table 5 Median rank of treatments according to PASI 75 response in each of the five networks
Full size table

The network of primarily white patients also had slightly different treatment rankings; secukinumab ranked higher and guselkumab ranked lower than in the other networks, although there was large uncertainty for the guselkumab result. Data on ethnicity was often not reported in the included studies, so some assumptions had to be made based on the location of the study when extracting data from primary studies, adding further uncertainty to the results for this network.

Sensitivity analysis

Some studies of the earlier treatments for psoriasis, adalimumab, etanercept and infliximab, did not report prior biologic use; however, they may have had largely biologic-naïve patient populations as biologics were not widely available at the time they were conducted. Therefore, all the studies not already included in the network of patients who had no prior biologic exposure (< 25% patients) were screened and studies conducted prior to 2007, where prior biologic use was not reported, were added to the network. The cut-off of 2007 was chosen to ensure that all the earliest studies were included. Six studies conducted prior to 2007 were identified and included in the network: Gottlieb et al. (2003) [16], Leonardi et al. (2003) [17], Papp et al. (2005) [18], Reich et al. (2005) [19], Gordon et al. (2006) [20] and Tyring et al. (2006) [21]. The random effects model with a log-normal prior distribution was chosen for the network of patients with no previous biologic use (< 25% patients had previous biologic use) (see Additional file 3, Table 1).

The results from the sensitivity analysis were very similar to the main results (see Additional file 3, Table 2). There were minimal changes to the risk ratios, with very little difference in the anti-TNF drugs adalimumab, infliximab and etanercept. There were a few small changes to other treatments. The median ranking of guselkumab changed from 3 to 4, with the same credible interval of 1–7. The median ranking of apremilast and DMF dropped one rank each, with the addition of etanercept 25 mg to the network, making the total number of treatments 17, rather than 16.

Discussion

The smaller networks investigated were less heterogeneous, with between-study standard deviation ranging from 0.14 (95% CrI 0.09–0.23) for the network of patients with no previous biologic use to 0.17 (95% CrI 0.10–0.25) for the network of predominantly white patients, in comparison with the network of all studies with licensed doses (0.31, 95% CrI 0.17–0.45). The reduction in heterogeneity in the network of patients with no previous biologic use could be due to the population being more clinically homogenous. Previous biologic use may be an important effect modifier and so excluding patients with previous biologic use may have removed a significant source of heterogeneity.

Results for most of the NMAs were consistent, in terms of treatment rankings for PASI 75 response. The main exception was the NMA of studies in which ≤ 25% patients had prior exposure to a biologic therapy; in this network, results were better for the anti-TNF therapies certolizumab pegol and infliximab than in the other networks. Whilst this could simply reflect the fact that studies in which a higher proportion of patients had prior exposure to a biologic therapy had used an anti-TNF as the prior therapy (i.e. adalimumab or etanercept), this may be an important effect modifier. Prior biologic therapy exposure was the most commonly conducted sensitivity analysis amongst the NICE STAs of systemic therapies for psoriasis that included a NMA (see Table 1) and our results confirm the importance of considering this as a potential effect modifier.

Meta-regression is another method commonly used to adjust for effect modifiers. However, this requires a sufficient number of studies in order to estimate independent coefficients for each treatment comparison. Additional file 4 presents the number of studies that reported each continuous covariate for each treatment comparison. This shows that there are not enough studies between comparisons to estimate independent coefficients and a common regression coefficient would need to be assumed, which may not be clinically credible. Therefore, analyses were simplified by dichotomising variables according to clinically relevant cut-offs and creating separate networks. Previous work has investigated the effect of baseline risk using meta-regression [22]. Baseline risk is often a proxy for multiple observed and unobserved effect modifiers and does not describe specific individual patient-related treatment effect modifiers. Adjusting for baseline risk in this analysis may not be clinically meaningful for decision making since it is uncertain what determines the baseline risk. Our aim was to characterise heterogeneity based on known and previously hypothesised study-level characteristics that translate to individual patient characteristics, which can be used to focus decision-making on more specific, homogeneous populations.

A limitation of our analysis is the variation in time point at which PASI 75 was assessed in the included studies. In most included studies, the time point for the primary efficacy assessment was week 12, although in some studies it was week 16; adalimumab, apremilast, certolizumab pegol, tildrakizumab and ustekinumab were assessed at week 12 in some studies and week 16 in others. The primary efficacy assessment was week 10 in placebo-controlled trials of infliximab, reflecting the shorter time to treatment effect for this therapy.

Our findings could be investigated further using individual patient data meta-analysis accounting for different important covariates. However, this preliminary approach has highlighted potential differences in treatment response for patients with prior exposure to biologic therapy. Where individual patient data are available, a better characterisation of patients’ prior biologic use could be used to further explore the differences identified.

Comparison with other results

Treatment rankings for the ‘licensed doses’ NMA were broadly consistent with the results of the NMA undertaken by the guideline development group for the BAD guidelines for biologic therapy for psoriasis, published in April 2017 [23]. The BAD NMA compared ixekizumab, secukinumab, infliximab, ustekinumab, adalimumab, etanercept, methotrexate and placebo. Interventions were ranked in order of efficacy using the surface under the cumulative ranking (SUCRA) curve method. For the outcome, PASI 75 at 3–4 months ixekizumab ranked best (SUCRA 96.4, mean rank 1.3), followed by infliximab (SUCRA 81.2, mean rank 2.3), secukinumab (SUCRA 79.0, mean rank 2.5), ustekinumab (SUCRA 51.9, mean rank 4.4), adalimumab (SUCRA 48.7, mean rank 4.6), etanercept (SUCRA 28.4, mean rank 6.0), methotrexate (SUCRA 14.5, mean rank 7.0) and placebo (SUCRA 0, mean rank 8.0). However, the BAD NMA pooled licensed and unlicensed doses [24]. It included many unlicensed doses that were not included in this analysis as they are not relevant for decision-making. Naïve pooling across doses, without accounting for possible differential dose effects, is not recommended as it can increase heterogeneity due to different treatment definitions. Furthermore, the aim of this analysis was to characterise heterogeneity in networks used by NICE; therefore, only licensed doses were relevant.

A recent article evaluated the association between patient characteristics and response to biologic therapies for psoriasis, using a multicentre longitudinal cohort study; the British Association of Dermatologists Biologic Interventions Register (BADBIR) [25]. This study also found little evidence for predictors of differential treatment response, although only biologic-naïve patients were included in the study.

Network structure

There was some overlap between networks in terms of included studies (see Table 2). In particular, many of the studies excluded from the ≥ 90% white patients network were included in the network of studies with lighter patients (≤ 90 kg). Only ten studies included patients with a mean weight below 80 kg, nine of which were conducted in Japanese, Chinese or mixed Taiwanese, Chinese and Korean patients (see Additional file 1).

Recommendations for future research

NMAs of psoriasis treatments undertaken in the future should investigate heterogeneity within the networks and include clinically relevant subgroups to further investigate effect modification related to certain patient characteristics. This recommendation is also appropriate for NMAs in other clinical areas and other fields outside of medicine.

Conclusions

This work has highlighted potential differences in relative treatment effectiveness for biologic-naïve patients receiving psoriasis treatment. Our results support the assumption that prior exposure to biologic therapy is associated with psoriasis treatment response and confirm the importance of considering this as a potential effect modifier. Future decision-making on psoriasis treatments should consider patients’ prior exposure to biologic therapies.

More broadly, we have demonstrated the importance of assessing heterogeneity in patient characteristics and adjusting for effect modifiers in a NMA, which can be done by restricting inclusion in the NMA to certain subgroups of patients with similar characteristics. Focusing the inclusion criteria to produce smaller, more homogenous networks can reduce the risk of both heterogeneity and inconsistency, and give more valid results.