Dear Editor,

We read with interest the recent article by Wingerchuk et al. [1] reporting the results of a fixed-effects proportional hazards Bayesian meta-analysis of treatments for neuromyelitis optica spectrum disorder (NMOSD), a rare autoimmune neuroinflammatory central nervous system disease predominantly affecting the optic nerves and the spinal cord, that often results in accumulated permanent disability caused by successive attacks [2].

Three US Food and Drug Administration (FDA)-approved treatments for adults with aquaporin-4 IgG seropositive (AQP4+) NMOSD were included in the meta-analysis. Eculizumab, an inhibitor of C5, a component of the complement cascade, was studied in the PREVENT trial in which add-on immunosuppression was allowed [3]. Satralizumab, which targets the interleukin-6 receptor, was studied as monotherapy (SAkuraStar) and in combination with immunosuppressives (SAkuraSky) [4, 5]. Inebilizumab, a monoclonal antibody to CD19 expressed on the surface of B cells, was studied as a monotherapy in N-MOmentum [6].

In the absence of head-to-head trials of these agents, an indirect network meta-analysis might potentially fill a gap in comparing relative treatment effects on preventing attacks in these patients; however, a matching-adjusted indirect comparison (MAIC) is the gold standard for such analyses. MAIC analyses require that the populations and outcomes of the comparator studies being evaluated are similar enough to be matched by way of factors that influence these outcomes. This is not possible with the trials being evaluated here, in our view, as key methodological issues, including differences in sample size, the populations under study, trial design, and outcome measurements, not only limit the conclusions that can be drawn but also raise serious questions as to whether this analysis should have been conducted because of systematic bias.

Sample Size

Notably, inebilizumab and satralizumab were studied as monotherapy options for NMOSD in 161 AQP4+ participants in the treatment arm of the N-MOmentum and 41 in the SAkuraStar trials. In contrast, eculizumab was studied as an add-on therapy to baseline immunosuppression. In the treatment arm (N = 96) of the PREVENT trial, only 21 participants in the randomized controlled trial were not receiving concomitant immunosuppression. For those receiving concomitant immunosuppression, 75 AQP4+ participants were enrolled in the PREVENT and 27 in the SAkuraSky trials. The small number of participants leaves considerable uncertainty about the significance of differences in outcomes in a diverse NMOSD population.

Differences in Enrolled Patient Populations

The entry criteria across the studies targeted different populations resulting in significant differences in prior attack history, baseline disability, and disease duration. Perhaps the most noteworthy difference is the rituximab exclusion criteria in the PREVENT trial compared to the other pivotal NMOSD trials. In the PREVENT trial, patients could be enrolled in the study if their last dose of rituximab was at least 3 months prior to screening. The N-MOmentum, SAkuraStar, and SAkuraSky trials required that the last dose of rituximab must have been administered at least 6 months prior to screening. Thus, in the PREVENT trial, patients could still be protected by therapeutic doses of rituximab during the first several months of the trial, a potentially critical issue since the primary outcome evaluated only time to first attack. Moreover, 27% of patients in the treatment arm of the PREVENT trial had a history of rituximab use compared to just 13% in the SAkuraSky trial and 7% in the N-MOmentum trial. Differences in residual effects from prior treatments could strongly influence the on-study likelihood of experiencing attacks across the trials. Specifically, the likelihood of experiencing a clinical attack in the PREVENT trial could have been lower in the first 3 months of the study compared to that of the other studies because of the rituximab exclusion criterion resulting in systematic bias of attack risk.

Attack Criteria and Adjudication

The definition of an NMOSD attack and how it was operationally determined varied substantially in the four clinical trials (Table 1). In the PREVENT trial, attacks were defined as any worsening of neurological symptoms preceded by at least 30 days of clinical stability. Like the PREVENT trial, the satralizumab trials defined attacks and worsening of neurological symptoms preceded by 30 days of stability but required objective worsening of the Expanded Disability Status Scale (EDSS) and/or Functional Systems Scores (FSS). In contrast, the N-MOmentum trial used 18 prospectively pre-defined attack criteria in relevant functional domains (myelitis, optic neuritis, area postrema syndrome, and cerebral syndromes). The N-MOmentum trial also consistently obtained magnetic resonance imaging data to supplement the clinical attack criteria. The definition of attack becomes particularly important when the primary outcome is time to first attack since only a single event is measured. Because different attack definitions were used, cross-trial comparisons are particularly perilous and misleading. Differences in assessment of outcome definition and adjudication render comparisons meaningless.

Table 1 Attack criteria across clinical trials
Full size table

The NMOSD clinical trials illustrated that attack definitions in NMOSD are not straightforward. While the SAkuraSky, SAkuraStar, and N-MOmentum trial designs prospectively included an independent relapse adjudication committee who reviewed all physician-determined relapses, in the PREVENT trial, adjudication was retrospective; attacks were initially physician determined and an attack adjudication committee was established through a protocol amendment after 88 participants were already enrolled. Adjudication in the N-MOmentum study was performed “in real time” so that when uncertainty about the adjudication existed, the trial sites were able to explain or collect additional data if necessary. Further, there was poor correlation between physician-determined and adjudicated attacks in PREVENT (Table 2), yet only adjudicated attacks were considered in this network meta-analysis (NMA).

Table 2 Investigator-determined and committee-adjudicated attacks in AQP4 seropositive participants
Full size table

The different attack designations in PREVENT enable an understanding of the impact of differences in attack definition on the time to first attack primary outcome. As an example, if one uses physician-determined attacks, 14 patients in the eculizumab group and 29 in the placebo group experienced attacks, yielding a hazard ratio of 0.18. If one uses attacks defined by adjudication, 3 patients in the eculizumab group and 20 out of 47 placebo-treated participants experienced attacks, resulting in a hazard ratio of 0.06. This example clearly shows that different definitions of attacks can result in substantially different point estimates of therapeutic benefit. Furthermore, the duration of the randomized controlled period in the N-MOmentum trial ended at 197 days to limit placebo exposure, whereas the randomized controlled period in PREVENT lasted until the end of study and in the satralizumab studies the double-blind period was limited to 1.5 years. In summary, the attack definition, attack adjudication, and timing of the controlled periods for attack inclusion differed across these trials and are not easily comparable.

Additional Methodological Considerations

Beyond the large differences in patient characteristics and outcome definitions between underlying trials, it is critical to emphasize the considerable uncertainty associated with results for eculizumab underpinning this NMA. The sample sizes of monotherapy and combination therapy populations for eculizumab in PREVENT trial are small (N = 21 and N = 75, respectively). Further, patients in this small eculizumab monotherapy cohort did not experience an adjudicated attack during the PREVENT study so a hazard ratio of 0.025 was imputed by the authors and used in NMA. Therefore, the conclusions drawn for eculizumab monotherapy by the authors are based on an imputed HR from a trial with small sample size and should be interpreted with extreme caution. Indeed, leading NMA methodologists emphasize that NMAs based on a small number of trials with few events drawn from different populations are prone to considerable bias.

The NMA results clearly and overwhelmingly favor eculizumab for the monotherapy analysis. When one considers the combination therapy analysis comparing satralizumab to eculizumab the confidence interval crosses unity, indicating that there is considerable uncertainty as to whether eculizumab is superior to satralizumab in participants receiving concomitant immune suppressive medications. Because there is no indication that immune suppressants directly interfere with the efficacy of eculizumab, or enhance that of satralizumab, this curious finding must have some other explanation. Further, the sample of participants for the combination treatment arms is considerably larger than for the monotherapy comparisons. In summary, the limitations of this NMA study as outlined preclude an unbiased assessment and future studies should attempt to adjust for the key differences in demographic and disease variables affecting outcomes between studies where possible using accepted techniques such as MAIC.