Dear Editor,

We read with interest the article “Treatment of relapsed/refractory multiple myeloma in the bortezomib and lenalidomide era: a systematic review and network meta-analysis” (Ann Hematol 100, 725–734 [2021]). Such an article is important for decision-making to inform clinicians of the most effective and safe treatments in a complex setting such as relapsed refractory multiple myeloma (MM). However, all such studies should be conducted in line with recognized guidance [1, 2], and caution is warranted in the interpretation of this study as there are a number of serious methodological and data concerns that limit the validity of the conclusions.

The systematic literature review that was conducted to identify phase III studies was originally restricted to include only lenalidomide or bortezomib in the control arm. Fifteen such studies were identified. However, this protocol was violated when “two studies with pomalidomide and one with carfilzomib in the control arm were also included.”

Furthermore, the principles which underpin the network meta-analysis (NMA) methodology are violated on multiple levels:

  • Trials evaluating different populations without any adjustment for treatment effect modifiers are compared.

  • Studies were conducted over at least a 15-year period, during which drug availability and standard of care treatment regimens varied considerably.

  • Different backbone therapies are “considered equivalent therapies” and combined as a single “control” group.

  • Severe adverse events (SAE) are assumed to be comparable with grade III/IV events.

It is well documented that patient characteristics (i.e., age, performance status, cytogenetic risk), number of prior treatment lines, type of therapies received, and refractoriness to treatment options may act as prognostic factors and treatment effect modifiers. However, study inclusion criteria vary from no refractory patients (DOXIL-MMY-3001) to all patients were refractory (ICARIA), and the median number of prior lines of therapy ranged from 1 to 3 (see Table 1).

Table 1 Overview of the studies included in the NMA by Arcuri and Americo (2021)
Full size table

Lenalidomide and bortezomib are routinely used in clinical practice for MM treatment, including in combination. Assuming all control backbone therapies are “equivalent” when the efficacy differs substantially between studies invalidates this approach, as shown for example by the PFS of CASTOR (Vd) vs POLLUX (Rd). It fails to accurately capture their respective benefit and the role they may play in the efficacy of the regimen. This is also relevant for ICARIA, CANDOR, and KEYNOTE-183, which included neither lenalidomide nor bortezomib in the treatment regimens, but still had their backbone therapies combined for inclusion in the NMA.

This equivalency assumption is likely to also skew the interventional treatment’s adverse event (AE) profile, which may be over- or underestimated based on the backbone regimen. Further, results of the toxicity comparison are rendered misleading by using grade III/IV events interchangeably with SAEs. These two terminologies have distinct definitions and capture different aspects of treatment toxicity. AEs are graded I–V, and seriousness of the event is determined independently of the grade. Generally, the occurrence of grade III/IV AEs is higher than SAEs.

In conclusion, we commend the authors in attempting to address an important open question in the treatment of relapsed myeloma patients; however, we recommend an updated analysis be conducted taking into account the points mentioned above, and validated with clinical experts and experts in evidence synthesis.