Avoid common mistakes on your manuscript.
To the Editor
Network meta-analysis (NMA) approaches are increasingly being used to compare treatments across multiple trials by incorporating results of both direct and indirect comparisons. To inform sound evidence-based treatment decisions, NMA studies must be based on methodologically rigorous analyses that are presented and interpreted in the context of important limitations to ensure robust scientific conclusions. Armstrong et al. [1] recently published an NMA evaluating the benefit–risk profile of treatments for plaque psoriasis. A similar analysis was previously published by the same group (Shear et al. [2]). This publication was subject to scientific criticism by Pettitt et al. [3], as it utilized an analytic approach that is inconsistent with prevailing methodological conventions. At that time, the authors (Shear et al. [2]) chose not to respond nor defend their methodological choices and assumptions. As the same methodological flaws persist in the current manuscript, further scientific discourse on the matter is needed.
Both publications [1, 2] conclude that “Risankizumab was associated with the most favorable long-term benefit-risk profile.” However, this statement is not supported by statistical analysis. The authors present risk–benefit results as surface under the cumulative rank (SUCRA) scores, which vary from 0 to 1 and can be interpreted as the proportion of therapies to which a given treatment compares favorably [4]. There are several important limitations of the SUCRA approach that can be misleading if results are not interpreted correctly. As a ranking strategy, SUCRA is agnostic to the magnitude of treatment effects and therefore artificially inflates apparent differences between therapies [5]. Indeed, SUCRA can make a treatment look favorable to comparators even if it is neither statistically nor clinically significantly superior to any other treatment on any outcome. In Armstrong’s analysis of any serious adverse event (SAE), no statistically significant differences were evident in the frequencies between risankizumab and any other treatment examined [1].
To address the limitations and provide important context for the results of SUCRA analyses, experts encourage the use of absolute risks in such studies to align with best practices [6], as well as displaying the point estimates and their credible intervals in figures. To illustrate, Fig. 1 shows how SUCRA scores magnify small differences when plotting SAEs against psoriasis area and severity index (PASI) 100 response. Plots of absolute outcomes reveal that safety trade-offs are small and uncertain, with widely overlapping credible intervals. A formal test of significant improvement in both outcomes could be achieved using commonly available multivariate methods [7], which was not presented by the authors.
Although the authors did provide 95% credible intervals around the posterior median estimates for PASI, and separately for the proportions of patients that experienced SAEs [1], fixed effects models were used to conduct the NMA. Fixed effects models do not account for and appropriately reflect variation due to between-study heterogeneity, which is apparent in the wide range of placebo responses in the induction period across trials. Best practice when faced with between-trial heterogeneity is to use a random effects model and a meta-regression to adjust for variations in placebo response across trials (i.e., baseline-risk adjustment) to produce unbiased estimates [8]. Since a random effects model and baseline-risk adjustment are not possible in sparse networks, such as the one in Armstrong et al. [1], credible intervals can be misleading and unadjusted point estimates are likely confounded, meaning that the NMA should not be used for decision-making.
The issues raised here reiterate our previous concerns with the NMA conducted by Shear et al. [3]. We continue to emphasize the importance of aligning to common NMA standards and best practices to provide reliable evidence and appropriately guided clinical decision-making. Avoiding perceptions that NMAs are untrustworthy and that results are malleable is critical to their ongoing clinical utility, especially since the volume of overlapping NMAs in psoriasis and the appearance of “spin” in conclusions has recently been highlighted [9]. We invite an open dialogue with interdisciplinary NMA stakeholders on how to align on common standards for conducting and interpreting NMAs.
References
Armstrong AW, Soliman AM, Betts KA, et al. Long-term benefit–risk profiles of treatments for moderate-to-severe plaque psoriasis: a network meta-analysis. Dermatol Ther (Heidelb). 2022;12:167–84.
Shear NH, Betts KA, Soliman AM, et al. Comparative safety and benefit-risk profile of biologics and oral treatment for moderate-to-severe plaque psoriasis: a network meta-analysis of clinical trial data. J Am Acad Dermatol. 2021;85:572–81.
Pettitt D, Plotnick M, Miller LS, Berlin JA. Letter to the editor concerning the article: “Comparative safety and benefit-risk profile of biologics and oral treatments for moderate-to-severe plaque psoriasis: a network meta-analysis of clinical trial data.” J Am Acad Dermatol. 2021;85:e305–6.
Salanti G, Nikolakopoulou A, Efthimiou O, Mavridis D, Egger M, White IR. Introducing the treatment hierarchy question in network meta-analysis. Am J Epidemiol. 2022;191:930–38. https://doi.org/10.1093/aje/kwab278.
Mbuagbaw L, Rochwerg B, Jaeschke R, et al. Approaches to interpreting and choosing the best treatments in network meta-analyses. Syst Rev. 2017;6:79.
van Valkenhoef G, Tervonen T, Zhao J, de Brock B, Hillege HL, Postmus D. Multicriteria benefit–risk assessment using network meta-analysis. J Clin Epidemiol. 2012;65:394–403.
Mavridis D, Porcher R, Nikolakopoulou A, Salanti G, Ravaud P. Extensions of the probabilistic ranking metrics of competing treatments in network meta-analysis to reflect clinically important relative differences on many outcomes. Biom J. 2020;62:375–85.
Cameron C, Hutton B, Druchok C, et al. Importance of assessing and adjusting for cross-study heterogeneity in network meta-analysis: a case study of psoriasis. J Comp Eff Res. 2018;7:1037–51.
Guelimi R, Afach S, Régnaux JP, et al. Overlapping network meta-analyses on psoriasis systemic treatments, an overview: quantity does not make quality. Br J Dermatol. 2022;187:29–41. https://doi.org/10.1111/bjd.20908.
Acknowledgements
Funding
This work was supported by Janssen Research and Development.
Medical Writing and Editorial Assistance
The authors would like to acknowledge EVERSANA for medical writing support (Tim Disher and Barkha Patel) and analytic support, which included generation of figures (Tim Disher), and Lumanity for editorial and administrative support (Jenna Zeringo and Mike Henretty).
Author Contributions
DP, MP, and JG contributed to the conception and design, article revisions of critically important intellectual content, and the final review and approval of the article.
Disclosures
Dan Pettitt, Michael Plotnick, and Joshua Gagne are employees of Janssen Pharmaceuticals and Joshua Gagne is an employee of Johnson & Johnson. All are shareholders of Johnson & Johnson.
Compliance with Ethics Guidelines
This article is based on previously conducted studies and does not contain any new studies with human participants or animals performed by any of the authors.
Data Availability
Data sharing is not applicable to this article as no datasets were generated or analyzed during the current study.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which permits any non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc/4.0/.
About this article
Cite this article
Pettitt, D., Plotnick, M. & Gagne, J. Letter to the Editor Regarding Long-Term Benefit–Risk Profiles of Treatments for Moderate-to-Severe Plaque Psoriasis: A Network Meta-analysis. Dermatol Ther (Heidelb) 12, 2863–2866 (2022). https://doi.org/10.1007/s13555-022-00809-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13555-022-00809-8