Avoid common mistakes on your manuscript.
Network meta-analysis (NMA) is increasingly recognized as a promising evidence synthesis method commonly allowing stronger conclusions on the comparative effectiveness of healthcare interventions than conventional pairwise meta-analysis [1]. Its strength arises from the fact that it allows to synthesize both direct and indirect evidence from randomized trials. It is hence timely that Hui et al. recently published an NMA in the European Journal of Nutrition [2], comparing the effects of different whole grains (oat, brown rice, barley, and wheat) and brans (oat bran and wheat bran) on blood lipids (total cholesterol, LDL-C, HDL-C, and triacylglycerols), using data from 55 trials. NMA allows inference on every possible pairwise comparison of interventions within a connected network. For example, in the paper by Hui et al. [2], oat bran and barley have not been directly compared in a randomized trial, but each has been compared with wheat (Fig. 1). As such, an indirect comparison between oat bran and barley can be obtained. Sometimes, the relative effects estimated by the network may rely to a notable extent on indirect comparisons (i.e., for which no trials were ever conducted); the influence of direct and indirect evidence on the results can be seen using the contribution matrix [3, 4]. In fact, in the NMA by Hui et al., the contribution of direct evidence to the relative effects estimated by the network was very low ranging from 0.3% (oat vs. wheat) to 15.9% (wheat vs. control).
Nutrition research can substantially profit from the potential of NMA. However, it is crucial that authors meticulously plan, conduct, and report NMA [5, 6]; in particular, authors should follow a study protocol published a priori so as to improve transparency and perform a rigorous risk of bias assessment within and across studies as well as an evaluation of the quality of evidence. As Hui et al. are among the pioneers of applying NMA to the field of nutrition research, we draw on their article to highlight some methodological challenges that require specific attention when performing NMA.
Summary effects from NMA are usually presented in a league table including all comparisons: Hui et al. [2] identified oat bran as the most effective intervention strategy, revealing clinically relevant mean differences (MD) in comparison with the control diet [improvements in total cholesterol (TC) (MD: − 0.35 mmol/L, 95% CI − 0.47, − 0.23 mmol/L) and LDL-C (MD: − 0.32 mmol/L, 95% CI − 0.44, − 0.19 mmol/L)]. Another unique feature of NMA is its ability to rank interventions in relation with the studied outcomes, using the distribution of the ranking probabilities and the surface under the cumulative ranking curve (SUCRA) [7]. SUCRA ranges from 0%, i.e., the treatment always ranks last without uncertainty, to 100%, i.e., the treatment always ranks first without uncertainty. In the NMA by Hui et al. [2] oat bran ranked as the best treatment for TC (SUCRA: 97%), LDL-C (SUCRA: 97%), and triacylglycerols (TG) (SUCRA: 78%), followed by oat (SUCRA: 79% for TC, 64% for LDL-C, 76% for TG).
The extent to which NMA allows valid indirect inference depends on the extent to which the fundamental assumption of NMA usually called the ‘transitivity’, assumption is likely to be plausible. Transitivity requires that the trials comparing different sets of interventions are appreciably comparable in characteristics (other than the interventions being compared) which may affect the outcome [8, 9]. Transitivity should be evaluated prior to conducting NMA [8, 9], e.g., by examining whether the distributions of potential effect modifiers are comparable across the direct treatment comparisons. Transitivity would, for example, be violated if changes in body weight would differ strongly between the two direct comparisons, as depicted in Fig. 1. An important limitation of the NMA by Hui et al. is the lack of a formal investigation of the distributions of potential effect modifiers across the available direct comparisons.
They did, however, apply a number of approaches (loop-specific approach, node-splitting approach, and design-by-treatment interaction model) to examine statistical incoherence [2]. The coherence assumption suggests that direct and indirect evidence are in statistical agreement and its assessment is mandatory. Incoherence can be tested locally (i.e., in parts of the network) and globally (i.e., in the entire network) and the use of both types of tests is highly recommended [6]. If incoherence is identified, subgroup analyses and network meta-regression may be used to investigate potential sources. It should be noted that the absence of statistically significant incoherence—as in the case of Hui et al. [2]—is not necessarily evidence of the absence of incoherence. In particular in networks that built on scarce evidence—as in the NMA of Hui et al.—tests for incoherence have low power; thus, their results should be interpreted cautiously [10]. It is hence welcome that the authors performed several sensitivity analyses confirming the findings of their main analyses.
Addressing publication bias in NMA is as difficult as in pairwise meta-analysis; thus, priority must be given to the exhaustive search for unpublished studies. Assessment of small-study effects is usually the first step in the formal investigation of reporting biases [11]. Hui el al used the ‘comparison adjusted funnel plot’, a modified funnel plot for application to a network of trials [4]. Should funnel plot asymmetry be detected, meta-regression allows the estimation of the magnitude of small-study effects, while ‘selection models’ can investigate the potential for publication bias [12].
Hui and colleagues [2] also assessed the confidence of evidence, using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework that considers the following items: study limitations, imprecision, inconsistency, indirectness, and publication bias. While the confidence on evidence was low for most comparisons, for comparisons of oat bran vs. the control diet it was judged as high for TC and moderate for LDL-C and TG. Hence, further research will provide important evidence on the majority of the comparisons included in this NMA, yet confidence is appreciable for oat bran vs. control diets. As this is an important conclusion for future research, we believe that a confidence on evidence statement should always be incorporated in the conclusion section of any manuscript. The use of the CINeMA (Confidence In Network Meta-Analysis: http://cinema.ispm.ch/) framework, which is an improvement of a previously suggested approach [3], can greatly facilitate judgements about the confidence that can be placed in results obtained by NMA. CINeMA modified and extended the five GRADE domains for use in NMA and is transparent and applicable to any network structure [3].
In view of its potentials, there is a risk that NMA is applied in cases, where the scarcity of available data precludes the estimation of precise results and the evaluation of its assumptions. Specifically, in the absence of direct evidence for several comparisons transitivity and incoherence cannot be formally tested, although transitivity can always be evaluated clinically and epidemiologically. In that case, the resulting relative effects would possibly be estimated with large uncertainty and the relative rankings might be meaningless.
In spite of these risks, we expect that high-quality NMAs combining the results of dietary intervention trials will become the new evidence synthesis norm also in nutrition research. While widely applied in many medical fields [13], its use in the field of nutrition is at present surprisingly rare. A quick search in PubMed (September 10th, 2018) using search-terms network meta-analysis[tiab] OR multiple treatments meta-analysis[tiab] OR mixed-treatment comparison[tiab] AND (diet*[tiab] OR nutrition[tiab]) yielded only 38 hits. Out of these, only ~ 50% original NMA papers dealt with a nutrition-related topic and only two of them [14, 15] were published in a nutrition journal. Yet, NMAs have the potential to advance the knowledge in the field of nutrition as they provide insights that cannot be obtained by individual trials or pairwise meta-analysis: for example, the DASH dietary approach proved to be the most effective dietary approach to reduce blood pressure among hypertensive and pre-hypertensive patients [15], while the Mediterranean diet emerged as the most effective dietary approach to improve glycaemic control in type 2 diabetes patients [16]. Finally, butter and lard were ranked worst for reducing LDL-C, whereas safflower-, rapeseed-, and sunflower oil performed best [17].
Beyond this, NMA has recently been proposed also as a tool to plan the optimal design and the required sample size of new trials [18]. Finally, NMA may be used to close the gap between evidence stemming from meta-analyses of prospective observational studies and missing evidence from RCTs. Using dietary exposures comparable to those examined in observational studies on hard clinical endpoints in NMAs on intervention trials with intermediate disease markers could ultimately strengthen the credibility of nutrition research findings [19, 20].
References
Nikolakopoulou A, Mavridis D, Furukawa TA, Cipriani A, Tricco AC, Straus SE, Siontis GCM, Egger M, Salanti G (2018) Living network meta-analysis compared with pairwise meta-analysis in comparative effectiveness research: empirical study. BMJ 360:k585. https://doi.org/10.1136/bmj.k585
Hui S, Liu K, Lang H, Liu Y, Wang X, Zhu X, Doucette S, Yi L, Mi M (2018) Comparative effects of different whole grains and brans on blood lipid: a network meta-analysis. Eur J Nutr. https://doi.org/10.1007/s00394-018-1827-6
Salanti G, Del Giovane C, Chaimani A, Caldwell DM, Higgins JP (2014) Evaluating the quality of evidence from a network meta-analysis. PloS One 9(7):e99682. https://doi.org/10.1371/journal.pone.0099682
Chaimani A, Higgins JP, Mavridis D, Spyridonos P, Salanti G (2013) Graphical tools for network meta-analysis in STATA. PLoS One 8(10):e76654. https://doi.org/10.1371/journal.pone.0076654
Hutton B, Salanti G, Caldwell DM, Chaimani A, Schmid CH, Cameron C, Ioannidis JP, Straus S, Thorlund K, Jansen JP (2015) The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann Intern Med 162. https://doi.org/10.7326/m14-2385
Chaimani A, Caldwell DM, Li T, Higgins JP, Salanti G (2017) Additional considerations are required when preparing a protocol for a systematic review with multiple interventions. J Clin Epidemiol 83:65–74. https://doi.org/10.1016/j.jclinepi.2016.11.015
Salanti G, Ades AE, Ioannidis JP (2011) Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial. J Clin Epidemiol 64(2):163–171. https://doi.org/10.1016/j.jclinepi.2010.03.016
Salanti G (2012) Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Res Synth Methods 3(2):80–97. https://doi.org/10.1002/jrsm.1037
Mavridis D, Giannatsi M, Cipriani A, Salanti G (2015) A primer on network meta-analysis with emphasis on mental health. Evid Based Mental Health 18(2):40–46. https://doi.org/10.1136/eb-2015-102088
Veroniki AA, Vasiliadis HS, Higgins JPT, Salanti G (2013) Evaluation of inconsistency in networks of interventions. Int J Epidemiol 42(3):919–919. https://doi.org/10.1093/ije/dyt106
Chaimani A, Salanti G (2012) Using network meta-analysis to evaluate the existence of small-study effects in a network of interventions. Res Synth Methods 3(2):161–176. https://doi.org/10.1002/jrsm.57
Dimitris M, Alex S, Andrea C, Georgia S (2013) A fully Bayesian application of the Copas selection model for publication bias extended to network meta-analysis. Stat Med 32(1):51–66. https://doi.org/10.1002/sim.5494 doi
Petropoulou M, Nikolakopoulou A, Veroniki A-A, Rios P, Vafaei A, Zarin W, Giannatsi M, Sullivan S, Tricco AC, Chaimani A, Egger M, Salanti G (2017) Bibliographic study showed improving statistical methodology of network meta-analyses published between 1999 and 2015. J Clin Epidemiol 82:20–28. https://doi.org/10.1016/j.jclinepi.2016.11.002
Carter P, Achana F, Troughton J, Gray LJ, Khunti K, Davies MJ (2014) A Mediterranean diet improves HbA1c but not fasting blood glucose compared to alternative dietary strategies: a network meta-analysis. J Hum Nutr Diet 27(3):280–297. https://doi.org/10.1111/jhn.12138
Schwingshackl L, Chaimani A, Schwedhelm C, Toledo E, Punsch M, Hoffmann G, Boeing H (2018) Comparative effects of different dietary approaches on blood pressure in hypertensive and pre-hypertensive patients: a systematic review and network meta-analysis. Crit Rev Food Sci Nutr:1–14. https://doi.org/10.1080/10408398.2018.1463967
Schwingshackl L, Chaimani A, Hoffmann G, Schwedhelm C, Boeing H (2018) A network meta-analysis on the comparative efficacy of different dietary approaches on glycaemic control in patients with type 2 diabetes mellitus. Eur J Epidemiol 33(2):157–170. https://doi.org/10.1007/s10654-017-0352-x
Schwingshackl L, Bogensberger B, Bencic A, Knuppel S, Boeing H, Hoffmann G (2018) Effects of oils and solid fats on blood lipids: a systematic review and network meta-analysis. J Lipid Res 59(9):1771–1782. https://doi.org/10.1194/jlr.P085522
Salanti G, Nikolakopoulou A, Sutton AJ, Reichenbach S, Trelle S, Naci H, Egger M (2018) Planning a future randomized clinical trial based on a network of relevant past trials. Trials 19(1):365. https://doi.org/10.1186/s13063-018-2740-2
Schwingshackl L, Schlesinger S, Devleesschauwer B, Hoffmann G, Bechthold A, Schwedhelm C, Iqbal K, Knuppel S, Boeing H (2018) Generating the evidence for risk reduction: a contribution to the future of food-based dietary guidelines. Proc Nutr Soc. https://doi.org/10.1017/s0029665118000125
Schwingshackl L, Hoffmann G, Iqbal K, Schwedhelm C, Boeing H (2018) Food groups and intermediate disease markers: a systematic review and network meta-analysis of randomized trials. Am J Clin Nutr 108(3):576–586. https://doi.org/10.1093/ajcn/nqy151
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
AEB is a member of the Editorial board of the European Journal of Nutrition (Section Epidemiology) and a member of the International Carbohydrate Quality Consortium (ICQC). LS and AC declare that they have no conflict of interest.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Schwingshackl, L., Buyken, A. & Chaimani, A. Network meta-analysis reaches nutrition research. Eur J Nutr 58, 1–3 (2019). https://doi.org/10.1007/s00394-018-1849-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00394-018-1849-0