The question asked by Nelson et al. is relevant and timely. The incidence of surgical site infection (SSI) following colorectal surgery remains high with one-fifth of hospital acquired infections attributed to SSI. SSI are associated with substantial postoperative morbidity and are used as a metric for hospital performance ranking and reimbursements [1, 2]. Dr. Nelson et al. performed a systematic review of randomized-controlled trials (RTCs) that associated the use topical antimicrobials, excluding antimicrobial ointments, with SSI reduction. They performed a meta-analysis of the literature at large, and subgroup analyses for use of gentamicin sponges/beads, chlorhexidine impregnated suture, and antibiotic powders/lavages/injection into the wound calculating relative risk reduction of SSI with the use of these agents. The methodology is thorough, attentive to bias, and well executed. However, in limiting their meta-analysis to RTCs, the interpretation is subject to significant limitations. These limitations include limited external validity, lack of power, the heterogeneity of the aggregate study population and intervention, and publication bias [3]. RCTs are the gold standard for detecting an effect of interventions. However, this effect is seen in patients who are stringently selected by inclusion criteria, and compliance with the intervention is strictly overseen. These parameters limit the external validity of RTCs in practice where the patients are heterogeneous and compliance with established protocols may be lax. With these limitations, the effects of RTCs must be confirmed in studies that have improved generalizability to contemporary practice patterns.

The use of meta-analysis is commonplace as it increases the sample size by which to detect an effect if the RTCs are underpowered, as well as perform subgroup analysis. The authors’ meta-analysis of the data is well performed; however, their inclusion criteria for the analysis are underpowered for the question which they seek to answer. The combined SSI rate for the study was 665/5511 (12%). The outcome of interest, not the denominator population, drives the power of the study and statistical plan. The underpowering permeates each of their analyses, leading to low-quality conclusions. In addition, the heterogeneity of the interventions is such that even their aggregate conclusion of risk reduction cannot be applied, as it is unclear which of the topicals drives the effect size. Another challenge to the use of meta-analysis is the publication bias of positive effect studies. It is well documented across domains that studies with null results are less likely to be published than those with a demonstrated effect which can make aggregate results challenging to interpret.

Ultimately, this study is an academic exercise in meta-analysis, without providing robust interpretation by which to adopt new practice patterns for topical antimicrobials. However, this study does inform study design for those seeking to determine if these interventions have a role in the management of colorectal surgery patients in the prevention of SSI. A meta-analysis that uses both prospective and retrospective studies may provide increased power to allow for meaningful sub-analyses. Alternatively, analysis of a large data set may lead to improved understanding of the impact of topical interventions at large. The National Surgical Quality Improvement Program (NSQIP) has provided granular data regarding perioperative quality for SSI, driving improvements both within institutions and nationally [4, 5]. In its current iteration, it does not include a field for use of topical antimicrobials; however, given the impact that these interventions may have, it could be considered for inclusion in the colorectal surgical outcomes data extraction. Alternatively, a multi-institutional study using prospective or retrospective data could foreseeably be high powered enough to run these sub-analyses without the heterogeneity found in meta-analysis [6]. This methodology has been used across institutions in implementing SSI prevention bundles and using NSQIP to track quality outcomes over time [7]. The granularity of the data set, homogeneity of the outcomes, and risk adjustment for population differences allow NSQIP-based retrospective studies to substantially contribute to the improvement of surgical care, and allow for the measurement of iterative changes to surgeons’ practice patterns.

In conclusion, Dr. Nelson et al. have investigated an interesting question about the effect of topical antimicrobials on overall SSI reduction in colorectal surgery. Their methodology and sub-analyses are thoughtful and well performed. However, limiting their inclusion exclusively to RCTs dramatically underpowered their meta-analysis, preventing meaningful conclusions from being drawn. There is benefit to the inclusion of large retrospective data from national sets or from multi-institutional groups that would augment their study and allow for practice improving conclusions to be drawn.