Niv et al. [1] present a meta-analysis on colonoscopy as primary screening for colorectal cancer (CRC) in asymptomatic people. Their aim was to quantify diagnostic yield and safety of primary colonoscopy in CRC screening. We feel obliged to draw the attention of readers to the facts that this meta-analysis was not performed on thoroughly reviewed data and not according to some basic principles for meta-analysis (available, for example, as open learning material from The Cochrane Collaboration: http://www.cochrane-net.org/openlearning/HTML/mod0.htm).

First, according to Table 1, the diagnostic yield of colonoscopy for CRC was 0.08 in the study from Regula et al. [2], which in fact is ten times the actual figure of 0.008. Also in Fig. 1 an enormous outlier for CRC from the same study is presented. This outlier might have made both the authors as well as peer reviewers concerned about a possible mistake. Given the erroneous CRC detection rate and that the study by Regula constitutes over 70% of the pooled study population, the authors surprisingly present a relatively low pooled proportion of 0.0078. However, we suspect that the pooled analysis has also been performed with incorrect data. The pooled proportion of 0.0078 is presented with a 95% confidence interval of 1.29E-05 to 0.0297, which is wider (i.e., has less precision) than any of the confidence intervals in any of the included studies. Meta-analysis is performed to increase, not to decrease precision.

Several meta-analysis principles are not completely met or evaluated by the authors. Due to space limitations we will not focus on the appropriateness of data pooling and concentrate on publication bias. Although publication bias should be minimized, it is difficult to measure and usually impossible to avoid completely. One should always start by creating a comprehensive database of literature with broad but accurate search terms. The authors propose two search terms. Without wanting to ignore all other databases, in PubMed the term “screening colorectal cancer” between quotes (i.e., searching as typed) returns no hits and without quotes (i.e., free search) returns >60,000 hits. Both queries seem inaccurate. On the other hand the search term “screening colonoscopy” between quotes in PubMed alone returns 305 hits, which is ten times more than the 31 hits that the authors report. The authors should have reported all references considered and all the reasons for exclusion to reach the remaining studies. An additional method to avoid publication bias is to exclude only studies with real shortcomings. The authors hardly clarify their reasons for inclusion or exclusion, and some of the reasons seem incorrect. Usually in meta-analyses authors limit themselves to randomized controlled trials (RCTs). For the aim of this study, the authors do not specifically need RCTs. However, that does not imply that they should exclude RCTs, as they obviously do: “… using another method in addition to colonoscopy.” In the first part of the complete sentence, the exclusion of primary screening with other tests (e.g., fecal occult blood tests) is covered. Therefore in the cited part of the sentence the authors can only mean to exclude primary colonoscopy screening studies in which alternative tests were offered in parallel (e.g., RCTs). The authors offer no explanations why these studies, such as the study by Segnan et al. [3] from Italy with 1,597 individuals with colonoscopy, should be excluded. Also only using studies published in English, as the authors do, is only acceptable if the authors can convincingly claim that no relevant studies in other languages are missed. However, at least one study was missed from Germany, that by Hüppe et al. [4] with 5,066 individuals with colonoscopy. Possibly some other relevant studies on colonoscopy screening might have been missed, such as Chinese or Japanese studies. Incapability to read languages other than English should never be an argument to accept publication bias.

We conclude that, because of the inappropriate review of the original data and the poor implementation of meta-analysis methodology, the results and the conclusions based on these results of this meta-analysis should be reviewed with much caution.

Reply

We like to thank Mr. Leo GM van Rossum and colleagues for reading our paper and finding a typo, that is, the yield of screening colonoscopy for colorectal cancer in Regula’s study was 0.008 instead of 0.08. We included in the meta-analysis only papers describing cohort of screening colonoscopy; thus, the paper of Segnan et al. comparing attendance rate of colonoscopy, sigmoidoscopy, and FIT was not included [3]. We included only papers written in the English language.

Yaron Niv, MD, FACG, AGAF

Rachel Hazazi, MSc

Zohar Levi, MD

Gerald Fraser, MD