The inadequacy of published random control trials of antibacterial prophylaxis in colorectal surgery

Abstract

Fifty-six papers published between 1979 and 1986 that tested regimens of antibacterial bowel preparation before elective colorectal operations were studied using a numerical score devised for assessing publications of random control clinical trials. A maximum score of 100 was allotted: 50 for 15 aspects of design, 30 for ten aspects of analysis, and 20 for eight aspects of presentation. The 56 papers scored from 33 to 89, (mean, 61.6, standard deviation, 11.9). Only 13 (23 percent) reached a score of more than 70. The most frequent errors in design were the use of placebos in the control group (27 percent) and faulty methods of randomization (36 percent). Errors in analysis resulted in penalization of more papers than any other aspect; these included the almost universal omission of confidence limits, confusion of exclusions and withdrawals (46 percent), not recording the fate of withdrawals (80 percent) and incorrect use of statistical tests (55 percent). Ten papers reported results showing important clinical differences that did not achieve statistical significance, but only two mentioned the Type II error. Defects in presentation were less frequently encountered; the most common were inaccessibility of raw data (66 percent), lack of sufficient information to allow replication of methods (43 percent), and the drawing of firm conclusions from shaky data (50 percent). It was particularly disappointing that no evidence of improvement in the standard of these reports over the seven years studied was found.