Dear Editor,

We read with interest the recent paper by Ghoniem et al. [1] exploring urinary cytokine profiles of women with overactive bladder (OAB) in comparison to both healthy controls and women with UTI. While we applaud their efforts to apply such innovative techniques, we wish to raise concerns regarding the statistical methods used to analyse data from the cytokine antibody arrays.

The authors report p values <0.05 in association with twofold overexpression for six of the cytokines assayed comparing controls and women with OAB, and for 13 of the cytokines assayed comparing controls and women with UTI. Despite failing to reproduce the well-recognised association between NGF and OAB, the authors concluded that a subset of the cytokines were upregulated in both OAB and UTI.

We believe that these results can more plausibly be explained by type I error. Simultaneously assaying 120 cytokines in three groups of women introduced the risk of type I error. The authors chose a significance level of 0.05, which would be expected to give 12 false positives among the 240 hypotheses they tested. As an example, using the observed mean for NGF among controls and the pooled standard deviation, and applying the conservative Bonferroni correction, we estimate the power of the study to detect a twofold change as only 14%. Even adjusting the p values, using the less conservative Benjamini and Hochberg false discovery rate method [2], suggests that the “top hits” MCP-2 and MCP-1 were no longer significantly different between controls and cases, with p = 0.106 and p = 0.364, respectively.

As we increasingly see application of “'omics” technologies to urogynaecology, it is important that we consider the perils of multiple hypothesis testing. Just as in randomised interventional trials, we should expect to see careful consideration of sample size, and cautious interpretation of results that have not been replicated.