To the Editors:

Abu-Elneel et al. [1] compare microRNA expression levels in autistic individuals to those of a control group. Unfortunately, their results are compromised by an inappropriate statistical method.

For 227 miRNAs, Abu-Elneel et al. fit normal distributions for expression levels among the control group and confirm, with the Kolmogorov–Smirnov test, that the observed distributions are consistent with normal distributions. Of course, the means and standard deviations are estimated using the data on the 13 controls. They then check whether miRNAs from individual autistic subjects are dysregulated compared to the control group. To do so, they use a statistic defined, for miRNA i and individual j, by (my notation)

$$t_{{ij}} = {{\left( {{\left( {{\text{miRNA}}_{i} \;{\text{level}}\;{\text{in autistic}}\;{\text{individual}}\;j} \right)} - {\left( {{\text{mean}}\;{\text{miRNA}}_{i} \;{\text{in}}\;{\text{control}}\;{\text{group}}} \right)}} \right)}} \mathord{\left/ {\vphantom {{{\left( {{\left( {{\text{miRNA}}_{i} \;{\text{level}}\;{\text{in autistic}}\;{\text{individual}}\;j} \right)} - {\left( {{\text{mean}}\;{\text{miRNA}}_{i} \;{\text{in}}\;{\text{control}}\;{\text{group}}} \right)}} \right)}} {{\left( {{\text{standard}}\;{\text{deviation}}\;{\text{of}}\;{\text{miRNA}}_{i} \;{\text{in control}}\;{\text{group}}} \right)}.}}} \right. \kern-\nulldelimiterspace} {{\left( {{\text{standard}}\;{\text{deviation}}\;{\text{of}}\;{\text{miRNA}}_{i} \;{\text{in control}}\;{\text{group}}} \right)}.} $$

Unfortunately, the authors compare this statistic to the standard normal distribution. With a Bonferroni correction for 227 miRNAs × 13 individuals and a two-sided test, the standard normal distribution gives a critical value of 4.30 for | t ij  |. (The authors appear to have applied a one-sided test, giving a critical value of 4.15, but that is secondary to the issue here.) However, the use of the normal distribution requires the standard deviation to be known, not estimated. When the standard deviation is estimated, Student’s t distribution should be used.

In essence, Abu-Elneel et al. have actually performed 2,951 pooled-error two-sided two-sample t tests where only one sample has non-zero degrees of freedom. Based on the t distribution with 12 degrees of freedom, the Bonferroni-corrected critical value is actually 6.88. This leads not to 28 dysregulated miRNAs over nine out of 13 individuals, as reported in Abu-Elneel et al., but five dysregulated miRNAs in two individuals. Permutation testing suggests that even that result may be over optimistic.