Statistics in Brief: The Importance of Sample Size in the Planning and Interpretation of Medical Research

Abstract

The increasing volume of research by the medical community often leads to increasing numbers of contradictory findings and conclusions. Although the differences observed may represent true differences, the results also may differ because of sampling variability as all studies are performed on a limited number of specimens or patients. When planning a study reporting differences among groups of patients or describing some variable in a single group, sample size should be considered because it allows the researcher to control for the risk of reporting a false-negative finding (Type II error) or to estimate the precision his or her experiment will yield. Equally important, readers of medical journals should understand sample size because such understanding is essential to interpret the relevance of a finding with regard to their own patients. At the time of planning, the investigator must establish (1) a justifiable level of statistical significance, (2) the chances of detecting a difference of given magnitude between the groups compared, ie, the power, (3) this targeted difference (ie, effect size), and (4) the variability of the data (for quantitative data). We believe correct planning of experiments is an ethical issue of concern to the entire community.

Appendices

Appendix 1

The sample size (n) per group for comparing two means with a two-sided two-sample t test is

$$ \hbox{n} = \frac{{2 \times ({\hbox{z}}_{{1 - \upalpha /2}} + \hbox{z}_{{1 - \upbeta }} )^{2} }} {{\hbox{d}_{\rm t} ^{2} }} + 0.25 \times \hbox{z}^{2}_{{1 - \upalpha /2}} $$

where z_1−α/2 and z_1−β are standard normal deviates for the probability of 1 − α/2 and 1 − β, respectively, and d_t = (μ₀ − μ₁)/σ is the targeted standardized difference between the two means.

The following values correspond to the example:

• α = 0.05 (statistical significance level)

• β = 0.10 (power of 90%)

• |μ₀ − μ₁| = 10 (difference in the mean score between the two groups)

• σ = 20 (standard deviation of the score in each group)

• z_1−α/2 = 1.96

• z_1−β = 1.28

Therefore:

$$ \hbox{n} = \frac{{2 \times (1.96 + 1.28)^{2} }} {{(10/20)^{2} }} + 0.25 \times 1.96^{2} = 85. $$

Two-sided tests which do not assume the direction of the difference (ie, that the mean value in one group would always be greater than that in the other) are generally preferred. The null hypothesis makes the assumption that there is no difference between the treatments compared, and a difference on one side or the other therefore is expected.

Appendix 2

Computation of Confidence Interval

To determine the estimation of a parameter, or alternatively the confidence interval, we use the distribution of the parameter estimate in repeated samples of the same size. For instance, consider a parameter with observed mean, m, and standard deviation, sd, in a given sample. If we assume that the distribution of the parameter in the sample is close to a normal distribution, the means, x_n, of several repeated samples of the same size have true mean, μ, the population mean, and estimated standard deviation,

$$ {\text{se}} = {{\text{sd}}} \mathord{\left/ {\vphantom {{{\text{sd}}} {{\surd{\text{n}} }}}} \right. \kern-\nulldelimiterspace} {{\surd{\text{n}} }}, $$

(1)

also known as standard error of the mean, and

$$ {{\left( {{\text{x}}_{{\text{n}}} - \upmu } \right)}} \mathord{\left/ {\vphantom {{{\left( {{\text{x}}_{{\text{n}}} - \mu } \right)}} {{\text{se}}}}} \right. \kern-\nulldelimiterspace} {{\text{se}}} $$

(2)

follows a t distribution. For a large sample, the t distribution becomes close to the normal distribution; however, for a smaller sample size the difference is not negligible and the t distribution is preferred. The precision of the estimation is

$$ 2 \times {\text{t}}_{{{\left( {1 - \upalpha /2} \right)},\;{\text{n}} - 1}} \times {\text{se}}, $$

(3)

and the confidence interval for μ is the range of values extending either side of the sample mean m by

$$ {\text{t}}_{{{\left( {1 - \upalpha /2} \right)},\;{\text{n}} - 1}} \times {\text{se}}. $$

(4)

For example, Handl et al. [11] in a biomechanical study of 21 fresh-frozen cadavers reported a mean ultimate load failure of 4-strand hamstring tendon constructs of 4546 N under dynamic loading with standard deviation of 1500 N. If we were to plan an experiment, the anticipated precision of the estimation at the 95% level would be

$$ 2 \times {\left( {{2.78 \times 1500} \mathord{\left/ {\vphantom {{2.78 \times 1500} {{\surd 5 }}}} \right. \kern-\nulldelimiterspace} {{\surd 5 }}} \right)} = 3725 $$

(5)

for five specimens,

$$ 2 \times {\left( {{2.26 \times 1500} \mathord{\left/ {\vphantom {{2.26 \times 1500} {{\surd {10} }}}} \right. \kern-\nulldelimiterspace} {{\surd {10} }}} \right)} = 2146\;{\text{for}}\;10\;{\text{specimens}}, $$

(6)

$$ 2 \times {\left( {{2.06 \times 1500} \mathord{\left/ {\vphantom {{2.06 \times 1500} {{\surd {25} }}}} \right. \kern-\nulldelimiterspace} {{\surd {25} }}} \right)} = 1238\;{\text{for}}\;25\;{\text{specimens}}, $$

(7)

$$ 2 \times {\left( {{2.01 \times 1500} \mathord{\left/ {\vphantom {{2.01 \times 1500} {{\surd {50} }}}} \right. \kern-\nulldelimiterspace} {{\surd {50} }}} \right)} = 853\;{\text{for}}\;50\;{\text{specimens}}, $$

(8)

$$ {\text{and}}\;2 \times {\left( {{1.98 \times 1500} \mathord{\left/ {\vphantom {{1.98 \times 1500} {{\surd {100} }}}} \right. \kern-\nulldelimiterspace} {{\surd {100} }}} \right)} = 595\;{\text{for}}\;100\;{\text{specimens}}{\text{.}} $$

(9)

The values 2.78, 2.26, 2.06, 2.01, and 1.98 correspond to the t distribution deviates for the probability of 1 − α/2, with 4, 9, 24, 49, and 99 (n − 1) degrees of freedom; the well known corresponding standard normal deviate is 1.96. Given an estimated mean of 4546 N, the corresponding 95% confidence intervals are 2683 N to 6408 N for five specimens, 3473 N to 5619 N for 10 specimens, 3927 N to 5165 N for 25 specimens, 4120 N to 4972 N for 50 specimens, and 4248 N to 4844 N for 100 specimens (Fig. 2).

Similarly, for a proportion p in a given sample with sufficient sample size to assume a nearly normal distribution, the confidence interval extends either side of the proportion p by

$$ {{{\text{z}}_{{\left( {1-\upalpha/2} \right)}}}} \times {\text{se}}\;{\text{with}}\;{\text{se}} = {\surd {{{\text{p}}{\left( {1 - {\text{p}}} \right)}} \mathord{\left/ {\vphantom {{{\text{p}}{\left( {1 - {\text{p}}} \right)}} {\text{n}}}} \right. \kern-\nulldelimiterspace} {\text{n}}} }. $$

(10)

For a small sample size, exact confidence interval for proportions should be used.