# How to guarantee finding a statistically significant difference: the use and abuse of subgroup analyses

- 224 Downloads
- 9 Citations

In 1988 the Lancet published a very large randomised clinical trial of intravenous streptokinase, oral aspirin, both, or neither for treatment of suspected acute myocardial infarction [1]. The ISIS-2 trial recruited 17,187 patients from 417 hospitals. The authors concluded that there were benefits both from streptokinase and from aspirin. This paper contained a complex table reporting subgroup analyses, and rather intriguingly, the first analysis was by astrological birth sign. The results suggested that for people born under the star signs Gemini and Libra, aspirin was no better than placebo; for others, aspirin had a strongly beneficial effect. Why, one might wonder, did a highly respected journal publish such arrant nonsense?

The use of patient reported outcomes (PROs) in clinical trials can lead to problems with ‘multiple testing’, that is, the testing of multiple hypotheses and the associated problem of how to interpret resultant *P* values [2]. Most commonly, this problem arises...

## Keywords

Migraine Subgroup Analysis Streptokinase Rizatriptan Interaction Test## References

- 1.ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. (1988). Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2.
*Lancet,**ii*, 349–360.Google Scholar - 2.Bland, J. M., & Altman, D. G. (1995). Multiple significance tests: The Bonferroni method.
*British Medical Journal,**310*, 170.PubMedGoogle Scholar - 3.Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance.
*Biometrika,**75*, 800–802. doi: 10.1093/biomet/75.4.800.CrossRefGoogle Scholar - 4.Sankoh, A. J., Huque, M. F., & Dubey, S. D. (1997). Some comments on frequently used multiple endpoint adjustment methods in clinical trials.
*Statistics in Medicine,**16*, 2529–2542. doi: 10.1002/(SICI)1097-0258(19971130)16:22<2529::AID-SIM692>3.0.CO;2-J.PubMedCrossRefGoogle Scholar - 5.Assmann, S. F., Pocock, S. J., Enos, L. E., & Kasten, L. E. (2000). Subgroup analysis and other (mis)uses of baseline data in clinical trials.
*Lancet,**355*, 1064–1069. doi: 10.1016/S0140-6736(00)02039-0.PubMedCrossRefGoogle Scholar - 6.Brookes, S. T., Whitely, E., Egger, M., Smith, G. D., Mulheran, P. A., & Peters, T. J. (2004). Subgroup analyses in randomized trials: Risks of subgroup-specific analyses; power and sample size for the interaction test.
*Journal of Clinical Epidemiology,**57*, 229–236. doi: 10.1016/j.jclinepi.2003.08.009.PubMedCrossRefGoogle Scholar - 7.Horton, R. (2000). Commentary: From star signs to trial guidelines.
*Lancet,**355*, 1033–1034. doi: 10.1016/S0140-6736(00)02031-6.PubMedCrossRefGoogle Scholar - 8.Peto, R. (1990). Misleading subgroup analyses in GISSI.
*The American Journal of Cardiology,**66*, 771–772. doi: 10.1016/0002-9149(90)91149-Z.PubMedCrossRefGoogle Scholar - 9.Yusuf, S., Wittes, J., Probstfield, J., & Tyroler, H. A. (1991). Analysis and interpretation of treatment effects in subgroups of patients in randomised clinical trials.
*Journal of the American Medical Association,**266*, 93–98. doi: 10.1001/jama.266.1.93.PubMedCrossRefGoogle Scholar - 10.Brookes, S. T., Whitley, E., Peters, T. J., Mulheran, P. A., Egger, M., & Davey Smith, G. (2001). Subgroup analyses in randomised controlled trials: Quantifying the risks of false-positives and false-negatives.
*Health Technology Assessment,**5*(33), 1–56. From http://www.hta.nhs.uk/fullmono/mon533.pdf. Accessed 12 March 2009. - 11.Grouin, J.-M., Coste, M., & Lewis, J. (2005). Subgroup analyses in randomized clinical trials: Statistical and regulatory issues.
*Journal of Biopharmaceutical Statistics,**15*, 869–882. doi: 10.1081/BIP-200067988.PubMedCrossRefGoogle Scholar - 12.Wang, R., Lagakos, S. W., Ware, J. H., Hunter, D. J., & Drazen, J. M. (2007). Statistics in medicine—reporting of subgroup analyses in clinical trials.
*The New England Journal of Medicine,**357*, 2189–2194. doi: 10.1056/NEJMsr077003.PubMedCrossRefGoogle Scholar - 13.Martin, V., Cady, R., Mauskop, A., Seidman, L. S., Rodgers, A., Hustard, C. M., et al. (2008). Efficacy of rizatriptan for menstrual migraine in an early intervention model: A prospective subgroup analysis of the rizatriptan TAME (Treat a Migraine Early) studies.
*Headache,**48*, 226–235.PubMedCrossRefGoogle Scholar