How to guarantee finding a statistically significant difference: the use and abuse of subgroup analyses
Rent the article at a discountRent now
* Final gross prices may vary according to local VAT.Get Access
In 1988 the Lancet published a very large randomised clinical trial of intravenous streptokinase, oral aspirin, both, or neither for treatment of suspected acute myocardial infarction . The ISIS-2 trial recruited 17,187 patients from 417 hospitals. The authors concluded that there were benefits both from streptokinase and from aspirin. This paper contained a complex table reporting subgroup analyses, and rather intriguingly, the first analysis was by astrological birth sign. The results suggested that for people born under the star signs Gemini and Libra, aspirin was no better than placebo; for others, aspirin had a strongly beneficial effect. Why, one might wonder, did a highly respected journal publish such arrant nonsense?
The use of patient reported outcomes (PROs) in clinical trials can lead to problems with ‘multiple testing’, that is, the testing of multiple hypotheses and the associated problem of how to interpret resultant P values . Most commonly, this problem arises fro ...
- ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. (1988). Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2. Lancet, ii, 349–360.
- Bland, J. M., & Altman, D. G. (1995). Multiple significance tests: The Bonferroni method. British Medical Journal, 310, 170.
- Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance. Biometrika, 75, 800–802. doi:10.1093/biomet/75.4.800. CrossRef
- Sankoh, A. J., Huque, M. F., & Dubey, S. D. (1997). Some comments on frequently used multiple endpoint adjustment methods in clinical trials. Statistics in Medicine, 16, 2529–2542. doi:10.1002/(SICI)1097-0258(19971130)16:22<2529::AID-SIM692>3.0.CO;2-J. CrossRef
- Assmann, S. F., Pocock, S. J., Enos, L. E., & Kasten, L. E. (2000). Subgroup analysis and other (mis)uses of baseline data in clinical trials. Lancet, 355, 1064–1069. doi:10.1016/S0140-6736(00)02039-0. CrossRef
- Brookes, S. T., Whitely, E., Egger, M., Smith, G. D., Mulheran, P. A., & Peters, T. J. (2004). Subgroup analyses in randomized trials: Risks of subgroup-specific analyses; power and sample size for the interaction test. Journal of Clinical Epidemiology, 57, 229–236. doi:10.1016/j.jclinepi.2003.08.009. CrossRef
- Horton, R. (2000). Commentary: From star signs to trial guidelines. Lancet, 355, 1033–1034. doi:10.1016/S0140-6736(00)02031-6. CrossRef
- Peto, R. (1990). Misleading subgroup analyses in GISSI. The American Journal of Cardiology, 66, 771–772. doi:10.1016/0002-9149(90)91149-Z. CrossRef
- Yusuf, S., Wittes, J., Probstfield, J., & Tyroler, H. A. (1991). Analysis and interpretation of treatment effects in subgroups of patients in randomised clinical trials. Journal of the American Medical Association, 266, 93–98. doi:10.1001/jama.266.1.93. CrossRef
- Brookes, S. T., Whitley, E., Peters, T. J., Mulheran, P. A., Egger, M., & Davey Smith, G. (2001). Subgroup analyses in randomised controlled trials: Quantifying the risks of false-positives and false-negatives. Health Technology Assessment, 5(33), 1–56. From http://www.hta.nhs.uk/fullmono/mon533.pdf. Accessed 12 March 2009.
- Grouin, J.-M., Coste, M., & Lewis, J. (2005). Subgroup analyses in randomized clinical trials: Statistical and regulatory issues. Journal of Biopharmaceutical Statistics, 15, 869–882. doi:10.1081/BIP-200067988. CrossRef
- Wang, R., Lagakos, S. W., Ware, J. H., Hunter, D. J., & Drazen, J. M. (2007). Statistics in medicine—reporting of subgroup analyses in clinical trials. The New England Journal of Medicine, 357, 2189–2194. doi:10.1056/NEJMsr077003. CrossRef
- Martin, V., Cady, R., Mauskop, A., Seidman, L. S., Rodgers, A., Hustard, C. M., et al. (2008). Efficacy of rizatriptan for menstrual migraine in an early intervention model: A prospective subgroup analysis of the rizatriptan TAME (Treat a Migraine Early) studies. Headache, 48, 226–235. CrossRef
- How to guarantee finding a statistically significant difference: the use and abuse of subgroup analyses
Quality of Life Research
Volume 18, Issue 5 , pp 527-530
- Cover Date
- Print ISSN
- Online ISSN
- Springer Netherlands
- Additional Links
- Industry Sectors
- Author Affiliations
- 1. Institute of Applied Health Sciences, University of Aberdeen Medical School, Foresterhill, Aberdeen, AB25 2ZD, UK
- 2. Department of Cancer Research and Molecular Medicine, Faculty of Medicine, Norwegian University of Science and Technology, Trondheim, Norway
- 3. Quality of Life Office, Psycho-Oncology Cooperative Research Group, School of Psychology, University of Sydney, Sydney, NSW, Australia