Statistical Tools for Subset Analysis in Clinical Trials

Abstract

The topic of subset analysis is complex, yet it has evoked simplistic solutions and strong feelings as a results of these simplistic solutions. There are some clinical investigators who ransack their data to find a subset of patients in which their new therapy looks effective. They report the findings in apparent ignorance of the problems of multiple comparisons, and their statistically significant finding looks good to the mass of statistically unsophisticated readers. Many of us react strongly to this, and the phrase “subset analysis” itself acquires a connotation of deception and sophistry. We hear individuals say that it is all right to look at subset results, but don’t believe them. We hear that subset analyses can only generate hypotheses to be tested in other clinical trials. Does this mean that one must entirely ignore subset results regardless of how strong they are? Is it practical to design future studies and treat future patients on this basis? Does this also hold for clinical trials of cardiovascular disease or cancer prevention which may involve enormous numbers of patients and cost tens of millions of dollars? Is this sensible statistically?