Abstract
This chapter starts with a thorough discussion of different multiple comparison error rates, including weak and strong control for multiple tests and noncoverage probability for confidence sets. With multiple endpoints as an example, it describes which error rate controls would translate to incorrect decision rate controls. Then, using targeted therapy as the context, this chapter discusses a potential issue with some efficacy measures in terms of respecting logical relationships among the subgroups. A statistical principle that helps avoid this issue is described. As another example of multiplicity-induced issues to be aware of, it is shown that permutation test for patient targeting may not control Type I error rate in some situations. Finally, a list of the key points and a summary of the conclusions are given.
This is a preview of subscription content, log in via an institution to check access.
References
Buyse M, Péron J (2020) Generalized pairwise comparisons for prioritized outcomes. In: Principles and practice of clinical trials. Springer Nature, Cham
Cox DR, Oakes D (1984) Analysis of Survival Data. Chapman and Hall
Ding Y, Lin H-M, Hsu JC (2016) Subgroup mixable inference on treatment efficacy in mixture populations, with an application to time-to-event outcomes. Stat Med 35:1580–1594
Finner H, Strassburger K (2002) The partitioning principle: a powerful tool in multiple decision theory. Ann Stat 30:1194–1213
Finner H, Strassburger K (2007) Step-up related simultaneous confidence intervals for MCC and MCB. Biom J 49(1):40–51
Greenland S, Robins JM, Pearl J (1999) Confounding and collapsibility in causal inference. Stat Sci 14(1):29–46
Hayter AJ, Hsu JC (1994) On the relationship between stepwise decision procedures and confidence sets. J Am Stat Assoc 89:128–136
Hsu JC (1996) Multiple comparisons: theory and methods. Chapman & Hall, London
Hsu JC, Berger RL (1999) Stepwise confidence intervals without multiplicity adjustment for dose response and toxicity studies. J Am Stat Assoc 94:468–482
Huang Y, Hsu JC (2007) Hochberg’s step-up method: cutting corners off Holm’s step-down method. Biometrika 22:2244–2248
Jiang W, Freidlin B, Simon R (2007) Biomarker-adaptive threshold design: a procedure for evaluating treatment with possible biomarker-defined subset effect. J Natl Cancer Inst 99:1036–1043
Kaizar EE, Li Y, Hsu JC (2011) Permutation multiple tests of binary features do not uniformly control error rates. J Am Stat Assoc 106:1067–1074
Lin H-M, Xu H, Ding Y, Hsu JC (2019) Correct and logical inference on efficacy in subgroups and their mixture for binary outcomes. Biom J 61:8–26
Lipkovich I, Dmitrienko A, Denne J, Enas G (2011) Subgroup identification based on differential effect search – a recursive partitioning method for establishing response to treatment in patient subpopulations. Stat Med 30:2601–2621
Martinussen T, Vansteelandt S, Andersen PK (2018) Subtleties in the interpretation of hazard ratios. arXiv:1810.09192v1
Miller R, Siegmund D (1982) Maximally selected Chi-square statistics. Biometrics 38:1011–1016
Stefansson G, Kim W, Hsu JC (1988) On confidence sets in multiple comparisons. In: Gupta SS, Berger JO (eds) Statistical decision theory and related topics IV, vol 2. Springer, New York, pp 89–104
Strassburger K, Bretz F (2008) Compatible simultaneous lower confidence bounds for the holm procedure and other Bonferroni-based closed tests. Stat Med 27(24):4914–4927
Takeuchi K (1973) Studies in some aspects of theoretical foundations of statistical data analysis (in Japanese). Toyo Keizai Shinposha, Tokyo
Takeuchi K (2010) Basic ideas and concepts for multiple comparison procedures. Biom J 52:722–734
Tukey JW (1953) The problem of multiple comparisons. Dittoed manuscript of 396 pages, Department of Statistics, Princeton University
Tukey JW (1994) The problem of multiple comparisons, Chapter 1. In: Braun HI (ed) The collected works of John W. Tukey, vol VIII. Chapman & Hall, New York/London, pp 1–300
Woodcock J (2015) FDA Voice. Posted 23 Mar 2015
Xu H, Hsu JC (2007) Using the partitioning principle to control the generalized family error rate. Biom J 49:52–67
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Section Editor information
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this entry
Cite this entry
Kil, S., Kaizar, E., Tang, SY., Hsu, J.C. (2020). Confident Statistical Inference with Multiple Outcomes, Subgroups, and Other Issues of Multiplicity. In: Piantadosi, S., Meinert, C. (eds) Principles and Practice of Clinical Trials. Springer, Cham. https://doi.org/10.1007/978-3-319-52677-5_116-1
Download citation
DOI: https://doi.org/10.1007/978-3-319-52677-5_116-1
Received:
Accepted:
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-52677-5
Online ISBN: 978-3-319-52677-5
eBook Packages: Springer Reference MathematicsReference Module Computer Science and Engineering