Abstract
We consider comparing two treatments using a given hypothesis test on the full sample and on all possible subsets, and we separately consider restricting the subsets considered to be those defined based on half-intervals of a covariate. Rather than treating this as a family of hypothesis tests, we instead choose the minimum p-value from the group of hypothesis tests as our test statistic. Simulation is employed to find an approximate critical value to control the type I error for our novel test statistic. These techniques may be used as a rule of thumb for judging the potential significance of a result after a “fishing expedition” has been caried out on a dataset, i.e., a large number of tests of hypothesis were performed on subsets of the data or a subset was selected after inspecting the data. When the technique is restricted to subsets defined based on half-intervals of a covariate, it may be useful as a planned methodology for analyzing an experiment.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Fleiss, J. L. (1986). The Design and Analysis of Clinical Experiments, John Wiley & Sons, New York.
Hsu, J. C. (1996). Multiple Comparisons, Chapman and Hall, New York.
Koziol, J. A. and Wu, S. H. (1996). Changepoint statistics for assessing a treatment-vovariate interaction, Biometrics, 52, 1147–1152.
Mamounas, E. P. (1997). NSABP Protocol B-27: Preoperative doxorubicin plus cyclophosphamide followed by preoperative or postoperative docetaxel, Oncology, 11 (Suppl. No. 6), 37–40.
Miller R. G. (1981). Simultaneous Statistical Inference, Springer-Verlag, New York.
Potthoff, R. F. (1964). On the Johnson-Neyman technique and some extensions thereof, Psychometrika, 29, 241–256.
Worsley, K. J. (1992). A three dimensional statistical analysis for CBF activation studies in human brain, Journal of Cerebral Blood Flow and Metabolism, 12, 900–918.
Yothers, G. (2003). Methodologies for Identifying Subsets of the Population Where Two Treatments Differ, Ph.D. Dissertation, University of Pittsburgh, Pittsburgh, Pennsylvania.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Birkhäuser Boston
About this chapter
Cite this chapter
Yothers, G., Sampson, A.R. (2005). Inference Guided Data Exploration. In: Balakrishnan, N., Nagaraja, H.N., Kannan, N. (eds) Advances in Ranking and Selection, Multiple Comparisons, and Reliability. Statistics for Industry and Technology. Birkhäuser Boston. https://doi.org/10.1007/0-8176-4422-9_3
Download citation
DOI: https://doi.org/10.1007/0-8176-4422-9_3
Publisher Name: Birkhäuser Boston
Print ISBN: 978-0-8176-3232-8
Online ISBN: 978-0-8176-4422-2
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)