Abstract
Multiple testing refers to any instance that involves the simultaneous testing of more than one hypothesis. If decisions about the individual hypotheses are based on the unadjusted marginal p-values, then there is typically a large probability that some of the true null hypotheses will be rejected. Unfortunately, such a course of action is still common. In this article, we describe the problem of multiple testing more formally and discuss methods which account for the multiplicity issue. In particular, recent developments based on resampling result in an improved ability to reject false hypotheses compared to classical methods such as Bonferroni.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Bibliography
Davison, A.C., and D.V. Hinkley. 1997. Bootstrap Methods and their Application. Cambridge: University Press, Cambridge.
Hansen, P.R. 2005. A test for superior predictive ability. Journal of Business and Economics Statistics 23: 365–80.
Hochberg, Y., and A. Tamhane. 1987. Multiple comparison procedures. New York: Wiley.
Holm, S. 1979. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics 6: 65–70.
Lahiri, S.N. 2003. Resampling methods for dependent data. New York: Springer.
Romano, J.P., and M. Wolf. 2005. Exact and approximate stepdown methods for multiple hypothesis testing. Journal of the American Statistical Association 100: 94–108.
Romano, J.P., A.M. Shaikh, and M. Wolf. 2008a. Control of the false discovery rate under dependence using the bootstrap and subsampling (with discussion). Test 17: 417–42.
Romano, J.P., A.M. Shaikh, and M. Wolf. 2008b. Formalized data snooping based on generalized error rates. Econometric Theory 24: 404–47.
Sarkar, S.K. 1998. Some probability inequalities for ordered M T P2 random variables: A proof of simes conjecture. Annals of Statistics 26: 494–504.
Westfall, P.H., and S.S. Young. 1993. Resampling-based multiple testing: Examples and methods for p-value adjustment. New York: Wiley.
White, H.L. 2000. A reality check for data snooping. Econometrica 68: 1097–126.
Author information
Authors and Affiliations
Editor information
Copyright information
© 2018 Macmillan Publishers Ltd.
About this entry
Cite this entry
Romano, J.P., Shaikh, A.M., Wolf, M. (2018). Multiple Testing. In: The New Palgrave Dictionary of Economics. Palgrave Macmillan, London. https://doi.org/10.1057/978-1-349-95189-5_2914
Download citation
DOI: https://doi.org/10.1057/978-1-349-95189-5_2914
Published:
Publisher Name: Palgrave Macmillan, London
Print ISBN: 978-1-349-95188-8
Online ISBN: 978-1-349-95189-5
eBook Packages: Economics and FinanceReference Module Humanities and Social SciencesReference Module Business, Economics and Social Sciences