Distinguishing true from false positives in genomic studies: p values
- 816 Downloads
Distinguishing true from false positive findings is a major challenge in human genetic epidemiology. Several strategies have been devised to facilitate this, including the positive predictive value (PPV) and a set of epidemiological criteria, known as the “Venice” criteria. The PPV measures the probability of a true association, given a statistically significant finding, while the Venice criteria grade the credibility based on the amount of evidence, consistency of replication and protection from bias. A vast majority of journals use significance thresholds to identify the true positive findings. We studied the effect of p value thresholds on the PPV and used the PPV and Venice criteria to define usable thresholds of statistical significance. Theoretical and empirical analyses of data published on AlzGene show that at a nominal p value threshold of 0.05 most “positive” findings will turn out to be false if the prior probability of association is below 0.10 even if the statistical power of the study is higher than 0.80. However, in underpowered studies (0.25) with a low prior probability of 1 × 10−3, a p value of 1 × 10−5 yields a high PPV (>96 %). Here we have shown that the p value threshold of 1 × 10−5 gives a very strong evidence of association in almost all studies. However, in the case of a very high prior probability of association (0.50) a p value threshold of 0.05 may be sufficient, while for studies with very low prior probability of association (1 × 10−4; genome-wide association studies for instance) 1 × 10−7 may serve as a useful threshold to declare significance.
KeywordsVenice Criteria Significance thresholds “-Omics” Alzheimer’s disease
The study was supported by grants from the Centre for Medical Systems Biology (CMSB) and ENGAGE Consortium, grant agreement HEALTH-F4-2007-201413. The genetics databases used for this project have been made possible by the kind support of the Cure Alzheimer’s Fund (CAF), the Michael J. Fox Foundation (MJFF) for Parkinson’s Research, the National Alliance for Research on Schizophrenia and Depression (NARSAD), Prize4Life, and EMD Serono. C.M.L. is supported by the Fidelity Biosciences Research Initiative. L.Bertram is supported by the German Ministry for Education and Research (BMBF).
Conflict of interest
The authors declare no competing interests.
- 8.Rebbeck TR, Ambrosone CB, Bell DA, Chanock SJ, Hayes RB, Kadlubar FF, et al. SNPs, haplotypes, and cancer: applications in molecular epidemiology. Cancer Epidemiol Biomark Prev. 2004;13(5):681–7.Google Scholar
- 13.Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical. J R Statist Soc B. 1995;57(1):289–300.Google Scholar
- 21.Kavvoura FK, McQueen MB, Khoury MJ, Tanzi RE, Bertram L, Ioannidis JP. Evaluation of the potential excess of statistically significant findings in published genetic association studies: application to Alzheimer’s disease. Am J Epidemiol. 2008;168(8):855–65. doi: 10.1093/aje/kwn206.PubMedCrossRefGoogle Scholar
- 24.Rothman KJ. Epidemiology: an introduction. 1st ed. Oxford: Oxford University Press; 2002.Google Scholar
- 28.Sanna S, Li B, Mulas A, Sidore C, Kang HM, Jackson AU, et al. Fine mapping of five Loci associated with low-density lipoprotein cholesterol detects variants that double the explained heritability. PLoS Genet. 2011;7(7):e1002198. doi: 10.1371/journal.pgen.1002198PGENETICS-D-11-00557.PubMedCrossRefGoogle Scholar
- 29.Boseley S. Six men in intensive care after drug trial goes wrong. The Guardian. 2006.Google Scholar
- 30.Farrer LA, Cupples LA, Haines JL, Hyman B, Kukull WA, Mayeux R, et al. Effects of age, sex, and ethnicity on the association between apolipoprotein E genotype and Alzheimer disease. A meta-analysis. APOE and Alzheimer disease meta analysis consortium. JAMA. 1997;278(16):1349–56.PubMedCrossRefGoogle Scholar