Skip to main content
Log in

Distinguishing true from false positives in genomic studies: p values

  • Published:
European Journal of Epidemiology Aims and scope Submit manuscript


Distinguishing true from false positive findings is a major challenge in human genetic epidemiology. Several strategies have been devised to facilitate this, including the positive predictive value (PPV) and a set of epidemiological criteria, known as the “Venice” criteria. The PPV measures the probability of a true association, given a statistically significant finding, while the Venice criteria grade the credibility based on the amount of evidence, consistency of replication and protection from bias. A vast majority of journals use significance thresholds to identify the true positive findings. We studied the effect of p value thresholds on the PPV and used the PPV and Venice criteria to define usable thresholds of statistical significance. Theoretical and empirical analyses of data published on AlzGene show that at a nominal p value threshold of 0.05 most “positive” findings will turn out to be false if the prior probability of association is below 0.10 even if the statistical power of the study is higher than 0.80. However, in underpowered studies (0.25) with a low prior probability of 1 × 10−3, a p value of 1 × 10−5 yields a high PPV (>96 %). Here we have shown that the p value threshold of 1 × 10−5 gives a very strong evidence of association in almost all studies. However, in the case of a very high prior probability of association (0.50) a p value threshold of 0.05 may be sufficient, while for studies with very low prior probability of association (1 × 10−4; genome-wide association studies for instance) 1 × 10−7 may serve as a useful threshold to declare significance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Similar content being viewed by others


  1. Hirschhorn JN, Lohmueller K, Byrne E, Hirschhorn K. A comprehensive review of genetic association studies. Genet Med. 2002;4(2):45–61.

    Article  PubMed  CAS  Google Scholar 

  2. Abou-Sleiman PM, Hanna MG, Wood NW. Genetic association studies of complex neurological diseases. J Neurol Neurosurg Psychiatr. 2006;77(12):1302–4. doi:10.1136/jnnp.2005.082024.

    Article  PubMed  CAS  Google Scholar 

  3. Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2(8):e124. doi:10.1371/journal.pmed.0020124.

    Article  PubMed  Google Scholar 

  4. Wacholder S, Chanock S, Garcia-Closas M, El Ghormli L, Rothman N. Assessing the probability that a positive report is false: an approach for molecular epidemiology studies. J Natl Cancer Inst. 2004;96(6):434–42.

    Article  PubMed  Google Scholar 

  5. Weitkunat R, Kaelin E, Vuillaume G, Kallischnigg G. Effectiveness of strategies to increase the validity of findings from association studies: size versus replication. BMC Med Res Methodol. 2010;10:47. doi:10.1186/1471-2288-10-47.

    Article  PubMed  Google Scholar 

  6. Lucke JF. A critique of the false-positive report probability. Genet Epidemiol. 2009;33(2):145–50. doi:10.1002/gepi.20363.

    Article  PubMed  Google Scholar 

  7. Matullo G, Berwick M, Vineis P. Gene-environment interactions: how many false positives? J Natl Cancer Inst. 2005;97(8):550–1. doi:10.1093/jnci/dji122.

    Article  PubMed  Google Scholar 

  8. Rebbeck TR, Ambrosone CB, Bell DA, Chanock SJ, Hayes RB, Kadlubar FF, et al. SNPs, haplotypes, and cancer: applications in molecular epidemiology. Cancer Epidemiol Biomark Prev. 2004;13(5):681–7.

    CAS  Google Scholar 

  9. Thomas DC, Clayton DG. Betting odds and genetic associations. J Natl Cancer Inst. 2004;96(6):421–3.

    Article  PubMed  Google Scholar 

  10. Moonesinghe R, Khoury MJ, Janssens AC. Most published research findings are false-but a little replication goes a long way. PLoS Med. 2007;4(2):e28. doi:10.1371/journal.pmed.0040028.

    Article  PubMed  Google Scholar 

  11. Ioannidis JP, Boffetta P, Little J, O’Brien TR, Uitterlinden AG, Vineis P, et al. Assessment of cumulative evidence on genetic associations: interim guidelines. Int J Epidemiol. 2008;37(1):120–32. doi:10.1093/ije/dym159.

    Article  PubMed  Google Scholar 

  12. International HapMap C. The international HapMap project. Nature. 2003;426(6968):789–96. doi:10.1038/nature02168nature02168.

    Article  Google Scholar 

  13. Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical. J R Statist Soc B. 1995;57(1):289–300.

    Google Scholar 

  14. Gordon A, Glazko G, Qiu X, Yakovlev A. Control of the mean number of false discoveries, Bonferroni and stability of multiple testing. Ann Appl Stat. 2007;1:179–90.

    Article  Google Scholar 

  15. Khoury MJ, Bertram L, Boffetta P, Butterworth AS, Chanock SJ, Dolan SM, et al. Genome-wide association studies, field synopses, and the development of the knowledge base on genetic variation and human diseases. Am J Epidemiol. 2009;170(3):269–79. doi:10.1093/aje/kwp119.

    Article  PubMed  Google Scholar 

  16. Ridley J, Kolm N, Freckelton RP, Gage MJ. An unexpected influence of widely used significance thresholds on the distribution of reported p values. J Evol Biol. 2007;20(3):1082–9. doi:10.1111/j.1420-9101.2006.01291.x.

    Article  PubMed  CAS  Google Scholar 

  17. Bertram L, McQueen MB, Mullin K, Blacker D, Tanzi RE. Systematic meta-analyses of Alzheimer disease genetic association studies: the AlzGene database. Nat Genet. 2007;39(1):17–23. doi:10.1038/ng1934.

    Article  PubMed  CAS  Google Scholar 

  18. Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557–60. doi:10.1136/bmj.327.7414.557327/7414/557.

    Article  PubMed  Google Scholar 

  19. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7(3):177–88. doi:0197-2456(86)90046-2.

    Article  PubMed  CAS  Google Scholar 

  20. Harbord RM, Egger M, Sterne JA. A modified test for small-study effects in meta-analyses of controlled trials with binary endpoints. Stat Med. 2006;25(20):3443–57. doi:10.1002/sim.2380.

    Article  PubMed  Google Scholar 

  21. Kavvoura FK, McQueen MB, Khoury MJ, Tanzi RE, Bertram L, Ioannidis JP. Evaluation of the potential excess of statistically significant findings in published genetic association studies: application to Alzheimer’s disease. Am J Epidemiol. 2008;168(8):855–65. doi:10.1093/aje/kwn206.

    Article  PubMed  Google Scholar 

  22. Ioannidis JP. Calibration of credibility of agnostic genome-wide associations. Am J Med Genet B Neuropsychiatr Genet. 2008;147B(6):964–72. doi:10.1002/ajmg.b.30721.

    Article  PubMed  Google Scholar 

  23. Ioannidis JP, Tarone R, McLaughlin JK. The false-positive to false-negative ratio in epidemiologic studies. Epidemiology. 2011;22(4):450–6. doi:10.1097/EDE.0b013e31821b506e.

    Article  PubMed  Google Scholar 

  24. Rothman KJ. Epidemiology: an introduction. 1st ed. Oxford: Oxford University Press; 2002.

    Google Scholar 

  25. Panagiotou OA, Ioannidis JP, Genome-Wide Significance Project. What should the genome-wide significance threshold be? Empirical replication of borderline genetic associations. Int J Epidemiol. 2011;. doi:10.1093/ije/dyr178.

    PubMed  Google Scholar 

  26. Biotechnology Kaiser J. Researcher, two universities sued over validity of prostate cancer test. Science. 2009;325(5947):1484. doi:10.1126/science.325_1484.

    Article  Google Scholar 

  27. van Duijn CM. STROBE-ME too! Eur J Epidemiol. 2011;26(10):761–2. doi:10.1007/s10654-011-9628-8.

    Article  PubMed  Google Scholar 

  28. Sanna S, Li B, Mulas A, Sidore C, Kang HM, Jackson AU, et al. Fine mapping of five Loci associated with low-density lipoprotein cholesterol detects variants that double the explained heritability. PLoS Genet. 2011;7(7):e1002198. doi:10.1371/journal.pgen.1002198PGENETICS-D-11-00557.

    Article  PubMed  CAS  Google Scholar 

  29. Boseley S. Six men in intensive care after drug trial goes wrong. The Guardian. 2006.

  30. Farrer LA, Cupples LA, Haines JL, Hyman B, Kukull WA, Mayeux R, et al. Effects of age, sex, and ethnicity on the association between apolipoprotein E genotype and Alzheimer disease. A meta-analysis. APOE and Alzheimer disease meta analysis consortium. JAMA. 1997;278(16):1349–56.

    Article  PubMed  CAS  Google Scholar 

Download references


The study was supported by grants from the Centre for Medical Systems Biology (CMSB) and ENGAGE Consortium, grant agreement HEALTH-F4-2007-201413. The genetics databases used for this project have been made possible by the kind support of the Cure Alzheimer’s Fund (CAF), the Michael J. Fox Foundation (MJFF) for Parkinson’s Research, the National Alliance for Research on Schizophrenia and Depression (NARSAD), Prize4Life, and EMD Serono. C.M.L. is supported by the Fidelity Biosciences Research Initiative. L.Bertram is supported by the German Ministry for Education and Research (BMBF).

Conflict of interest

The authors declare no competing interests.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Cornelia M. van Duijn.

Additional information

Linda Broer, Christina M. Lill, Maaike Schuur are shared first author.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOC 364 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Broer, L., Lill, C.M., Schuur, M. et al. Distinguishing true from false positives in genomic studies: p values. Eur J Epidemiol 28, 131–138 (2013).

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: