European Journal of Epidemiology

, Volume 28, Issue 2, pp 131–138

Distinguishing true from false positives in genomic studies: p values

  • Linda Broer
  • Christina M. Lill
  • Maaike Schuur
  • Najaf Amin
  • Johannes T. Roehr
  • Lars Bertram
  • John P. A. Ioannidis
  • Cornelia M. van Duijn
METHODS

DOI: 10.1007/s10654-012-9755-x

Cite this article as:
Broer, L., Lill, C.M., Schuur, M. et al. Eur J Epidemiol (2013) 28: 131. doi:10.1007/s10654-012-9755-x

Abstract

Distinguishing true from false positive findings is a major challenge in human genetic epidemiology. Several strategies have been devised to facilitate this, including the positive predictive value (PPV) and a set of epidemiological criteria, known as the “Venice” criteria. The PPV measures the probability of a true association, given a statistically significant finding, while the Venice criteria grade the credibility based on the amount of evidence, consistency of replication and protection from bias. A vast majority of journals use significance thresholds to identify the true positive findings. We studied the effect of p value thresholds on the PPV and used the PPV and Venice criteria to define usable thresholds of statistical significance. Theoretical and empirical analyses of data published on AlzGene show that at a nominal p value threshold of 0.05 most “positive” findings will turn out to be false if the prior probability of association is below 0.10 even if the statistical power of the study is higher than 0.80. However, in underpowered studies (0.25) with a low prior probability of 1 × 10−3, a p value of 1 × 10−5 yields a high PPV (>96 %). Here we have shown that the p value threshold of 1 × 10−5 gives a very strong evidence of association in almost all studies. However, in the case of a very high prior probability of association (0.50) a p value threshold of 0.05 may be sufficient, while for studies with very low prior probability of association (1 × 10−4; genome-wide association studies for instance) 1 × 10−7 may serve as a useful threshold to declare significance.

Keywords

Venice Criteria Significance thresholds “-Omics” Alzheimer’s disease 

Supplementary material

10654_2012_9755_MOESM1_ESM.doc (364 kb)
Supplementary material 1 (DOC 364 kb)

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  • Linda Broer
    • 1
  • Christina M. Lill
    • 2
    • 3
  • Maaike Schuur
    • 4
  • Najaf Amin
    • 1
  • Johannes T. Roehr
    • 2
  • Lars Bertram
    • 2
  • John P. A. Ioannidis
    • 5
    • 6
    • 7
    • 8
  • Cornelia M. van Duijn
    • 1
  1. 1.Department of EpidemiologyErasmus University Medical Center RotterdamRotterdamThe Netherlands
  2. 2.Neuropsychiatric Genetics Group, Department of Vertebrate GenomicsMax Planck Institute for Molecular GeneticsBerlinGermany
  3. 3.Department of NeurologyUniversity Medical Center of the Johannes Gutenberg-UniversityMainzGermany
  4. 4.Department of NeurologyErasmus University Medical Center RotterdamRotterdamThe Netherlands
  5. 5.Department of MedicineStanford Prevention Research CenterStanfordUSA
  6. 6.Department of Health Research and PolicyStanford University School of MedicineStanfordUSA
  7. 7.Department of StatisticsStanford University School of Humanities and SciencesStanfordUSA
  8. 8.Department of Hygiene and EpidemiologyUniversity of Ioannina School of MedicineIoanninaGreece

Personalised recommendations