Human Genetics

, Volume 125, Issue 2, pp 131–151

Strengthening the reporting of genetic association studies (STREGA): an extension of the STROBE Statement

  • Julian Little
  • Julian P. T. Higgins
  • John P. A. Ioannidis
  • David Moher
  • France Gagnon
  • Erik von Elm
  • Muin J. Khoury
  • Barbara Cohen
  • George Davey-Smith
  • Jeremy Grimshaw
  • Paul Scheet
  • Marta Gwinn
  • Robin E. Williamson
  • Guang Yong Zou
  • Kim Hutchings
  • Candice Y. Johnson
  • Valerie Tait
  • Miriam Wiens
  • Jean Golding
  • Cornelia van Duijn
  • John McLaughlin
  • Andrew Paterson
  • George Wells
  • Isabel Fortier
  • Matthew Freedman
  • Maja Zecevic
  • Richard King
  • Claire Infante-Rivard
  • Alex Stewart
  • Nick Birkett
Open AccessOriginal Investigation

DOI: 10.1007/s00439-008-0592-7

Cite this article as:
Little, J., Higgins, J.P.T., Ioannidis, J.P.A. et al. Hum Genet (2009) 125: 131. doi:10.1007/s00439-008-0592-7

Abstract

Making sense of rapidly evolving evidence on genetic associations is crucial to making genuine advances in human genomics and the eventual integration of this information in the practice of medicine and public health. Assessment of the strengths and weaknesses of this evidence, and hence the ability to synthesize it, has been limited by inadequate reporting of results. The STrengthening the REporting of Genetic Association studies (STREGA) initiative builds on the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement and provides additions to 12 of the 22 items on the STROBE checklist. The additions concern population stratification, genotyping errors, modeling haplotype variation, Hardy–Weinberg equilibrium, replication, selection of participants, rationale for choice of genes and variants, treatment effects in studying quantitative traits, statistical methods, relatedness, reporting of descriptive and outcome data, and the volume of data issues that are important to consider in genetic association studies. The STREGA recommendations do not prescribe or dictate how a genetic association study should be designed but seek to enhance the transparency of its reporting, regardless of choices made during design, conduct, or analysis.

Keywords

Gene-disease associations Genetics Gene-environment interaction Systematic review Meta-analysis Reporting recommendations Epidemiology Genome-wide association 

Introduction

The rapidly evolving evidence on genetic associations is crucial to integrating human genomics into the practice of medicine and public health (Khoury et al. 2004; Genomics Health and Society Working Group 2004). Genetic factors are likely to affect the occurrence of numerous common diseases, and therefore identifying and characterizing the associated risk (or protection) will be important in improving the understanding of etiology and potentially for developing interventions based on genetic information. The number of publications on the associations between genes and diseases has increased tremendously; with more than 34,000 published articles, the annual number has more than doubled between 2001 and 2008 (Lin et al. 2006; Yu et al. 2008). Articles on genetic associations have been published in about 1,500 journals and in several languages.

Despite many similarities between genetic association studies and “classical” observational epidemiologic studies (that is, cross-sectional, case–control, and cohort) of lifestyle and environmental factors, genetic association studies present several specific challenges including an unprecedented volume of new data (Lawrence et al. 2005; Thomas 2006) and the likelihood of very small individual effects. Genes may operate in complex pathways with gene-environment and gene–gene interactions (Khoury et al. 2007). Moreover, the current evidence base on gene-disease associations is fraught with methodological problems (Little et al. 2003; Ioannidis et al. 2005, 2006). Inadequate reporting of results, even from well-conducted studies, hampers assessment of a study’s strengths and weaknesses, and hence the integration of evidence (von Elm and Egger 2004).

Although several commentaries on the conduct, appraisal and/or reporting of genetic association studies have so far been published (Nature Genetics 1999; Cardon and Bell 2001; Weiss 2001; Weiss et al. 2001; Cooper et al. 2002; Hegele 2002; Little et al. 2002; Romero et al. 2002; Colhoun et al. 2003; van Duijn and Porta 2003; Crossman and Watkins 2004; Huizinga et al. 2004; Little 2004; Rebbeck et al. 2004; Tan et al. 2004; Anonymous 2005; Ehm et al. 2005; Freimer and Sabatti 2005; Hattersley and McCarthy 2005; Manly 2005; Shen et al. 2005; Vitali and Randolph 2005; Wedzicha and Hall 2005; Hall and Blakey 2005; DeLisi and Faraone 2006; Saito et al. 2006; Uhlig et al. 2007; NCI-NHGRI Working Group on Replication in Association Studies et al. 2007), their recommendations differ. For example, some papers suggest that replication of findings should be part of the publication (Nature Genetics 1999; Cardon and Bell 2001; Cooper et al. 2002; Hegele 2002; Huizinga et al. 2004; Tan et al. 2004; Wedzicha and Hall 2005; Hall and Blakey 2005; DeLisi and Faraone 2006), whereas others consider this suggestion unnecessary or even unreasonable (van Duijn and Porta 2003; Begg 2005; Byrnes et al. 2005; Pharoah et al. 2005; Wacholder 2005; Whittemore 2005). In many publications, the guidance has focused on genetic association studies of specific diseases (Weiss 2001; Weiss et al. 2001; Hegele 2002; Romero et al. 2002; Crossman and Watkins 2004; Huizinga et al. 2004; Rebbeck et al. 2004; Tan et al. 2004; Manly 2005; Shen et al. 2005; Vitali and Randolph 2005; Wedzicha and Hall 2005; Hall and Blakey 2005; DeLisi and Faraone 2006; Saito et al. 2006; Uhlig et al. 2007) or the design and conduct of genetic association studies (Cardon and Bell 2001; Weiss 2001; Weiss et al. 2001; Hegele 2002; Romero et al. 2002; Colhoun et al. 2003; Crossman and Watkins 2004; Huizinga et al. 2004; Rebbeck et al. 2004; Hattersley and McCarthy 2005; Manly 2005; Shen et al. 2005; Hall and Blakey 2005; DeLisi and Faraone 2006) rather than on the quality of the reporting.

Despite increasing recognition of these problems, the quality of reporting genetic association studies needs to be improved (Bogardus et al. 1999; Peters et al. 2003; Clark and Baudouin 2006; Lee et al. 2007; Yesupriya et al. 2008). For example, an assessment of a random sample of 315 genetic association studies published from 2001 to 2003 found that most studies provided some qualitative descriptions of the study participants (for example, origin and enrollment criteria), but reporting of quantitative descriptors such as age and sex was variable (Yesupriya et al. 2008). In addition, completeness of reporting of methods that allow readers to assess potential biases (for example, number of exclusions or number of samples that could not be genotyped) varied (Yesupriya et al. 2008). Only some studies described methods to validate genotyping or mentioned whether research staff was blinded to outcome. The same problems persisted in a smaller sample of studies published in 2006 (Yesupriya et al. 2008). Lack of transparency and incomplete reporting have raised concerns in a range of health research fields (von Elm and Egger 2004; Reid et al. 1995; Brazma et al. 2001; Pocock et al. 2004; Altman and Moher 2005) and poor reporting has been associated with biased estimates of effects in clinical intervention studies (Gluud 2006).

The main goal of this article is to propose and justify a set of guiding principles for reporting results of genetic association studies. The epidemiology community has recently developed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement for cross-sectional, case–control, and cohort studies (von Elm et al. 2007; Vandenbroucke et al. 2007). Given the relevance of general epidemiologic principles for genetic association studies, we propose recommendations in an extension of the STROBE statement called the STrengthening the REporting of Genetic Association studies (STREGA) Statement. The recommendations of the STROBE Statement have a strong foundation because they are based on the empirical evidence on the reporting of observational studies, and they involved extensive consultations in the epidemiologic research community (Vandenbroucke et al. 2007). We have sought to identify gaps and areas of controversy in the evidence regarding potential biases in genetic association studies. With the recommendations, we have indicated available empirical or theoretical work that has demonstrated or suggested that a methodological feature of a study can influence the direction or magnitude of the association observed. We acknowledge that for many items, no such evidence exists. The intended audience for the reporting guideline is broad and includes epidemiologists, geneticists, statisticians, clinician scientists, and laboratory-based investigators who undertake genetic association studies. In addition, it includes “users” of such studies who wish to understand the basic premise, design, and limitations of genetic association studies in order to interpret the results. The field of genetic associations is evolving very rapidly with the advent of genome-wide association investigations, high-throughput platforms assessing genetic variability beyond common single-nucleotide polymorphisms (SNPs) (for example, copy number variants, rare variants), and eventually routine full sequencing of samples from large populations. Our recommendations are not intended to support or oppose the choice of any particular study design or method. Instead, they are intended to maximize the transparency, quality and completeness of reporting of what was done and found in a particular study.

Methods

A multidisciplinary group developed the STREGA Statement using literature review, workshop presentations and discussion, and iterative electronic correspondence after the workshop. Thirty-three of 74 invitees participated in the STREGA workshop in Ottawa, Ontario, Canada, in June, 2006. Participants included epidemiologists, geneticists, statisticians, journal editors, and graduate students.

Before the workshop, an electronic search was performed to identify existing reporting guidance for genetic association studies. Workshop participants were also asked to identify any additional guidance. They prepared brief presentations on existing reporting guidelines, empirical evidence on reporting of genetic association studies, the development of the STROBE Statement, and several key areas for discussion that were identified on the basis of consultations before the workshop. These areas included the selection and participation of study participants, rationale for choice of genes and variants investigated, genotyping errors, methods for inferring haplotypes, population stratification, assessment of Hardy–Weinberg equilibrium (HWE), multiple testing, reporting of quantitative (continuous) outcomes, selectively reporting study results, joint effects and inference of causation in single studies. Additional resources to inform workshop participants were the HuGENet handbook (Little and Higgins 2006; Higgins et al. 2007), examples of data extraction forms from systematic reviews or meta-analyses, articles on guideline development (Altman et al. 2001; Moher et al. 2001) and the checklists developed for STROBE. To harmonize our recommendations for genetic association studies with those for observational epidemiologic studies, we communicated with the STROBE group during the development process and sought their comments on the STREGA draft documents. We also provided comments on the developing STROBE Statement and its associated explanation and elaboration document (Vandenbroucke et al. 2007).

Results

In Table 1, we present the STREGA recommendations, an extension to the STROBE checklist (von Elm et al. 2007) for genetic association studies. The resulting STREGA checklist provides additions to 12 of the 22 items on the STROBE checklist. During the workshop and subsequent consultations, we identified five main areas of special interest that are specific to, or especially relevant in, genetic association studies: genotyping errors, population stratification, modeling haplotype variation, HWE, and replication. We elaborate on each of these areas, starting each section with the corresponding STREGA recommendation, followed by a brief outline of the issue and an explanation for the recommendations. Complementary information on these areas and the rationale for additional STREGA recommendations relating to selection of participants, choice of genes and variants selected, treatment effects in studying quantitative traits, statistical methods, relatedness, reporting of descriptive and outcome data, and issues of data volume, are presented in Table 2.
Table 1

STREGA reporting recommendations, extended from STROBE Statement

Item

Item number

STROBE guideline

Extension for Genetic Association Studies (STREGA)

Title and Abstract

1

(a) Indicate the study’s design with a commonly used term in the title or the abstract

 

(b) Provide in the abstract an informative and balanced summary of what was done and what was found

 

Introduction

 

 Background rationale

2

Explain the scientific background and rationale for the investigation being reported

 

 Objectives

3

State specific objectives, including any pre-specified hypotheses

State if the study is the first report of a genetic association, a replication effort, or both

Methods

 

 Study design

4

Present key elements of study design early in the paper

 

 Setting

5

Describe the setting, locations and relevant dates, including periods of recruitment, exposure, follow-up, and data collection

 

 Participants

6

(a) Cohort study: give the eligibility criteria, and the sources and methods of selection of participants. Describe methods of follow-up

Case–control study: give the eligibility criteria, and the sources and methods of case ascertainment and control selection. Give the rationale for the choice of cases and controls

Cross-sectional study: give the eligibility criteria, and the sources and methods of selection of participants

Give information on the criteria and methods for selection of subsets of participants from a larger study, when relevant

(b) Cohort study: for matched studies, give matching criteria and number of exposed and unexposed

Case–control study: for matched studies, give matching criteria and the number of controls per case

 

 Variables

7

(a) Clearly define all outcomes, exposures, predictors, potential confounders, and effect modifiers. Give diagnostic criteria, if applicable

(b) Clearly define genetic exposures (genetic variants) using a widely-used nomenclature system. Identify variables likely to be associated with population stratification (confounding by ethnic origin)

 Data sources/measurement

8a

(a) For each variable of interest, give sources of data and details of methods of assessment (measurement). Describe comparability of assessment methods if there is more than one group

(b) Describe laboratory methods, including source and storage of DNA, genotyping methods and platforms (including the allele calling algorithm used, and its version), error rates and call rates. State the laboratory/center where genotyping was done. Describe comparability of laboratory methods if there is more than one group. Specify whether genotypes were assigned using all of the data from the study simultaneously or in smaller batches

 Bias

9

(a) Describe any efforts to address potential sources of bias

(b) For quantitative outcome variables, specify if any investigation of potential bias resulting from pharmacotherapy was undertaken. If relevant, describe the nature and magnitude of the potential bias, and explain what approach was used to deal with this

 Study size

10

Explain how the study size was arrived at

 

 Quantitative variables

11

Explain how quantitative variables were handled in the analyses. If applicable, describe which groupings were chosen, and why

If applicable, describe how effects of treatment were dealt with

 Statistical methods

12

(a) Describe all statistical methods, including those used to control for confounding

State software version used and options (or settings) chosen

(b) Describe any methods used to examine subgroups and interactions

 

(c) Explain how missing data were addressed

 

Cohort study: if applicable, explain how loss to follow-up was addressed

Case–control study: if applicable, explain how matching of cases and controls was addressed

Cross-sectional study: if applicable, describe analytical methods taking account of sampling strategy

 

(e) Describe any sensitivity analyses

 
   

(f) State whether Hardy–Weinberg equilibrium was considered and, if so, how

   

(g) Describe any methods used for inferring genotypes or haplotypes

   

(h) Describe any methods used to assess or address population stratification

   

(i) Describe any methods used to address multiple comparisons or to control risk of false-positive findings

   

(j) Describe any methods used to address and correct for relatedness among subjects

Results

 

 Participants

13a

(a) Report the numbers of individuals at each stage of the study—e.g., numbers potentially eligible, examined for eligibility, confirmed eligible, included in the study, completing follow-up, and analyzed

Report numbers of individuals in whom genotyping was attempted and numbers of individuals in whom genotyping was successful

(b) Give reasons for non-participation at each stage

 

(c) Consider use of a flow diagram

 

 Descriptive data

14a

(a) Give characteristics of study participants (e.g., demographic, clinical, social) and information on exposures and potential confounders

Consider giving information by genotype

(b) Indicate the number of participants with missing data for each variable of interest

 

(c) Cohort study: summarize follow-up time, e.g., average and total amount

 

 Outcome data

15a

Cohort study: report numbers of outcome events or summary measures over time

Report outcomes (phenotypes) for each genotype category over time

Case–control study: report numbers in each exposure category, or summary measures of exposure

Report numbers in each genotype category

Cross-sectional study: report numbers of outcome events or summary measures

Report outcomes (phenotypes) for each genotype category

 Main results

16

(a) Give unadjusted estimates and, if applicable, confounder-adjusted estimates and their precision (e.g., 95% confidence intervals). Make clear which confounders were adjusted for and why they were included

 

(b) Report category boundaries when continuous variables were categorized

 

(c) If relevant, consider translating estimates of relative risk into absolute risk for a meaningful time period

 
   

(d) Report results of any adjustments for multiple comparisons

 Other analyses

17

(a) Report other analyses done—e.g., analyses of subgroups and interactions, and sensitivity analyses

 
   

(b) If numerous genetic exposures (genetic variants) were examined, summarize results from all analyses undertaken

   

(c) If detailed results are available elsewhere, state how they can be accessed

Discussion

 

 Key results

18

Summarize key results with reference to study objectives

 

 Limitations

19

Discuss limitations of the study, taking into account sources of potential bias or imprecision. Discuss both direction and magnitude of any potential bias

 

 Interpretation

20

Give a cautious overall interpretation of results considering objectives, limitations, multiplicity of analyses, results from similar studies, and other relevant evidence

 

 Generalizability

21

Discuss the generalizability (external validity) of the study results

 

Other information

 

 Funding

22

Give the source of funding and the role of the funders for the present study and, if applicable, for the original study on which the present article is based

 

STREGA Strengthening the REporting of Genetic Association studies, STROBE Strengthening the Reporting of Observational Studies in Epidemiology

aGive information separately for cases and controls in case–control studies and, if applicable, for exposed and unexposed groups in cohort and cross-sectional studies

Table 2

Rationale for inclusion of topics in the STREGA recommendations

Specific issue in genetic association studies

Rationale for inclusion in STREGA

Item(s) in STREGA

Specific suggestions for reporting

Main areas of special interest

 Genotyping errors (misclassification of exposure)

Non-differential genotyping errors will usually bias associations towards the null (Rothman et al. 1993; Garcia-Closas et al. 2004). When there are systematic differences in genotyping according to outcome status (differential error), bias in any direction may occur

8(b) Describe laboratory methods, including source and storage of DNA, genotyping methods and platforms (including the allele calling algorithm used, and its version), error rates and call rates. State the laboratory/center where genotyping was done. Describe comparability of laboratory methods if there is more than one group. Specify whether genotypes were assigned using all of the data from the study simultaneously or in smaller batches

13(a) Report numbers of individuals in whom genotyping was attempted and numbers of individuals in whom genotyping was successful

Factors affecting the potential extent of misclassification (information bias) of genotype include the types and quality of samples, timing of collection, and the method used for genotyping (Little et al. 2002; Pompanon et al. 2005; Steinberg and Gallagher 2004)

When high-throughput platforms are used, it is important to report not only the platform used but also the allele calling algorithm and its version. Different calling algorithms have different strengths and weaknesses [(McCarthy et al. 2008) and supplementary information in (Wellcome Trust Case Control Consortium 2007)]. For example, some of the currently used algorithms are notably less accurate in assigning genotypes to single-nucleotide polymorphisms with low minor allele frequencies (<0.10) than to single nucleotide polymorphisms with higher minor allele frequencies (Pearson and Manolio 2008). Algorithms are continually being improved. Reporting the allele calling algorithm and its version will help readers to interpret reported results, and it is critical for reproducing the results of the study given the same intermediate output files summarizing intensity of hybridization

For some high-throughput platforms, the user may choose to assign genotypes using all of the data from the study simultaneously, or in smaller batches, such as by plate (Clayton et al. 2005; Plagnol et al. 2007) and supplementary information (Wellcome Trust Case Control Consortium 2007)). This choice can affect both the overall call rate and the robustness of the calls

For case–control studies, whether genotyping was done blind to case–control status should be reported, along with the reason for this decision

 Population stratification (confounding by ethnic origin)

When study sub-populations differ both in allele (or genotype) frequencies and disease risks, then confounding will occur if these sub-populations are unevenly distributed across exposure groups (or between cases and controls)

12(h) Describe any methods used to assess or address population stratification

In view of the debate about the potential implications of population stratification for the validity of genetic association studies, transparent reporting of the methods used, or stating that none was used, to address this potential problem is important for allowing the empirical evidence to accrue

Ethnicity information should be presented (see for example (Winker 2006)), as should genetic markers or other variables likely to be associated with population stratification. Details of case-family control designs should be provided if they are used

As several methods of adjusting for population stratification have been proposed (Balding 2006), explicit documentation of the methods is needed

 Modeling haplotype variation

In designs considered in this article, haplotypes have to be inferred because of lack of available family information. There are diverse methods for inferring haplotypes.

12(g) Describe any methods used for inferring genotypes or haplotypes.

When discrete “windows” are used to summarize haplotypes, variation in the definition of these may complicate comparisons across studies, as results may be sensitive to choice of windows. Related “imputation” strategies are also in use (Wellcome Trust Case Control Consortium 2007; Scott et al. 2007; Scuteri et al. 2007).

It is important to give details on haplotype inference and, when possible, uncertainty. Additional considerations for reporting include the strategy for dealing with rare haplotypes, window size and construction (if used) and choice of software

 Hardy–Weinberg equilibrium (HWE)

Departure from Hardy–Weinberg equilibrium may indicate errors or peculiarities in the data (Salanti et al. 2005). Empirical assessments have found that 20–69% of genetic associations were reported with some indication about conformity with Hardy–Weinberg equilibrium, and that among some of these, there were limitations or errors in its assessment (Salanti et al. 2005)

12(f) State whether Hardy–Weinberg equilibrium was considered and, if so, how

Any statistical tests or measures should be described, as should any procedure to allow for deviations from Hardy–Weinberg equilibrium in evaluating genetic associations (Zou and Donner 2006)

 Replication

Publications that present and synthesize data from several studies in a single report are becoming more common

3: State if the study is the first report of a genetic association, a replication effort, or both

The selected criteria for claiming successful replication should also be explicitly documented

Additional issues

 Selection of participants

Selection bias may occur if (i) genetic associations are investigated in one or more subsets of participants (sub-samples) from a particular study; or (ii) there is differential non-participation in groups being compared; or, (iii) there are differential genotyping call rates in groups being compared

6(a) Give information on the criteria and methods for selection of subsets of participants from a larger study, when relevant

13(a) Report numbers of individuals in whom genotyping was attempted and numbers of individuals in whom genotyping was successful

Inclusion and exclusion criteria, sources and methods of selection of sub-samples should be specified, stating whether these were based on a priori or post hoc considerations

 Rationale for choice of genes and variants investigated

Without an explicit rationale, it is difficult to judge the potential for selective reporting of study results. There is strong empirical evidence from randomised controlled trials that reporting of trial outcomes is frequently incomplete and biased in favor of statistically significant findings (Chan et al. 2004a, b; Chan and Altman 2005). Some evidence is also available in pharmacogenetics (Contopoulos-Ioannidis et al. 2006)

7(b) Clearly define genetic exposures (genetic variants) using a widely-used nomenclature system. Identify variables likely to be associated with population stratification (confounding by ethnic origin)

The scientific background and rationale for investigating the genes and variants should be reported

For genome-wide association studies, it is important to specify what initial testing platforms were used and how gene variants are selected for further testing in subsequent stages. This may involve statistical considerations (for example, selection of P value threshold), functional or other biological considerations, fine mapping choices, or other approaches that need to be specified

Guidelines for human gene nomenclature have been published by the Human Gene Nomenclature Committee (Wain et al. 2002a, b). Standard reference numbers for nucleotide sequence variations, largely but not only SNPs are provided in dbSNP, the National Center for Biotechnology Information’s database of genetic variation (Sherry et al. 2001). For variations not listed in dbSNP that can be described relative to a specified version, guidelines have been proposed (Antonarakis 1998; den Dunnen and Antonarakis 2000)

 Treatment effects in studies of quantitative traits

A study of a quantitative variable may be compromised when the trait is subjected to the effects of a treatment for example, the study of a lipid-related trait for which several individuals are taking lipid-lowering medication. Without appropriate correction, this can lead to bias in estimating the effect and loss of power

9(b) For quantitative outcome variables, specify if any investigation of potential bias resulting from pharmacotherapy was undertaken. If relevant, describe the nature and magnitude of the potential bias, and explain what approach was used to deal with this

11: If applicable, describe how effects of treatment were dealt with

Several methods of adjusting for treatment effects have been proposed (Tobin et al. 2005). As the approach to deal with treatment effects may have an important impact on both the power of the study and the interpretation of the results, explicit documentation of the selected strategy is needed

Statistical methods

Analysis methods should be transparent and replicable, and genetic association studies are often performed using specialized software

12(a) State software version used and options (or settings) chosen

 

 Relatedness

The methods of analysis used in family-based studies are different from those used in studies that are based on unrelated cases and controls. Moreover, even in the studies that are based on apparently unrelated cases and controls, some individuals may have some connection and may be (distant) relatives, and this is particularly common in small, isolated populations, for example, Iceland. This may need to be probed with appropriate methods and adjusted for in the analysis of the data

12(j) Describe any methods used to address and correct for relatedness among subjects

For the great majority of studies in which samples are drawn from large, non-isolated populations, relatedness is typically negligible and results would not be altered depending on whether relatedness is taken into account. This may not be the case in isolated populations or those with considerable inbreeding. If investigators have assessed for relatedness, they should state the method used (Lynch and Ritland 1999; Slager and Schaid 2001; Voight and Pritchard 2005) and how the results are corrected for identified relatedness

 Reporting of descriptive and outcome data

The synthesis of findings across studies depends on the availability of sufficiently detailed data

14(a) Consider giving information by genotype

15: Cohort study: Report outcomes (phenotypes) for each genotype category over time

Case-control study: Report number in each genotype category

Cross-sectional study: Report outcomes (phenotypes) for each genotype category

 

 Volume of data

The key problem is of possible false-positive results and selective reporting of these. Type I errors are particularly relevant to the conduct of genome-wide association studies. A large search among hundreds of thousands of genetic variants can be expected by chance alone to find thousands of false-positive results (odds ratios significantly different from 1.0)

12(i) Describe any methods used to address multiple comparisons or to control risk of false-positive findings

16(d) Report results of any adjustments for multiple comparisons

17(b) If numerous genetic exposures (genetic variants) were examined, summarize results from all analyses undertaken

17(c) If detailed results are available elsewhere, state how they can be accessed

Genome-wide association studies collect information on a very large number of genetic variants concomitantly. Initiatives to make the entire database transparent and available online may supply a definitive solution to the problem of selective reporting (Khoury et al. 2007)

Availability of raw data may help interested investigators reproduce the published analyses and also pursue additional analyses. A potential drawback of public data availability is that investigators using the data second-hand may not be aware of limitations or other problems that were originally encountered, unless these are also transparently reported. In this regard, collaboration of the data users with the original investigators may be beneficial. Issues of consent and confidentiality (Homer et al. 2008; Zerhouni and Nabel 2008) may also complicate what data can be shared, and how. It would be useful for published reports to specify not only what data can be accessed and where, but also briefly mention the procedure. For articles that have used publicly available data, it would be useful to clarify whether the original investigators were also involved and if so, how

The volume of data analyzed should also be considered in the interpretation of findings

Examples of methods of summarizing results include giving distribution of P values (frequentist statistics), distribution of effect sizes and specifying false discovery rates

Genotyping errors

Recommendation for reporting of methods (Table 1, item 8(b)): Describe laboratory methods, including source and storage of DNA, genotyping methods and platforms (including the allele calling algorithm used, and its version), error rates, and call rates. State the laboratory/center where genotyping was done. Describe comparability of laboratory methods if there is more than one group. Specify whether genotypes were assigned using all of the data from the study simultaneously or in smaller batches.

Recommendation for reporting of results (Table 1, item 13(a)): Report numbers of individuals in whom genotyping was attempted and numbers of individuals in whom genotyping was successful.

Genotyping errors can occur as a result of effects of the DNA sequence flanking the marker of interest, poor quality or quantity of the DNA extracted from biological samples, biochemical artefacts, poor equipment precision or equipment failure, or human error in sample handling, conduct of the array or handling the data obtained from the array (Pompanon et al. 2005). A commentary published in 2005 on the possible causes and consequences of genotyping errors observed that an increasing number of researchers were aware of the problem, but that the effects of such errors had largely been neglected (Pompanon et al. 2005). The magnitude of genotyping errors has been reported to vary between 0.5 and 30% (Pompanon et al. 2005; Akey et al. 2001; Dequeker et al. 2001; Mitchell et al. 2003). In high-throughput centers, an error rate of 0.5% per genotype has been observed for blind duplicates that were run on the same gel (Mitchell et al. 2003). This lower error rate reflects an explicit choice of markers for which genotyping rates have been found to be highly repeatable and whose individual polymerase chain reactions (PCR) have been optimized. Non-differential genotyping errors, that is, those that do not differ systematically according to outcome status, will usually bias associations towards the null (Rothman et al. 1993; Garcia-Closas et al. 2004), just as for other non-differential errors. The most marked bias occurs when genotyping sensitivity is poor and genotype prevalence is high (>85%) or, as the corollary, when genotyping specificity is poor and genotype prevalence is low (<15%) (Rothman et al. 1993). When measurement of the environmental exposure has substantial error, genotyping errors of the order of 3% can lead to substantial under-estimation of the magnitude of an interaction effect (Wong et al. 2004). When there are systematic differences in genotyping according to outcome status (differential error), bias in any direction may occur. Unblinded assessment may lead to differential misclassification. For genome-wide association studies of SNPs, differential misclassification between comparison groups (for example, cases and controls) can occur because of differences in DNA storage, collection or processing protocols, even when the genotyping itself meets the highest possible standards (Clayton et al. 2005). In this situation, using samples blinded to comparison group to determine the parameters for allele calling could still lead to differential misclassification. To minimize such differential misclassification, it would be necessary to calibrate the software separately for each group. This is one of the reasons for our recommendation to specify whether genotypes were assigned using all of the data from the study simultaneously or in smaller batches.

Population stratification

Recommendation for reporting of methods (Table 1, item 12(h)): Describe any methods used to assess or address population stratification.

Population stratification is the presence within a population of subgroups among which allele (or genotype; or haplotype) frequencies and disease risks differ. When the groups compared in the study differ in their proportions of the population subgroups, an association between the genotype and the disease being investigated may reflect the genotype being an indicator identifying a population subgroup rather than a causal variant. In this situation, population subgroup is a confounder because it is associated with both genotype frequency and disease risk. The potential implications of population stratification for the validity of genetic association studies have been debated (Knowler et al. 1988; Gelernter et al. 1993; Kittles et al. 2002; Thomas and Witte 2002; Wacholder et al. 2002; Cardon and Palmer 2003; Wacholder et al. 2000; Ardlie et al. 2002; Edland et al. 2004; Millikan 2001; Wang et al. 2004; Ioannidis et al. 2004; Marchini et al. 2004; Freedman et al. 2004; Khlat et al. 2004). Modeling the possible effect of population stratification (when no effort has been made to address it) suggests that the effect is likely to be small in most situations (Wacholder et al. 2000; Ardlie et al. 2002; Millikan 2001; Wang et al. 2004; Ioannidis et al. 2004). Meta-analyses of 43 gene-disease associations comprising 697 individual studies showed consistent associations across groups of different ethnic origin (Ioannidis et al. 2004), and thus provide evidence against a large effect of population stratification, hidden or otherwise. However, as studies of association and interaction typically address moderate or small effects and hence require large sample sizes, a small bias arising from population stratification may be important (Marchini et al. 2004). Study design (case-family control studies) and statistical methods (Balding 2006) have been proposed to address population stratification, but so far few studies have used these suggestions (Yesupriya et al. 2008). Most of the early genome-wide association studies used family-based designs or such methods as genomic control and principal components analysis (Wellcome Trust Case Control Consortium 2007; Ioannidis 2007) to control for stratification. These approaches are particularly appropriate for addressing bias when the identified genetic effects are very small (odds ratio < 1.20), as has been the situation in many recent genome-wide association studies (Wellcome Trust Case Control Consortium 2007; Parkes et al. 2007; Todd et al. 2007; Zeggini et al. 2007; Diabetes Genetics Initiative of Broad Institute of Harvard and MIT, Lund University, and Novartis Institutes of BioMedical Research et al. 2007; Scott et al. 2007; Helgadottir et al. 2007; McPherson et al. 2007; Easton et al. 2007; Hunter et al. 2007; Stacey et al. 2007; Gudmundsson et al. 2007; Haiman et al. 2007b; Yeager et al. 2007; Zanke et al. 2007; Tomlinson et al. 2007; Haiman et al. 2007a; Rioux et al. 2007; Libioulle et al. 2007; Duerr et al. 2006). In view of the debate about the potential implications of population stratification for the validity of genetic association studies, we recommend transparent reporting of the methods used, or stating that none was used, to address this potential problem. This reporting will enable empirical evidence to accrue about the effects of population stratification and methods to address it.

Modeling haplotype variation

Recommendation for reporting of methods (Table 1, item 12(g)): Describe any methods used for inferring genotypes or haplotypes.

A haplotype is a combination of specific alleles at neighboring genes that tends to be inherited together. There has been a considerable interest in modeling haplotype variation within candidate genes. Typically, the number of haplotypes observed within a gene is much smaller than the theoretical number of all possible haplotypes (Zhao et al. 2003; International HapMap Consortium et al. 2007). Motivation for utilizing haplotypes comes, in large part, from the fact that multiple SNPs may “tag” an untyped variant more effectively than a single typed variant. The subset of SNPs used in such an approach is called “haplotype tagging” SNPs. Implicitly, an aim of haplotype tagging is to reduce the number of SNPs that have to be genotyped, while maintaining statistical power to detect an association with the phenotype. Maps of human genetic variation are becoming more complete, and large-scale genotypic analysis is becoming increasingly feasible. In consequence, it is possible that modeling haplotype variation will become more focussed on rare causal variants, because these may not be included in the genotyping platforms.

In most current large-scale genetic association studies, data are collected as unphased multilocus genotypes (that is, which alleles are aligned together on particular segments of chromosome is unknown). It is common in such studies to use statistical methods to estimate haplotypes (Stephens et al. 2001; Qin et al. 2002; Scheet and Stephens 2006; Browning 2008), and their accuracy and efficiency have been discussed (Huang et al. 2003; Kamatani et al. 2004; Zhang et al. 2004; Carlson et al. 2004; van Hylckama Vlieg et al. 2004). Some methods attempt to make use of a concept called haplotype “blocks” (Greenspan and Geiger 2004; Kimmel and Shamir 2005), but the results of these methods are sensitive to the specific definitions of the “blocks” (Cardon and Abecasis 2003; Ke et al. 2004). Reporting of the methods used to infer individual haplotypes and population haplotype frequencies, along with their associated uncertainties should enhance our understanding of the possible effects of different methods of modeling haplotype variation on study results as well as enabling comparison and syntheses of results from different studies.

Information on common patterns of genetic variation revealed by the International Haplotype Map (HapMap) Project (International HapMap Consortium et al. 2007) can be applied in the analysis of genome-wide association studies to infer genotypic variation at markers not typed directly in these studies (Servin and Stephens 2007; Marchini et al. 2007). Essentially, these methods perform haplotype-based tests but make use of information on variation in a set of reference samples (for example, HapMap) to guide the specific tests of association, collapsing a potentially large number of haplotypes into two classes (the allelic variation) at each marker. It is expected that these techniques will increase power in individual studies, and will aid in combining data across studies, and even across differing genotyping platforms. If imputation procedures have been used, it is useful to know the method, accuracy thresholds for acceptable imputation, how imputed genotypes were handled or weighted in the analysis, and whether any associations based on imputed genotypes were also verified on the basis of direct genotyping at a subsequent stage.

Hardy–Weinberg equilibrium

Recommendation for reporting of methods (Table 1, item 12(f)): State whether HWE was considered and, if so, how.

Hardy–Weinberg equilibrium has become widely accepted as an underlying model in population genetics after (Hardy 1908) and (Weinberg 1908) proposed the concept that genotype frequencies at a genetic locus are stable within one generation of random mating; the assumption of HWE is equivalent to the independence of two alleles at a locus. Views differ on whether testing for departure from HWE is a useful method to detect errors or peculiarities in the data set, and also the method of testing (Minelli et al. 2008). In particular, it has been suggested that deviation from HWE may be a sign of genotyping errors (Xu et al. 2002; Hosking et al. 2004; Salanti et al. 2005). Testing for departure from HWE has a role in detecting gross errors of genotyping in large-scale genotyping projects such as identifying SNPs for which the clustering algorithms used to call genotypes have broken down (Wellcome Trust Case Control Consortium 2007; Pearson and Manolio 2008). However, the statistical power to detect less important errors of genotyping by testing for departure from HWE is low (McCarthy et al. 2008) and, in hypothetical data, the presence of HWE was generally not altered by the introduction of genotyping errors (Zou and Donner 2006). Furthermore, the assumptions underlying HWE, including random mating, lack of selection according to genotype, and absence of mutation or gene flow, are rarely met in human populations (Shoemaker et al. 1998; Ayres and Balding 1998). In five of 42 gene-disease associations assessed in meta-analyses of almost 600 studies, the results of studies that violated HWE significantly differed from the results of studies that conformed to the model (Trikalinos et al. 2006). Moreover, the study suggested that the exclusion of HWE-violating studies may result in loss of the statistical significance of some postulated gene-disease associations and that adjustment for the magnitude of deviation from the model may also have the same consequence for some other gene-disease associations. Given the differing views about the value of testing for departure from HWE and about the test methods, transparent reporting of whether such testing was done and, if so, the method used, is important for allowing the empirical evidence to accrue.

For massive-testing platforms, such as genome-wide association studies, it might be expected that many false-positive violations of HWE would occur if a lenient P value threshold were set. There is no consensus on the appropriate P value threshold for HWE-related quality control in this setting. Hence, we recommend that investigators state which threshold they have used, if any, to exclude specific polymorphisms from further consideration. For SNPs with low minor allele frequencies, substantially more significant results than expected by chance have been observed, and the distribution of alleles at these loci has often been found to show departure from HWE.

For genome-wide association studies, another approach that has been used to detect errors or peculiarities in the data set (due to population stratification, genotyping error, HWE deviations or other reasons) has been to construct quantile–quantile (Q/Q) plots whereby observed association statistics or calculated P values for each SNP are ranked in order from smallest to largest and plotted against the expected null distribution (Pearson and Manolio 2008; McCarthy et al. 2008). The shape of the curve can lend insight into whether or not systematic biases are present.

Replication

Recommendation: state if the study is the first report of a genetic association, a replication effort, or both (Table 1, item 3).

Articles that present and synthesize data from several studies in a single report are becoming more common. In particular, many genome-wide association analyses describe several different study populations, sometimes with different study designs and genotyping platforms, and in various stages of discovery and replication (Pearson and Manolio 2008; McCarthy et al. 2008). When data from several studies are presented in a single original report, each of the constituent studies and the composite results should be fully described. For example, a discussion of sample size and the reason for arriving at that size would include clear differentiation between the initial group (those that were typed with the full set of SNPs) and those that were included in the replication phase only (typed with a reduced set of SNPs) (Pearson and Manolio 2008; McCarthy et al. 2008). Describing the methods and results in sufficient detail would require substantial space in print, but options for publishing additional information on the study online make this possible.

Discussion

The choices made for study design, conduct and data analysis potentially influence the magnitude and direction of results of genetic association studies. However, the empirical evidence on these effects is insufficient. Transparency of reporting is, thus, essential for developing a better evidence base (Table 2). Transparent reporting helps address gaps in empirical evidence (Bogardus et al. 1999), such as the effects of incomplete participation and genotyping errors. It will also help assess the impact of currently controversial issues such as population stratification, methods of inferring haplotypes, departure from HWE and multiple testing on effect estimates under different study conditions.

The STREGA Statement proposes a minimum checklist of items for reporting genetic association studies. The statement has several strengths. First, it is based on existing guidance on reporting observational studies (STROBE). Second, it was developed from discussions of an interdisciplinary group that included epidemiologists, geneticists, statisticians, journal editors, and graduate students, thus reflecting a broad collaborative approach in terminology accessible to scientists from diverse disciplines. Finally, it explicitly describes the rationale for the decisions (Table 2) and has a clear plan for dissemination and evaluation.

The STREGA recommendations are available at www.strega-statement.org. We welcome comments, which will be used to refine future versions of the recommendations. We note that little is known about the most effective ways to apply reporting guidelines in practice, and that therefore it has been suggested that editors and authors collect, analyze, and report their experiences in using such guidelines (Davidoff et al. 2008). We consider that the STREGA recommendations can be used by authors, peer reviewers and editors to improve the reporting of genetic association studies. We invite journals to endorse STREGA, for example by including STREGA and its Web address in their Instructions for Authors and by advising authors and peer reviewers to use the checklist as a guide. It has been suggested that reporting guidelines are most helpful if authors keep the general content of the guideline items in mind as they write their initial drafts, then refer to the details of individual items as they critically appraise what they have written during the revision process (Davidoff et al. 2008). We emphasize that the STREGA reporting guidelines should not be used for screening submitted manuscripts to determine the quality or validity of the study being reported. Adherence to the recommendations may make some manuscripts longer, and this may be seen as a drawback in an era of limited space in a print journal. However, the ability to post information on the Web should alleviate this concern. The place in which supplementary information is presented can be decided by authors and editors of the individual journal.

We hope that the recommendations stimulate transparent and improved reporting of genetic association studies. In turn, better reporting of original studies would facilitate the synthesis of available research results and the further development of study methods in genetic epidemiology with the ultimate goal of improving the understanding of the role of genetic factors in the cause of diseases.

Acknowledgments

The authors thank Kyle Vogan and Allen Wilcox for their participation in the workshop and for their comments; Michele Cargill (Affymetrix Inc) and Aaron del Duca (DNA Genotek) for their participation in the worshop as observers: and the Public Population Project in Genomics (P3G), hosted by the University of Montreal and supported by Genome Canada and Genome Quebec. This article was made possible thanks to input and discussion by the P3G International Working Group on Epidemiology and Biostatistics, discussion held in Montreal, May 2007. The authors also thank the reviewers for their very thoughtful feedback, and Silvia Visentin, Rob Moriarity, Morgan Macneill and Valery L’Heureux for administrative support. We were unable to contact Barbara Cohen to confirm her involvement in the latest version of this article. This article was supported by the Institutes of Genetics and of Nutrition, Metabolism and Diabetes, Canadian Institutes of Health Research; Genome Canada; Biotechnology, Genomics and Population Health Branch, Public Health Agency of Canada; Affymetrix; DNA Genotek; TrialStat!; and GeneSens. The funders had no role in the decision to submit the article or in its preparation.

Open Access

This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Copyright information

© The Author(s) 2009

Authors and Affiliations

  • Julian Little
    • 1
    • 2
  • Julian P. T. Higgins
    • 3
  • John P. A. Ioannidis
    • 4
    • 5
  • David Moher
    • 2
  • France Gagnon
    • 6
  • Erik von Elm
    • 7
    • 8
  • Muin J. Khoury
    • 9
  • Barbara Cohen
    • 10
  • George Davey-Smith
    • 11
  • Jeremy Grimshaw
    • 12
  • Paul Scheet
    • 13
  • Marta Gwinn
    • 9
  • Robin E. Williamson
    • 14
  • Guang Yong Zou
    • 15
    • 16
  • Kim Hutchings
    • 2
  • Candice Y. Johnson
    • 2
  • Valerie Tait
    • 2
  • Miriam Wiens
    • 2
  • Jean Golding
    • 17
  • Cornelia van Duijn
    • 18
  • John McLaughlin
    • 19
    • 20
  • Andrew Paterson
    • 21
  • George Wells
    • 22
  • Isabel Fortier
    • 23
  • Matthew Freedman
    • 24
  • Maja Zecevic
    • 25
  • Richard King
    • 26
  • Claire Infante-Rivard
    • 27
  • Alex Stewart
    • 28
  • Nick Birkett
    • 2
  1. 1.Canada Research Chair in Human Genome EpidemiologyOttawaCanada
  2. 2.Department of Epidemiology and Community MedicineUniversity of OttawaOttawaCanada
  3. 3.MRC Biostatistics UnitCambridgeUK
  4. 4.Department of Hygiene and Epidemiology, School of MedicineUniversity of IoanninaIoanninaGreece
  5. 5.Center for Genetic Epidemiology and ModelingTufts University School of MedicineBostonUSA
  6. 6.CIHR New Investigator and Canada Research Chair in Genetic EpidemiologyUniversity of TorontoTorontoCanada
  7. 7.Institute of Social and Preventive MedicineUniversity of BernBernSwitzerland
  8. 8.Department of Medical Biometry and Medical Informatics, German Cochrane CentreUniversity Medical CentreFreiburgGermany
  9. 9.National Office of Public Health GenomicsCenters for Disease Control and PreventionAtlantaUSA
  10. 10.Public Library of ScienceSan FranciscoUSA
  11. 11.Department of Social Medicine, MRC Centre for Causal Analyses in Translational EpidemiologyUniversity of BristolBristolUK
  12. 12.Canada Research Chair in Health Knowledge Transfer and Uptake, Clinical Epidemiology Program, Department of Medicine, Ottawa Health Research InstituteUniversity of OttawaOttawaCanada
  13. 13.Department of Epidemiology, MD Anderson Cancer CenterUniversity of TexasHoustonUSA
  14. 14.American Journal of Human GeneticsBostonUSA
  15. 15.Department of Epidemiology and BiostatisticsUniversity of Western OntarioLondonCanada
  16. 16.Robarts Clinical TrialsRobarts Research InstituteLondonCanada
  17. 17.Paediatric and Perinatal EpidemiologyBristolUK
  18. 18.European Journal of EpidemiologyRotterdamThe Netherlands
  19. 19.Cancer Care OntarioTorontoCanada
  20. 20.Prosserman Centre for Health Research, Samuel Lunenfeld Research InstituteTorontoCanada
  21. 21.Canada Research Chair in Genetics of Complex DiseasesHospital for Sick Children (SickKids)TorontoCanada
  22. 22.Cardiovascular Research Methods CentreUniversity of Ottawa Heart InstituteOttawaCanada
  23. 23.Genome Quebec and P3G ObservatoryMcGill University and Genome Quebec Innovation CenterMontréalCanada
  24. 24.Dana-Farber Cancer InstituteBostonUSA
  25. 25.LancetNew YorkUSA
  26. 26.Genetics in MedicineMinneapolisUSA
  27. 27.Department of Epidemiology, Biostatistics and Occupational Health, Faculty of MedicineMcGill UniversityMontrealCanada
  28. 28.University of Ottawa Heart InstituteOttawaCanada