Skip to main content

Table 3 Summary table of key study design recommendations

From: Analysis of common methodological flaws in the highest cited e-cigarette epidemiology research

Area of study Recommendation Comment
Impact of vaping on smoking cessation/reduction Ensure allocation and randomization of research participants to make sure to avoid selection bias, including (if possible) analyzing of those who have already quit by vaping Limiting the enrollment to a subgroup of smokers (e.g., active smokers) omits those who had successfully quit by vaping, and prevents gathering generalizable findings
Consider possible causal pathways towards smoking quitting/reduction, which could be attributable to vaping initiation Data collection and analysis should be designed to investigate the quit/reduction attempts attributable to vaping, particularly in individuals with a history of previous unsuccessful attempts and “accidental quitters.”
Impact of vaping on smoking initiation Detail vaping (and smoking) habits and history, in terms of their duration, amount, and frequency The phrases “tried vaping” or “was a vaper” are limited as proxy indicator of levels of vaping exposure, and unreliable to support the gateway claim
Analyze population-level trends in vaping incidence and prevalence together with smoking trends, and “triangulate” the findings across multiple types of evidence “Gateway” studies are inconsistent with actual population-level trends, as “gateway” hypothesis would predict more smokers, but population-level trends show faster declines in smoking. Thus, the role of vaping in preventing smoking initiation should be considered
Health outcomes Acknowledge the health consequences of previous smoking history in the evaluation non-acute effects of vaping, accounting for duration of smoking, time since quitting, or frequency and quantity of tobacco use The vast majority of vapers are former smokers, and possible health events should be weighted as a function of previous smoking exposure (in particular for those conditions whose onset continues for longer after quitting)
Ensure temporal relationships are consistent with the association being tested Outcomes cannot logically occur before the exposure being tested, yet such data points are often included, especially in cross-sectional studies
General Specify the exact causal pathway being tested—including particular exposure, outcome, and potential mediator variables—and think through plausible causal mechanisms Different causal mechanisms are involved for outcomes of experimentation vs. regular use. Also, an association could include possible multiple mechanisms, specify which one is being tested (e.g., vaping impacts on number of quit attempts vs the success of each quit attempt). For health outcomes, specifying the exact causal pathway can inform about biological plausibility
Use sufficiently robust methods to measure and control for relevant confounding factors, including multivariate statistics and analyses, and propensity score techniques if possible All studies should control for detailed smoking and/or vaping history (frequency, quantity, duration of use). Additionally, cessation/reduction studies should control for number and methods of quit attempts, goals, etc.; initiation studies should control for peer and family tobacco use, personality characteristics, mental health factors, demographic characteristics; and health outcome studies should control for other relevant environmental or exposure factors
Ensure that implications and conclusions do not assume a causal relationship, unless causality has been established Implications that suggest altering one variable to change another, assume a causal relationship; this is inappropriate when only an association has been established
Discuss biases, limitations, and alternate explanations honestly and transparently, and discuss how they impact findings Confounding is a serious limitation in observational studies, and can render the entire set of results inconclusive. Biases from sample definition should be discussed (e.g., omitting those who previously quit by vaping). Alternate explanations such as diagnostic bias and reverse causality should be discussed