Introduction

There is significant interest in the potential causal impact that ultra-processed food (UPF) consumption has on health. Observational studies have identified that higher UPF consumption tends to be associated with worse health outcomes, such as weight gain and obesity [1, 2]. Prospective studies on this topic are particularly importance as they better allow for inferences on temporal order of associations.

A major challenge in all observational research are unmeasured confounding variables. In line with this, confounding (e.g., residual confounding by social class or lifestyle behaviours) has been discussed as a limitation in numerous studies and reviews on UPFs and health [3,4,5]. To date, there have been limited attempts to quantify how feasible it is that unmeasured confounding could in part attenuate or fully explain prospective relationships between UPF consumption and health outcomes, such as weight gain. A rare exception is a negative control outcome analysis, which found some evidence to suggest that confounding could explain why UPF consumption and cancer were associated in a prospective observational study [4].

In the present article we introduce a recently developed analysis approach to obesity researchers - the E value [6] – to estimate the plausibility that unmeasured confounding could explain observational findings linking UPF consumption with weight gain. The E value is defined as the minimum strength of association that one or more (combined) unaccounted for confounding variables would need to have with an exposure (UPF consumption) and outcome (weight gain) to explain the association between the exposure and outcome of interest.

Methods

Calculation and interpretation of an E value

When an outcome is predicted using a risk ratio (RR), the E value is calculated as:

$${\rm{E}}={\rm{RR}}+\surd {[{\rm{RR}}\times ({\rm{RR}}-1)]}$$

For outcomes predicted using odds ratios and hazard ratios the same equation is used, although some adjustments are made, based on how common the outcome is (e.g. >15% of individuals have an outcome at the end of follow-up).

Specific E values should not be considered as generically ‘likely’ vs. ‘unlikely’ evidence of confounding potentially explaining exposure-outcome observations, as inference should be based on a case-by-case basis. For example, Gaster et al. [7] conducted a meta-analysis on the association between anti-depressant use and miscarriage risk, concluding that risk of miscarriage was higher among anti-depressant users (RR = 1.41). For this RR, the E value = 2.17. Alcohol use was considered as a potential confounder because pregnant women who use anti-depressants are at much higher risk of excessive alcohol consumption than pregnant women who do not. The authors went on to conclude that alcohol use could explain the association between anti-depressant use and miscarriage risk because the relationship (expressed as a risk ratio) between anti-depressant use and alcohol, and alcohol and miscarriage risk are both known to be >2.17. Yet, it is important to note that the strengths of the association between confounder and exposure and confounder and outcome do not both need to exceed an E-value to provide statistical evidence of potential ‘total’ confounding.

Used alongside the E-value, is the joint bounding factor, B:

$$B=({RR}_{UD}\,{\ast}\,{RR}_{EU})/({RR}_{EU}+{RR}_{UD}-1)$$

RRUD is the size of association between the potential confounder and outcome and RREU is the size of association between the potential confounder and exposure. To explore how combinations of confounder exposure and outcome relationships could combine to create statistical conditions for ‘total’ confounding, one sets B (bounding factor) to the E value. In simple language, if the likely size of association between the potential confounder and outcome (or exposure) is larger than E value but the potential confounder and exposure (or out outcome) association is smaller than the E value, the two may still combine to be equal to or exceed the E value and therefore contribute to ‘total’ confounding. For instance, if an E value is 2.00 (RR ~ 1.33), a stronger association between confounder and outcome (RR = 3.00) and a weaker association between confounder and exposure could exist (RRs > 1.60) to produce ‘total’ confounding and explain away the effect (see Fig. 1).

Fig. 1: Example visualisation of the bounding factor.
figure 1

Red dotted lines show a hypothetical example combination of associations between confounder and outcome (RRUD) and confounder and exposure (RRUE) that would explain away an effect of RR = 1.33. Number in brackets is E Value [RR = 2] and denoted by a black circle on the curve.

Full size image

This equation can also be used to estimate by how much (% of effect) combinations of known RRUD and RREU could account for an observed effect (RR). B = the observed RR, would indicate combinations of RRUD and RREU could feasibly create ‘total’ confounding and explain away the observed effect. B = 50% of observed RR, would indicate that combinations of RRUD and RREU could feasibly explain half of the size of the observed effect (‘partial’ confounding).

UPF consumption and weight gain: identifying potential confounders

Higher UPF consumption has been identified as a potential causal contributor to weight gain and obesity largely on the basis of observational research. Because both UPF consumption and obesity have sociodemographic and personal characteristic patterning, variables such as age, gender, social class, physical activity and smoking status are typical control variables in study analyses [2, 8] due to concerns over potential confounding. However, other potential confounders could be ‘unmeasured’ and therefore not statistically controlled for in any given study. In the present analyses we consider a person’s trait tendency to overeat and experience depression symptoms, as two examples of ‘unmeasured confounders’.

Socioeconomic status (SES) is a particularly important control variable in diet and health studies. Statistically controlling for standard SES measures like education level may not fully capture the various ways by which social class could indirectly contribute to both UPF consumption and weight gain, resulting in ‘residual’ confounding [9]. Residual confounding (confounding that remains after statistically controlling for a variable) is typically very difficult to measure, but could be relevant to diet because low SES greatly increases likelihood of decreased access to healthy nutritious food, also known as food insecurity [10]. SES measures are somewhat associated with food insecurity [10], but unlikely to capture the negative consequences of food insecurity, resulting in residual risk. Here we therefore treat food insecurity as a quantifiable example of ‘residual’ confounding.

Analyses

To provide an effect estimate of the association between UPF consumption and weight gain, we identified prospective studies examining UPF consumption and risk of weight gain among adults from two recent systematic and one recent narrative review on the topic [1, 11, 12]. Five prospective studies were identified and meta-analysed. We focused on study effect estimates from analyses relating to weight gain from baseline. See Table 1. We extracted results from models that allowed for prospective weight gain to be accurately quantified and adjusted for the most comprehensive collection of potential confounders, which included demographics (social class, age, sex) and personal characteristics (physical activity, sleep, smoking status) across studies.

Table 1 Prospective studies included in meta-analysis.
Full size table

Random effects meta-analysis using a Restricted Maximum Likelihood estimator were conducted using the ‘metafor’ package in R. Hazard Ratios were converted to Risk Ratios using the ‘toRR’ function from the ‘EValue’ package. We conducted separate meta-analyses to determine the effects of daily UPF intake on weight gain outcomes across different quartiles of UPF consumption (see Table 1), with quartile 1 (lowest consumption of UPF) used as a comparator in each meta-analysis. For the purpose of E value analysis we focused on Q1 vs. Q4 effect estimates, as this constituted the largest association between UPF consumption and risk of weight gain.

E values were calculated using the ‘evalues.RR’ function from the EValue package (see Table 2). To characterise size of associations potential confounding variables have with UPF consumption and risk of weight gain, we drew on available published meta-analyses and high quality large epidemiology studies. To convert Odds Ratios to Risk Ratios for the confounding effects we used the formula RR = OR/(1 – p0 + (p0 * OR)) where p0 is the baseline risk [6]. To do this we used the ‘ORToRelRisk’ function from the ‘DescTools’ package [13]. We used baseline risk estimates from relevant studies or conservative estimates if not available directly. Data and R code can be found here https://osf.io/z89pa/.

Table 2 Meta-analysis of UPF consumption and risk of weight gain studies.
Full size table

Table 2 presents the meta-analysis estimates. Figure 2 illustrates the RRUD and RREU values that equate to E values. Meta-analysis revealed statistically significant and small associations between UPF consumption and weight gain outcomes, based on effect size interpretation guidance [14]. For the largest effect estimate (UPF quartile 4 vs. 1), E values for the point estimate and its lower confidence interval = 1.55 and 1.43, respectively, indicating that unmeasured confounders associated with both UPF consumption and weight gain to a similar degree (small to medium effect sizes) could nullify associations.

Fig. 2: Visualisation of the Bounding Factor for UPF consumption and risk of weight gain.
figure 2

Red line is RR association of trait overeating and risk of weight gain, and green line represents the strength of RR association between trait overeating and UPF consumption needed to fully explain the association between UPF consumption and weight gain. Number in brackets is E Value [1.55] and denoted by a black circle on the curve.

Full size image

Potential unmeasured confounding

No studies controlled for mental health or depression symptoms as potential confounders. Previously published meta-analyses indicates that higher depression symptoms are associated with UPF consumption [15] (OR = 1.44 ~ RR = 1.39) and predict development of obesity [16] (OR = 1.58 ~RR = 1.48). The joint bounding factor (RRUD * RREU / (RRUD + RREU − 1)) = 1.10, suggesting that depressive symptoms could account for 71% (effect estimate) or as much as 100% (lower confidence interval of effect estimate) of the meta-analysed association between the highest vs. lowest UPF consumption groups and weight gain.

No studies controlled for eating traits, such as overeating. Tendency to overeat has a genetic basis and is typically characterised as either general disinhibited overeating or emotional-based overeating. Both are associated with risk of higher BMI across multiple previously published meta-analyses and effect sizes are medium in size [17, 18]. For instance, the pooled association between disinhibited overeating and BMI is r = 0.28 (~OR = 2.88 [95% CI: 2.02–4.44] ~RR = 1.49 [95% CI: 1.34–1.63). Prospective studies of the association between tendency to overeat and weight gain produce similar estimates [19, 20]. A positive relationship between tendency to overeat and higher UPF consumption would seem plausible, but there is a lack of data to confidently estimate effect size with precision and we therefore concluded E value calculation was not feasible. However, as denoted in Fig. 2 an RR = 1.62 (small to medium in size) would be needed to fully explain the meta-analysed association between the highest vs. lowest UPF consumption groups and weight gain.

Potential residual confounding

All studies controlled for SES indicators, but not access to healthy nutritious food or food insecurity specifically. Food insecurity is associated with higher UPF consumption [21] and a recent epidemiological survey study estimates participants with the highest UPF consumption have a 60% higher prevalence of food insecurity (RR = 1.60) [22]. Food insecurity is associated with elevated obesity risk in a previously published meta-analysis (OR = 1.53 ~RR 1.42) [23] and effects appear similar when examined prospectively in studies [24]. Taken together, it suggests that residual confounding of this nature could account for ~86% of the effect estimate, or 100% based on the lower bound confidence interval.

Additional analyses

In an additional meta-analysis we included only studies with the same outcome (risk of developing overweight/obesity, n = 3). Results were similar to the main meta-analysis (see Table 2), though the effect estimate and E-value was slightly smaller, suggesting that confounding would be more plausible for this sub-set of studies. Corresponding Q1 vs. Q4 E value = 1.50 (Lower bound CI = 1.45).

Discussion

We provide an illustration of how the E value can be used to examine the plausibility of confounding in obesity research. Using this approach we show that confounding variables with small-to-moderate sized associations with UPF consumption and weight gain could theoretically attenuate or completely nullify associations, and we identify a number of examples of potential confounding variables that could meet these conditions.

The present work also highlights some of the limitations of the E value. Accurate calculation is based on having sufficient statistical information about the potential confounders’ association with both outcome (RRUD) and exposure (RREU) variables of interest. Overeating tendencies were identified as partially meeting the conditions for total confounding (RRUD > lower bound E value confidence interval), but due to a lack of data to calculate a robust estimate RREU, we were unable to formally apply the E value approach with confidence, though we were able to estimate associations between UPF consumption and overeating tendencies which would create statistical conditions that could nullify UPF-weight gain associations. This observation underscores that appropriate use of the E value is contextually specific and reliant on various effect size estimations, which may not always be available.

As the most extreme E values we identified were relatively small in size (based on effect size interpretation guidance), this suggests that plausible confounders would not need to be very strongly associated with UPF consumption and weight gain to substantially attenuate or fully nullify associations. Furthermore, as a number of plausible confounders were identified that we reason could collectively attenuate the meta-analysed UPF consumption – weight gain association observed to non-significance, unmeasured confounding is of significant concern. Yet, it is important to note that the E value approach provides evidence on whether unaccounted confounding factors could hypothetically explain away observed associations. It is also plausible that more complete measurement of potential confounding variables could increase size of the UPF and weight gain association. This highlights the need for further confirmatory high quality observational research that is better able to control for potential confounders of concern and evidence from study types devoid of confounding (e.g., randomised controlled trials).

There is debate about how UPFs could causally contribute to weight gain. One proposed explanation is that the unfavourable macronutrient profile of UPFs is responsible for weight gain. A different, but not mutually exclusive explanation is that UPF consumption may also harm health independent to macronutrient profile and this is proposed by some observational studies finding an association between UPFs and weight gain remains when macronutrient factors are controlled for [25]. Meta-analysed studies did not consistently control for macronutrient profile or diet quality. Therefore, from the present analyses it is unclear the extent to which the macronutrient profile of diets higher in UPFs could in part explain the meta-analysed associations observed and/or in combination with confounding variables, fully explain association between higher UPF consumption and weight gain.

There are limitations to the present research and the E value approach. We examined a select number of example potential confounders for illustrative purposes and other potential confounders may warrant investigation (e.g. shared genetic risk for weight gain and unhealthy diet). Food insecurity was examined as a quantifiable hypothetical example of residual confounding from measurement of SES. Most residual confounding by its nature is due to measurement imprecision and therefore unquantifiable. Food insecurity (based on prevalence) is a relatively rare event (dependent on country) and therefore itself may be unlikely to fully explain UPF and weight gain associations, but less extreme limited access to healthy nutrition will be more common and therefore a more likely potential confounder.

We based meta-analysis study inclusion on recent systematic reviews and not a formal search procedure, as this was beyond the scope of this technical report. A small number of studies were suitable for meta-analysis and they may be prone to publication bias. If so, the size of association between UPF consumption and weight gain and E-values may be overestimated. The E value approach provides information on hypothetical confounding and the accuracy of estimates are dependent on the underlying statistical assumptions, as well as assumptions made when converting effect sizes. Critiques of the statistical application of the E value [6, 26] underscore that it can be at best considered as an analysis tool to inform thinking about potential confounding and not a blunt instrument to draw definitive conclusions from.

Unaccounted for confounding could hypothetically explain the prospective association between UPF consumption and weight gain in adults. High quality observational research controlling for a wider range of potential confounders and evidence from study types devoid of confounding are now needed.