Advertisement

Hunger and socioeconomic background additively predict impulsivity in humans

  • C. AllenEmail author
  • D. Nettle
Open Access
Article

Abstract

Impulsivity refers to the valuation of future rewards relative to immediate ones. From an evolutionary perspective, we should expect impulsivity to be sensitive to the current state of the organism (for example, hunger), and also its long-term developmental history. There is evidence that both current hunger and childhood socioeconomic deprivation are individually associated with impulsivity, but it is not known how these combine. For example, acute hunger might over-ride social gradients in baseline impulsivity, or alternatively, individuals who have experienced greater deprivation might respond more strongly to acute hunger. We aimed to investigate whether hunger and childhood socioeconomic deprivation act additively or interactively in three studies utilising delay discounting tasks. Childhood socioeconomic deprivation was measured using childhood postcode and a self-report measure. In two studies hunger was experimentally manipulated (n = 95 & n = 93 respectively), and in the third we simply measured natural variation. We employed a standard hypothetical delay discounting task in two studies, and a behavioural task with experienced delays in the third (n = 330). Although the individual studies varied in which predictors were statistically significant, when we meta-analysed them, a clear pattern emerged. Hunger predicted greater impulsivity; childhood socioeconomic deprivation predicted greater impulsivity; and these two effects were additive rather than interactive.

Keywords

Impulsivity Hunger Developmental history Childhood socioeconomic deprivation Delay discounting 

Introduction

Impulsivity is a complex concept encompassing multiple behavioural components. High levels of impulsivity have been implicated in a range of negative health related behaviours, including, but not limited to, sexual risk-taking (Donohew et al. 2000), smoking and heavy drinking (Granö et al. 2004), and drug abuse (de Wit 2009). Impulsivity has also been associated with a range of disordered eating habits, which can have negative consequences for health, such as increased food consumption in a lab environment (Ely et al. 2015), obesity and percentage body fat (Weller et al. 2008, Rasmussen et al. 2010), and binge-eating disorder and anorexia nervosa (Steward et al. 2017). Finally, there is evidence that those who show lower levels of impulsivity may be protected against weight gain (Duckworth et al. 2010), suggesting that interventions focussing on manipulating impulsivity could be beneficial for public health. For this reason it is important to understand factors which may underlie the emergence of impulsive behaviours. Delay discounting is one component of impulsivity that we focus on in this paper. Delay discounting involves the systematic devaluation of an outcome as the delay to its delivery increases; more impulsive individuals devalue at a higher rate. Measurement of delay discounting involves repeated choices between smaller but more immediately available rewards (smaller-sooner rewards - SSRs) and larger but more delayed rewards (larger-later rewards - LLRs).

The valuation of SSRs and LLRs by an individual can be expected to be a product of both the current state of said individual, and their developmental history, because both of these factors will affect their capacity to endure unrewarded delay. Evidence to date suggests that this is indeed the case, as illustrated in a recent paper investigating levels of impulsivity in European starlings. Bateson and colleagues (Bateson et al. 2015) found that developmental history, in this case a biomarker based on telomere attrition through early life, and current state (body condition, which in these birds is a measure of energetic reserves), both significantly predicted impulsivity measured using a food-based delay discounting task. Birds that experienced a more adverse developmental history, and birds with lower current energetic reserves, were more impulsive on the task. The two influences were additive.

The literature investigating human behaviour also supports the idea that both current state and developmental history are associated with impulsivity. When considering long term developmental history for example, childhood abuse (Brodsky et al. 2001), childhood family unpredictability (Hill et al. 2008), and early-life environmental risk and uncertainty (Chisholm 1999) have all been found to be associated with increased impulsivity measured in adulthood. When considering current state, the variable that has most reliably been found to affect impulsivity is hunger: individuals who are hungrier are also more impulsive (Kirk and Logue 1997; Wang & Dvorak 2010; Loeber et al. 2013). This links directly to the starling finding on energetic reserves. Interestingly, the human literature provides evidence for hunger affecting both impulsivity of responses specifically related to food stimuli (Loeber et al. 2013; Kirk and Logue 1997), and for non-food stimuli such as money (Wang and Dvorak 2010; though see De Ridder et al. 2014).

Given the above findings, the aim of the current study was to investigate the ways in which developmental history and current state contribute to impulsivity in humans, and how they may combine in order to predict this. We employed hunger as our measure of current state and childhood socioeconomic deprivation as our measure of developmental history. Childhood socioeconomic deprivation was operationalized primarily in the form of neighbourhood deprivation assessed from childhood postcodes, for which data exists publicly in the UK. This provides a broad and simple summary measure of likely childhood experience of deprivation, incorporating information concerning family income, employment, education, health-care access, crime, barriers to housing, and living environment. This measure has previously been found to be significantly associated with levels of impulsivity measured in adults, albeit using a different type of impulsivity task to the delay discounting we focus on here (Paál et al. 2015).

If both current hunger and childhood socioeconomic deprivation are related to adult impulsivity, they might combine in several different ways (Fig. 1). Their effects might simply be additive (Fig. 1a), as seen in Bateson and colleagues’ (Bateson et al. 2015) starling findings. Alternatively, we might find an interaction between the two predictors. For example, it could be the case that in the absence of hunger we find a deprivation gradient in impulsivity, but hunger over-rides other individual differences, making all individuals highly impulsive regardless of developmental history (Fig. 1b). The opposite of this would be finding that there was no deprivation gradient in impulsivity when individuals were satiated, but that hunger reinstates this gradient, for example because childhood deprivation has sensitized individuals to adult cues of hunger (Fig. 1c). Our search for interactive influences is motivated by previous research. For example, Nettle and Bateson (2017) found that the negative health effects of low adult socioeconomic position were more marked if individuals had experienced childhood deprivation; and Griskevicius et al. (2011) found that a current cue of environmental adversity had different effects on the impulsivity of people who had experienced different levels of childhood deprivation (though see Pepper et al. 2017).
Fig. 1

Hypothesised combined effects of hunger and childhood socioeconomic deprivation on impulsivity. a Individuals with greater experience of deprivation are more impulsive; hunger increases impulsivity by a constant amount regardless of deprivation, and hence the two influences are additive. b In satiated individuals, those with greater experience of deprivation show greater levels of impulsivity; hunger makes all individuals highly impulsive, removing the deprivation gradient. c There is no difference between individuals based on their experiences of deprivation when satiated; individuals with greater experience of deprivation respond more strongly to cues of hunger, producing a deprivation gradient only when hungry. Further interactions other than those shown in b and c are also possible

It is important to understand how hunger and socioeconomic background may interact to influence delay discounting as it has been suggested that people from poorer backgrounds may also be exposed to greater hunger (Nettle 2017). For example, pupils from deprived areas are more likely to attend school without eating breakfast first (Hoyland et al. 2012). If hunger interacts with other aspects of socioeconomic background, there could be a multiplier effect. Investigating the interaction of hunger and socioeconomic background may also be beneficial for predicting the efficacy of potential nutritional interventions for specific groups (for example, breakfast clubs, Hoyland et al. 2009). If we find an effect similar to 1c, this could suggest that the differences in impulsivity seen between people of high and low socioeconomic background could be circumvented with simple interventions aimed at reducing hunger. The pattern seen in Fig. 1b, by contrast, would suggest that such interventions would not reduce socioeconomic discrepancies at all.

Overview of the Studies

In order to investigate whether hunger and childhood socioeconomic deprivation have additive or interactive effects on impulsivity, we conducted three studies incorporating delay discounting tasks. Two of these were laboratory studies in which hunger was experimentally manipulated by having participants attend sessions in the morning after having eaten, or having abstained, from breakfast. The third was a survey study which simply measured variation in current hunger.

The first lab study (study 1) investigated a delay discounting task called the experiential discounting task, or EDT (Reynolds and Schiffbauer 2004). This computer-based task is distinctive in that participants actually receive the rewards they choose, and have to wait to receive these rewards as well, unlike in hypothetical discounting tasks which are often employed in the human literature. Because the delays are really experienced, this task can be considered to be analogous to the behavioural delay discounting task employed by Bateson and colleagues (Bateson et al. 2015) when investigating impulsivity in starlings. To our knowledge the EDT has yet to be employed to investigate hunger or childhood socioeconomic deprivation, though there are findings exploring EDT performance and other current states. Both sleep deprivation (Reynolds & Schiffbauer 2004) and alcohol consumption (Reynolds et al. 2006) were found to increase impulsivity in the EDT, with the latter having no effect on performance when measured using a standard hypothetical delay discounting task. This suggests that the EDT may be more sensitive to some current state changes than standard hypothetical measures. In order to investigate this, the second lab study (study 2) had a similar design to study 1, but used a more conventional hypothetical discounting task. The third study consisted of a survey in which hunger was measured but not manipulated, again incorporating a hypothetical discounting task. The aim of this study was to recruit a larger sample, with a broader range of childhood socioeconomic deprivation, something that was somewhat restricted in studies 1 and 2.

It should be noted that the studies were conducted in the order 1, 3, 2, but for ease of interpretation have been presented here in the order 1, 2, and 3. Although the studies have differences of design, they all measure impulsivity (delay discounting), childhood socioeconomic deprivation, and current hunger. Thus, the evidence from the three can be combined to give greater inferential precision about our overarching question than from each study alone. As well as the analyses of each individual dataset, we therefore also present meta-analyses of the comparable measures from the three studies.

Study 1

Introduction

In study 1, we manipulated hunger by assigning volunteers to either breakfast as normal, or to abstain from breakfast, on the day of the testing session. The participants were drawn from a university participant pool, and we relied on finding sufficient variation in childhood socioeconomic deprivation in this pool, as we have done before (Paál et al. 2015). Impulsivity was assessed using the EDT.

Methods

Ethical Approval

All studies received ethical approval from Newcastle University Faculty of Medical Sciences Ethics Committee (application No: 1281/14850 & 01224/9886). All participants gave informed consent to participate.

Participants

Ninety-five participants were recruited via a university participant pool (mostly non students, age range 18–77, consisting of 68 women and 27 men). Hunger was manipulated in the study by having participants attend the experimental session either after having eaten their breakfast as normal (breakfast condition), or after skipping breakfast (no breakfast condition). All sessions were held in the morning, and were scheduled within 1–3 h of participants’ usual waking time. Participants were assigned to the breakfast or no breakfast conditions on an alternate sign up basis (no breakfast: n = 48, Mean age = 32.6, SD = 16.4, 18–67; breakfast: n = 47, Mean age = 33.6, SD = 17.1, 18–77).

Materials

All of the following measures were presented on a computer using Inquisit (Millisecond, Seattle, WA; www.millisecond.com) software. In order to check our manipulation, self-reported hunger was measured using a 7 point Likert scale (1 = ‘very full’, 7 = ‘very hungry’), and participants reported approximately how many hours it had been since they had last eaten. In addition to age and sex, participants were asked to provide their childhood postcode (specifically from age 5) – this was later used to establish scores of childhood socioeconomic deprivation using the English Indices of Multiple Deprivation (IMD). These scores are calculated by the UK government for small geographic areas based on income, employment, education, health, crime, barriers to housing and services, and finally, living environment. A higher IMD score indicates greater neighbourhood socioeconomic deprivation. Wales, England, Scotland, and Northern Ireland have their own IMD scores, and while these measure similar information, they are not directly comparable, and so for the purposes of this study we included IMD scores only from England. Participants growing up in the other parts of the UK, or elsewhere, have missing values for IMD.

Participants completed the computer-based Experiential Discounting Task (EDT). Participants are presented with repeated choices between monetary SSRs and LLRs. The script used for this task was obtained from the Inquisit test library (http://www.millisecond.com/download/library/), and was based on the EDT task described by Reynolds and Schiffbauer (2004). Participants are initially presented with instructions for the task. For each trial of the game participants are presented with two light bulbs on the screen, underneath which there are two different amounts of money. One amount is always 30p (the LLR) and the other is less than 30p (the SSR). The LLR is delayed by either 0, 7, 14 or 28 s depending on the round. The SSR is received immediately. Participants click a green start button to begin the trial, at which point the lightbulbs light up and the participant has to select one. Reward delivery for the LLR is probabilistic (probability of delivery 0.3). If participants receive the money from the bulb they have selected then a bank symbol lights up and they then click this to end the trial. Their cumulative winnings are displayed at the bottom of the screen. An adjusting-amount procedure is used to ascertain stable indifference points (IP) for each participant - every time the LLR is chosen the value of the SSR increases, and every time the SSR is chosen the SSR value decreases. The IP is the point at which the SSR and LLR are of equal subjective value. Therefore a smaller IP shows greater impulsivity. If participants choose the same lightbulb 4 times in a row they are forced to choose the other lightbulb in the next trial. Each round consists of a minimum of 16 trials. If after 16 trials the IP can be established (If the participant chose the same number of SSR and LLRs across the last 6 trials) then the round ends. If the IP cannot be established the trials continue, with the programme checking after each new trial if an IP can be established. Each round has a set duration (20 x the delay length of that round). The round will end either when this has been reached or an IP has been found. If the round ends before the maximum duration then the additional time is added to an inter-round interval. For these reasons the number of trials which each participant completes varies for each round. As the script had been written for an American audience we changed dollars to pounds at a ratio of 1:1. Each participant completed 5 rounds of the game, with each round containing multiple trials of SSR/LLR choices; 1 practice round, followed by 4 rounds with varying delays to reward delivery (0 s, 7 s, 14 s, 28 s), presented in ascending order. After the practice round participants are presented with a shortened version of the instructions before they begin the task.

Procedure

Participants were allocated to a condition on an alternate sign up basis, and given instructions regarding breakfast in advance of the sessions via email. In the breakfast condition participants were told: ‘We would like you to come to the session having eaten breakfast – please do not skip breakfast before the study, and try to eat within 1-1.5 hours before the study. You will be asked when you last ate.’ In the no breakfast condition they were told: ‘We would like you to come to the session having not eaten breakfast, having not eaten anything since the evening before. You will be asked what time you last ate. Drinking water/tea/coffee is fine, but please avoid any high sugar energy drinks such as smoothies/protein shakes/fizzy drinks/fruit juice/milk.’ Sessions lasted between 45 and 60 min. Participants completed demographic information and self-reported hunger, provided their childhood postcode, and then completed the EDT. Participants received their winnings from this game in cash (between £6 and £15). Finally, participants completed the same self-report hunger measure again before being debriefed.

Data Analysis

Our main outcome variable was IP. Utilising the area under the curve formed by the four indifference points (AuC) has been posited as a useful measure of discounting (see Myerson et al. 2001). However, we were unable to do this with our data set. Calculating AuC assumes that each IP is less than the previous one, and should produce a value less than or equal to 1. Some of the discounting functions displayed by our participants did not follow this pattern, resulting in AuC values of greater than 1. For this reason we decided to use the individual IP for each delay length as our measure of impulsivity and our dependent variable.

There were a few cases for individual participants where the EDT was unable to establish an IP. In delay A (0 s) 9 participants did not establish an IP, but they had all plateaued at an SSR of 0.2399999999999999911 and so we gave them all a value of 0.25 which we believe to be a conservative estimate of their actual IP (0.25 is 0.1 higher than the highest observed IP). Two participants did not establish an IP for delay B (7 s) and these were left missing. Finally, 3 participants failed to establish an IP in delay D (28 s). These participants were mostly only selecting the SSR, resulting in the SSR value plateauing at 0.0600000000000000047. We gave all of these participants an IP of 0.05, which we believe to be a conservative estimate of their true valuation.

The main predictor variables were childhood socioeconomic deprivation (IMD score) and experimental condition. We were unable to get IMD scores for 28 participants for a variety of reasons. Some participants did not provide a valid postcode, some postcodes provided were not from England, and for some postcodes an IMD score was not available (this can happen with very old postcodes). In some instances participants had provided a street name, and if this could be identified then the postcode for that street was substituted. In instances where a street had multiple postcodes an average IMD score was calculated. Participants without an IMD score were excluded from any analyses that incorporated this measure (this was done across all three studies).

We were unable to establish an IMD score for 28 participants and so analyses below involving IMD were run on a sample of 67 individuals. In the breakfast condition there were 25 women and 12 men (mean age = 35.2, SD = 18.57, range = 18–73), who had an average IMD of 15.51 (SD = 11.72, range = 2–39). In the no breakfast condition there were 25 women and 5 men (mean age = 33.07, SD = 16.97, range = 18–77), with a mean IMD of 19 (SD = 17, range = 3–42). We found no significant difference in the IMD scores of our participants in the two conditions, t(49.67) = −.95, p = .35.

A combination of linear models (using base package in R) and linear mixed models (using the nlme package in R; Pinheiro et al. 2018) were fitted to see if childhood socioeconomic deprivation and experimental condition predicted impulsivity. All models used in the analyses (and the analyses for the subsequent studies) satisfied the assumptions of normally distributed residuals and homogenous variance of residuals within the fitted values of the models. Raw data and R scripts for the analysis are freely available via the Zenodo repository at: https://zenodo.org/record/1402599#.XECslFz7Sck.

We did not control for age or sex in any of the models we ran, as we found no consistent effects of either across the three studies. In study 1 we did find that IP from the longest delay was significantly predicted by age (with older people being less impulsive). However, including age as an additional predictor in our models did not alter any of the conclusions presented below.

Results

Manipulation Check

The no breakfast group had a significantly higher mean hunger score than the breakfast group at both the start, t(92) = −11.53, p < .001 (no breakfast M = 5.04 SD = 1.27, Breakfast M = 2.40 SD = 0.92), and end of the study, t(93) = −9.72, p < .001 (no breakfast M = 5.31 SD = 1.37, breakfast M = 2.67 SD = 1.29). These differences in hunger between the groups were maintained when only analysing data from participants who we had an IMD score for (hunger at start, t(50.18) = −9.69, p = <.001; hunger at end, t(58.98) = −8.67, p = <.001).

Models Predicting Impulsivity

As it has previously been suggested that hunger and measures of socioeconomic status may be associated (Nettle 2017) we initially ran a model to see if hunger was predicted by childhood IMD, whilst controlling for time since participants had last eaten, finding that time since a person had eaten did predict hunger but childhood IMD did not (Appendix, Table 5, Model 1).

For the main analysis, the outcome variable was indifference point. We fitted a linear mixed model with delay length, condition, IMD and their interactions as fixed predictors, and participant as a random effect (see Table 1; model adjusted R2 .023). As impulsivity measured via delay discounting is defined as the systematic devaluation of an outcome as the delay to its delivery increases, impulsivity is captured by the rate with which indifference point reduces as the delay increases. Therefore, significant interactions between condition and delay length, or IMD and delay length, would provide evidence that condition and IMD respectively were related to impulsivity. Any non-additive influence of condition and IMD on impulsivity would manifest as a three-way interaction between condition, IMD, and delay length. We found no evidence for such a three-way interaction (see Table 1). We did, however, find a main effect of delay length, and a marginally non-significant interaction between condition and delay length (see Table 1 and Fig. 2). As Fig. 2 shows, no breakfast participants had slightly lower indifference points than breakfast participants when the delay was long, but not when there was no delay.
Table 1

Parameter estimates for predictors of indifference point in Study 1

Outcome variable

Random effects

Predictors

B(±SE)

p

Indifference point

Participant ID

Condition(breakfast)

.001(.02)

.94

  

Delay Length

−.01(.003)

<.001*

  

IMD

−.001(.001)

.13

  

Condition * Delay Length

−.01(.01)

.059

  

Condition * IMD

<.001(<.001)

.62

  

Delay Length * IMD

<−.001(<.001)

.51

  

Condition * Delay Length * IMD

<.001(<.001)

.18

*p < 0.001

Fig. 2

Mean indifference points (±1SEM) across EDT delays for the breakfast and no breakfast conditions in study 1

In order to investigate the near-significant interaction between condition and delay length further, we fitted a linear model using only the indifference points from the longest delay length (delay D, 28 s), and our two predictors of interest, IMD and condition. This model yielded a significant interaction between condition and IMD, as well as significant main effects of condition and IMD, with an adjusted R2 of .048 (see Table 2 and Fig. 3). Figure 3 suggests that at low levels of deprivation, breakfasted individuals are less impulsive (they have a higher indifference point) than no breakfast individuals. However, as childhood socioeconomic deprivation increases, the difference in impulsivity between the two groups becomes less clear.
Table 2

Parameter estimates for predictors of indifference points in delay D (28 s)

Outcome variable

Predictors

B(±SE)

p

Indifference point (28 s delay only)

Condition (breakfast)

−.05(.02)

.02*

IMD

−.001(<.001)

.0498*

Condition * IMD

.002(<.01)

.03*

*p < 0.05

Fig. 3

Scatterplot of the indifference point recorded at the longest delay (28 s) against IMD score for the two experimental groups in study 1 Fit lines represent simple regressions and shaded areas represent 95% confidence intervals

Discussion

Our breakfast manipulation appears to have been successful in bringing about a substantial difference in hunger between the two groups. The findings from study 1 provided some support for the literature, albeit marginally non-significant in the main analysis involving all the delays: the participants in the no breakfast condition had lower indifference points as the delay became long, suggesting greater impulsivity. When considering the longest delay alone we found significant main effects of condition and of IMD. These were in the predicted directions: no breakfast condition, and greater deprivation, both had negative parameter estimates. Thus, greater hunger or greater deprivation were associated with lower indifference points and hence greater impulsivity. The model considering the longest delay alone also provided some evidence of an interaction between IMD and hunger condition in predicting impulsivity, though this was not seen (in the form of a three-way interaction) in the larger model involving all delays. It seems that at low levels of childhood socioeconomic deprivation, the hunger manipulation affected indifference points in the way that we would expect, but that as deprivation increased our experimental manipulation appears to have less of an effect on indifference points. In terms of our initial models, this is closest to that illustrated in Fig. 1b. However, this interpretation should be taken with caution, and readers should note that we recruited only a small number of participants who had high levels of childhood socioeconomic deprivation, as seen in Fig. 3.

Study 2

Introduction

Previous findings have reported effects of current state on impulsivity measured using the EDT, but not when measured using a hypothetical choice task (Reynolds et al. 2006). However, hypothetical choice tasks, where the participant states a preference between a specified SSR and a specified LLR, but does not in fact have to endure the delay, are much more widely used. Study 2 aimed to replicate study 1 using an identical procedure, with the substitution of a hypothetical monetary delay discounting choice task (HMDT), with the aim of comparing findings across the two studies. A difference between the EDT and HDMT is that the HDMT provides just one variable per participant (the number of LLRs chosen), rather than the four indifference points per participant produced by the EDT.

Methods

Participants

Ninety-three Psychology Undergraduate students (14 men and 79 women) took part in exchange for required course credits (no breakfast: n = 46, Mean age = 19.36, SD = 11.18, range = 18–22; breakfast: n = 47, Mean age = 19.57, SD = 1.28, range = 18–25).

Materials

In place of the EDT of study 1, participants completed a pen and paper HMDT which contained 20 questions asking participants to choose between a variable small amount of money which they could receive immediately (SSR) and a larger amount of £100 which they could receive in a year’s time (LLR). The values chosen for the choices reflected the range of discount rates that we have used in previous studies (Pepper & Nettle, 2013) and are available via the Zenodo repository https://zenodo.org/record/1402599#.XECslFz7Sck.

Procedure

Hunger was experimentally manipulated using the same method as described in study 1. Participants were assigned to conditions on an alternate sign up basis and attended a lab session lasting approximately 30 min. They completed basic demographic measures, as well as a Likert current hunger scale, and amount of time since they had last eaten, as described in the previous study. Participants then completed the HMDT. At the end of the study they completed a final Likert rating of current hunger before being debriefed.

Data Analysis

The dependent variable was the number of LLRs chosen. Hence, a higher number indicates lower impulsivity. As in study 1, the main predictor variables were childhood socioeconomic deprivation (IMD score) and experimental condition – breakfast or no breakfast. We were unable to obtain IMD scores for 18 participants, for reasons discussed in study 1, and so analyses below involving IMD were run on a sample of 75 participants. In the breakfast group there were 31 women and 9 men (mean age19.6, SD = 1.35, range = 18–25) with an average IMD score of 16.8 (SD = 14.59, range = 4–68). In the no breakfast group there were 33 women and 2 men (mean age = 19.28, SD = 1.1, range = 18–22), with an average IMD score of 12.4 (SD = 7.08, range = 2–31). There was no significant difference between the IMD scores of our two experimental groups, t(58) = 1.68, p = .098. Data were analysed using linear models from the base package in R.

Results

Manipulation Check

Independent samples t-tests indicated that the no breakfast group had a significantly higher mean hunger score than the breakfast group at both the start, t(89) = −10.28, p < .001 (no breakfast M = 4.84 SD =1.03, breakfast M = 2.52 SD = 1.13), and end of the study, t(88) = −12.47, p < .001 (no breakfast M = 5.29 SD =1 .05, breakfast M = 2.60 SD = 0.99). These differences in hunger between the groups were maintained when only analysing data from participants who we had an IMD score for (hunger at start, t(63.77) = −8.77, p = <.001; hunger at end, t(70.52) = −10.52, p = <.001).

Models Predicting Impulsivity

Participants in the breakfast group had a mean LLR of 12.11 (SD = 4.8, range = 1–20) and participants in the no breakfast group had a mean LLR of 11.78 (SD = 5.16, range = 4–20). As in study 1 we first ran a model to see if current hunger was predicted by childhood socioeconomic deprivation, finding that it was not, though it was significantly predicted by the amount of time it had been since a participant had last eaten (Appendix, Table 5, Model 2). We then fitted a linear model with an outcome variable of number of LLRs chosen and two predictor variables, IMD and condition. There were no significant main effects or interactions (R2 = <−.001, Table 3).
Table 3

Parameter estimates for models predicting impulsivity (number of larger later rewards selected on the hypothetical monetary decision task) from condition, childhood socioeconomic deprivation (IMD), and their interaction in study 2

Outcome variable

Predictors

B(±SE)

p

Number of larger later rewards chosen

Condition(breakfast)

2.02(2.17)

.36

IMD

.06(.05)

.28

Condition * IMD

−.21(.14)

.14

Discussion

Study 2, considered individually, failed to find any clear support for an association between either childhood socioeconomic deprivation or hunger condition and impulsivity, as measured using the HMDT. As the sample size and hence power was similar to study 1, this may suggest that the experiential EDT task of study 1 is a more sensitive measure of impulsivity than the HMDT of study 2. This could be because the delays actually have to be endured in the EDT, or because the delays are much shorter: it is possible that current state and developmental differences in impulsivity are most evident when measuring the discount rate over very short intervals. It should be noted that study 1 and 2 have samples with quite different age ranges, and this may also underlie the differences in our findings. However, as mentioned previously, we found no consistent effects of age on impulsivity across our studies. We return to this issue in the meta-analysis where we formally compare the results of the three studies.

Study 3

Introduction

Study 3 was conducted with the main aim of recruiting a broader sample of participants in the hope of increasing the number of individuals with higher childhood socioeconomic deprivation scores, which we were lacking in the first two studies. This would provide greater power to replicate the interaction between hunger and socioeconomic deprivation that was suggested by the analysis of the longest delay from study 1. In order to do this we decided to recruit participants from public spaces (shopping centres) in the hopes that by removing the effort and cost required to attend a lab session, and by going outside the university, we would recruit a broader, as well as larger, sample. To make the study as quick and easy as possible for participants we used the HDMT, as in study 2. As this was not a laboratory-based study we employed a survey design wherein rather than manipulating hunger and time since eating, we measured the naturally-occurring variation in them (data collection took place across 7 days between the hours of 10 am and 5 pm).

Methods

Participants

An opportunity sample of 330 participants (241 women, 89 men, Mean age: 35.84, SD: 13.71, range 16–79) were recruited from two large UK shopping centres. Participants were entered into a prize draw for taking part, for which there was one prize of £100 in shopping vouchers.

Materials

Participants completed a pen and paper survey. Current hunger was measured using the 7 point Likert scale as before, and participants also recorded approximately how many hours it had been since they had last eaten. Childhood (age 5) postcode was collected in order to obtain IMD score, and participants additionally completed a self-report Material Needs Scale (Conger et al. 1994), thinking back to when they were 5 years old. This included 7 Likert questions related to whether they felt that their family had enough money for specific items (food, clothing, bills etc.). We included this additional measure as an insurance policy in case we had a large number of participants without a childhood postcode from England (or an English postcode that we were unable to obtain an IMD score for). Finally, Participants completed the same HMDT used in study 2, consisting of 20 choices between SSRs and LLRs, which contained a fixed delay to reward of 1 year.

Data Analysis

The dependent variable was again the number of LLRs chosen on the HMDT (more LLRs being less impulsive). As in the previous two studies, the main predictor variables were childhood socioeconomic deprivation and hunger. In this study there were two variables for each of these measures; hence we present all four possible models in parallel. Linear models were run using the base package in R.

Self-reported hunger (Hunger), and hours since eating (Hours) were moderately positively correlated with one another, r(328) = .49, p < .001. IMD and the material needs score were weakly positively correlated, r(250) = .23, p < .001. Every participant completed the material needs scale (mean = 14.58, SD = 6.95, range = 7–35), but we were unable to establish IMD scores for 78 participants, and so analyses below involving IMD were run on a sample of 252 participants (190 women, 62 men, mean age = 35.48, SD = 13.73, range = 16–79, mean IMD = 24.16, SD = 15.68, range = 1–74). Power is therefore greater for analyses using the material needs scale, though this may be measuring something rather different from IMD score. Two participants failed to record their age but were included in the analysis as they had completed the other measures.

Results

Models Predicting Impulsivity

As in the previous two studies we initially investigated whether there was any relationship between current hunger, time since eating, and childhood socioeconomic deprivation. Two separate models were run, one including IMD and the other including material needs, both of which found that only time since eating significantly predicted current hunger (Appendix, Table 5, models 3 & 4).

In this study we had two measures of childhood socioeconomic deprivation (IMD and Material Needs score) and two measures of hunger (self-reported hunger and Hours since eating) and so four linear models were run in order to incorporate all of these using the outcome variable number of LLRs (Table 4). The two measures of childhood socioeconomic deprivation, IMD and material needs score, significantly predicted impulsivity in each model in which they appeared, and in the predicted direction: people from more deprived childhood backgrounds chose fewer LLRs. Self-reported hunger did not significantly predict number of LLRs chosen in either model in which it appeared, although the parameter estimates were in the predicted (negative) direction. Hours since eating significantly predicted LLRs chosen when coupled with IMD; and was marginally non-significant when coupled with material needs scores. The direction in both cases was as predicted (more hours since eating predicts fewer LLRs chosen). There was no evidence of interactions between the hunger variable and the childhood deprivation variable in any of the four models.
Table 4

Parameter estimates for models investigating predictors of number of larger later rewards chosen in the HMDT in study 3. Significant predictors are shown in bold

Model

Outcome

Predictors

B(±SE)

p

Model R2

1

Number of larger later rewards

Hunger

−.39(.34)

.25

.024

  

IMD

−.08(.04)

.047*

 
  

Hunger * IMD

.01(.01)

.56

 

2

Number of larger later rewards

Hunger

−.51(.37)

.17

.015

  

Material Needs

−.18(.07)

.02*

 
  

Hunger * Material Needs

.03(.02)

.26

 

3

Number of larger later rewards

Hours

−.33(.18)

.07

.036

  

IMD

−.08(.03)

.01*

 
  

Hours * IMD

.01(.01)

.41

 

4

Number of larger later rewards

Hours

−.45(.20)

.03*

.027

  

Material Needs

−.17(.06)

.004*

 
  

Hours * Material Needs

.02(.01)

.10

 

*p < 0.05

Discussion

Study 3 provided consistent evidence that childhood socioeconomic deprivation was associated with more impulsive decision making, whether this was measured by postcode IMD or by responses on the material needs scale. There was some evidence that hunger was also associated with impulsivity, though this was only found with hours since eating, and not with self-reported hunger score, and then was marginally non-significant in one of the two models run. However, it should be noted that parameter estimates for hunger and hours since eating were always in the predicted direction, with increases in time and hunger predicting more impulsive responses. Study 3 provided no evidence of an interactive effect of hunger and early life adversity on impulsivity.

Meta-Analysis of Studies 1, 2 & 3

In order to establish whether the data from our studies combined supported interactive or additive effects of hunger and childhood socioeconomic deprivation on impulsivity, we meta-analysed the findings from studies 1, 2, and 3. We only used continuous variables which were available from all three of the studies (these were scaled). The outcome variable used was always impulsivity (either indifference point from the EDT which was taken from the longest delay, or number of LLRs from the HMDTs – all these variables are aligned in the same direction, with a higher indifference point and a greater number of LLRs both indicating a lower level of impulsivity). The predictor variable used for childhood socioeconomic deprivation was IMD score, which was available across the three studies. For hunger we used both the measure of self-reported hunger, and the number of hours since people had last eaten as our predictor variables, since these were both also available across the three studies.

Two linear models were run for each study, the first investigating whether impulsivity was predicted by self-reported hunger score and IMD, including the interaction effect, and the second investigating whether impulsivity was predicted by hours since food and IMD, again including the interaction effect. Parameter estimates from the individual models were combined and subjected to meta-analysis using R package ‘metafor’ (Viechtbauer, 2010). We chose a random-effects model to obtain the meta-analytic parameter estimates, since there is variation in the designs of the studies, and they cannot therefore be considered exact replications of the same measurement.

As can be seen from Fig. 4 the combined data provide evidence for significant effects of hunger (Fig. 4a) and time since eating (Fig. 4b) on impulsivity, with greater hunger associated with greater impulsivity. There is also evidence for a significant overall association between IMD and impulsivity (greater IMD, greater impulsivity; Fig. 4 c, d). However, there is no overall evidence for any interaction between hunger/h and IMD (Fig. 4e, f). Furthermore, I2 and Cochrane’s Q values indicate that findings relating to hunger, time since eating, and IMD are homogenous across the three studies (Fig. 4). Despite the marginally non-significant Cochrane’s Q values, the high I2 values for the interaction models suggest that the interaction effects have a high level of heterogeneity across the three studies (Fig. 4).
Fig. 4

Forest plots from meta-analyses across the three studies. All models have impulsivity as the outcome variable. Central estimates of effect size and 95% confidence intervals are shown. The outcome variable in all cases is represented so that a lower number indicates greater impulsivity. Therefore, a negative parameter estimate (to the left of the 0 line) indicates that the predictor was associated with greater impulsivity. Models show the main effects of self-reported hunger (a), time since eating in hours (b), IMD in models with self-reported hunger (c), IMD in models with time since eating in hours (d), the interaction between self-reported hunger and IMD (e), and the interaction between time since eating in hours and IMD (f)

General Discussion

We have reported three separate studies aiming to establish whether current state (hunger) and developmental history (childhood socioeconomic deprivation) have interactive or additive effects on adult levels of impulsivity, measured via delay discounting tasks.

Studies 1 and 2 were both experimental studies in which hunger (current state) was manipulated, and study 3 was a survey in which current hunger was simply measured. Studies 1 and 2 yielded seemingly contradictory findings, with tentative evidence of an interactive effect shown in study 1 (for the longest delay) but evidence of no effect of either hunger or childhood deprivation seen in study 2. While these studies followed a similar procedure they differed in the discounting measure employed, which may account for this discrepancy. Study 1 utilised an experiential task (EDT) in which participants actually had to endure delays and also received the rewards, unlike in the hypothetical task used in study 2. Previous research investigating other current states has also found this discrepancy between delay discounting measured hypothetically and experientially (Reynolds et al. 2006), suggesting that hypothetical measures are perhaps not as sensitive to state induced changes in discounting. In the meta-analysis we subsequently performed, the standardized parameter estimates for hunger and childhood deprivation were similar for studies 1 and 2. However, their precision was greater in study 1 (explaining the significant effects in that study). Thus, it may be that the experiential measure is subject to less random variability and is thus more powerful for capturing associations of other variables with impulsivity. It should also be noted that these two tasks also differ in the delay lengths used, with the EDT delays ranging from 7 to 28 s, and the hypothetical task having a fixed delay of 1 year; this large difference in range of delays may also contribute to the seemingly inconsistent results seen. The difference between the hypothetical or real nature of the rewards may also explain our different findings. Wang and Dvorak (2010) had participants complete a monetary discounting task where they rolled a dice for a chance to win some of their choices. They found an effect of blood glucose differences on impulsivity, even though their delays ranged from 4 to 939 days, suggesting that it is the quality of the reward, real or hypothetical, rather than the delay length which may explain the differences between our findings in study 1 and study 2. We did find an effect of both current state and developmental history (though no interaction between the two) in study 3 which utilised the same hypothetical measure as study 1 but had a much larger sample size.

All three studies were moderately sized and so meta-analyses were conducted in order to increase statistical power, with the aim of resolving the variation in findings seen across the studies individually. The analyses conducted yielded evidence for the additive model shown in Fig. 1 a, and analogous to the findings reported by Bateson and colleagues (Bateson et al. 2015). Both hunger and childhood socioeconomic deprivation predicted delay discounting: hungrier people and those who were more deprived showed increased discounting of delayed rewards. However, there was no conclusive evidence for an interactive effect of these two variables on delay discounting when combining the three studies. It should be noted that while levels of heterogeneity across the three studies were extremely low for the main effects of hunger and childhood socioeconomic deprivation, these were higher for the interaction models which indicates that the pooled estimates shown are not reliable. Models E and F in Fig. 4 had acceptable Cochrane’s Q values, though this test has been found to be poor at detecting heterogeneity when only a small number of studies are employed (Higgins et al. 2003). I2 has been found to be a more robust measure of heterogeneity in these instances and indeed was found to be high in both of the interaction models, but zero for the main effects. The heterogeneity for the interaction effect may be driven by the differences between the experiential and hypothetical tasks, with tentative evidence of an interaction only being found in the former. It is also important to note that the models included in the meta-analysis had quite small effect sizes (adjusted R2 values ranging from −.001 to .036). Therefore, while our meta-analysis does suggest that hunger and socioeconomic background both explain variation in impulsivity measured across our studies, they do only explain a small proportion of the variance. This is likely why we failed to see consistent significant effects across our studies individually.

Though the evidence from the meta-analysis strongly suggests support for the additive model the authors would urge caution in completely ruling out the possibility of an interactive effect existing between hunger and childhood socioeconomic deprivation. The EDT has not been as widely used in the literature as traditional hypothetical measures, and indeed, our paper is the first to investigate delay discounting using this task and manipulating the current state of hunger. Furthermore, study 1 was moderately sized, and also failed to recruit a large number of individuals with a high level of childhood socioeconomic deprivation, something which is necessary in order to identify and clearly interpret an interaction if one exists. We recommend that further investigation is needed into the differences in response seen in these two types of tasks. It has also previously been hypothesised that individuals of lower socioeconomic status may simply be hungrier than other individuals, which may underlie differences seen in impulsivity (Nettle, 2017). We tested for this across all three of our studies finding no evidence of an association between current hunger and childhood socioeconomic statues when controlling for the amount of time since people had last eaten.

As discussed in the introduction, there is a wealth of evidence that higher levels of delay discounting have been linked to numerous negative health behaviours, including smoking and heavy drinking (Granö et al. 2004), and drug abuse (de Wit, 2009). Perhaps most interestingly in light of the current study investigating hunger, there is also a growing literature linking delay discounting to a range of disordered eating behaviours, all of which are costly both for the individual and for society. This coupled with the finding that more moderate levels of delay discounting seem to have protective effects on adolescent weight gain (Duckworth et al. 2010) really highlight the importance of understanding factors which underlie delay discounting in the population. If we can establish a way to manipulate delay discounting then this could facilitate the development of public health interventions. Our study has investigated two factors which we believe to be important in underlying delay discounting, finding evidence across our studies that both hunger and socioeconomic background seem to additively predict discounting behaviour in our samples. We now need to think about the practical implications of this additive finding. Our findings could be taken to suggest that the effects of pre-school breakfast programmes or improved school meals (for example) should be similar for children of all social backgrounds (at least, if levels of hunger are similar). However, evidence of strong interactions, such as those illustrated in Fig. 1b and c, would have suggested that such programmes might have very different levels of benefit, even at the same level of hunger, for different social groups within the population. In order to develop effective interventions future research should continue to focus on taking a multi-factor approach when investigating factors relating to impulsivity.

To conclude, we found evidence across three studies for additive weak effects of hunger and childhood socioeconomic deprivation on adult levels of delay discounting, taken here as a measure of impulsivity. It is important to understand factors which may be underpinning delay discounting, which along with other measures of impulsivity has been linked to a number of behaviours that are detrimental to the individual, and costly for society at large. The question of whether immediate state variables (such as hunger) and longer-term developmental influences (such as childhood socioeconomic deprivation) combine additively or interactively is an important one for individual differences research. It relates to the general issue of how life-course experiences accumulate to influence psychological processes and can be valuable in informing social intervention programmes.

Notes

Acknowledgements

The authors would like to thank Melissa Bateson, Gillian Pepper, Clare Andrews, Jonathon Dunn, and Amy Wallace for useful discussions of the studies, as well as Hannah Gardner, Louise Cunningham, Georgia Houston, Esme Bloomfield, and Lizzie Wilson for collecting data for Study 2.

This research has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No AdG 666669 COMSTAR).

Compliance with Ethical Standards

Conflict of Interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

References

  1. Bateson, M., Brilot, B. O., Gillespie, R., Monaghan, P., & Nettle, D. (2015). Developmental telomere attrition predicts impulsive decision-making in adult starlings. Proceedings of the Royal Society of London B: Biological Sciences, 282(1799), 20142140.CrossRefGoogle Scholar
  2. Brodsky, B. S., Ph, D., Oquendo, M., Ellis, S. P., Ph, D., Haas, G. L., et al. (2001). The relationship of childhood abuse to impulsivity and suicidal behavior in adults with major depression. American Journal of Psychiatry, 158, 1871–1877.CrossRefGoogle Scholar
  3. Chisholm, J. S. (1999). Attachment and time preference. Relations between early stress and sexual behavior in a sample of American university women. Human Nature, 10(l), 51–83.CrossRefGoogle Scholar
  4. Conger, R. D., Ge, X., Elder, G. H., Lorenz, F. O., & Simons, R. L. (1994). Economic stress, coercive family process, and developmental problems of adolescents. Child Development, 65(2), 541–561.CrossRefGoogle Scholar
  5. de Ridder, D., Kroese, F., Adriaanse, M., & Evers, C. (2014). Always gamble on an empty stomach: Hunger is associated with advantageous decision making. PLoS One, 9(10), e111081.CrossRefGoogle Scholar
  6. de Wit, H. (2009). Impulsivity as a determinant and consequence of drug use: A review of underlying processes. Addiction Biology, 14(1), 22–31.CrossRefGoogle Scholar
  7. Donohew, L., Zimmerman, R., Cupp, P. S., Novak, S., Colon, S., & Abell, R. (2000). Sensation seeking, impulsive decision-making, and risky sex: Implications for risk-taking and design of interventions. Personality and Individual Differences, 28(6), 1079–1091.CrossRefGoogle Scholar
  8. Duckworth, A. L., Tsukayama, E., & Geier, A. B. (2010). Self-controlled children stay leaner in the transition to adolescence. Appetite, 54(2), 304–308.CrossRefGoogle Scholar
  9. Ely, A. V., Howard, J., & Lowe, M. R. (2015). Delayed discounting and hedonic hunger in the prediction of lab-based eating behavior. Eating Behaviors, 19, 72–75.CrossRefGoogle Scholar
  10. Granö, N., Virtanen, M., Vahtera, J., Elovainio, M., & Kivimäki, M. (2004). Impulsivity as a predictor of smoking and alcohol consumption. Personality and Individual Differences, 37(8), 1693–1700.CrossRefGoogle Scholar
  11. Griskevicius, V., Tybur, J. M., Delton, A. W., & Robertson, T. E. (2011). The influence of mortality and socioeconomic status on risk and delayed rewards: A life history theory approach. Journal of Personality and Social Psychology, 100(6), 1015–1026.CrossRefGoogle Scholar
  12. Higgins, J. P., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. BMJ: British Medical Journal, 327(7414), 557–560.CrossRefGoogle Scholar
  13. Hill, E. M., Jenkins, J., & Farmer, L. (2008). Family unpredictability, future discounting, and risk taking. Journal of Socio-Economics, 37(4), 1381–1396.CrossRefGoogle Scholar
  14. Hoyland, A., Dye, L., & Lawton, C. L. (2009). A systematic review of the effect of breakfast on the cognitive performance of children and adolescents. Nutrition Research Reviews, 22, 220–243.  https://doi.org/10.1017/S0954422409990175.CrossRefPubMedGoogle Scholar
  15. Hoyland, A., Mcwilliams, K. A., Duff, R. J., & Walton, J. L. (2012). Breakfast consumption in UK schoolchildren and provision of school breakfast clubs. Nutrition Bulletin, 37, 232–240.  https://doi.org/10.1111/j.1467-3010.2012.01973.x.CrossRefGoogle Scholar
  16. Kirk, J. M., & Logue, A. W. (1997). Effects of deprivation level on humans' self-control for food reinforcers. Appetite, 28(3), 215–226.Google Scholar
  17. Loeber, S., Grosshans, M., Herpertz, S., Kiefer, F., & Herpertz, S. C. (2013). Hunger modulates behavioral disinhibition and attention allocation to food-associated cues in normal-weight controls. Appetite, 71, 32–39.CrossRefGoogle Scholar
  18. Myerson, J., Green, L., & Warusawitharana, M. (2001). Area under the curve as a measure of discounting. Journal of the Experimental Analysis of Behavior, 76(2), 235–243.CrossRefGoogle Scholar
  19. Nettle, D. (2017). Does hunger contribute to socioeconomic gradients in behaviour? Frontiers in Psychology, 8, 358.CrossRefGoogle Scholar
  20. Nettle, D., & Bateson, M. (2017). Childhood and adult socioeconomic position interact to predict health in mid life in a cohort of British women. PeerJ, 5, e3528.CrossRefGoogle Scholar
  21. Paál, T., Carpenter, T., & Nettle, D. (2015). Childhood socioeconomic deprivation, but not current mood, is associated with behavioural disinhibition in adults. PeerJ, 3, e964.CrossRefGoogle Scholar
  22. Pepper, G. V., & Nettle, D. (2013). Death and the time of your life: Experiences of close bereavement are associated with steeper financial future discounting and earlier reproduction. Evolution and Human Behavior, 34(6), 433–439.CrossRefGoogle Scholar
  23. Pepper, G. V., Corby, D. H., Bamber, R., Smith, H., Wong, N., & Nettle, D. (2017). The influence of mortality and socioeconomic status on risk and delayed rewards: A replication with British participants. PeerJ, 5, e3580.CrossRefGoogle Scholar
  24. Pinheiro J, Bates D, DebRoy S, Sarkar D and R Core Team (2018). Nlme: Linear and nonlinear mixed effects models. R package version 3.1-131. https://CRAN.R-project.org/package=nlme. Accessed May 2018.
  25. Rasmussen, E. B., Lawyer, S. R., & Reilly, W. (2010). Percent body fat is related to delay and probability discounting for food in humans. Behavioural Processes, 83(1), 23–30.CrossRefGoogle Scholar
  26. Reynolds, B., & Schiffbauer, R. (2004). Measuring state changes in human delay discounting: An experiential discounting task. Behavioural Processes, 67(3), 343–356.CrossRefGoogle Scholar
  27. Reynolds, B., Richards, J. B., & de Wit, H. (2006). Acute-alcohol effects on the experiential discounting task (EDT) and a question-based measure of delay discounting. Pharmacology Biochemistry and Behavior, 83(2), 194–202.CrossRefGoogle Scholar
  28. Steward, T., Mestre-Bach, G., Vintró-Alcaraz, C., Agüera, Z., Jiménez-Murcia, S., Granero, R., & Fernández-Aranda, F. (2017). Delay discounting of reward and impulsivity in eating disorders: From anorexia nervosa to binge eating disorder. European Eating Disorders Review, 25(6), 601–606.CrossRefGoogle Scholar
  29. Viechtbauer, W. (2010). Conducting meta-analyses in R with the metaphor package. Journal of Statistical Software, 36(3), 1–48.CrossRefGoogle Scholar
  30. Wang, A. X. T., & Dvorak, R. D. (2010). Sweet future: Fluctuating blood glucose levels affect future discounting. Psychological Science, 21(2), 183–188.CrossRefGoogle Scholar
  31. Weller, R. E., Cook, E. W., III, Avsar, K. B., & Cox, J. E. (2008). Obese women show greater delay discounting than healthy-weight women. Appetite, 51(3), 563–569.CrossRefGoogle Scholar

Copyright information

© The Author(s) 2019

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.School of PsychologyNewcastle UniversityNewcastle upon TyneUK
  2. 2.Institute of NeuroscienceNewcastle UniversityNewcastle upon TyneUK

Personalised recommendations