Advertisement

The effects of life stress and neural learning signals on fluid intelligence

  • Eva Friedel
  • Florian Schlagenhauf
  • Anne Beck
  • Raymond J. Dolan
  • Quentin J.M. Huys
  • Michael A. Rapp
  • Andreas HeinzEmail author
Open Access
Original Paper

Abstract

Fluid intelligence (fluid IQ), defined as the capacity for rapid problem solving and behavioral adaptation, is known to be modulated by learning and experience. Both stressful life events (SLES) and neural correlates of learning [specifically, a key mediator of adaptive learning in the brain, namely the ventral striatal representation of prediction errors (PE)] have been shown to be associated with individual differences in fluid IQ. Here, we examine the interaction between adaptive learning signals (using a well-characterized probabilistic reversal learning task in combination with fMRI) and SLES on fluid IQ measures. We find that the correlation between ventral striatal BOLD PE and fluid IQ, which we have previously reported, is quantitatively modulated by the amount of reported SLES. Thus, after experiencing adversity, basic neuronal learning signatures appear to align more closely with a general measure of flexible learning (fluid IQ), a finding complementing studies on the effects of acute stress on learning. The results suggest that an understanding of the neurobiological correlates of trait variables like fluid IQ needs to take socioemotional influences such as chronic stress into account.

Keywords

Reinforcement learning Prediction error signal Ventral striatum Stress Intelligence 

Introduction

Fluid intelligence (fluid IQ) [1, 2] describes the capacity for rapid problem solving and flexible adjustment to an ever-changing environment. Its expression is a general factor comprising attributes such as attention, cognitive speed, working memory, reasoning, and episodic memory that have manifold impacts on learning. One important neurobiological correlate of learning is the phasic activation of dopamine neurons in the VTA [3, 4, 5] which is reflected in BOLD fMRI studies in humans as a prediction error (PE) in a number of regions including the ventral striatum (VS) [6]. The prominent place of the VS PE in neurobiological accounts of learning derives from the fact that it is a particular type of teaching signal, which indicates the need for a change in expectation, as well as the direction and quantity of change necessary to acquire habits [7, 8, 9]. Animal experiments have shown that PE signaling is associated with phasic dopamine firing with the size of PE reflecting the amount of dopamine release [4, 10]. In humans, PE signaling has also been shown to phasically release striatal dopamine (indirectly measured via displacement of dopamine D2 receptor ligands) [11] and to be modulated by more tonic aspects of dopamine synthesis capacity measured over 1 hour [12].

Recently, we reported that individual differences in fluid IQ are associated with VS BOLD PE signals, with stronger VS BOLD PE correlates in subjects with higher IQ [13]. At the same time, there is recent evidence that acute stress increases BOLD responses elicited by aversive PE signals in the VS [14]. Furthermore, fluid IQ and stress are well known to interact, with stress having a strong moderating influence on cognitive abilities [15, 16], reward learning [17, 18], risk taking [19], reward responsivity [20], and decision-making speed [19]. Also, stress due to social exclusion situations impairs cognitive speed and accuracy [21]. However, despite strong evidence for an impact of stress on cognition and decision-making processes, little is known about the specific patterns and moderating trait components underlying these changes on a behavioral and neuronal level. Interestingly, it has recently been observed that (on a behavioral level) acute stress does not impair so-called model-free reward learning, while more cognitively demanding model-based reward learning is more affected by acute stress when working memory capacity is lower [16].

With respect to dopaminergic transmission, a series of animal experiments indicates that acute as well as chronic stress moderated dopamine release and may thus interact with dopamine-dependent PE signaling in the striatum [22, 23]. For instance, changes in cortisol levels during an acute stressor were correlated with increases in striatal responses during a decision-making task [24]. Given these interactions between stress, dopaminergic PE signals, and aspects of fluid intelligence, we wanted to explore the effects of chronic stress on PE signaling and fluid IQ. One possibility is that stress increases dopamine release, which in turn increases PE signaling, and in accordance with our previous observation [12], fluid IQ. However, this is unlikely given the predominantly negative interactions between stress exposure and fluid IQ (a measure closely related to cognitive capacity and cognitive speed). We therefore tested the hypothesis that stress exposure modulates PE signaling above and beyond the previously observed correlation between PE signaling and fluid IQ.

To assess PE-related VS activity, we regressed VS BOLD signals onto PEs derived from a simple learning algorithm in which PEs are accumulated over time to form predictions. We note that this simple learning algorithm is ‘model-free,’ in that it only requires subjects to iteratively track what outcomes are observed after their choices, but it does not require subjects to have any explicit model or understanding of the task. We then asked whether this correlation, henceforth termed the VS BOLD PE signal, (1) is associated with measures of chronic life stress and (2) whether chronic life stress and VS BOLD PE signal alone or in interaction contribute to individual differences in fluid IQ.

Materials and methods

Subjects and screening instruments

A group of 16 right-handed healthy men with a mean age of 38.4 years (SD = 11.9; range 22–61) underwent fMRI and neuropsychological testing as a subgroup of a sample previously reported [13]. Subjects with Axis I and II psychiatric disorders according to DSM IV were excluded through the Structured Clinical Interview, and drug abuse was further excluded with urine tests.

Neuropsychological assessment, intelligence measures, and stressful life events

A neuropsychological battery was administered within 2 months of fMRI measurements. Components of fluid and crystallized IQ were measured with an adaptation of the standard battery used in the Berlin Aging Study [25]. Fluid IQ was measured with a battery of nine tests comprising cognitive speed, attention and executive function, working memory, episodic memory, and reasoning. Cognitive speed was measured using the Digit Symbol Substitution test and the Reitan Trailmaking test, part A. Attention and executive function was measured using the Reitan Trailmaking test, part B [26], and Stroop [27] tests. Working memory was measured using forward and backward digit span tests [28]. Episodic memory was assessed using story recall with the German version of the Rivermead Behavioral Memory Test [29] and a German version of the auditory verbal learning test. Reasoning was measured using a test of figural analogies [30]. Fluid IQ was derived from a factorial analysis of the raw scores of each of these tests. Specifically, we used a Varimax rotation with an Eigenvalue cutoff set to 1.0; the final (single factor) solution accounted for 74.3 % of the variance in cognitive speed, attention, working memory, memory, and reasoning. Crystallized IQ was estimated using a verbal knowledge test, during which subjects are required to identify each single meaningful word from a total of 42 lists of five words.

Stressful life events (SLES) were assessed using the Life Events Scale [31], in which subjects were asked to self-report the presence of a stressful life event during the past 2 years from a list of 42 life events. We used the number of SLES as the main outcome measure.

Reversal learning task

During fMRI acquisition, subjects performed a reversal learning task (Fig. 1) known to evoke a BOLD PE signal in the striatum [13, 32]. In each of 200 trials (100 per session), subjects first saw two abstract targets on the screen and were asked to choose one of them as quickly as possible by pressing the left or right button with the left or right thumb on a response box (maximum response time: 2 s). A blue box surrounding their chosen target and feedback (either a green smiley face for positive feedback or a red frowning face for negative feedback) was simultaneously shown for 1 s. The trials were separated by a jittered interval of 1–6.5 s. Reward allocation was determined as follows: There were three types of reward allocation (i.e., block types): (1) 20 % for the left-hand response and 80 % for the right-hand response leading to reward, (2) 80 % for the left-hand response and 20 % for the right-hand response leading to reward, and (3) 50 % reward for the left-hand response and for the right-hand response. Block types changed unpredictably for the subject when two criteria were fulfilled: (1) minimum of 10 trials and (2) minimum of 70 % correct responses in the entire block. If subjects did not reach learning criteria after 16 trials, the task went over to the next block automatically.
Fig. 1

Probabilistic reversal task. Subjects first saw two abstract stimuli for up to 2 s (or reaction time). After selecting one with a button press, a blue frame surrounded the chosen target along with either positive (reward) or negative (loss) feedback. Feedback was displayed for 1 s, followed by a fixation cross for 1–6.5 s

Computational modeling of reinforcement learning

The trial-by-trial sequence of choices for each subject was fit by a simple Rescorla-Wagner (RW) model in which behavior is driven by the expectation of rewards and in which trial-by-trial behavioral adaptation is proportional to the difference between the expected and obtained rewards [7]. More specifically, the model assumes that the likelihood of choosing action a on trial t is proportional to the reinforcement Q t (a) the subject expects to receive on that trial. The proportionality between the expected value and the choice probability is given by the softmax rule, which defines the probability p(a) of a certain action a as a function of the expected value, Q t , as follows:
$$p\left( {a|Q_{t} } \right) \, = \, \exp \left( {Q_{t} \left( a \right)} \right) \, / \, \left( {\varSigma_{{a^{\prime}}} \exp \left( {Q_{t} \left( {a^{\prime}} \right)} \right)} \right)$$
Note that this simply turns expectations of rewards for actions into action probabilities, ensuring that p(a) > p(a′) if and only if Q(a) > Q(a′) and that 0 ≤ p(a) ≤ 1. The expected value Q t (a chosen) in turn is updated iteratively:
$$Q_{t} \left( {a_{\text{chosen}} } \right) \, = \, Q_{t - 1} \left( {a_{\text{chosen}} } \right) \, + \varepsilon \left( {R_{t} {-} \, Q_{t - 1} \left( {a_{\text{chosen}} } \right)} \right)$$
where ε is the learning rate and R t the reward obtained. The difference R t Q t−1(a) is the reward PE. Thus, if R t > Q t (a chosen), then Q t (a chosen) is increased, leading to a higher probability of being chosen on the next trial. Note that, here, the variable R t represents the (effective) reinforcement sensitivity as expressed by the effect of the reinforcement on the subject’s choice behavior. This variable takes on value R t  = β rew if a reward was obtained and R t  = −β pun if a punishment was obtained. For each individual, a learning rate ε’ and the reinforcement sensitivity for reward and punishment (β rew′ and β pun′) were computed as the maximum a posteriori estimates of these parameters using a Gaussian prior.
Model fitting parameter estimation was performed in a hierarchical model with empirical priors treating parameters as a random effect. For an in-depth description, please compare [33, 34]. Briefly, prior to fitting the models, the learning rate was inverse sigmoid transformed and the reward sensitivities were log-transformed. This enforced the constraint that 0 ≤ ε ≤ 1 and that β pun ≥ 0 and β rew ≥ 0. Letting θ = [ε′, β pun′, β rew′] denote the vector of transformed parameters, we report the maximum a posteriori estimates of these parameters using a Gaussian prior with mean and variance parameters μ and Σ:
$$\theta^{i}_{est} = \, \arg \max_{\theta } \log \, p(A^{i} |\theta )p(\theta |\mu ,\varSigma ) = \arg \max_{\theta } [\varSigma_{t} \log \, p(a^{i}_{t} |Q_{t} ,\theta )]p(\theta |\mu ,\varSigma )$$
where A i represents all the actions by subject i and where the dependence of each individual action probability on the parameters θ determining the Q value was emphasized. Importantly, we set the prior parameters empirically using expectation maximization to find the maximum-likelihood estimates of μ and Σ given all the data by all the subjects. Based on the parameters θ i for each of the subjects, a temporal sequence of PEs was computed for each subject i as follows:
$${\text{PE}}^{i}_{t} = \, R^{i} \left( t \right) \, {-} \, Q^{i}_{t - 1} \left( {a_{t} } \right).$$

Thus, rather than doing an individual maximum-likelihood fit, we did a maximum-likelihood fit of the group mean via expectation–maximization and then inferred the posterior parameter estimates for each individual. This reduces variability in the parameter estimates. One alternative in the literature is to assume that all subjects share one and the same parameter. This reduces variability in the regressors, and thus, the SPM beta estimates. The present procedure is what we believe to be a reasonable tradeoff between the two extremes.

fMRI protocol

fMRI acquisition. Functional imaging was conducted using a 3.0-Tesla GE Signa scanner with an eight channel phase array head coil to acquire gradient echo T2*-weighted echo-planar images as previously described [35, 36]. For each of the two sessions, 310 EPI volumes (~12 min) containing 29 slices were acquired [repetition time (TR) = 2,300 ms, echo time (TE) = 27 ms, matrix size 128 × 128, and a field of view (FOV) 256 × 256 mm2, thus yielding an in-plane voxel resolution of 2.7 mm2, flip angle α = 90 degree). Slices were acquired interleaved with a thickness of 4 mm and no gap. The acquisition plane was tilted 30° from the anterior–posterior commissure. A 3D anatomical image of the entire brain was obtained by using a T1-weighted 3D spoiled-gradient echo pulse sequence (TR = 7.8 ms, TE = 3.2 ms, matrix size 256 × 256, FOV 256 x 256 mm2, 1 mm slice thickness, flip angle α = 20°, voxel size 1 mm × 1 mm × 1 mm).

fMRI data preprocessing. Functional imaging data were analyzed using SPM8 (Wellcome Department of Imaging Neuroscience, Institute of Neurology, London, UK; http://www.fil.ion.ucl.ac.uk/spm/). After de-noising with ArtRepair (http://cibsr.stanford.edu/tools/ArtRepair/ArtRepair.htm), the following preprocessing steps were performed: acquisition time and motion correction, coregistration of the mean EPI to the anatomical T1 image, spatial normalization and segmentation into tissue classes of the T1 image using the unified segmentation approach as implemented in SPM8 [37] application of the normalization parameters to all functional images, and spatial smoothing with an isotropic Gaussian kernel of 8 mm full width at half maximum (FWHM) kernel.

Small volume correction for multiple comparisons was used within a ventral striatal volume of interest (VOI). We created an fMRI literature-based probabilistic VOI for the VS [13]. We selected 16 recent papers containing data from 325 subjects [5, 6, 35, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50]. From each study, the coordinates of PE-related activation for right and the left VS were extracted.

Statistical analysis

The images were analyzed in an event-related manner using the general linear model approach (GLM) as implemented in SPM8; neuronal activity was modeled using a stick function at the onsets of the feedback. We used a parametric design [32, 51] in which the trial-by-trial PE values from the RW model modulated the amplitude of the trial-related stick. Regressors of interest for the BOLD responses corresponding to the trial-wise PEs were generated by convolving the modulated stimulus functions with the canonical hemodynamic response function (HRF), provided by SPM8. To account for signal fluctuations associated with the movement by susceptibility interaction, the six movement parameters from the realignment preprocessing step were included in the model as additional regressors. The individual contrast images for the contrast of the PE-modulated feedback were then taken to a random-effects group-level analysis using a one sample t test. To test for associations with measures of IQ and SLES, these measures were entered as covariates into additional random-effects analyses.

In addition, we performed stepwise multiple regression analyses to test the predictive effect of SLES and PE signaling on fluid IQ using SPSS. These were used to assess (1) the effects of PE signaling and SLES on fluid IQ and (2) tested for interaction effects (moderation) between SLES and PE signaling on fluid IQ by computing an interaction term (specifically, we assigned participants reporting below median SLES a value of one and participants reporting above median SLES a value of two and multiplied this dichotomous variable with the peak VS BOLD PE signal) and added this interaction term to the regression analysis predicting fluid IQ.

To minimize false-positive results due to median split of SLES (and thus reducing effective sample size), we performed an additional stepwise multiple regression analyses entering SLES as a continuous variable into the interaction term after z-transformation (and multiplied this continuous variable with the peak VS BOLD PE signal) and then added this interaction term to the regression analysis.

Results

On average, participants reported a mean of 16.31 (SD = 4.59) out of 42 SLES. Overall mean fluid IQ was 0.60 (SD = 0.64) after Varimax Rotation (z-transformed). Participants displayed a mean crystallized IQ of 32.31 (SD = 5.67), measured with verbal knowledge test. All subjects made an average of 70.8 ± 6.1 % correct responses and reached criterion (number of reversal stages) on 5.2 ± 1.3 conditions with a learning speed of 0.6 trials. For best-fitting parameter estimates, log likelihoods, and learning rates see Table 1. There was no correlation between fluid IQ and any of the performance parameters (percent correct responses, number of achieved reversal stages, learning speed: all p > 0.05). Fluid IQ declined with age (correlation between fluid IQ and age: Pearson’s correlation = −0.61, p = 0.013). Fluid IQ was not significantly associated with SLES (Pearson’s correlation = 0.24, p = 0.376), and the amount of SLES was not associated with age (Pearson’s correlation = −0.11, p = 0.694).
Table 1

Best-fitting parameter estimates are shown as median + quartiles across subjects

 

β rew

β pun

LL

LR

25th percentile

1.96

0.61

53.15

0.40

Median

2.73

0.84

73.10

0.62

75th percentile

5.03

1.10

109.07

0.77

Also shown are medians and quartiles for the log-likelihood (LL) of the data at the best-fitting parameters and the learning rate (LR). The variables reward/punishment sensitivities (β rew and −β pun) represent the effective reinforcement sensitivity as expressed by the effect of the reinforcement on the subject’s choice behavior. This variable took on value β rew if a reward was obtained and −β pun if a punishment was obtained

β rew reward-sensitivity, −β pun punishment-sensitivity, LL log-likelihood, LR learning rate

Prediction error signaling in the ventral striatum (VS Bold PE signal)

In the fMRI group of 16 healthy controls, we first observed a significant correlation between the model-free PEs and the BOLD response in the left VS (x/y/z = −18/3/−5, T = 3.38, p FWE corrected for VS VOI = 0.026). There was also a trendwise correlation in the right VS (x/y/z = 20/6/−5, T = 2.47, p FWE corrected for VS VOI = 0.057, for whole brain activation see Supplementary Figure 1). Thus, subjects’ VS BOLD activity correlated with the PE learning signal. We term this correlate the VS BOLD PE signal. Below, we examine correlations between this VS BOLD PE signal, SLES, and fluid IQ.

Correlations between fluid IQ, stressful life events, and VS BOLD PE signal

Fluid IQ was significantly and positively correlated with the VS BOLD PE signal both on the left (x/y/z = −11/8/−8, T = 3.89, p FWE corrected for VS VOI = 0.012) and on the right (x/y/z = 17/6/−8, T = 3.71, p FWE corrected for VS VOI = 0.013). Thus, VS functional activation was more closely correlated with model-free habitual PEs in subjects with higher IQ as reported previously [13]. Upon controlling for age by introducing age as an additional covariate into the SPM analysis, VS PE BOLD signal remained associated with fluid IQ (left VS: x/y/z = −16/3/−8, p FWE corrected for VS VOI = 0.045, T = 3.11, right VS: x/y/z = 20/3/−8, p FWE corrected for VS VOI = 0.006, T = 4.30), suggesting that this association was not simply explained by an age-related decline in fluid IQ.

In addition, SLES was significantly and positively correlated with the BOLD PE signal in the left VS (left VS: x/y/z = −16/3/−8, T = 3.02, p FWE corrected for VS VOI = 0.047 (Fig. 2), though not in the right VS (p > 0.2). This association remained significant when entering age as an additional covariate into the SPM analyses (left VS: x/y/z = −16/3/−8, p FWE corrected for VS VOI = 0.048, T = 3.10). As stated above, fluid IQ was not significantly associated with SLES (Pearson’s correlation = 0.24, p = 0.376).
Fig. 2

Stressful life events were significantly and positively correlated with the BOLD PE signal in the left VS (x/y/z = −16/3/−8, T = 3.02; p FWE corrected for VS VOI = 0.047). Color Scale represents T Values

Effects of stressful life events and VS BOLD PE signal on fluid IQ

Stress has been shown to increase the habitual, or model-free, component of behavior [52]. The PE regressor we have used is one such model-free, habitual, learning signal [8]. We were thus interested to know whether stress increases the contribution of this basic learning signal to the more general index of flexible learning measured by fluid IQ. We therefore performed a stepwise regression analysis, asking whether the product of SLES and left VS BOLD PE signal explains additional variance in the fluid IQ measure beyond the effects of SLES and VS BOLD PE signal alone.

The VS BOLD PE signal again correlated significantly with fluid IQ (β = 0.89, T = 3.61, p < 0.005), while SLES did not correlate with fluid IQ (p > 0.20). Thus, in our sample, fluid IQ and SLES were not directly correlated. However, there was a significant effect of the interaction of VS BOLD PE signal x SLES (β=1.19, T=2.53, p<0.05). This interaction term increased the fraction of variance in fluid IQ that was explained from 53 to 69 %—a significant change (F Change = 6.39, p < 0.05). In order to understand the direction of this effect, we plotted fluid IQ as a function of VS BOLD PE signals using a median split on SLES (above or below 17 reported events). Figure 3 shows that the association between VS PE BOLD signaling and fluid IQ was increased in subjects reporting more SLES (R 2 =0.76). This interaction term remained significant when adding SLES as a continuous variable (ß = 0.43, T = 2.53, p < 0.05). We performed an outlier analysis for each included data point (n = 16) revealing no relevant leverage effects (z value<3, Cook’s distance<1, max=0.218, SD=0.061).
Fig. 3

Effects of stressful life events and VS BOLD PE signal on fluid IQ. Subjects reporting stressful life events above the median are depicted in red solid triangles (R 2 = 0.759). Subjects reporting below the median in blue transparent circles (R 2 = 0.287). The interaction term is significant (β = 1.19, T = 2.53, p < 0.05), indicating that in subjects reporting more stressful life events, the VS BOLD PE signal correlates more strongly with fluid IQ. An outlier analysis was performed for each included data point (n = 16) revealing no relevant leverage effects (z value < 3, Cook’s distance < 1, max = 0.218, SD = 0.061)

Conclusion

We learn from making mistakes and need to adapt our predictions in the face of changing circumstances; iterative learning via PEs plays a major role in such learning processes [7, 53]. The present results are the first reported data showing that a basic neurobiological learning signal—the ventral striatal PE signal—and SLES interact to predict individual differences in fluid intelligence, even when correcting for the decline of fluid IQ with age. Several points merit comment.

First, it may seem surprising that the relationship of SLES to the BOLD PE signal is in the same, positive, direction as that of fluid IQ. However, stress is well known to promote habitual, model-free responses in humans and animals. The VS BOLD PE signal quantifies how closely the BOLD signal in the VS correlated with a standard model-free learning signal. This is a measure of the co-alignment of the two time-varying signals, not of the magnitude of either of them. As such, a stronger VS BOLD PE signal is in keeping with more habitual, model-free learning. On the other hand, fluid IQ among other cognitive attributes captures planning ability and other goal-directed and more challenging (model-based) cognitive functions [54]. The fact that higher fluid IQ is positively associated with a stronger alignment of VS activity with the model-free (habitual) learning signal is thus, at first sight, a counterintuitive finding. However, the model-free learning signal may be correlated with more complex model-based signals in this particular task, and there is evidence that a VS PE signal may in fact comprise both model-free and model-based components [55]. One possible explanation is that this alignment is an expression of the engagement of several different learning strategies in parallel (including goal-directed—model-based and habitual—model-free), which is in accordance with our previous interpretation of study results. We had observed that fluid IQ is associated with the VS BOLD PE signal, a putative signature of habitual (model-free) learning even when controlling for behavioral fit, which indicates how strongly the observed behavior is accounted for by PE-driven learning [13]. This issue needs re-visiting in the light of the current data, as here, subjects who had not experienced much life stress showed only a small correlation between IQ and VS BOLD PE signal. Thus, our finding of a correlate between fluid IQ and VS BOLD PE signal is driven by those subjects who have experienced substantial life stress, which may have gradually shifted their flexible cognitive capacities toward more model-free strategies.

Our results complement previous studies on the effect of (acute) stress on dopamine signaling and ventral striatal PE encoding. Stress exposure facilitates ventral striatal dopamine release [22, 23, 56], and Robinson et al. [14] recently observed increased ventral striatal PE signaling (model-free) of negative errors of reward prediction (i.e. when received outcome is smaller than expected) during acute stress. Additionally, Otto et al. [16] reported that acute stress exposure reduces the amount of model-based learning in subjects with low working memory capacity (a measure closely related to fluid intelligence and general cognitive ability), while model-free learning was unaffected and thus shifts the balance from goal-directed (model-based) toward more habitual (model-free) learning strategies during decision making. We now show that self-reported SLES, which reflect chronic rather than acute stressors, are positively associated with ventral striatal BOLD PE signal and that SLES interact with the neuronal learning signature to predict fluid IQ. Thus, it seems feasible that chronic stressors have an effect on VS PE signaling in that they induce a stronger encoding of the PE signal during habitual learning, effects which interact to predict individual differences in fluid IQ—above and beyond the (negative) effect of age on fluid IQ.

Despite the link between stress and the development of various psychopathologies [57, 58], few studies have compared the neurophysiological effects of acute and repeated stressors. There is ample evidence to indicate both types of stress influence the dopaminergic system, pointing in the direction of differential effects of acute and chronic stress [14, 16]. These modulations of dopaminergic firing have been seen in the ventral tegmental area and the VS [22, 23, 56] core areas for the neuronal representation of reward PEs, reward anticipation, learning from reinforcement [59, 60], and flexible behavioral adaptation (fluid IQ). Furthermore, studies suggest that genetic variation in the dopamine system moderates the effects of acute and chronic stress [58, 61] and are associated with individual differences in working memory capacity and other aspects of fluid intelligence [62, 63, 64, 65].

Our data indicate that the above reported VS BOLD PE signal works as an indirect neuronal signature of stress experience that is in part driven by a dopaminergic modulation which might shift learning strategies in highly flexible subjects (with high levels of fluid IQ) towards more habitual rather than goal-directed learning strategies. These data might allow for the speculation that it may be ecologically salient to encode errors of reward prediction more strongly when life experiences are mainly adverse and thus reduce cognitive demands in complex and threatening situations.

An important limitation to our findings is that the results are based on a small sample and therefore will require replication. Nevertheless, the direction of the stress effects is consistent with a recent report of the impact of acute stress on PE signals in a reversal learning task [14]. Also, recent evidence [24] points toward the direction of gender differences in reward-related decision processing under stress. Therefore, our study will have to be repeated in an independent female sample. The findings are correlational, and hence, no statements about causality can be made. The various directional interpretations are, however, worth disentangling. It appears that stress alters how IQ relates to the VS PEs. The importance of the findings derive from the fact that this is the first study showing that an association between reward PE signaling and intelligence is moderated by life stress experience and this suggests stressful life experiences may sensitize the dopaminergic system toward more habitual decision making.

Notes

Acknowledgments

This work was supported by the German Research Foundation (Deutsche Forschungsgemeinschaft, DFG, FOR 1617: Grants HE2597/13-1, RA1047/2-1, and SM80/7-1).

Conflict of interest

None.

Supplementary material

406_2014_519_MOESM1_ESM.tif (298 kb)
Supplementary Figure 1: Striatal activation at p <0.001 whole brain activation (left striatum x/y/z = -26/1/-8, p<0.001, F=27.91, cluster size (voxel) =17; right striatum: x/y/z = 25/6/-3, p<0.001, F=23.74, cluster size (voxel) = 18). (TIFF 298 kb)

References

  1. 1.
    Horn JL, Cattell RB (1966) Age differences in primary mental ability factors. J Gerontol 21(2):210–220PubMedCrossRefGoogle Scholar
  2. 2.
    Sternberg RJ (2000) The holey grail of general intelligence. Science 289(5478):399–401PubMedCrossRefGoogle Scholar
  3. 3.
    Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16(5):1936–1947PubMedGoogle Scholar
  4. 4.
    Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275(5306):1593–1599PubMedCrossRefGoogle Scholar
  5. 5.
    D’Ardenne K, McClure SM, Nystrom LE, Cohen JD (2008) BOLD responses reflecting dopaminergic signals in the human ventral tegmental area. Science 319(5867):1264–1267. doi: 10.1126/science.1150605 PubMedCrossRefGoogle Scholar
  6. 6.
    O’Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304(5669):452–454. doi: 10.1126/science.1094285 PubMedCrossRefGoogle Scholar
  7. 7.
    Sutton RBA (1998) Reinforcement Learning: An Introduction. A Bradford Book, CambridgeGoogle Scholar
  8. 8.
    Daw ND, Niv Y, Dayan P (2005) Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci 8(12):1704–1711. doi: 10.1038/nn1560 PubMedCrossRefGoogle Scholar
  9. 9.
    Morris G, Nevet A, Arkadir D, Vaadia E, Bergman H (2006) Midbrain dopamine neurons encode decisions for future action. Nat Neurosci 9(8):1057–1063. doi: 10.1038/nn1743 PubMedCrossRefGoogle Scholar
  10. 10.
    Schultz W (2010) Multiple functions of dopamine neurons. F1000 biology reports 2. doi: 10.3410/B2-2
  11. 11.
    Schott BH, Minuzzi L, Krebs RM, Elmenhorst D, Lang M, Winz OH, Seidenbecher CI, Coenen HH, Heinze HJ, Zilles K, Duzel E, Bauer A (2008) Mesolimbic functional magnetic resonance imaging activations during reward anticipation correlate with reward-related ventral striatal dopamine release. J Neurosci 28(52):14311–14319. doi: 10.1523/JNEUROSCI.2058-08.2008 PubMedCrossRefGoogle Scholar
  12. 12.
    Schlagenhauf F, Rapp MA, Huys QJ, Beck A, Wustenberg T, Deserno L, Buchholz HG, Kalbitzer J, Buchert R, Bauer M, Kienast T, Cumming P, Plotkin M, Kumakura Y, Grace AA, Dolan RJ, Heinz A (2013) Ventral striatal prediction error signaling is associated with dopamine synthesis capacity and fluid intelligence. Hum Brain Mapp 34(6):1490–1499. doi: 10.1002/hbm.22000 PubMedCentralPubMedCrossRefGoogle Scholar
  13. 13.
    Schlagenhauf F, Rapp MA, Huys QJ, Beck A, Wustenberg T, Deserno L, Buchholz HG, Kalbitzer J, Buchert R, Bauer M, Kienast T, Cumming P, Plotkin M, Kumakura Y, Grace AA, Dolan RJ, Heinz A (2012) Ventral striatal prediction error signaling is associated with dopamine synthesis capacity and fluid intelligence. Hum Brain Mapp. doi: 10.1002/hbm.22000 PubMedCentralPubMedGoogle Scholar
  14. 14.
    Robinson OJ, Overstreet C, Charney DR, Vytal K, Grillon C (2013) Stress increases aversive prediction error signal in the ventral striatum. Proc Natl Acad Sci USA 110(10):4129–4133. doi: 10.1073/pnas.1213923110 PubMedCentralPubMedCrossRefGoogle Scholar
  15. 15.
    Garrett DD, Grady CL, Hasher L (2010) Everyday memory compensation: the impact of cognitive reserve, subjective memory, and stress. Psychol Aging 25(1):74–83. doi: 10.1037/a0017726 PubMedCrossRefGoogle Scholar
  16. 16.
    Otto AR, Raio CM, Chiang A, Phelps EA, Daw ND (2013) Working-memory capacity protects model-based learning from stress. Proc Natl Acad Sci USA 110(52):20941–20946. doi: 10.1073/pnas.1312011110 PubMedCentralPubMedCrossRefGoogle Scholar
  17. 17.
    Cavanagh JF, Frank MJ, Allen JJ (2011) Social stress reactivity alters reward and punishment learning. Soc Cogn Affect Neurosci 6(3):311–320. doi: 10.1093/scan/nsq041 PubMedCentralPubMedCrossRefGoogle Scholar
  18. 18.
    Petzold A, Plessow F, Goschke T, Kirschbaum C (2010) Stress reduces use of negative feedback in a feedback-based learning task. Behav Neurosci 124(2):248–255. doi: 10.1037/a0018930 PubMedCrossRefGoogle Scholar
  19. 19.
    Porcelli AJ, Delgado MR (2009) Acute stress modulates risk taking in financial decision making. Psychol Sci 20(3):278–283. doi: 10.1111/j.1467-9280.2009.02288.x PubMedCrossRefGoogle Scholar
  20. 20.
    Bogdan R, Pizzagalli DA (2006) Acute stress reduces reward responsiveness: implications for depression. Biol Psychiatry 60(10):1147–1154. doi: 10.1016/j.biopsych.2006.03.037 PubMedCentralPubMedCrossRefGoogle Scholar
  21. 21.
    Baumeister RF, Twenge JM, Nuss CK (2002) Effects of social exclusion on cognitive processes: anticipated aloneness reduces intelligent thought. J Pers Soc Psychol 83(4):817–827PubMedCrossRefGoogle Scholar
  22. 22.
    Valenti O, Lodge DJ, Grace AA (2011) Aversive stimuli alter ventral tegmental area dopamine neuron activity via a common action in the ventral hippocampus. J Neurosci 31(11):4280–4289. doi: 10.1523/JNEUROSCI.5310-10.2011 PubMedCentralPubMedCrossRefGoogle Scholar
  23. 23.
    Cao JL, Covington HE 3rd, Friedman AK, Wilkinson MB, Walsh JJ, Cooper DC, Nestler EJ, Han MH (2010) Mesolimbic dopamine neurons in the brain reward circuit mediate susceptibility to social defeat and antidepressant action. The Journal of neuroscience : the official journal of the Society for Neuroscience 30(49):16453–16458. doi: 10.1523/JNEUROSCI.3177-10.2010 CrossRefGoogle Scholar
  24. 24.
    Lighthall NR, Sakaki M, Vasunilashorn S, Nga L, Somayajula S, Chen EY, Samii N, Mather M (2012) Gender differences in reward-related decision processing under stress. Soc Cogn Affect Neurosci 7(4):476–484. doi: 10.1093/scan/nsr026 PubMedCentralPubMedCrossRefGoogle Scholar
  25. 25.
    Lindenberger U, Baltes PB (1997) Intellectual functioning in old and very old age: cross-sectional results from the Berlin aging study. Psychol Aging 12(3):410–432PubMedCrossRefGoogle Scholar
  26. 26.
    Reitan RM (1955) The relation of the trail making test to organic brain damage. J Consult Psychol 19:393–394PubMedCrossRefGoogle Scholar
  27. 27.
    Stroop JR (1935) Studies of interference in serial verbal reactions. J Exp Psychol 18:643–662CrossRefGoogle Scholar
  28. 28.
    Wechsler D (1955) Wechsler adult intelligence scale manual. Psychological Corporation, New YorkGoogle Scholar
  29. 29.
    Beckers K, Behrends U, Canavan A (1992) Deutsche version des river mead behavioral memory test. Thames Valley Test Company, Bury St EdmundsGoogle Scholar
  30. 30.
    Horn W (1983) L-P-S Leistungsprüfsystem, vol 2. Auflage. Hogrefe, GöttingenGoogle Scholar
  31. 31.
    Holmes THRR (1967) The social readjustment rating scale. J Psychosom Res 11(2):213–218PubMedCrossRefGoogle Scholar
  32. 32.
    O’Doherty JP, Hampton A, Kim H (2007) Model-based fMRI and its application to reward learning and decision making. Ann N Y Acad Sci 1104:35–53. doi: 10.1196/annals.1390.022 PubMedCrossRefGoogle Scholar
  33. 33.
    Huys QJ, Cools R, Golzer M, Friedel E, Heinz A, Dolan RJ, Dayan P (2011) Disentangling the roles of approach, activation and valence in instrumental and pavlovian responding. Plos Comput Biol 7(4):e1002028. doi: 10.1371/journal.pcbi.1002028 PubMedCentralPubMedCrossRefGoogle Scholar
  34. 34.
    Huys QJ, Eshel N, O’Nions E, Sheridan L, Dayan P, Roiser JP (2012) Bonsai trees in your head: how the pavlovian system sculpts goal-directed choices by pruning decision trees. Plos Comput Biol 8(3):e1002410. doi: 10.1371/journal.pcbi.1002410 PubMedCentralPubMedCrossRefGoogle Scholar
  35. 35.
    Kahnt T, Park SQ, Cohen MX, Beck A, Heinz A, Wrase J (2009) Dorsal striatal-midbrain connectivity in humans predicts how reinforcements are used to guide decisions. J Cogn Neurosci 21(7):1332–1345. doi: 10.1162/jocn.2009.21092 PubMedCrossRefGoogle Scholar
  36. 36.
    Park SQ, Kahnt T, Beck A, Cohen MX, Dolan RJ, Wrase J, Heinz A (2010) Prefrontal cortex fails to learn from reward prediction errors in alcohol dependence. J Neurosci 30(22):7749–7753. doi: 10.1523/JNEUROSCI.5587-09.2010 PubMedCentralPubMedCrossRefGoogle Scholar
  37. 37.
    Ashburner J, Friston KJ (2005) Unified segmentation. NeuroImage 26(3):839–851. doi: 10.1016/j.neuroimage.2005.02.018 PubMedCrossRefGoogle Scholar
  38. 38.
    Pessiglione M, Seymour B, Flandin G, Dolan RJ, Frith CD (2006) Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442(7106):1042–1045. doi: 10.1038/nature05051 PubMedCentralPubMedCrossRefGoogle Scholar
  39. 39.
    Bray S, O’Doherty J (2007) Neural coding of reward-prediction error signals during classical conditioning with attractive faces. J Neurophysiol 97(4):3036–3045. doi: 10.1152/jn.01211.2006 PubMedCrossRefGoogle Scholar
  40. 40.
    Cohen MX, Ranganath C (2005) Behavioral and neural predictors of upcoming decisions. Cogn Affect Behav Neurosci 5(2):117–126PubMedCrossRefGoogle Scholar
  41. 41.
    Cohen MX (2007) Individual differences and the neural representations of reward expectation and reward prediction error. Cogn Affect Behav Neurosci 2(1):20–30. doi: 10.1093/scan/nsl021 CrossRefGoogle Scholar
  42. 42.
    Gershman SJ, Pesaran B, Daw ND (2009) Human reinforcement learning subdivides structured action spaces by learning effector-specific values. Journal Neurosci 29(43):13524–13531. doi: 10.1523/JNEUROSCI.2469-09.2009 CrossRefGoogle Scholar
  43. 43.
    Krugel LK, Biele G, Mohr PN, Li SC, Heekeren HR (2009) Genetic variation in dopaminergic neuromodulation influences the ability to rapidly and flexibly adapt decisions. Proc Natl Acad Sci USA 106(42):17951–17956. doi: 10.1073/pnas.0905191106 PubMedCentralPubMedCrossRefGoogle Scholar
  44. 44.
    Murray GK, Corlett PR, Clark L, Pessiglione M, Blackwell AD, Honey G, Jones PB, Bullmore ET, Robbins TW, Fletcher PC (2008) Substantia nigra/ventral tegmental reward prediction error disruption in psychosis. Molecular psychiatry. 13 (3): 239, 267–276. doi: 10.1038/sj.mp.4002058
  45. 45.
    O’Doherty JP, Dayan P, Friston K, Critchley H, Dolan RJ (2003) Temporal difference models and reward-related learning in the human brain. Neuron 38(2):329–337PubMedCrossRefGoogle Scholar
  46. 46.
    Palminteri S, Boraud T, Lafargue G, Dubois B, Pessiglione M (2009) Brain hemispheres selectively track the expected value of contralateral options. J Neurosci 29(43):13465–13472. doi: 10.1523/JNEUROSCI.1500-09.2009 PubMedCrossRefGoogle Scholar
  47. 47.
    Rodriguez PF, Aron AR, Poldrack RA (2006) Ventral-striatal/nucleus-accumbens sensitivity to prediction errors during classification learning. Hum Brain Mapp 27(4):306–313. doi: 10.1002/hbm.20186 PubMedCrossRefGoogle Scholar
  48. 48.
    Schonberg T, O’Doherty JP, Joel D, Inzelberg R, Segev Y, Daw ND (2010) Selective impairment of prediction error signaling in human dorsolateral but not ventral striatum in Parkinson’s disease patients: evidence from a model-based fMRI study. NeuroImage 49(1):772–781. doi: 10.1016/j.neuroimage.2009.08.011 PubMedCrossRefGoogle Scholar
  49. 49.
    Tobler PN, O’Doherty JP, Dolan RJ, Schultz W (2006) Human neural learning depends on reward prediction errors in the blocking paradigm. J Neurophysiol 95(1):301–310. doi: 10.1152/jn.00762.2005 PubMedCentralPubMedCrossRefGoogle Scholar
  50. 50.
    Valentin VV, O’Doherty JP (2009) Overlapping prediction errors in dorsal striatum during instrumental learning with juice and money reward in the human brain. J Neurophysiol 102(6):3384–3391. doi: 10.1152/jn.91195.2008 PubMedCrossRefGoogle Scholar
  51. 51.
    Buchel C, Holmes AP, Rees G, Friston KJ (1998) Characterizing stimulus-response functions using nonlinear regressors in parametric fMRI experiments. NeuroImage 8(2):140–148. doi: 10.1006/nimg.1998.0351 PubMedCrossRefGoogle Scholar
  52. 52.
    Schwabe L, Wolf OT (2009) Stress prompts habit behavior in humans. J Neurosci 29(22):7191–7198. doi: 10.1523/JNEUROSCI.0979-09.2009 PubMedCrossRefGoogle Scholar
  53. 53.
    O’Doherty JP (2004) Reward representations and reward-related learning in the human brain: insights from neuroimaging. Curr Opin Neurobiol 14(6):769–776. doi: 10.1016/j.conb.2004.10.016 PubMedCrossRefGoogle Scholar
  54. 54.
    Hampton AN, Bossaerts P, O’Doherty JP (2006) The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. The Journal of neuroscience : the official journal of the Society for Neuroscience 26(32):8360–8367. doi: 10.1523/JNEUROSCI.1010-06.2006 CrossRefGoogle Scholar
  55. 55.
    Daw ND, Gershman SJ, Seymour B, Dayan P, Dolan RJ (2011) Model-based influences on humans’ choices and striatal prediction errors. Neuron 69(6):1204–1215. doi: 10.1016/j.neuron.2011.02.027 PubMedCentralPubMedCrossRefGoogle Scholar
  56. 56.
    Morgan D, Grant KA, Gage HD, Mach RH, Kaplan JR, Prioleau O, Nader SH, Buchheimer N, Ehrenkaufer RL, Nader MA (2002) Social dominance in monkeys: dopamine D2 receptors and cocaine self-administration. Nat Neurosci 5(2):169–174. doi: 10.1038/nn798 PubMedCrossRefGoogle Scholar
  57. 57.
    Patchev AV, Rodrigues AJ, Sousa N, Spengler D, Almeida OF (2013) The future is now: early life events pre-set adult behavior. Acta Physiol (Oxf). doi: 10.1111/apha.12140 Google Scholar
  58. 58.
    Bouma EM, Riese H, Doornbos B, Ormel J, Oldehinkel AJ (2012) Genetically based reduced MAOA and COMT functioning is associated with the cortisol stress response: a replication study. Mol Psychiatry 17(2):119–121. doi: 10.1038/mp.2011.115 PubMedCrossRefGoogle Scholar
  59. 59.
    Breiter HC, Aharon I, Kahneman D, Dale A, Shizgal P (2001) Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron 30(2):619–639PubMedCrossRefGoogle Scholar
  60. 60.
    Cooper JC, Knutson B (2008) Valence and salience contribute to nucleus accumbens activation. NeuroImage 39(1):538–547. doi: 10.1016/j.neuroimage.2007.08.009 PubMedCentralPubMedCrossRefGoogle Scholar
  61. 61.
    Jabbi M, Korf J, Kema IP, Hartman C, van der Pompe G, Minderaa RB, Ormel J, den Boer JA (2007) Convergent genetic modulation of the endocrine stress response involves polymorphic variations of 5-HTT COMT and MAOA. Mol Psychiatry 12(5):483–490. doi: 10.1038/sj.mp.4001975 PubMedGoogle Scholar
  62. 62.
    Nisbett RE, Aronson J, Blair C, Dickens W, Flynn J, Halpern DF, Turkheimer E (2012) Intelligence: new findings and theoretical developments. Am Psychol 67(2):130–159. doi: 10.1037/a0026699 PubMedCrossRefGoogle Scholar
  63. 63.
    Heinz A, Smolka MN (2006) The effects of catechol O-methyltransferase genotype on brain activation elicited by affective stimuli and cognitive tasks. Rev Neurosci 17(3):359–367PubMedCrossRefGoogle Scholar
  64. 64.
    Barnett JH, Heron J, Goldman D, Jones PB, Xu K (2009) Effects of catechol-O-methyltransferase on normal variation in the cognitive function of children. Am J Psychiatry 166(8):909–916. doi: 10.1176/appi.ajp.2009.08081251 PubMedCentralPubMedCrossRefGoogle Scholar
  65. 65.
    Thompson JM, Sonuga-Barke EJ, Morgan AR, Cornforth CM, Turic D, Ferguson LR, Mitchell EA, Waldie KE (2012) The catechol-O-methyltransferase (COMT) Val158Met polymorphism moderates the effect of antenatal stress on childhood behavioural problems: longitudinal evidence across multiple ages. Dev Med Child Neurol 54(2):148–154. doi: 10.1111/j.1469-8749.2011.04129.x PubMedCrossRefGoogle Scholar

Copyright information

© The Author(s) 2014

Open AccessThis article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.

Authors and Affiliations

  • Eva Friedel
    • 1
  • Florian Schlagenhauf
    • 1
    • 2
  • Anne Beck
    • 1
  • Raymond J. Dolan
    • 3
  • Quentin J.M. Huys
    • 3
    • 4
    • 5
  • Michael A. Rapp
    • 1
    • 6
  • Andreas Heinz
    • 1
    • 7
    Email author
  1. 1.Department of Psychiatry and PsychotherapyCharité – Universitätsmedizin BerlinBerlinGermany
  2. 2.Max Planck Institute for Human Cognitive and Brain SciencesLeipzigGermany
  3. 3.Gatsby Computational Neuroscience UnitUniversity College LondonLondonUK
  4. 4.Translational Neuromodeling Unit, Institute for Biomedical EngineeringUniversity of Zurich and ETH ZurichZurichSwitzerland
  5. 5.Department of Psychiatry, Psychotherapy and PsychosomaticsUniversity Hospital of PsychiatryZurichSwitzerland
  6. 6.Social and Preventive MedicineUniversity of PotsdamPotsdamGermany
  7. 7.Cluster of Excellence NeuroCureCharite-Universitätsmedizin BerlinBerlinGermany

Personalised recommendations