The effects of life stress and neural learning signals on fluid intelligence
- 1.6k Downloads
Fluid intelligence (fluid IQ), defined as the capacity for rapid problem solving and behavioral adaptation, is known to be modulated by learning and experience. Both stressful life events (SLES) and neural correlates of learning [specifically, a key mediator of adaptive learning in the brain, namely the ventral striatal representation of prediction errors (PE)] have been shown to be associated with individual differences in fluid IQ. Here, we examine the interaction between adaptive learning signals (using a well-characterized probabilistic reversal learning task in combination with fMRI) and SLES on fluid IQ measures. We find that the correlation between ventral striatal BOLD PE and fluid IQ, which we have previously reported, is quantitatively modulated by the amount of reported SLES. Thus, after experiencing adversity, basic neuronal learning signatures appear to align more closely with a general measure of flexible learning (fluid IQ), a finding complementing studies on the effects of acute stress on learning. The results suggest that an understanding of the neurobiological correlates of trait variables like fluid IQ needs to take socioemotional influences such as chronic stress into account.
KeywordsReinforcement learning Prediction error signal Ventral striatum Stress Intelligence
Fluid intelligence (fluid IQ) [1, 2] describes the capacity for rapid problem solving and flexible adjustment to an ever-changing environment. Its expression is a general factor comprising attributes such as attention, cognitive speed, working memory, reasoning, and episodic memory that have manifold impacts on learning. One important neurobiological correlate of learning is the phasic activation of dopamine neurons in the VTA [3, 4, 5] which is reflected in BOLD fMRI studies in humans as a prediction error (PE) in a number of regions including the ventral striatum (VS) . The prominent place of the VS PE in neurobiological accounts of learning derives from the fact that it is a particular type of teaching signal, which indicates the need for a change in expectation, as well as the direction and quantity of change necessary to acquire habits [7, 8, 9]. Animal experiments have shown that PE signaling is associated with phasic dopamine firing with the size of PE reflecting the amount of dopamine release [4, 10]. In humans, PE signaling has also been shown to phasically release striatal dopamine (indirectly measured via displacement of dopamine D2 receptor ligands)  and to be modulated by more tonic aspects of dopamine synthesis capacity measured over 1 hour .
Recently, we reported that individual differences in fluid IQ are associated with VS BOLD PE signals, with stronger VS BOLD PE correlates in subjects with higher IQ . At the same time, there is recent evidence that acute stress increases BOLD responses elicited by aversive PE signals in the VS . Furthermore, fluid IQ and stress are well known to interact, with stress having a strong moderating influence on cognitive abilities [15, 16], reward learning [17, 18], risk taking , reward responsivity , and decision-making speed . Also, stress due to social exclusion situations impairs cognitive speed and accuracy . However, despite strong evidence for an impact of stress on cognition and decision-making processes, little is known about the specific patterns and moderating trait components underlying these changes on a behavioral and neuronal level. Interestingly, it has recently been observed that (on a behavioral level) acute stress does not impair so-called model-free reward learning, while more cognitively demanding model-based reward learning is more affected by acute stress when working memory capacity is lower .
With respect to dopaminergic transmission, a series of animal experiments indicates that acute as well as chronic stress moderated dopamine release and may thus interact with dopamine-dependent PE signaling in the striatum [22, 23]. For instance, changes in cortisol levels during an acute stressor were correlated with increases in striatal responses during a decision-making task . Given these interactions between stress, dopaminergic PE signals, and aspects of fluid intelligence, we wanted to explore the effects of chronic stress on PE signaling and fluid IQ. One possibility is that stress increases dopamine release, which in turn increases PE signaling, and in accordance with our previous observation , fluid IQ. However, this is unlikely given the predominantly negative interactions between stress exposure and fluid IQ (a measure closely related to cognitive capacity and cognitive speed). We therefore tested the hypothesis that stress exposure modulates PE signaling above and beyond the previously observed correlation between PE signaling and fluid IQ.
To assess PE-related VS activity, we regressed VS BOLD signals onto PEs derived from a simple learning algorithm in which PEs are accumulated over time to form predictions. We note that this simple learning algorithm is ‘model-free,’ in that it only requires subjects to iteratively track what outcomes are observed after their choices, but it does not require subjects to have any explicit model or understanding of the task. We then asked whether this correlation, henceforth termed the VS BOLD PE signal, (1) is associated with measures of chronic life stress and (2) whether chronic life stress and VS BOLD PE signal alone or in interaction contribute to individual differences in fluid IQ.
Materials and methods
Subjects and screening instruments
A group of 16 right-handed healthy men with a mean age of 38.4 years (SD = 11.9; range 22–61) underwent fMRI and neuropsychological testing as a subgroup of a sample previously reported . Subjects with Axis I and II psychiatric disorders according to DSM IV were excluded through the Structured Clinical Interview, and drug abuse was further excluded with urine tests.
Neuropsychological assessment, intelligence measures, and stressful life events
A neuropsychological battery was administered within 2 months of fMRI measurements. Components of fluid and crystallized IQ were measured with an adaptation of the standard battery used in the Berlin Aging Study . Fluid IQ was measured with a battery of nine tests comprising cognitive speed, attention and executive function, working memory, episodic memory, and reasoning. Cognitive speed was measured using the Digit Symbol Substitution test and the Reitan Trailmaking test, part A. Attention and executive function was measured using the Reitan Trailmaking test, part B , and Stroop  tests. Working memory was measured using forward and backward digit span tests . Episodic memory was assessed using story recall with the German version of the Rivermead Behavioral Memory Test  and a German version of the auditory verbal learning test. Reasoning was measured using a test of figural analogies . Fluid IQ was derived from a factorial analysis of the raw scores of each of these tests. Specifically, we used a Varimax rotation with an Eigenvalue cutoff set to 1.0; the final (single factor) solution accounted for 74.3 % of the variance in cognitive speed, attention, working memory, memory, and reasoning. Crystallized IQ was estimated using a verbal knowledge test, during which subjects are required to identify each single meaningful word from a total of 42 lists of five words.
Stressful life events (SLES) were assessed using the Life Events Scale , in which subjects were asked to self-report the presence of a stressful life event during the past 2 years from a list of 42 life events. We used the number of SLES as the main outcome measure.
Reversal learning task
Computational modeling of reinforcement learning
Thus, rather than doing an individual maximum-likelihood fit, we did a maximum-likelihood fit of the group mean via expectation–maximization and then inferred the posterior parameter estimates for each individual. This reduces variability in the parameter estimates. One alternative in the literature is to assume that all subjects share one and the same parameter. This reduces variability in the regressors, and thus, the SPM beta estimates. The present procedure is what we believe to be a reasonable tradeoff between the two extremes.
fMRI acquisition. Functional imaging was conducted using a 3.0-Tesla GE Signa scanner with an eight channel phase array head coil to acquire gradient echo T2*-weighted echo-planar images as previously described [35, 36]. For each of the two sessions, 310 EPI volumes (~12 min) containing 29 slices were acquired [repetition time (TR) = 2,300 ms, echo time (TE) = 27 ms, matrix size 128 × 128, and a field of view (FOV) 256 × 256 mm2, thus yielding an in-plane voxel resolution of 2.7 mm2, flip angle α = 90 degree). Slices were acquired interleaved with a thickness of 4 mm and no gap. The acquisition plane was tilted 30° from the anterior–posterior commissure. A 3D anatomical image of the entire brain was obtained by using a T1-weighted 3D spoiled-gradient echo pulse sequence (TR = 7.8 ms, TE = 3.2 ms, matrix size 256 × 256, FOV 256 x 256 mm2, 1 mm slice thickness, flip angle α = 20°, voxel size 1 mm × 1 mm × 1 mm).
fMRI data preprocessing. Functional imaging data were analyzed using SPM8 (Wellcome Department of Imaging Neuroscience, Institute of Neurology, London, UK; http://www.fil.ion.ucl.ac.uk/spm/). After de-noising with ArtRepair (http://cibsr.stanford.edu/tools/ArtRepair/ArtRepair.htm), the following preprocessing steps were performed: acquisition time and motion correction, coregistration of the mean EPI to the anatomical T1 image, spatial normalization and segmentation into tissue classes of the T1 image using the unified segmentation approach as implemented in SPM8  application of the normalization parameters to all functional images, and spatial smoothing with an isotropic Gaussian kernel of 8 mm full width at half maximum (FWHM) kernel.
Small volume correction for multiple comparisons was used within a ventral striatal volume of interest (VOI). We created an fMRI literature-based probabilistic VOI for the VS . We selected 16 recent papers containing data from 325 subjects [5, 6, 35, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50]. From each study, the coordinates of PE-related activation for right and the left VS were extracted.
The images were analyzed in an event-related manner using the general linear model approach (GLM) as implemented in SPM8; neuronal activity was modeled using a stick function at the onsets of the feedback. We used a parametric design [32, 51] in which the trial-by-trial PE values from the RW model modulated the amplitude of the trial-related stick. Regressors of interest for the BOLD responses corresponding to the trial-wise PEs were generated by convolving the modulated stimulus functions with the canonical hemodynamic response function (HRF), provided by SPM8. To account for signal fluctuations associated with the movement by susceptibility interaction, the six movement parameters from the realignment preprocessing step were included in the model as additional regressors. The individual contrast images for the contrast of the PE-modulated feedback were then taken to a random-effects group-level analysis using a one sample t test. To test for associations with measures of IQ and SLES, these measures were entered as covariates into additional random-effects analyses.
In addition, we performed stepwise multiple regression analyses to test the predictive effect of SLES and PE signaling on fluid IQ using SPSS. These were used to assess (1) the effects of PE signaling and SLES on fluid IQ and (2) tested for interaction effects (moderation) between SLES and PE signaling on fluid IQ by computing an interaction term (specifically, we assigned participants reporting below median SLES a value of one and participants reporting above median SLES a value of two and multiplied this dichotomous variable with the peak VS BOLD PE signal) and added this interaction term to the regression analysis predicting fluid IQ.
To minimize false-positive results due to median split of SLES (and thus reducing effective sample size), we performed an additional stepwise multiple regression analyses entering SLES as a continuous variable into the interaction term after z-transformation (and multiplied this continuous variable with the peak VS BOLD PE signal) and then added this interaction term to the regression analysis.
Best-fitting parameter estimates are shown as median + quartiles across subjects
Prediction error signaling in the ventral striatum (VS Bold PE signal)
In the fMRI group of 16 healthy controls, we first observed a significant correlation between the model-free PEs and the BOLD response in the left VS (x/y/z = −18/3/−5, T = 3.38, p FWE corrected for VS VOI = 0.026). There was also a trendwise correlation in the right VS (x/y/z = 20/6/−5, T = 2.47, p FWE corrected for VS VOI = 0.057, for whole brain activation see Supplementary Figure 1). Thus, subjects’ VS BOLD activity correlated with the PE learning signal. We term this correlate the VS BOLD PE signal. Below, we examine correlations between this VS BOLD PE signal, SLES, and fluid IQ.
Correlations between fluid IQ, stressful life events, and VS BOLD PE signal
Fluid IQ was significantly and positively correlated with the VS BOLD PE signal both on the left (x/y/z = −11/8/−8, T = 3.89, p FWE corrected for VS VOI = 0.012) and on the right (x/y/z = 17/6/−8, T = 3.71, p FWE corrected for VS VOI = 0.013). Thus, VS functional activation was more closely correlated with model-free habitual PEs in subjects with higher IQ as reported previously . Upon controlling for age by introducing age as an additional covariate into the SPM analysis, VS PE BOLD signal remained associated with fluid IQ (left VS: x/y/z = −16/3/−8, p FWE corrected for VS VOI = 0.045, T = 3.11, right VS: x/y/z = 20/3/−8, p FWE corrected for VS VOI = 0.006, T = 4.30), suggesting that this association was not simply explained by an age-related decline in fluid IQ.
Effects of stressful life events and VS BOLD PE signal on fluid IQ
Stress has been shown to increase the habitual, or model-free, component of behavior . The PE regressor we have used is one such model-free, habitual, learning signal . We were thus interested to know whether stress increases the contribution of this basic learning signal to the more general index of flexible learning measured by fluid IQ. We therefore performed a stepwise regression analysis, asking whether the product of SLES and left VS BOLD PE signal explains additional variance in the fluid IQ measure beyond the effects of SLES and VS BOLD PE signal alone.
We learn from making mistakes and need to adapt our predictions in the face of changing circumstances; iterative learning via PEs plays a major role in such learning processes [7, 53]. The present results are the first reported data showing that a basic neurobiological learning signal—the ventral striatal PE signal—and SLES interact to predict individual differences in fluid intelligence, even when correcting for the decline of fluid IQ with age. Several points merit comment.
First, it may seem surprising that the relationship of SLES to the BOLD PE signal is in the same, positive, direction as that of fluid IQ. However, stress is well known to promote habitual, model-free responses in humans and animals. The VS BOLD PE signal quantifies how closely the BOLD signal in the VS correlated with a standard model-free learning signal. This is a measure of the co-alignment of the two time-varying signals, not of the magnitude of either of them. As such, a stronger VS BOLD PE signal is in keeping with more habitual, model-free learning. On the other hand, fluid IQ among other cognitive attributes captures planning ability and other goal-directed and more challenging (model-based) cognitive functions . The fact that higher fluid IQ is positively associated with a stronger alignment of VS activity with the model-free (habitual) learning signal is thus, at first sight, a counterintuitive finding. However, the model-free learning signal may be correlated with more complex model-based signals in this particular task, and there is evidence that a VS PE signal may in fact comprise both model-free and model-based components . One possible explanation is that this alignment is an expression of the engagement of several different learning strategies in parallel (including goal-directed—model-based and habitual—model-free), which is in accordance with our previous interpretation of study results. We had observed that fluid IQ is associated with the VS BOLD PE signal, a putative signature of habitual (model-free) learning even when controlling for behavioral fit, which indicates how strongly the observed behavior is accounted for by PE-driven learning . This issue needs re-visiting in the light of the current data, as here, subjects who had not experienced much life stress showed only a small correlation between IQ and VS BOLD PE signal. Thus, our finding of a correlate between fluid IQ and VS BOLD PE signal is driven by those subjects who have experienced substantial life stress, which may have gradually shifted their flexible cognitive capacities toward more model-free strategies.
Our results complement previous studies on the effect of (acute) stress on dopamine signaling and ventral striatal PE encoding. Stress exposure facilitates ventral striatal dopamine release [22, 23, 56], and Robinson et al.  recently observed increased ventral striatal PE signaling (model-free) of negative errors of reward prediction (i.e. when received outcome is smaller than expected) during acute stress. Additionally, Otto et al.  reported that acute stress exposure reduces the amount of model-based learning in subjects with low working memory capacity (a measure closely related to fluid intelligence and general cognitive ability), while model-free learning was unaffected and thus shifts the balance from goal-directed (model-based) toward more habitual (model-free) learning strategies during decision making. We now show that self-reported SLES, which reflect chronic rather than acute stressors, are positively associated with ventral striatal BOLD PE signal and that SLES interact with the neuronal learning signature to predict fluid IQ. Thus, it seems feasible that chronic stressors have an effect on VS PE signaling in that they induce a stronger encoding of the PE signal during habitual learning, effects which interact to predict individual differences in fluid IQ—above and beyond the (negative) effect of age on fluid IQ.
Despite the link between stress and the development of various psychopathologies [57, 58], few studies have compared the neurophysiological effects of acute and repeated stressors. There is ample evidence to indicate both types of stress influence the dopaminergic system, pointing in the direction of differential effects of acute and chronic stress [14, 16]. These modulations of dopaminergic firing have been seen in the ventral tegmental area and the VS [22, 23, 56] core areas for the neuronal representation of reward PEs, reward anticipation, learning from reinforcement [59, 60], and flexible behavioral adaptation (fluid IQ). Furthermore, studies suggest that genetic variation in the dopamine system moderates the effects of acute and chronic stress [58, 61] and are associated with individual differences in working memory capacity and other aspects of fluid intelligence [62, 63, 64, 65].
Our data indicate that the above reported VS BOLD PE signal works as an indirect neuronal signature of stress experience that is in part driven by a dopaminergic modulation which might shift learning strategies in highly flexible subjects (with high levels of fluid IQ) towards more habitual rather than goal-directed learning strategies. These data might allow for the speculation that it may be ecologically salient to encode errors of reward prediction more strongly when life experiences are mainly adverse and thus reduce cognitive demands in complex and threatening situations.
An important limitation to our findings is that the results are based on a small sample and therefore will require replication. Nevertheless, the direction of the stress effects is consistent with a recent report of the impact of acute stress on PE signals in a reversal learning task . Also, recent evidence  points toward the direction of gender differences in reward-related decision processing under stress. Therefore, our study will have to be repeated in an independent female sample. The findings are correlational, and hence, no statements about causality can be made. The various directional interpretations are, however, worth disentangling. It appears that stress alters how IQ relates to the VS PEs. The importance of the findings derive from the fact that this is the first study showing that an association between reward PE signaling and intelligence is moderated by life stress experience and this suggests stressful life experiences may sensitize the dopaminergic system toward more habitual decision making.
This work was supported by the German Research Foundation (Deutsche Forschungsgemeinschaft, DFG, FOR 1617: Grants HE2597/13-1, RA1047/2-1, and SM80/7-1).
Conflict of interest
- 7.Sutton RBA (1998) Reinforcement Learning: An Introduction. A Bradford Book, CambridgeGoogle Scholar
- 10.Schultz W (2010) Multiple functions of dopamine neurons. F1000 biology reports 2. doi: 10.3410/B2-2
- 11.Schott BH, Minuzzi L, Krebs RM, Elmenhorst D, Lang M, Winz OH, Seidenbecher CI, Coenen HH, Heinze HJ, Zilles K, Duzel E, Bauer A (2008) Mesolimbic functional magnetic resonance imaging activations during reward anticipation correlate with reward-related ventral striatal dopamine release. J Neurosci 28(52):14311–14319. doi: 10.1523/JNEUROSCI.2058-08.2008 PubMedCrossRefGoogle Scholar
- 12.Schlagenhauf F, Rapp MA, Huys QJ, Beck A, Wustenberg T, Deserno L, Buchholz HG, Kalbitzer J, Buchert R, Bauer M, Kienast T, Cumming P, Plotkin M, Kumakura Y, Grace AA, Dolan RJ, Heinz A (2013) Ventral striatal prediction error signaling is associated with dopamine synthesis capacity and fluid intelligence. Hum Brain Mapp 34(6):1490–1499. doi: 10.1002/hbm.22000 PubMedCentralPubMedCrossRefGoogle Scholar
- 13.Schlagenhauf F, Rapp MA, Huys QJ, Beck A, Wustenberg T, Deserno L, Buchholz HG, Kalbitzer J, Buchert R, Bauer M, Kienast T, Cumming P, Plotkin M, Kumakura Y, Grace AA, Dolan RJ, Heinz A (2012) Ventral striatal prediction error signaling is associated with dopamine synthesis capacity and fluid intelligence. Hum Brain Mapp. doi: 10.1002/hbm.22000 PubMedCentralPubMedGoogle Scholar
- 23.Cao JL, Covington HE 3rd, Friedman AK, Wilkinson MB, Walsh JJ, Cooper DC, Nestler EJ, Han MH (2010) Mesolimbic dopamine neurons in the brain reward circuit mediate susceptibility to social defeat and antidepressant action. The Journal of neuroscience : the official journal of the Society for Neuroscience 30(49):16453–16458. doi: 10.1523/JNEUROSCI.3177-10.2010 CrossRefGoogle Scholar
- 28.Wechsler D (1955) Wechsler adult intelligence scale manual. Psychological Corporation, New YorkGoogle Scholar
- 29.Beckers K, Behrends U, Canavan A (1992) Deutsche version des river mead behavioral memory test. Thames Valley Test Company, Bury St EdmundsGoogle Scholar
- 30.Horn W (1983) L-P-S Leistungsprüfsystem, vol 2. Auflage. Hogrefe, GöttingenGoogle Scholar
- 44.Murray GK, Corlett PR, Clark L, Pessiglione M, Blackwell AD, Honey G, Jones PB, Bullmore ET, Robbins TW, Fletcher PC (2008) Substantia nigra/ventral tegmental reward prediction error disruption in psychosis. Molecular psychiatry. 13 (3): 239, 267–276. doi: 10.1038/sj.mp.4002058
- 48.Schonberg T, O’Doherty JP, Joel D, Inzelberg R, Segev Y, Daw ND (2010) Selective impairment of prediction error signaling in human dorsolateral but not ventral striatum in Parkinson’s disease patients: evidence from a model-based fMRI study. NeuroImage 49(1):772–781. doi: 10.1016/j.neuroimage.2009.08.011 PubMedCrossRefGoogle Scholar
- 54.Hampton AN, Bossaerts P, O’Doherty JP (2006) The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. The Journal of neuroscience : the official journal of the Society for Neuroscience 26(32):8360–8367. doi: 10.1523/JNEUROSCI.1010-06.2006 CrossRefGoogle Scholar
- 65.Thompson JM, Sonuga-Barke EJ, Morgan AR, Cornforth CM, Turic D, Ferguson LR, Mitchell EA, Waldie KE (2012) The catechol-O-methyltransferase (COMT) Val158Met polymorphism moderates the effect of antenatal stress on childhood behavioural problems: longitudinal evidence across multiple ages. Dev Med Child Neurol 54(2):148–154. doi: 10.1111/j.1469-8749.2011.04129.x PubMedCrossRefGoogle Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.