“Wanted!” The effects of reward on face recognition: electrophysiological correlates
The aim of the present study was to uncover the temporal dynamics of face recognition as a function of reward. Event-related potentials (ERPs) were recorded during the encoding and the subsequent old/new memory test in response to faces that could be associated with a monetary reward. The behavioral results showed that faces associated with reward at both encoding and retrieval were recognized better than the unrewarded ones. ERP responses highlighted that successful encoding predictive of subsequent memory was greater for faces associated with reward than for faces without reward-driven motivational learning. At retrieval, an early positive-going component was elicited for potentially rewarded faces on frontal regions, while the occipito-temporal N170 component showed priming effects as a function of reward. Later on, larger centro-parietal ERP components, related to recognition memory, were found selectively for reward-associated faces. Remarkably, electrophysiological responses varied in a graded manner, with the largest amplitude yielded by faces with double reward, followed by faces associated with reward only at encoding. Taken together, the present data show that the processing of outcome expectations affects face structural encoding and increases memory efficiency, yielding a robust and sustained modulation over frontal and temporal areas where reward and memory mechanisms operate in conjunction.
KeywordsERP face recognition memory reward
An important function of our cognitive system is to evaluate the motivational significance of stimuli we encounter in the environment. A likely assumption is that evolution has forced the brain to develop special mechanisms to provide a rapid assessment of the valence, the magnitude, and other aspects of the reward value of events. These mechanisms can shape our behavior by enhancing specific perceptual and cognitive functions (for a review, see Pessoa & Engelmann, 2010; Raymond, 2009). Thus, events with emotional and motivational content are often remembered better than those of lesser relevance.
It is well known that a mild reward, such as that associated with everyday experiences like receiving a small gift, might improve cognitive performance in a subsequent task by strengthening motivation, attention, and learning (e.g., Estrada, Young, & Isen, 1994; Nielson & Bryant, 2005). It has been proposed that an enhanced neural interaction between reward-related regions and areas involved in perceptual and cognitive processing improves behavioral performance to maximize the reward outcome (Pessoa, 2009). For example, some studies have provided evidence that motivation and attention jointly influence visual orienting by modulating early sensory processes (Engelmann & Pessoa, 2007; Kiss, Driver, & Eimer, 2009; Rutherford, O'Brien & Raymond 2010). In this regard, Rutherford and coworkers suggested that prior experience of reward might influence visual selection.
Encoding relevant stimuli, while suppressing inappropriate inputs, can be considered as a core process in cognition. Such motivation-driven mechanisms might also have a pivotal role in memory encoding and retrieval. In this respect, a key question is how memory can be promoted and enhanced by motivation. One factor that might increase attention, motivation, and memory is monetary reward. Imagine seeing the face of a “wanted” person associated with a monetary incentive (as in western movies). It is unlikely that you will ever forget the “wanted” face.
Interestingly, it seems that long-lasting motivational effects depend on the type of reward. Behavioral findings have demonstrated that the positive effect generated by an extrinsic reward (e.g., money) can influence both learning and retention more than an intrinsic reward (e.g., a smile, a prize, or an attractive face; Nielson & Bryant 2005).
Neuroimaging studies have shown that monetary incentives promote memory encoding and consolidation, through the involvement of the hippocampus and its interactions with the mesolimbic reward system, with an increase in dopamine release (Adcock, Thangavel, Whitfield-Gabrieli, Knutson, & Gabrieli, 2006; Duzel, Bunzeck, Guitart-Masip, & Duzel, 2010; Hampton, Adolphs, Tyszka, & O’Doherty, 2007; Rossato, Bevilacqua, Izquierdo, Medina, & Cammarota, 2009; Shohamy & Adcock, 2010; Wittmann, Schiltz, Boehler & Duezel 2008; Wittmann et al. 2005). Specifically, Adcock et al. found that monetary reward, by enhancing hippocampal activity during encoding and subsequent memory (Adcock et al. 2006), might have a crucial role in facilitating learning.
Thus, we have a special ability to remember past rewards, probably because the positive arousal induced by reward anticipation promotes and enhances long-term memory formation (Knutson & Adcock, 2005).
Although functional imaging studies have led to a crucial advancement in disclosing the brain areas involved in reward processing, they have failed so far to uncover when these reward-related processes might occur. Undoubtedly, the dynamics of reward effects on visual encoding and memory can be studied with greater accuracy by using event-related potentials (ERPs), given their high temporal resolution. ERP studies have shown that emotionally and motivationally salient stimuli can be associated to a sustained increase in attention and in cognitive processing starting around 300 ms after stimulus onset; these effects are well described by an amplitude enhancement of the P300 and LPP (late positive component) ERP components.
The P300 has been associated with attention, motivation, and stimulus salience (for a review, see Hajcak, MacNamara, & Olvet, 2010; Polich, 2007), which are linked to reward processing (Ressler, 2004), and it has also been related to various aspects of outcome evaluation (Kamarajan, Porjesz, Rangaswamy, Tang, Chorlian, Padmanabhapillai and Begleiter 2009; Mennes, Wouters, van den Bergh, Lagae & Stiers 2008; Yeung, Holroyd, & Cohen, 2005; Yeung & Sanfey, 2004). Importantly, a correlation between an enhanced centro-parietal P300 and the magnitude of the reward was recently found by Goldstein and coworkers (2006), showing that this component is sensitive to reward value.
Furthermore, a later positive component known as LPP, starting around 300 ms after stimulus onset, has been found to be selective for stimuli with high relevance for the observer (Schupp, Flaisch, Stockburger, & Junghöfer, 2006; Schupp, Junghoefer, Weike and Hamm 2003). This component is larger in response to both pleasant and unpleasant, as compared with neutral, pictures and words, as well as to highly arousing stimuli (Foti & Hajcak, 2008; Hajcak, Dunning, & Foti, 2009; Hajcak, MacNamara and Olvet 2010; Schupp et al. 2004a, 2004b).
Although the onset latency of P300 and LPP is similar, the shorter duration of P300, as well as its sensitivity to nonemotional manipulations, led to the suggestion that the P300 might reflect the initial allocation of attention to motivationally salient stimuli, whereas the later LPP might be more specifically related to the stimulus significance (Foti & Hajcak, 2008; Hajcak et al., 2009; Schupp et al., 2006). Moreover, the LPP has also been linked to memory encoding and storage (Dolcos & Cabeza, 2002).
The modulations of the ERP responses produced by a reward have recently been investigated by means of a reward delivery omission procedure during a gambling task (Nahum, Gabriel, & Achnider, 2011; Schnider, Mohr, Morand, & Michel, 2007). An early brain modulation at frontal leads (peaking at about 250 ms) has been associated to the value of an expected reward (Schnider et al., 2007). This activity has been considered as an index of the suppression of a previously valid memory. Similarly, a frontal activation (peaking from 200 to 300 ms) emerged when the anticipated outcome (reward) failed to occur (Nahum et al., 2011). In this context, it is important to stress that evidence concerning the temporal dynamics of the neural correlates of the effects of monetary rewards on memory retrieval processes is rather scanty.
Taking into account the findings above and considering the neuroimaging evidence showing that reward anticipation might boost recognition memory, the aim of the present study was to study the temporal aspects of the effects of monetary reward on encoding and retrieval processes. For this purpose, an old/new paradigm was used as a suitable means to cast light on the physiological correlates of the encoding and recognition memory stages. On the basis of the neuroimaging evidence, our prediction was that reward and valuation processes might have a strong influence in shaping perceptual, encoding, and recognition memory functions (Pessoa & Engelmann, 2010).
To assess such a prediction, we tested face recognition with an old/new memory paradigm in which cues signaled a potential reward for the upcoming faces during the encoding and the retrieval phases. It is noteworthy to report that this experimental procedure is highly similar to the paradigm used by Adcock et al. (2006), with the difference that, in the present design, faces were used instead of scenes. This experimental design enabled us to identify the electrophysiological patterns occurring during encoding, when a link between a face and an extrinsic reward is established, as well as during retrieval, when such an association may promote access to memory representations. Furthermore, this old/new procedure might help in finding out whether (or not) the ERP correlates of suppression or enhancement of previously formed memories, shown in the experiments focused on short delays, hold for longer intervals, thus suggesting long-term memory consequences. In this respect, a recent behavioral study has shown that monetary rewards improve memory only after a delay (Murayama & Kuhbandner, 2011).
We used faces as stimuli because they are very suitable for tapping the time course of different processing stages characterized by well-known face-related ERP components. Specifically, ERPs studies have shown that face stimuli elicit a particularly large negative occipito-temporal component that peaks at around 170 ms (N170; see Bentin, Allison, Puce, Perez, & McCarthy, 1996). This component coincides in time with a positive component maximal in amplitude at centro-frontal sites, the vertex positive potential (VPP; Jeffreys, 1989; Jeffreys, Tukmachi, & Rockley, 1992). The N170 is supposed to reflect structural encoding processing—that is, the extraction of a perceptual representation of the face (Eimer, 2000)—and therefore, it might shed light on the possible influences of reward and motivated behavior on earlier perceptual and structural processes. Whereas the influence of top-down (i.e., cognitive) factors on the N170 has recently been ascertained by several studies (Galli, Feurra, & Viggiano, 2006; Jacques, d’Arripe, & Rossion, 2007; Jacques & Rossion, 2006; Jemel, Pisani, Calabria, Crommelink, & Bruyer, 2003; Jemel, Pisani, Rousselle, Crommelinck & Bruyer 2005; Marzi & Viggiano, 2007, 2010a, 2010b), showing a form of cognitive “penetrability,” it remains to be established whether this component is sensitive to motivational and reward mechanisms. Furthermore, for what concerns recognition memory processes, the well-established ERP old/new effects, which consist of an enhanced amplitude positivity for hits, as compared with correct rejections (for reviews, see Friedman & Johnson, 2000; Paller, Voss, & Boehm, 2007; Rugg & Curran, 2007), enabled us to investigate the reward-driven modulation of memory processes at retrieval.
Recently, it has been shown that the ERP old/new responses might be influenced by the emotional value of faces, such as facial expression (Johansson, Mecklinger, & Treese, 2004) and facial beauty (Marzi & Viggiano, 2010b). However, it is still unclear when and how monetary reward expectancy might interact with face recognition memory.
It appears reasonable to hypothesize that behaviorally relevant stimuli, such as reward-associated faces, might create enriched mnemonic representations, determining which episodic memories are formed and how they are represented. In particular, our prediction was that a potential rewarding outcome would facilitate successful memory encoding and reactivation of the experience that could lead to that outcome. To further assess the influence of motivation on encoding and successful memory formation, we also investigated the possible differences in study phase activity as a function of subsequent memory performance. ERP responses during encoding were compared for items subsequently remembered, relative to items later forgotten, to search for the presence of a differential neural activity based on memory—that is, a “Dm (difference in memory) effect” (Lucas, Chiao, & Paller, 2011; Paller, Kutas, & Mayes, 1987; Paller & Wagner, 2002).
In sum, the general aim of the present study was to try to clarify the temporal interplay between motivation, encoding, and memory, in order to uncover how the brain codes and stores information about the value of a stimulus.
Sixteen young healthy adults participated in this experiment. Two participants were excluded from analysis because of a large number of eye blinks during EEG registration (over 50% of the trials). The remaining 14 participants (8 females; mean age, 24.4 years; range, 21–28 years) were all right-handed (Viggiano, Borelli, Vannucci, & Rocchetti, 2001), had normal or corrected-to-normal vision, and had no history of neurological or psychiatric disorders. All participants were naïve regarding the aim of the study and gave their informed consent to participate in this experiment. This study was approved by the departmental ethics committee.
Stimuli and procedure
The stimuli consisted of 504 photographs of front-viewed faces of adult males and females. They were presented in grayscale in a 480 × 480 pixel window in the middle of the screen, on a uniform gray background. Participants were comfortably seated in front of the screen, at a distance of 100 cm, in a dimly illuminated, sound-attenuated, and electrically shielded room. The procedure consisted of an intentional encoding task, followed by a recognition memory test after a 5-min delay in which participants just rested. The experiment included 14 blocks of stimuli, each containing a study (24 trials) and a retrieval (36 trials) phase.
The time course of each trial was as follows: A fixation point appeared for 800 ms, followed by a reward/no-reward cue for 300 ms; then a delay of 1000 ms preceded the face, presented for 500 ms. The ERP data were time-locked to the presentation of the face.
During the retrieval phase, participants performed an old/new recognition task. As for the study phase, during the old/new task, each face was preceded by the reward/no-reward cue. Old faces were presented intermixed with new faces in a random sequence (for each block, 24 old and 12 new faces were presented). Participants indicated their responses by pressing one of two buttons on a response box with the index and middle fingers of the right hand. The time course of each trial was the same as in the study phase. The timeline of encoding and retrieval stimuli presentation is shown in Fig. 1.
Participants were told that the reward cue represented “real money” and that, at the end of the experiment, they would be paid, in cash, the total amount of money they had accrued during the memory test.
There were six experimental conditions: old faces associated with a reward cue both at encoding and at retrieval (RR), old faces cued only at encoding (R0), old faces cued only at retrieval (0R), old never-cued faces (00), new cued faces (NR), and new uncued faces (N0). Participants were told that they would receive a monetary reward for each correct old/new answer for faces associated with the monetary cue during encoding, during retrieval, or at both stages. Moreover, participants were punished with a monetary loss for each new face identified as old (false alarm errors), in order to make their choice of answer not excessively oriented toward an always-old decisional criterion that, otherwise, would have permitted them to maximize their reward earnings. The amount of reward was +0.20 € for each correctly remembered old face, and the amount of punishment was ˗0.10 € for each false alarm. Participants did not receive feedback regarding their earnings on each trial, but only at the end of the experiment. In addition to electrophysiological recordings, accuracy and reaction times (RTs) were recorded for each participant.
Apparatus and recordings
The stimuli presentation sequence was programmed with E-Prime E-Studio and displayed on a 19-in. LCD screen with a resolution of 1024 × 768 pixels and a refresh rate of 75 Hz. During both the encoding and the retrieval phases, we recorded continuous EEG, using an Intel Pentium IV computer with a Neuroscan 4.3 software package and a Nu-Amp system. The recordings were made using an elastic cap with 32 electrodes mounted according to the international 10–20 system. Scalp positions of the electrodes were the following: F7, F3, Fz, F4, F8, FT7, FC3, FCz, FC4, FT8, T3, C3, Cz, C4, T4, TP7, CP3, CPz, CP4, TP8, T5, P3, Pz, P4, T6, O1, Oz, O2. The right mastoid served as reference. ERP waveforms were rereferenced to right and left mastoid offline. The electrooculogram (EOG) was recorded in order to track horizontal eye movements with two electrodes placed in a bipolar montage at about 1-cm distance from the external canthus of each eye. Vertical eye movements and eye blinks were detected by 1 electrode positioned below the left eye. During acquisition, the EEG signals were amplified, filtered (0.01–100 Hz), and digitized at 1 kHz. The impedance of all electrodes was kept below 5 kΩ. Trials with eye blinks, eye movements (frontal electrodes exceeding 60 μV in the 1000-ms interval following the stimulus onset), and muscular or other artifacts (defined as a voltage deviation on any recording electrode exceeding ±60 μV in the 1000 ms following stimulus onset) were excluded from analysis. We also rejected trials on which horizontal or nonblink vertical movements occurred, as well as trials containing EEG drifts or A/D saturation. After removal of EEG and EOG artifacts, epochs beginning 200 ms prior to stimulus onset and ending 1000 ms after stimulus onset were used for the analysis. On each trial, potentials were baseline corrected using the signal during the 200 ms that preceded the onset of the stimulus. The EEG was averaged separately for each experimental condition, and the average waveforms computed for the different conditions were low-pass filtered at 15 Hz. Average waveforms were based on a minimum of 50 artifact-free trials.
Recognition accuracy was assessed by calculating the percentage of correct responses for old faces (hits) for the different reward value of each stimulus. Accuracy data were preliminarily adjusted with the arcsine transformation (Freeman & Tukey, 1950). For accuracy and RTs, 2 × 2 repeated measures ANOVAs were conducted considering old faces, with study–reward and retrieval–reward (each one with two levels: presence or absence of a reward cue) as main factors. Accuracies for new faces in the retrieval phase were separately compared by means of a t-test. Here and in all subsequent ANOVAs, to account for potential violations of the sphericity assumption, p values and degrees of freedom were adjusted using the Greenhouse–Geisser correction when appropriate (Keselman & Rogan, 1980). We also ran an analysis on sensitivity and criterion, using paired t-tests, in order to assess for a possible response bias driven by the reward cue in the retrieval phase.
ERP analyses were conducted on mean amplitude values for specific sets of electrodes within predefined time windows (100–150, 150–200, 200–300, 300–500, and 500–700 ms) chosen on the basis of visual inspection of grand-average amplitude and on the basis of previous literature.
Mean amplitude and peak latency values of the VPP and the N170 components, measured from 150 to 200 ms, were submitted to ANOVAs on electrodes T5 and T6 for the N170 and on electrodes FCZ and CZ for the VPP component, with reward (reward, no reward) and electrode (two electrodes) as factors. The P1 component was assessed on occipital (O1, Oz, and O2) and temporal (T5 and T6) sites in the 100- to 150-ms latency window.
For later time windows, separate ANOVAs were conducted on EEG mean amplitudes from a cluster of frontal-frontocentral electrodes (F3, FC3, FZ, FCZ, F4, FC4) and from a cluster of centroparietal-parietal electrodes (CP3, P3, CPZ, PZ, CP4, P4). The factors considered for the study phase were reward (reward, no-reward) side (left, middle, right), and electrode (two electrodes for each side).
Moreover, the subsequent-memory effect (Dm) was analyzed; brain activity during encoding was compared for items subsequently remembered with respect to items subsequently forgotten. To isolate the Dm effect within each condition, encoded faces were sorted into sets of later hits and later misses. An inclusion criterion of 20 artifact-free trials was established for each condition and for each participant. The Dm effect was computed for the R0 and 00 conditions. The rationale for this choice was that, during encoding, participants could not know whether, upon retrieval, faces would be associated with a reward cue or not. ANOVAs were performed with Dm (later hit, later miss), reward (reward, no reward), side (left, middle, right), and electrode (two electrodes for each side) as factors.
For each time interval, repeated measures ANOVAs were conducted separately for frontal-frontocentral electrodes (F3, FC3, FZ, FCZ, F4, FC4) and for centroparietal-parietal electrodes (CP3, P3, CPZ, PZ, CP4, P4), with study–reward (reward, no reward), retrieval–reward (reward, no reward), side (left, middle, right), and electrode (two electrodes for each side) as factors.
As for the study phase, mean amplitude and peak latencies values of the VPP and the N170 components were submitted to ANOVAs on electrodes T5 and T6 for the N170 and on electrodes FCZ and CZ for the VPP component. P1 was analyzed considering occipital (O1, Oz, and O2) and temporal (T5, T6) sites.
Furthermore, in order to compute the old/new effect, two subsidiary ANOVAs were performed on the selected time windows (the first with RR and R0 and the second with 0R and 00 as old conditions), with memory (old, new), retrieval–reward (reward, no reward), side (left, middle, right), and electrode (varying according to the considered time window, as for the other ANOVAs) as factors.
Behavioral analysis on accuracy scores for old faces showed an overall effect of the presence of reward on recognition memory, since both the main factors turned out to be significant [study–reward, F(1, 13) = 10.9, p < .007; retrieval–reward, F(1, 13) = 12.5, p < .004]. No significant interactions were observed.
These results showed that participants were more accurate in recognizing old faces associated, both during study and retrieval, with a reward cue, as compared with faces not associated with a reward.
The ANOVA performed on RTs showed effects similar to the ones found for accuracy. Participants were reliably faster in correctly recognizing old faces when a reward cue was associated, as compared with old faces without any potential monetary reward. Both main factors reached significance [study–reward, F(1, 13) = 8.9, p < .02; retrieval–reward, F(1, 13) = 6.7, p < .03]. No significant interaction emerged in this analysis.
New faces were judged more accurately when they were not associated with a reward, as compared with new faces associated with a reward, t(15) = 3.25, p < .006. No significant difference emerged in RTs between new reward-associated and not-associated faces.
An analysis on sensitivity (d-prime) did not show any effect of the reward cue during retrieval, t(15) = 0.7, p = .51. A shift in criterion was found in responses to faces associated to a reward during retrieval, as compared with nonrewarded faces, t(15) = 4.95, p < .001. Potentially rewarded faces showed a criterion of 0.08, as compared with 0.22 for nonrewarded faces.
In Fig. 2 (bottom), mean inverse efficiency scores are shown, defined as the RT divided by the percentage of correct trials (Townsend & Ashby, 1983). This measure allows one to correct for possible speed–accuracy trade-offs in the data (Shore, Barnes, & Spence, 2006).
No significant effects emerged for the 100- to 150-ms temporal window.
N170 A main effect of reward emerged in considering the mean amplitude of the N170 component, F(1, 13) = 6.1, p < .03. Furthermore, the significant reward × electrode interaction, F(1, 13) = 7.3, p < .02, indicated on T6 a smaller activation for faces associated with reward, as compared with unrewarded faces, F(1, 13) = 8.7, p < .02. No latency effects were found on the N170.
VPP The ANOVA performed in the early 150- to 200-ms time window on the mean amplitude of the VPP component showed a significant main effect of reward, F(1, 13) = 9.1, p < .02, with potentially rewarded faces eliciting enhanced amplitude, as compared with faces with no reward cue. No significant differences were found for the latency.
In the same time window, a significant effect of reward was found also on fronto-central and centro-parietal electrode clusters, F(1, 13) = 7.2, p < .02, and F(1, 13) = 9.1, p < .02, respectively, showing enhanced amplitude for potentially rewarded, as compared with nonrewarded, faces.
Time window 200–300 ms
Faces associated with reward continued to elicit enhanced amplitudes also in this time window, with a wide distribution, as shown by the significant main effect of reward on fronto-central and centro-parietal sites, F(1, 13) = 15.7, p < .003, and F(1, 13) = 9.8, p < .009, respectively.
Time window 300–500 ms
The ANOVAs showed a persistent effect of reward with a positive amplitude enhancement for potentially rewarded, as compared with nonrewarded, faces on fronto-central sites, F(1, 13) = 6.3, p < .03, and on centro-parietal and parietal sites, F(1, 13) = 4.8, p < .05.
Time window 500–700 ms
A significant effect of reward, F(1, 13) = 7.9, p < .02, emerged also in this later time window, possibly reflecting the LPP component. On centro-parietal and parietal electrodes, ERP responses to faces with potential reward elicited enhanced amplitudes with respect to faces not associated with any reward. No significant effects were found when considering the fronto-central electrode cluster.
From 500 to 700 ms on fronto-central sites, the Dm was still significant, F(1, 13) = 7.2, p < .02. Furthermore, on centro-parietal sites, a significant Dm × reward interaction emerged, F(1, 13) = 5.9, p < .04. Such an interaction indicates that, in the reward condition, the Dm effect showed larger positive ERPs for later-remembered, as compared with later-forgotten, faces. This effect was not significant for faces not associated with reward. Moreover, potentially rewarded later hits yielded enhanced amplitude, as compared with later hits not associated with reward during encoding. No further significant effects emerged. Subsequent memory effects were found beginning from 300 ms, and they took the form of an enhanced positivity for subsequently remembered, as compared with subsequently forgotten, faces. This effect was particularly evident for faces that, at encoding, were associated with the reward cue. Subsequent memory effects are depicted in Fig. 4.
Effects of reward on recognition of old faces
The first train of significant results occurred between 100 and 150 ms on the mean amplitudes at occipital and temporal sites, with a significant main factor of retrieval–reward, F(1, 13) = 5.6, p < .04, indicating an enhanced positivity for faces that, during retrieval, were not preceded by a reward cue. This effect can be seen in Fig. 5 on temporal electrodes T5 andT6.
The ANOVA revealed a significant study–reward × electrode interaction, F(1, 13) = 12.9, p < .04, indicating that the differences between conditions on the T6 electrode varied according to the presence or absence of a potential monetary reward during encoding. Post hoc comparisons showed that potentially rewarded old faces elicited reduced amplitude with respect to faces not associated to reward. No significant differences emerged for the latency of the N170 component. It is clearly visible in Fig. 7 that the RR and R0 conditions elicited a smaller N170 than did the 0R and 00 conditions.
No significant differences emerged on mean amplitude and latency for the VPP component. Time window 200–300 ms An enhanced positivity was found on fronto-central sites for faces associated with a monetary incentive during retrieval, as demonstrated by the significant retrieval–reward factor, F(1, 13) = 6.8, p < .03; no further significant effects emerged.
Time window 300–500 ms
The ANOVA carried out on the fronto-central electrode cluster revealed two significant main effects of study–reward, F(1, 13) = 7.7, p < .02, and retrieval–reward, F(1, 13) = 8.9, p < .02, as well as a study–reward × retrieval–reward × electrode interaction, F(1, 13) = 5.7, p < .004.
Post hoc comparisons showed an interesting gradual effect of amplitude enhancement as a function of reward (this effect can be seen in Fig. 7). All in all, the reinforced RR condition, with reward cues during both encoding and retrieval, elicited the greatest amplitude enhancement, followed by the R0 condition (with reward only at encoding), by the 0R condition (with reward only at retrieval), and finally by the 00 condition, which showed the smallest amplitude. These effects were reflected in a greater positivity for RR with respect to R0 (on electrodes F3, FC3, Fz, FC4; ps < .04) and with respect to 0R (on electrodes FCZ, F4, FC4; ps < .04). Moreover, electrophysiological responses in the R0 condition were enhanced, as compared with responses in the 00 condition (on electrodes F3, Fz, F4; ps < .04), and responses in the 0R condition were enhanced, as compared with the 00 condition responses (on electrodes F3 and F4; ps < .04). On centro-parietal sites, the study–reward factor was significant, F(1, 13) = 5.5, p < .04, with an amplitude enhancement in both RR and R0 with respect to 0R and 00.
Time window 500–700 ms
In this later latency window, the ANOVA revealed a main effect of Study–reward on both fronto-central, F(1, 13) = 16.4, p < .002, and centro-parietal, F(1, 13) = 6.4, p < .03, electrode clusters, indicating an overall greater positivity in the RR and R0 conditions with respect to R0 and 00.
For each time window, two separate ANOVAs were carried out; in the first one, the RR and R0 conditions were compared with the NR (new faces associated with reward) and N0 (new faces not associated with reward) conditions, whereas in the second one, the new faces were compared with the 0R and 00 conditions.
Interestingly, old/new effects emerged very early in the latency of both the N170 and the VPP components (see Figs. 6 and 7), while no effects were found on the mean amplitudes of both the components. Specifically, on the N170 component, shorter latencies were found for old with respect to new faces, as evidenced by the significant main effect of memory that emerged in both ANOVAs, F(1, 13) = 6.4, p < .03, and F(1, 13) = 11.3, p < .006. The same pattern of results was reliably observed on the VPP component, in which the memory factor was also significant for both the analyses, F(1, 13) = 43.2, p < .001, and F(1, 13) = 54.8, p < .001. These results reflect the fact that old faces elicited responses with shorter peak latencies, as compared with new faces. The old/new effects are clearly visible in Figs. 6 and 7.
Later on, from 300 to 500 ms, the RR and R0 condition faces still elicited enhanced positivity, as compared with new faces, on frontal sites [significant main effect of the memory factor, F(1, 13) = 16.2, p < .002], while no old/new differences were found for 0R and 00 condition faces, as compared with new faces. The same result, with enhanced positivity for old with respect to new faces, was found also on centro-parietal and parietal sites [significant main effect for the memory factor, F(1, 13) = 10.1, p < .008].
Finally, from 500 to 700 ms, an old/new effect was found on frontal and parietal regions, reflecting a more positive-going ERP response to old faces (RR and R0 conditions) with respect to new faces, F(1, 13) = 4.9, p < .05, and F(1, 13) = 15.7, p < .003. No further significant effects were found.
The topographies clearly show that beginning at about 300 ms after stimulus onset, an enhanced positivity, distributed over frontal and centro-parietal sites, emerges specifically for RR faces and also persists on later latencies (late old/new effect); on fronto-central sites also, condition R0 shows a remarkable enhanced positivity with respect to 0R and 00, but it becomes less conspicuous, as compared with RR, from 500 to 700 ms. In sum, a gradual effect occurs, with the greatest positivity for the RR condition, followed by the R0 condition, while no old/new differences are seen for the 0R and 00 conditions.
Unraveling the chronometry of reward processing can provide insight into fundamental principles of motivational learning and its effects on face recognition memory. In the present study, we investigated when and how a reward delivery affects the ERP components related to face encoding and recognition. The main purpose was to shed light on the time course of the effects exerted on memory for faces by an explicit reward cue signaled during study and retrieval.
Memory encoding and predictive reward
The ERP data for the study phase yielded several main findings. First of all, an enhanced early fronto-central positivity beginning around 150 ms was observed for faces associated with reward. This activation could reflect an enhanced allocation of attentional resources toward emotionally and motivationally significant faces, perhaps responsible for a subsequent enhancement of memory traces (Kilpatrick & Cahill, 2003). It might be the case that the reward anticipation leads to a motivated learning, thus ensuring an efficient allocation of cognitive resources. By the same token, an fMRI study that used a quite similar experimental procedure showed that monetary reward enhanced hippocampal activity during encoding and subsequent memory for scenes (Adcock et al., 2006), suggesting a reward-related role in facilitating learning.
A second important finding was that reward-related effects emerged also for a well-established neural signature of face processing, the N170 component (Bentin et al., 1996; Eimer, 2000). The modulation of the N170 might suggest that the processing of visual stimuli can be modulated by task relevance and by their motivational significance (see Vuilleumier & Driver, 2007, for a recent review). At a first glance, the increased amplitude in response to nonrewarded faces may suggest an increased difficulty during the structural encoding, probably due to an attentional inhibition process for faces that have to be ignored in order to maximize the subsequent outcome. That could represent an active cognitive process, such as the ability to suppress an undesired memory formation (Anderson & Green, 2001; Mecklinger, Parra, & Waldhauser, 2009). In contrast, a more efficient encoding process has been found for stimuli associated with reward, as indicated by the study of Adcock et al. (2006).
Stimuli that convey motivational significance are able to preferentially engage attention (Engelmann, Damaraju, Padmala, & Pessoa, 2009; Mogg & Bradley, 1999; Most, Smith, Cooter, Levy, & Zald, 2007). In this respect, it has been suggested that reward might promote the “fine-tuning” of attention, leading to preferential processing of specific events (Engelmann & Pessoa, 2007; Pessoa, 2009; see also Della Libera & Chelazzi, 2009).
Later on in the ERP response, a sustained ongoing increased positivity, reflecting the LPP, was found on central and parietal sites, from 300 up to 500 ms after stimulus onset, which was selective for potentially rewarding faces and is already known for being related to the processing of motivationally salient stimuli (Schupp et al., 2000; Schupp et al., 2006). These enhanced ERP amplitudes might represent the neural correlate of a later top-down process of outcome evaluation in which motivational evaluations come into play (Hajcak, Holroyd, Moser, & Simons, 2005; Hajcak, Moser, Holroyd, & Simons, 2007; Yeung & Sanfey, 2004).
The processing of faces associated with reward during the study phase could also be related to the encoding of value-prediction codes that are relevant for the subsequent recognition memory. When a particular face is known to potentially produce a reward, learning allows acquiring and storing specific neural codes for predicting which outcome is most likely, when the same face is encountered again. These value-prediction codes allow the comparison of diverse options with diverse outcomes (Montague & Berns, 2002; Padoa-Schioppa & Assad, 2006; Rutherford et al., 2010). The acquisition, updating, and use of value-prediction codes might represent crucial abilities needed to correctly perform the subsequent memory task. The expectations about potential rewards, taken together with the capability of reward to produce hedonic consequences, probably promotes motivation to learn and to efficiently encode reward-associated faces (Berridge & Robinson, 2003).
The reward/nonreward cues used in the present procedure motivated participants to memorize the presented faces and probably gave rise to an anticipatory mechanism that prepared for a more efficient encoding of the upcoming event (Haynes et al., 2007). Several studies have shown that neural activity elicited by a cue preceding an event can influence whether that event is later remembered (Adcock et al., 2006; Gruber & Otten, 2010; Otten, Quayle, Akram, Ditewig, & Rugg, 2006; Otten, Quayle, & Puvaneswaran, 2010; Park & Rugg, 2010). For example, neural activity elicited by a reward cue was found to predict later memory when the incentive to memorize a word was high (Gruber & Otten, 2010). These findings indicate that engaging neural activity that enhances the encoding of an upcoming event is under voluntary control, reflecting a preparatory state that anticipates the processing of an event (Gruber & Otten, 2010).
We suggest that reward motivation can play a pivotal role in driving successful encoding and memory formation. This effect clearly emerged, in the present study, by comparing brain potentials of successful versus unsuccessful encoding, in the Dm (Paller et al., 1987). Interestingly, we found that a widespread positivity starting from about 300 ms after stimulus onset predicted subsequent memory for faces preceded by a reward cue and that this effect was absent for faces not associated with a reward cue. This centro-parietal component probably reflects activity related to attentional enhancement at encoding that enables more detailed processing, thereby promoting later recognition. In keeping with previous studies (Lucas et al., 2011; Paller & Wagner, 2002; Yovel & Paller, 2004), we suggest that this amplitude enhancement for subsequent remembered faces represents an electrophysiological predictor of a successful encoding into long-term memory and that it is affected by reward.
Importantly, it has been shown the existence of a strong interaction between reward-related processes, involving the orbito-frontal cortex, and successful memory encoding, which relies on the hippocampus (Tsukiura & Cabeza, 2011).
Considering that the ERP response patterns emerged during the encoding phase, we propose that the association of faces with a potential monetary gain determines a more efficient allocation of attentional resources, a facilitated visual processing, and a highly motivated learning. Therefore, we claim that all these factors, which are related to an anticipated reward, contribute to promote successful encoding and storage of long-term memory information.
Reward-related recognition memory
The behavioral data showed enhanced recognition rate for faces associated with reward only at encoding or during both encoding and retrieval, with respect to faces with reward only during retrieval or unrewarded faces, similar to the recent findings regarding highly attractive faces (Marzi & Viggiano, 2010b). More specifically, the potentially rewarding faces yielded higher accuracies and shorter RTs, as compared with the nonrewarded faces. Accordingly, it has been shown that learning associated with rewarding stimuli enhances subsequent memory performance (Eppinger, Herbert, & Kray, 2010), possibly by means of an intent to remember (Block, 2009). Moreover, the ability to select or to ignore specific visual stimuli appears to be strongly biased by the delivery of monetary rewards also when the same objects are encountered after several days (Della Libera & Chelazzi, 2009).
However, while the reward-driven facilitation that we observed on behavioral responses appears to be a genuine effect with respect to the effect of the encoding-delivered reward cue, the improved performance elicited by the reward cue during retrieval is associated with a shift in response criterion. At retrieval, participants showed a tendency to adopt a more liberal response criterion for potentially rewarding faces (i.e., old judgments were more common) than for nonrewarding ones. This tendency emerged also in responses to new faces, with false alarms being more common when a reward cue was present. These findings could therefore represent a sort of familiarity bias toward reward-related stimuli. In a similar vein, Krebs, Schott, Schütze, and Düzel (2009) showed that recognition memory for novel and familiar stimuli was enhanced by reward anticipation and that a familiarity bias emerged for reward-associated stimuli.
An alternative explanation for this “feeling of familiarity” for potentially rewarded faces might be related to the fact that the punishment for false alarms might not have been fully effective in counteracting the participants’ heuristic tendency to respond old on reward trials, hence determining a shift in their response criterion.
Little is known about the processing stage at which a value prediction might affect simple decisions, such as judging whether a visually presented face is familiar or novel. To assess this issue, we compared the electrophysiological patterns during retrieval of previously seen faces with responses elicited by new faces.
An enhancement was observed on the P1 component for faces not associated with a reward during retrieval. This finding could be interpreted either as possible ERP signatures of attentional inhibition mechanisms to suppress unwanted memories or as an index of visual facilitation for faces associated with reward. Reduced P1 amplitudes have been previously found in a go/no-go task for go stimuli (Thomas, Gonsalvez, & Johnstone, 2009). Attentional inhibition was found for task-irrelevant distractors (Raymond, Fenske, & Tavassoli, 2003), and it has been proposed that distractor suppression may be sensitive to reward contingencies (Della Libera & Chelazzi 2009). These results suggest that motivation can affect early visual processing enhancing selective attentional processing, showing that the attentional selection of target stimuli can be modulated by their reward value (Kiss et al., 2009).
Similarly, it is possible to explain the enhanced negativity that we found at about 200 ms on fronto-central sites for faces without reward and with the reward cue only during encoding as an index of inhibitory mechanisms. An interaction between inhibitory processes and emotional stimuli has recently been reported on the N2 (Kiss, Raymond, Westoby, Nobre, & Eimer, 2008).
At early stages, we observed also an effect of familiarity on the latency of the VPP and N170 components. A significant delay was found for new faces, as compared with old faces, consistent with perceptual priming effects yielding faster responses for repeated faces (Guillaume et al., 2009; Itier & Taylor, 2004; Marzi & Viggiano, 2010a, 2010b). The reward-related effects found on the N170 were even more interesting, with a smaller amplitude in response to faces with the reward cue during both study and retrieval and to faces associated with the reward only during study, with respect to the other conditions. A better encoding and consolidation, due to reward-driven effects during study, might facilitate a subsequent processing of the face, hence giving rise to priming or repetition effects, with a consequent reduction in amplitude. Potentially rewarded faces may appear more familiar, thereby diminishing “deep” stimulus processing. Such a view seems to be supported by the greater false alarm rate associated with reward-related faces. Several lines of research have recently shown effects of familiarity, repetition priming, and recognition on N170 responses to faces (Caharel, Courtay, Bernard, Lalonde, & Rebai, 2005; Campanella et al., 2000; Guillaume et al., 2009; Heisz & Shedden, 2009; Herzmann & Sommer, 2010; Jacques & Rossion, 2006; Jemel et al., 2005; Jemel et al., 2003; Marzi & Viggiano, 2007, 2010a, 2010b), suggesting that, as early as 170 ms after stimulus onset, the brain is individuating previously encoded faces. In line with these studies, we further suggest that reward-related processes might interact with structural encoding processes that lead to face recognition. Hence, our results show that the speed and efficiency with which faces are processed can be modulated by their reward status. Importantly, the prospect of a potential reward at encoding, rather than at retrieval, appears to be crucial in the reward-driven enhancement of subsequent memory performance. At later processing stages, we found a robust and sustained effect of reward on memory processes. These effects begin at about 300 ms at frontal and fronto-central sites, and, subsequently, this increased activation for old potentially rewarded faces extends to more posterior areas (centro-parietal and parietal sites), in the time window from 500 to 700 ms. Overall, these results show that the monetary prospect promotes a correct face recognition and yields an enhanced activation for old, as compared with new, faces (with or without an associated reward). Interestingly, a gradual effect was found for old faces as a function of reward: The greatest positivity was elicited by faces with reward at encoding and retrieval (RR), followed by reward only during study (R0), by reward only during retrieval (0R), and finally by faces without any reward (00). An interesting outcome is that whereas old–new differences were found for faces associated with reward in both study and retrieval or only during study, faces with the reward cue only at retrieval or unrewarded showed almost the same amplitude as new faces. This means that reward motivation during learning favors encoding and promotes subsequent access to memory, while a reward delivered only during retrieval is less influential. That is in keeping with the results of Bunzeck, Doeller, Fuentemilla, Dolan, and Duzel (2009) showing that reward motivation at retrieval accelerates access to memory representations but does not substantially change the quality of the representations accessed. Thus, reward-related improvements in memory accuracy are more likely to be seen for an explicit reward manipulation at encoding, rather than at retrieval (Adcock et al., 2006; Wittmann et al., 2005).
On the basis of the findings above, one can suggest that reward motivation during learning influences top-down control of subsequent access to memory representations. The face encoding is strengthened by a related value-prediction information—specifically, the possibility of an outcome, which appears to provide a facilitatory top-down signal during later recognition. This enhanced reactivation in response to reward could be a mechanism for binding rewarding outcomes to the experiences that preceded them.
In conclusion, we showed that a prospective delivery of a monetary reward increases the efficiency of memory for faces. Given that rewarded events are often well remembered, we suggest that reward might modulate face memory encoding, with effects on the N170 and LPP components and storage mechanisms. In particular, we propose that a reward expectation might facilitate subsequent reactivation of previously encountered faces with an amplitude enhancement for old faces associated with reward. A crucial factor for these effects to take place is that the face–reward association is expected to be present during the learning phase. Reward during learning probably enhances motivation and promotes the strengthening of face representations. During retrieval, the previously established reward–face association competes with unrewarded faces in the old/new decision, and, possibly, top-down processes could facilitate the access to a specific path in which the reward-related information is stored.
Taken together, the present electrophysiological data show that the processing of outcome expectations interacts with both the earlier perceptual and structural encoding processes and with the later memory-related ERP components, showing a robust and sustained modulation over frontal and temporal areas where reward and memory mechanisms take place.
We thank two anonymous reviewers for their thorough and important comments on previous versions of the manuscript.