Decision-making is generally regarded as a series of mental processes resulting in the selection of an action (among different options) on the basis of the available information, including sensory input, internal states, and most importantly, the rules of the task in which a person is engaged. During the past three decades, many studies have examined the neural correlates of decision-making in human and nonhuman primates (for reviews, see Gold & Shadlen, 2007; Heekeren, Marrett, & Ungerleider, 2008). Recent neuroimaging studies have suggested that a domain-general mechanism of decision-making can be responsible for gathering and comparing sensory information and selecting the proper action, independent of the domain of the sensory input and the type of motor output (Heekeren, Marrett, Bandettini, & Ungerleider, 2004; Heekeren, Marrett, Ruff, Bandettini, & Ungerleider, 2006; Ho, Brown, & Serences, 2009; Kayser, Buchsbaum, Erickson, & D’Esposito, 2010). In parallel, studies of event-related potentials (ERPs) have shown that the neural processes underlying decision-making are linked to the late ERP, usually evoked around 300 ms after stimulus presentation within fronto-parietal electrode sites (Bankó, Gál, Körtvélyes, Kovács, & Vidnyánszky, 2011; Philiastides & Sajda, 2006).

To detect those processes that can be influenced by decision-making processes, previous studies have searched for ERP components whose variations in amplitude (or timing) are correlated to the level of task difficulty and/or to response accuracy (e.g., Bankó et al., 2011; Philiastides & Sajda, 2006). These studies have shown that the amplitudes of ERPs usually evoked around 300 ms after stimulus onset show a strong correlation to response accuracy, while this correlation is weaker, and usually nonsignificant, for earlier components (Bankó et al., 2011; Philiastides & Sajda, 2006).

The other major difference between the neural processes underlying perception and decision-making is that, while the mechanism of object perception can be “category specific” and can vary from one object category to another, decision-making processes seem to be “domain general” and act independently of object categories and/or response mechanisms (Heekeren et al., 2006; Heekeren et al., 2008; Ho et al., 2009). For instance, although early ERP components such as the occipito-temporal N170 (Bentin, Allison, Puce, Perez, & McCarthy, 1996; Botzel, Schulze, & Stodieck, 1995; Itier & Taylor, 2004; Nasr & Esteky, 2009), vertex positive potential (VPP; Jeffreys, 1989, 1993, 1996; Joyce & Rossion, 2005; Nasr & Esteky, 2009), and occipito-temporal N250 (Nasr & Esteky, 2009; Schweinberger, Huddy, & Burton, 2004) show selectivity to faces, late components are less sensitive to stimulus category and vary mainly on the basis of task difficulty and/or of the subjects’ confidence, irrespective of the test stimulus category (Philiastides & Sajda, 2006). Neuroimaging studies consistently have shown that those brain areas responsible for decision-making, including dorsolateral prefrontal cortex (Heekeren et al., 2006; Wenzlaff, Bauer, Maess, & Heekeren, 2011) and/or insular cortex (Ho et al., 2009), seem to act independent of stimulus categories and/or response modality.

Despite this evidence for the contributions of modules underlying late ERP components in decision-making, it is not clear whether these potentials index those processes that monitor task difficulty (and subjects’ confidence) or whether variation in the task rules can also affect these ERPs. In general, each cognitive task consists of a series of rules that link the outcome of sensory perception to a proper action, and it is possible to vary the task rules while subjects’ confidence and/or response accuracy remain at the same level. To the best of my knowledge, no previous studies have checked whether a change in the task rules also influences late potentials. In addition, it is unclear whether the potential effect of change in the task rules varies with stimulus categories (i.e., is category specific) or acts independently of stimulus categories (i.e., is domain general).

My main goal in this study was to detect those brain-evoked potentials that were influenced by the task rules and to check whether these potentials showed sensitivity to stimulus categories (face vs. nonface) or were evoked irrespective of the visual stimuli (i.e., category specific vs. domain general). Human subjects were also instructed to do matching tasks at the category and identity levels. Irrespective of the subjects’ task, each trial consisted of two stimuli (i.e., sample and test) that were (a) identical, (b) different but still selected from the same category, or (c) selected from different categories. While the subjects’ responses to the first and third trial types remained the same between the matching-to-category and matching-to-identity tasks, their responses to the second trial type varied depending on the task (see the Method section). The results showed that frontal potentials, evoked 228–328 ms after test stimulus onset, varied according to the task rules and indexed those processes responsible for decision-making. A preliminary version of this work was presented at the Society for Neuroscience annual conference (Nasr, Sanayei, & Esteky, 2008).

Method

Subjects

A total of 13 male undergraduate students (20–28 years of age) participated in this experiment and received monetary compensation. The subjects were right-handed and had normal or corrected-to-normal vision and no history of neurological or psychiatric disorders. Written informed consent in accordance with the principles of the Declaration of Helsinki was obtained from each of the subjects prior to the experiment. All of the procedures were approved by the ethics committee of Shaheed Beheshti University of Medical Sciences (Tehran, Iran), as well as by the Iranian Society for Physiology and Pharmacology.

Stimuli

A total of 50 face and 50 leaf images were used in this experiment (Fig. 1). Each image subtended 7.3 × 7.3 deg of visual angle at a distance of 70 cm. The face images were generated by Facegen Modeller (www.facegen.com) and represented White Caucasian individuals (the same race as the subjects) without any emotional expression and only eyebrows as facial hair. The leaf images were selected from the Hemera photo objects collection (Hemera Technologies Corporation, Seattle, WA). They were from the front view, and the longest axis of the leaf images was parallel to the longest axis of the faces. All of the images were in grayscale, and their luminance was adjusted. The task cues were small colored squares (blue vs. red) that subtended 1 × 1 deg of visual angle. The response cue was a small black “question mark” that subtended 1 × 2 deg of visual angle. The stimuli were all presented at the center of a 19-in. monitor (LG F900P) against a light gray background (14.8 cd/m2), using the MATLAB Psychophysics Toolbox (Brainard 1997; Pelli, 1997).

Fig. 1
figure 1

Examples of the face and leaf stimuli used in this study

Full size image

Tasks and procedures

Figure 2 shows a schematic representation of a trial. In each trial, four stimuli (sample, task cue, test, and response cue) were sequentially presented, each for 100 ms with a 900-ms blank interstimulus interval.

Fig. 2
figure 2

Schematic representation of the trial types. Each trial started with a sample presentation, followed by the task cue, test stimulus, and response cue. The sample and test stimuli could be either a face or a leaf image. The sample and task stimuli could be identical (top; i.e., matched identity, or MI), could belong to the same category but represent two different identities (middle; nonmatched identity, or NMI), or could belong to two different categories (bottom; nonmatched category, or NMC). The task cue instructed subjects to match either the identity or category. The relation between the task-cue colors (blue vs. red) and the subjects’ tasks was counterbalanced between subjects

Full size image

Previous studies had shown that perception of sample stimuli can affect the perception of a subsequent test stimulus, a phenomenon widely known as priming (Guillem, Bicu, & Debruille, 2001; Henson et al., 2003; Rugg, Mark, Gilchrist, & Roberts, 1997; Schweinberger, Pfütze, & Sommer, 1995; Trenner, Schweinberger, Jentzsch, & Sommer, 2004). It has also been shown that subjects’ awareness of the task at hand can affect both sample-stimulus perception (Curran, Tanaka, & Weiskopf, 2002) and its priming effect on the following test stimulus (i.e., a differential priming effect; Schweinberger, Pickering, Jentzsch, Burton, & Kaufmann, 2002). To make the perception of the sample stimulus independent from the subjects’ task, the task cue was presented 1,000 ms after the sample-stimulus onset, when perception of the sample stimulus was complete. This reduced the chance of a differential priming effect on perception of the test stimulus (see the Discussion below).

The experiment did not have a blocked design, and across trials the subjects’ task varied randomly between delayed matching to identity (DMI; 50 % of trials) and delayed matching to category (DMC; 50 % of trials). During the DMI task, the subjects reported whether the sample and test stimuli were identical. In the DMC task, the subjects reported whether the sample and test stimuli were selected from the same visual category (i.e., either leafs or faces), regardless of whether they belonged to the same individual. The relationship between the task-cue color (blue vs. red) and the tasks (DMI vs. DMC) was counterbalanced between subjects.

Irrespective of the subjects’ task, the sample and test stimuli were selected pseudorandomly for each trial, with the constraints that (a) for 33.3 % of the trials, the sample and test stimuli were identical (matched-identity, or MI, trials); (b) for another 33.3 % of the trials, the sample and test stimuli were selected from the same category, but had different identities (nonmatched-identity, or NMI, trials); and (c) for the rest of the trials, the sample and test stimuli were from different categories (nonmatched-category, or NMC, trials). This method of stimulus presentation resulted in a 2 × 2 × 3 design (i.e., Face Test vs. Leaf Test × DMI vs. DMC × MI vs. NMI vs. NMC) in which the subjects encountered all trial types with equal probability.

In each trial, the subjects had to report their responses by pressing one of the two keys on a keypad (i.e., a two-alternative forced choice) using two fingers of their dominant hand. To the best of my knowledge, no previous study has shown any difference between evoked brain activities when subjects are using two fingers of the same hand. To decrease the impacts of motor preparation/activity on evoked potentials, the subjects had to hold their response until the response cue appeared on the screen, 900 ms after the test stimulus offset. Any response prior to the appearance of the response cue was discarded ( < 2 % of trials), and no relationship was found between the number of discarded trials and the experimental parameters (i.e., task, stimulus category, and/or trial types). The subjects did not receive any feedback about the accuracy of their responses. They practiced for about 10 min with the tasks and stimuli, and we started the recording session only when a subject’s performance was stabilized around 80 % or more. During both the practice and recording sessions, response accuracy, not speed, was stressed. Each subject participated in 1,200 trials (30 blocks) during a single recording session.

ERP recording and analysis

EEG recording was performed with a Neuroscan system with 32 Ag/AgCl sintered electrodes mounted on an elastic cap (10–20 montage). Data were acquired continuously in AC mode (0.05–30 Hz), with a 1-kHz sampling rate. Since finding activity modulations in the fronto-parietal electrode sites was expected (Heekeren et al., 2004; Philiastides & Sajda, 2006), linked mastoid electrodes were selected as reference electrodes, grounded to AFz (see Joyce & Rossion, 2005, for a comprehensive review). To remove any residual activity from the sample and task-cue intervals, baseline activity was corrected on the basis of a pretest stimulus interval of 100 ms.

Four electrodes monitored horizontal and vertical eye movements for offline artifact rejection. Channel impedance was kept at < 5 kΩ during the recording of brain activity. Any trials with eye movements and/or eye blinks (determined by a peak voltage in either the horizontal or vertical eye movement channels that exceeded ± 30 μV) from −100 to 1,000 ms after the test stimulus onset were discarded (< 10 % of trials). Again, no relationship was found between the experiment parameters and the number of rejected trials.

The sliding-windows method (Nasr, Moeeny, & Esteky, 2008) was used to assess the onset latency and duration of brain activity modulations starting from 100 ms before test-stimulus onset. The length of the time windows for the initial search was set to 48 ms, with a 91.7 % overlap between adjacent windows. Since response accuracy was high (see the Results), all trials were used in the data analysis, irrespective of response accuracy. For each time window, a three-factor repeated measures analysis of variance (ANOVA; Task [DMI vs. DMC] × Trial Type [MI vs. NMI vs. NMC] × Test Stimulus Category [face vs. leaf]) and, where necessary, subsequent Greenhouse–Geisser correction for sphericity were applied to the averaged activity measured in that time window. Channels with similar patterns of activity that showed maximum differential activity between the experimental conditions were selected on the basis of maps of brain activity distributions (Figs. 3 and 4) and then averaged to improve the signal-to-noise ratio. Whenever a significant effect was found in consecutive time windows, the time window was enlarged to cover the whole interval, and the same three-factor repeated measures ANOVA was applied to the averaged activity measured in the extended interval. Thus, all of the brain activity reported here passed two different criteria, based first on applications of ANOVA to the short intervals, and second on application of ANOVA to the whole, long interval. However, only the results of the final analysis are reported in the Results section.

Fig. 3
figure 3

Evoked brain potentials across different experimental conditions, averaged over face- and leaf-test trials. Each box represents one lead, and those with a red outline showed systematic modulation during the tasks. In each box, the top and bottom panels represent evoked activity during the delayed-matching-to-category and delayed-matching-to-identity tasks, respectively

Full size image
Fig. 4
figure 4

Frontal ERPs evoked after test-stimulus onset when the subjects performed category-matching (top) and identity-matching (bottom) tasks. The ERPs were averaged over the face-test and leaf-test trials. The red, blue, and green lines represent the evoked potentials during the matched-identity (MI), nonmatched-identity (NMI), and nonmatched-category (NMC) trials, respectively. The brain activity maps represent the differential distributions of MI–NMI (left) and NMI–NMC (right) activity throughout the scalp during the 228- to 328-ms span after the onset of the test stimulus

Full size image

Results

The subjects showed very high response accuracy ( > 85 %) during both the DMC and DMI tasks and across all experimental conditions (Table 1). Application of a three-factor repeated measures ANOVA, similar to the one used for the ERP analysis (see the Method section), to the measured response accuracy showed only a significant Task × Trial Type × Test Stimulus Category interaction [F(1.18, 14.26) = 8.426, p < .01], while the other effects remained nonsignificant (p > .05). Follow-up application of a paired t test analysis showed that the interaction was due to a significant [t(12) = 4.24, p < .01] decrease in response accuracy on face-test NMI trials for the DMC task relative to the DMI task. Application of the same analysis did not show a significant effect for leaf-test trials [t(12) = 1.93, p > .05]. The subjects’ reaction times are not reported here because speed was not stressed and the subjects had to hold their responses until presented with the response cue (1,000 ms after test-stimulus onset). This method has been used widely to eliminate or reduce the impact of motor preparation and its related processes on decision-related ERPs (see, e.g., Curran et al., 2002).

Table 1 Subjects’ response accuracy (mean ± SD) measured during the different experimental conditions
Full size table

As mentioned previously, all brain potentials evoked after the test-stimulus onset were assessed, irrespective of the subjects’ response accuracy (Fig. 3). The most prominent activity modulation was found mainly in the frontal electrode sites (i.e., FP1, FP2, F7, F3, Fz, F4, F8, FC3, FCz, and FC4 electrodes) during the 228- to 328-ms interval after test-stimulus onset (Fig. 4). During this interval, the frontal activities in response to the three trial types diverged systematically from each other. The application of a three-factor repeated measures ANOVA to the averaged frontal activity yielded a significant effect of trial type [F(1.82, 21.84) = 17.43, p < .001] and also a significant interaction between trial type and task [F(1.83, 23.24) = 5.98, p < .01], without any significant effect of the task per se [F(1, 12) = 1.67, p > .05].

In the DMC task (Fig. 4 top), when the identities of the stimuli were irrelevant to the task at hand, the frontal activities during the MI and NMI trials were not differentiable from each other [t(12) = 0.594, p > .05]. During NMC trials, when subjects had to dissociate their responses from those on MI and NMI trials, the evoked frontal activity became significantly more negative than in the MI [t(12) = 3.55, p < .01] and NMI [t(12) = 3.09, p < .01] trials. In contrast to the subjects’ brain activity, no significant difference was found in their response accuracy during NMC trials, as compared with either MI [t(12) = 1.91, p > .05] or NMI [t(12) = 0.66, p > .05] trials. It is important to note that the subjects did not respond before the response-cue onset (i.e., around 700 ms after this interval) and that their motor responses were similar for all trial types (see the Method section).

During the DMI task (Fig. 4 bottom), the subjects had to respond differently to the MI trials than to the other two trial types (i.e., NMI and NMC trials). In other words, any difference between the sample and test stimuli, at either the category level (i.e., face vs. leaf) or the identity level (same identity vs. different identity), was relevant to the task at hand. Correlated to this task demand, frontal brain activity during the NMI [t(12) = 7.51, p < .001] and the NMC [t(12) = 14.34, p < .001] trials was significantly more negative than activity during MI trials, while NMI and NMC trials were not differentiable from each other [t(12) = 1.378, p > .05].

As mentioned previously, during the DMI task the subjects’ response accuracy varied between trial types, but two different analyses showed that this effect could not be responsible for frontal brain modulation. First, while no significant difference appeared between the brain activities evoked during NMI and NMC trials, the subjects’ response accuracy was significantly lower during MI trials [t(12) = 6.50, p < .001]. Second, since variation in the level of response accuracy was confined to the face-test trials, I checked whether this frontal activity modulation was also confined to the face-test trials, or whether it was similarly observable during face-test and leaf-test trials.

Figure 5 shows evoked activity within the same frontal leads during the face-test and leaf-test trials separately. Although a significant effect of test stimulus category occurred [F(1, 12) = 33.46, p < .001], due to an overall activity increase during face-test trials as compared to leaf-test trials, the same pattern of activity variation was observed during both face-test and leaf-test trials. Consistent with this result, the two-way interactions between test stimulus category and either task or trial type and the three-way interaction remained nonsignificant (p > .05). Thus, the variation in response accuracy during face-test NMI trials could not, by itself, be responsible for the frontal activity modulation reported here. This result also showed that activity variation due to changes in the task rules was not confined to the face trials, which differentiates this ERP component from the occipito-temporal N250, which is linked to the face-specific processes responsible for face identification (Nasr & Esteky, 2009; Schweinberger et al., 2004; see also the Discussion section).

Fig. 5
figure 5

Frontal activity during the face-test (left column) and leaf-test (right column) trials when subjects performed the category-matching (top) and identity-matching (bottom) tasks. The red, blue, and green lines illustrate the evoked potentials during the matched-identity (MI), nonmatched-identity (NMI), and nonmatched-category (NMC) trials, respectively

Full size image

Finally, I checked whether the frequency of a required response could change the pattern of brain activity. During the DMC task, 66.6 % of trials required a “matched” response, while only 33.3 % of the trials required a “nonmatched” response. This proportion was reversed during the DMI task. Despite this difference in response frequencies, the trials that required a “matched” response always generated a more positive frontal potential, as compared with trials that required a “nonmatched” response, irrespective of their frequency (Fig. 4). This pattern was similarly observed during face-test and leaf-test trials (Fig. 5). Thus, the pattern shows that response frequency cannot change the pattern of frontal evoked potentials.

In addition to the activity modulations mentioned above, an earlier activity modulation occurred at fronto-central electrode sites 168–216 ms after the test-stimulus onset (Fig. 6), which resembled the previously studied VPP component (Jeffreys, 1989, 1993, 1996; Joyce & Rossion, 2005; Nasr & Esteky, 2009). In contrast to the later frontal activity, which showed clear sensitivity to the interaction between the effects of task and trial type, application of the three-factor repeated measures of ANOVA only yielded a significant effect of the test stimulus category [F(1, 12) = 5.20, p < .05]. This effect was due to more-positive potentials being evoked during the face-test trials than during the leaf-test trials. A statistically nonsignificant tendency was also found (p > .05) for stronger face-related activity if the stimulus was preceded by a leaf (i.e., NMC trials) rather than by another face (i.e., NMI and MI trials). This effect seemed to be consistent with recent findings in studies of N170/VPP categorical adaptation (Kloth, Schweinberger, & Kovacs, 2010; Maurer, Rossion, & McCandliss, 2008), and any weakness of this effect seems to be due to the shorter sample presentation time in our paradigm (100 ms) than in those used for adaptation studies. For this ERP component, the effects of other factors, as well as the interactions between them, remained nonsignificant (p > .05).

Fig. 6
figure 6

Frontal (left) and central (right) activity during face-test (solid lines) and leaf-test (dashed lines) trials during the category-matching (top) and identity-matching (bottom) tasks. The red, blue, and green lines illustrate the evoked potentials during the matched-identity (MI), nonmatched-identity (NMI), and nonmatched-category (NMC) trials, respectively

Full size image

Except for these effects, no systematic modulation was found at other electrode sites (see Fig. 3). Central and parieto-central leads showed weak modulations during later intervals of the DMI task, but not during the DMC trials, and this activity did not resemble either the task rules or subjects’ responses. Although a P300 (P3b) was detected at the centro-parietal electrode sites (P3, Pz, P4, CP3, CPz, and CP4), it did not show any meaningful modulation in response to the experimental parameters. I further checked whether exclusion of incorrect trials would affect the findings, but no prominent or meaningful differences were found.

Finally, trials were also reanalyzed relative to the sample rather than the test stimuli, and the same method was used to explore possible activity modulations from 0 to 600 ms after test-stimulus onset. In this new exploration, Sample-Stimulus Category (face vs. leaf) was used as an independent factor rather than Test-Stimulus Category, in addition to the factors Task (DMI vs. DMC) and Trial Type (MI vs. NMI vs. NMC). Reapplication of the analysis yielded (a) the same effect of trial type and (b) an interaction between the effects of task and trial type on frontal potentials evoked 0–600 ms after test-stimulus onset, as we showed above.

Discussion

In this study, I have shown that part of the ERPs recorded at frontal electrode sites (228–328 ms after test-stimulus onset) were influenced by the changing task rules as subjects were instructed to randomly do DMC and DMI tasks. The results revealed that the amplitudes of these potentials varied systematically with the expected response (i.e., “same” vs. “different”) but independently of the test-stimulus categories (face vs. nonface). In contrast to these potentials, earlier activity was sensitive to the stimulus categories and showed no sensitivity to the task at hand and/or to the expected response. These results show that a change in the task rules, similar to the change in response accuracy, can be used to detect those brain potentials linked to decision-making processes. However, it is also important to note that the timing of the activity modulation reported here (228–328 ms) is based on average brain activity, and that the exact onset time can vary from one individual to another.

Perceptual decision-making and response selection

These results support the general idea that frontal ERP components evoked within this interval (228–328 ms after test-stimulus onset) are under the influence of the task rules and are linked to decision-making processes. However, decision-making may also be divided into different processes, such as stimulus comparison at the perceptual level, task switching, rule execution, and response selection (e.g., Badre & D’Esposito, 2009). Since the amplitudes of frontal ERPs during this interval vary with the expected response, one might suggest that these potentials are linked to those neural processes responsible for response selection. Alternatively, these frontal potentials might be linked to perceptual comparison of the sample and test stimuli. According to this alternative hypothesis, changes in the level of frontal activity during DMC trials are a result of switching the level of comparison between the category and identity levels, which can trigger extra processes in one task (probably, DMI) relative to the other.

If frontal potentials were linked to response selection, the amplitudes of these potentials could be strengthened if they were measured relative to subjects’ responses rather than to response stimulus onset (Makeig et al., 2004), while if frontal potentials were linked to perceptual comparison of the sample and test stimuli, measuring them relative to the subjects’ responses would not necessarily enhance their amplitudes, and might even reduce them. However, in the present study, measuring response-locked brain activity was problematic, since a response cue was used to delay subjects’ responses in order to lower the impact of motor responses on the late evoked potentials (see Curran et al., 2002, for the same approach). Since the main goal of this study was to check the effect of task rules on late ERPs, rather than assessing the neural processes underlying frontal potentials, further studies (with an optimized task) would seem to be necessary to test the nature of the processes underlying late frontal ERPs.

Early onset time of frontal potentials

With regard to the onset time of these frontal potentials, it is important to note that, to do either the DMI or the DMC task, subjects did not need to fully process the test stimulus. Rather, depending on the task at hand, subjects only needed to decide whether the test stimulus differed from the sample at the category or the identity level (in DMC and DMI tasks, respectively), while the identities and/or categories of the stimuli per se were not important. Also, the relatively early onset of these potentials could be partly due to the fact that all of the stimuli were presented from a front view, so the sample–test category or identity difference could easily be detected on the basis of basic features (e.g., from external contours and/or the size of individual features). Consistent with this hypothesis, previous studies of face recognition showed that different brain areas could be responsible for face matching and/or comparison, depending on the level of face similarity (Rotshtein, Henson, Treves, Driver, & Dolan, 2005). For instance, when all faces are presented from a similar viewpoint and are differentiable on the basis of their lower-level features, activity within the occipital face area is enough to achieve the task, but when faces differ from each other configurally, activity within higher-order areas, such as the fusiform face area, seems also to be necessary. Thus it would be quite expected that, when using more complicated stimuli, the onset times of frontal potentials would also be delayed. Similarly, changing the level of complexity or difficulty of tasks can delay frontal potentials (see the Complementary Rather Than Contradictory section below).

Comparison with the occipito-temporal N250

The timing of frontal activity (228–328 ms) reported here suggests an overlap between the onset of this frontal ERP and the onset of another component usually evoked at occipito-temporal sites, known as the N250 (Kaufmann et al., 2009; Nasr & Esteky, 2009; Pickering & Schweinberger, 2003; Schweinberger et al., 2004; Tanaka et al., 2006). Despite this similarity in onset times, it is unlikely that identical neural modules would evoke the occipito-temporal N250 and the frontal activity reported here, for two reasons: First, the frontal component reported in this study showed very similar activity modulations during face and nonface trials (Fig. 5), while the occipito-temporal N250 is a highly face-selective component that is not evoked by nonface objects (Schweinberger et al., 2004). For instance, previous studies have shown that the effect of visual priming on the N250 component is highly limited to faces (Schweinberger et al., 2004).

Second, although using mastoid electrodes can shift temporal activity more frontally (Joyce & Rossion, 2005; see also the Reduced Effect of Priming section below), a previous study has shown that (a) even when linked mastoid electrodes are used, the N250 is still detected in occipito-temporal electrodes and still shows stronger sensitivity to the saliency level of faces than of nonface objects, and (b) in contrast to the N250, the frontal potential recorded here in the same time window showed a comparable response (i.e., sensitivity) to changes in the saliency levels of face and nonface objects (Nasr & Esteky, 2009). Thus, with regard to these differences, it is unlikely that the occipito-temporal N250 and the frontal potentials reported here represent activity within entirely the same neural module.

Complementary rather than contradictory

To find those processes that index decision-making, previous studies have used response accuracy as an independent parameter. Therefore, it was not clear in previous reports whether late potentials index processes responsible for decision-making or only those processes responsible for monitoring task difficulty. Here, by using task and trial type as independent parameters, I detected those potentials that indexed decision-making processes. This result showed that at least part of late evoked potentials is particularly linked to those processes responsible for sample–test comparison (at either the identity or the category level) and/or for response selection, even when response accuracy and/or the subjects’ confidence remained mostly intact between conditions.

The other advantage of this study, as compared to previous ERP studies of decision-making, is that the neural modules underlying decision-making were studied while the level of stimulus salience remained intact between trial types. Although previous studies had usually changed the level of stimulus saliency to control task difficulty (e.g., Philiastides & Sajda, 2006), a recent study by Bankó et al. (2011) showed that this approach can increase the length of sensory neural processes. Consequently, in the previous studies, part of the variation in late ERP components could be due to extended sensory processing for noisier stimuli. In the present study, the stimuli were always presented at the same level of saliency, and therefore extended sensory processing of the test stimuli could not be responsible for the reported effects.

The findings reported here seem complementary to previous findings that task difficulty (Bankó et al., 2011; Nasr & Esteky, 2009; Philiastides & Sajda, 2006), task complexity (Bentin & McCarthy, 1994), and attentional demands (Eimer, 2000; Nasr, 2010) can affect early and late potentials. It is important to note that, although matching to identity may require more attentional resources than does matching to category, this difference does not seem to be responsible for modulation of the late potentials reported here. When subjects cannot predict the trial type before the test-stimulus onset, a different level of complexity and/or attentional demand between the two tasks would be expected to cause a general increase or decrease in the levels of late evoked potentials, irrespective of the trial type. But here, it was found that activity changed only for one particular trial type (i.e., NMI) between the two tasks. The lack of activity modulation during the MI and/or the NMC trial types could not be explained by activity saturation, because I showed that the level of activity differed greatly between the MI and NMC trials. Therefore, modulation could have occurred during at least one of these two trial types. In addition, task difficulty could not be responsible for this effect, because I have shown that no relationship existed between activity modulation and the subjects’ performance. Thus, neither task difficulty differences nor differential attention demands could be responsible for systematic variation in late ERPs.

Reduced effect of priming

I controlled the effect of sensory priming (Desimone, 1996; Henson et al., 2003; Wiggs & Martin, 1998) on the test stimulus by presenting task cues between the sample and test stimuli. Because of this, the sample stimulus was processed independently from the task, and its priming effects on the test stimuli were equal for all experimental conditions. In addition, presentation of an image (i.e., the task cue) between the sample and test stimuli reduced the impact of sample-stimulus encoding on test stimulus perception (Pfütze, Sommer, & Schweinberger, 2002; Schweinberger, Pickering, Burton, & Kaufmann, 2002). Thus, the priming effect on test-stimulus encoding should have been weak (if present at all) and independent of the task. Therefore, it could not be responsible for modulation of the late ERPs (during NMI trials) between the two tasks.

Possible effect of reference electrode selection

Previous studies have shown that using linked mastoid electrodes (as was done here) can change the brain activity distribution relative to when other cephalic reference electrode(s) or a grand average is used as the reference. For instance, face-selective activity evoked around 170 ms after face image onset is better represented by frontal potentials (VPP) rather than by occipito-temporal ones (N170), as linked mastoids are used rather than other cephalic electrodes. However, it is important to note that this change in the location and polarity of the peak signal does not affect the timing and functional properties of these components (e.g., face selectivity) between the two reference-electrode conditions. With regard to the findings here, three points should be noted:

First, the main findings are related to measuring the onset times of brain potential modulations as task rules changed, and not the anatomical location of the underlying modules.

Second, using mastoid electrodes affects the localization of brain potentials mainly when the underlying neural modules are located near the occipito-temporal area (i.e., near the mastoid electrodes; Joyce & Rossion, 2005). According to previous fMRI studies, the neural modules underlying reported frontal activity are most probably located within the frontal lobe (and not the occipito-temporal area). Thus, selection of mastoid electrodes seems to be less problematic here than when visual perception is assessed.

Third, the physical location of peak brain potentials (no matter what reference electrode has been used) does not necessarily correspond to the anatomical location of the underlying neural modules, and to localize the neural modules underlying any brain potentials, other techniques with better spatial resolution (e.g., functional MRI) should be used.

Thus, although a change in the location of the peak signal is possible as other reference electrodes are used, it seems very unlikely that this change would affect any of the main findings here about the onset times of the reported brain modulations or the function of their underlying neural modules (e.g., their correlation with subjects’ responses).

Verbal processes

Finally, verbal processes are unlikely to be responsible for the frontal modulation (228–328 ms) for two reasons. First, previous studies have shown that a conscious comparison of words at the semantic level usually occurs approximately 400 ms after stimulus onset, and not before (Kutas & Van Petten, 1988; Pickering & Schweinberger, 2003; Ruz, Madrid, Lupiáñez, & Tudela, 2003; Zhang, Begleiter, Porjesz, & Litke, 1997). Thus, any verbal comparison of the sample and test stimuli is unlikely to be responsible for the modulation of the frontal potentials (228–328 ms) observed in this study. Second, my face stimuli were selected from a set of 50 unfamiliar computer-generated facial images, which again indicates that the frontal activity modulation could not be due to verbal processes. Thus, the impact of verbal processing on these results was minimal.

Conclusion

In conclusion, these results show that parts of late potentials (from 228 to 328 ms) index those processes responsible for decision-making. Despite this evidence, and mainly due to the poor spatial resolution of ERPs, I could not localize the neural modules indexed by these potentials. Consequently, the second step in this line of study could be to localize those neural modules by taking advantage of other techniques with better spatial resolution (e.g., fMRI). Finally, in this study, the concept of domain generality was tested by using stimuli from different visual categories (faces vs. nonfaces). However, domain generality could be tested further by using stimuli from different sensory modalities (e.g., visual vs. tactile vs. auditory) after the level of task difficulty was adjusted across trial types.