Brain Topography

, 22:145

Steady State Visually Evoked Potential Correlates of Static and Dynamic Emotional Face Processing

Authors

  • A. K. Mayes
    • Brain Sciences InstituteSwinburne University of Technology
  • A. Pipingas
    • Brain Sciences InstituteSwinburne University of Technology
  • R. B. Silberstein
    • Brain Sciences InstituteSwinburne University of Technology
    • Brain Sciences InstituteSwinburne University of Technology
Original Paper

DOI: 10.1007/s10548-009-0106-5

Cite this article as:
Mayes, A.K., Pipingas, A., Silberstein, R.B. et al. Brain Topogr (2009) 22: 145. doi:10.1007/s10548-009-0106-5

Abstract

While the neural regions associated with facial identity recognition are considered to be well defined, the neural correlates of non-moving and moving images of facial emotion processing are less clear. This study examined the brain electrical activity changes in 26 participants (14 males M = 21.64, SD = 3.99; 12 females M = 24.42, SD = 4.36), during a passive face viewing task, a scrambled face task and separate emotion and gender face discrimination tasks. The steady state visual evoked potential (SSVEP) was recorded from 64-electrode sites. Consistent with previous research, face related activity was evidenced at scalp regions over the parieto-temporal region approximately 170 ms after stimulus presentation. Results also identified different SSVEP spatio-temporal changes associated with the processing of static and dynamic facial emotions with respect to gender, with static stimuli predominately associated with an increase in inhibitory processing within the frontal region. Dynamic facial emotions were associated with changes in SSVEP response within the temporal region, which are proposed to index inhibitory processing. It is suggested that static images represent non-canonical stimuli which are processed via different mechanisms to their more ecologically valid dynamic counterparts.

Keywords

Biological motionFace processingSocial perceptionSteady state topographyEmotion

Introduction

The human face conveys a wealth of information about an individual to the viewer relating to both stable characteristics such as identity, age, and gender as well as to more transient aspects such as mood and level of alertness. These information sources comprise central components of social interaction. Disorders of facial identity recognition, such as prosopagnosia, highlight the severity of the social deficits accompanied by the inability to remember familiar faces. Moreover, they suggest the existence of specialised brain circuitry that deals specifically with face processing (e.g. Humphreys et al. 1993). Deficits in social cognition appear to be a core feature in a number of psychiatric disorders including Autism (Baron-Cohen 1995) and Schizophrenia (e.g. Hoekert et al. 2007). In these disorders, problems in social interaction have been related to an inability to correctly infer the mental state and intentions of others—abilities which are at least partly reliant on the ability to correctly “read” faces (Baron-Cohen 1995).

A highly influential model of face processing, proposed by Bruce and Young (1986) and the later elaborated neurocognitive model by Haxby et al. (2000), considers identity and emotional processing to be subserved by parallel streams. According to these models, one neural stream is involved in the initial structural coding of invariant aspects of a face, such as identity and the other is involved in the processing of variant or changeable aspects of a face, such as emotion and eye gaze. With respect to the first neural stream, structural aspects of a face are suggested to be processed within the lateral fusiform gyrus. Evidence of the role of this neural region in the processing of structural aspects of a face stems from an influential paper by Kanwisher et al. (1997) which identified that the fusiform gyrus is primarily activated in response to faces rather than other objects. Further support has been gained through the numerous neuroimaging and event-related potential (ERP) studies investigating face and identity neural processing (Allison et al. 1999; Bentin et al. 1996; Bentin and Deouell 2000). ERP studies have identified a negative wave occurring approximately 170 ms after stimulus onset, commonly known as the N170. This component of the ERP waveform has been suggested to be involved in the processing of structural aspects of the face, whereby a change from a front on view to a profile position does not disrupt the N170 (e.g. Allison et al. 1999), however inverting the entire face results in a delay in the ERP response (Bentin et al. 1996; Eimer 2000).

The second neural processing stream has been suggested to assess variable aspects of the face. This process is considered to take place within the superior temporal sulcus (STS) (Haxby et al. 2000). Further assessment of variable aspects of a face is proposed to occur through projections from the STS to the intraparietal sulcus, involving attention, auditory cortex for speech, and subcortical structures such as the amygdala for emotion processing. The STS is involved in the processing of biological motion, or the ability to perceive actual or implied human body movement (Grossman et al. 2000). The STS has been shown to respond more to faces than to other body parts, highlighting its importance in face processing, especially changeable aspects of people’s faces such as emotions and eye gaze (e.g. Pelphrey et al. 2004).

The existence of these separate, albeit interconnected neural systems is most evident in patients with prosopagnosia and some individuals with brain lesions, whereby the ability to recognise an individual’s face is impaired. However these individuals are able to recognise facial emotions and identify people by moving aspects of their body such as gait (Adolphs et al. 2001; Adolphs et al. 2003; Humphreys et al. 1993). An important lesion study by Adolphs et al. (2003) has also provided an insight into different aspects of emotion processing. The single case study reported on an individual with damage to inferior temporal regions and subcortical limbic regions, however frontal regions, the inferior temporal cortex, MT/V5 and the occipito-parietal regions were spared. Results revealed that whilst this patient was able to process structural aspects of a face through a facial recognition test, this patient had difficulties recognising static emotions, with the exception of happiness. When the patient viewed dynamic images of emotions or was told stories relating to emotions, the individual was able to discriminate and identify emotions with the exception of disgust. This research suggests that static and dynamic emotions may be processed along different neural pathways extending the neurocognitive model of face processing described by Haxby et al. (2000).

Neuroimaging and electrophysiological research into the neural substrates of facial emotion processing has concentrated largely on the ability of individuals to discriminate facial emotions either by valence (Kemp et al. 2002; Turetsky et al. 2007) or by specific facial emotion (e.g. Eimer and Holmes 2007), however the majority of these studies have focussed on photographs or static images of facial emotions. Past behavioural emotion research has suggested that dynamic facial emotions provide the best representation of emotions (Ekman 1972), this has been largely ignored in the facial emotion research area until recently (e.g. Kilts et al. 2003; LaBar et al. 2003; Sato et al. 2004). Considering the current understanding of the role of neural regions such as the STS and its putative involvement in the processing of biological motion, naturalistic moving images of emotion may be imperative in understanding the neural underpinnings of facial emotions. In a positron emission tomography (PET) study, Kilts et al. (2003), found that during emotional processing, static stimuli evoked a greater response than dynamic stimuli in regions of the frontal cortex. This study also suggested that dynamic face stimuli were associated with greater temporal lobe activation than static stimuli. This latter finding is supported by two fMRI studies (LaBar et al. 2003; Sato et al. 2004), however, neither of these studies examined the possibility of greater regional brain activation evoked by static stimuli compared to dynamic stimuli. Therefore there is tentative neuroimaging evidence that the processing of static and dynamic depictions of facial emotion may rely upon dissociable brain systems. Nevertheless, extensive replications are needed to further evaluate and understand the roles these neural regions play in both static and dynamic facial emotion processing.

Event-related potential studies have demonstrated that there are approximate latencies that are associated with different elements of face processing, thus contributing significantly to research in the area of face processing, however these studies have relied on the sole use of static stimuli (Krolak-Salmon et al. 2001). More recently functional magnetic resonance imaging (fMRI) and position emission topography (PET) studies have begun to suggest specific neural regions involved in the processing of dynamic images, however due to the high spatial resolution but poor temporal resolution of fMRI and PET, the temporal enfolding of naturalistic facial emotions in EEG studies is not largely understood. ERPs rely on single point time locked averaging. This technique may not be optimal for analysing dynamic images rather, it may be considered more appropriate to study dynamic facial emotions through a technique that is able to monitor temporally extended cognitive processing (Sato and Yoshikawa 2007). One such technique is steady state topography (SST) (Silberstein 1995). SST assesses the modulation of cognitive processing with respect to a steady state visually evoked potential (SSVEP) that is elicited by a visual flicker. A definite advantage of the SSVEP is its enhanced resilience to artefact, inclusive of eye blink and electromyographic (EMG) noise contamination. This relative insensitivity is due to electro-oculographic and EMG artefact power being distributed over a spectrum of frequencies; however the signal power is restricted to the stimulus frequency (Silberstein 1995; Silberstein et al. 2000). Moreover, SSVEP is considered to have the fast temporal resolution needed to track neural changes whilst viewing a dynamic stimulus set. This has been evidenced by the use of SSVEP in the recoding of brain electrical activity during the viewing of television commercials (Rossiter et al. 2001).

The SSVEP is characterised by both amplitude and phase (latency) information, which is calculated from the SSVEP signal. SSVEP phase changes represent changes in latency. Phase advance (latency decrease) represents faster processing of neural activity elicited by the visual flicker. Phase lag (latency increase), represents slower processing, relative to a reference task (Silberstein et al. 1990). Previous studies have shown temporal changes in SSVEP amplitude and latency over time is associated with cognitive processes.

For example, reduced SSVEP amplitude has been shown to be associated with increases in visual vigilance (Silberstein et al. 1990); attenuation of SSVEP amplitude and increased SSVEP latency in response to the Wisconsin Card Sorting Test (Silberstein et al. 1995); reduction in SSVEP amplitude for perceptual component of a graded working memory tasks and an increase in SSVEP amplitude and reduction in SSVEP latency for the hold component of the working memory task (Silberstein et al. 2001). Changes in 13 Hz SSVEP amplitude may be related to changes in EEG alpha. As reported above, Silberstein et al. (1990) reported reduced SSVEP amplitude on a visual vigilance task, consistent with many studies showing reduced EEG alpha activity during perceptual attention processing (Ray and Cole 1985). EEG alpha has also been shown to increase during a memory task (Klimesch et al. 1999). These findings are consistent with the findings of Silberstein et al. (2001) where an increase in SSVEP amplitude during the hold component of a graded working memory task was reported. Changes in SSVEP latency is suggested to reflect altered information processing conduction, through either neural excitation or inhibition (Silberstein et al. 1998). In a study on the neural processing associated with the valence of emotions, Kemp et al. (2002, 2004), reported changes in SSVEP latency, suggestive of changes in neural information processing speed, within the frontal, parietal and temporal regions in response to positive and negative emotion contexts.

To our knowledge no study has utilised SST methodology to investigate the spatial temporal changes associated with processing of dynamic facial emotions. There is also a general lack of research investigating static and dynamic components of facial emotion processing with electrophysiology methodologies. The present study aimed to investigate two questions, firstly, if electrophysiological correlates of face processing, when employing the use of SST, are consistent with previous electrophysiological static face research. Secondly, if there are differences in neural processing of dynamic and static emotion face processing. SST, like fMRI, employs the use of a subtractive method of analysis, whereby an active condition is compared against a comparable control condition. The active and control condition vary only on the distinct element that is to be investigated. To assess general face processing, neural regions associated with viewing images of faces were compared to scrambled images of faces. In both conditions no overt decision regarding the stimuli was necessary, however participants were instructed to press a button to assist with controlling for button press. Luminance levels were consistent between the tasks. Consistent with previous research, to assess the neural regions associated with emotion face processing an active discrimination task was employed (Adolphs et al. 2001; Ambadar et al. 2005; Johnston et al. 2008; Krolak-Salmon et al. 2001; Lahaie et al. 2006; Miyahara et al. 2007; Pourtois et al. 2004; Schultz and Pilz 2009). The neural regions associated with gender face discrimination task were compared to the emotion face discrimination task. By employing the use of a gender task which requires the individual to make an active decision regarding the image, general aspects of face processing, motor response, attention, and any neural response related to actively engaging in a task were controlled.

Based on previous ERP studies that have identified the role of the occipito-temporal cortex in the processing of structural aspects of a face (Allison et al. 1999) and fMRI research implicating the role of the fusiform gyrus in face processing (Kanwisher et al. 1997) it was hypothesised that differences in SSVEP latency suggestive of inhibitory or excitatory processes would be observed over the parieto-temporal region approximately 170 ms after stimulus presentation for static and dynamic faces compared to scrambled images of faces.

Considering the evidence that static and dynamic emotional face processing rely on somewhat discrete neural systems (e.g. Adolphs et al. 2003; Kilts et al. 2003) it was predicted that when SSVEP changes for dynamic gender discrimination were controlled, specific dynamic facial emotion discrimination SSVEP changes would be evident within the temporal lobe. Furthermore it was predicted that when SSVEP changes for static gender discrimination was controlled, static facial emotion discrimination SSVEP changes would be evident within neural regions consistent with the frontal lobe. Since the existing evidence for differential involvement of these areas in static and dynamic facial emotion processing is derived from lesion and low temporal acuity imaging methods (i.e. PET and fMRI), we do not feel that there is sufficient a priori evidence to make highly specific predictions regarding the relative timing of processes subserved by these areas.

Method

Participants

Fourteen males (M = 21.64, SD = 3.99) and twelve females (M = 24.42, SD = 4.36) right-handed participants completed the study. Participants with a history of seizures or epilepsy were excluded. The Swinburne University Human Research Ethics Subcommittee approved the study. Participants were recruited by word of mouth and posters displayed at Swinburne University of Technology, Hawthorn Campus.

Stimuli

Facial stimuli employed in the current study were from the Nimstim series (Tottenham et al., 2009). Consistent with previous research morphing software was used to create the dynamic stimuli (LaBar et al. 2003). Abrosoft FantaMorph (V.3.0) was used to create the dynamic stimuli from two static images, for instance a neutral pose and a fear pose for the same actor (Fig. 1a, b). Approximately 45 key dots were used to identify corresponding spatial locations across the two images and were placed at key locations on the face, including, for example, the inner and outer canthi of the eyes, the centres of the pupils and multiple locations along the top and bottom of the upper and lower lip, this provided a significant matching procedure between the two faces. Morphed videos were presented at 24 fps, for 1 s. Emotion and gender morphs of varying intensity were also included to prevent ceiling effects. Static snap shots at 33 and 66% for the emotion and gender stimuli were taken to develop the static comparison. A black oval frame was placed over each of the videos to reduce luminance changes due to hair. Static images were taken from beginning and end of the morph video (Fig. 1b). This technique was used for all dynamic and static task sequences. Twelve different actors (6 males, 6 females) were used for the emotion task, and eight different actors were used for the gender task (4 males, 4 females).
https://static-content.springer.com/image/art%3A10.1007%2Fs10548-009-0106-5/MediaObjects/10548_2009_106_Fig1_HTML.gif
Fig. 1

Examples of static and dynamic surprise stimuli within the emotion task block and an example of the scrambled control stimuli. a The static surprise image is displayed for the entirety of the 1-s stimulus presentation period. b A representation of the dynamic image sequence is shown with the image morphing from neutral to surprise over the entirety of the 1 s. c. The static scrambled images are taken from the emotion and gender task, reordered and blurred

Blocks of static and dynamic stimuli were presented to participants randomly over four blocks inclusive of 88 scrambled control stimuli, 88 face control stimuli, 96 emotion stimuli and 104 gender stimuli. The number of stimuli in task blocks differed due to the inclusion of static and dynamic images of varying intensity to prevent ceiling effects. Each image was presented by Eprime Beta (V 2.0) for 1 s followed by a response period of 2 s.

Since SST is a subtractive technique whereby an active condition is contrasted against a control condition, the current task involved two control conditions with the aim of providing baselines for different contrasts. A scrambled faces control task (SCR) was used to provide a baseline condition involving visual stimuli that were well matched for faces in terms of luminance and visual complexity. Static and dynamic scrambled images were created by dividing the original image into 4 by 5 grids and randomly reordering the image (Fig. 1c). Images were blurred to reduce contour effects. This control task provided a baseline for the passive face viewing task (FACE) to explore the spatiotemporal dynamics of processing relating to active discrimination of faces. The FACE task included a random selection of images from the emotion and gender face tasks.

The second baseline task was a gender discrimination task (GEND). This task involved a series of male and female static images and dynamic neutral mouth open, mouth closed images. The comparison task was an emotion discrimination task (EMOT) consisting of both male and female static fear and surprise images and dynamic neutral to fear and surprise images.

Procedure

Brain electrical activity was recorded from a 64-channel electrode cap corresponding to the International 10–20 system positions with additional electrode positions located in between. Linked earlobes were used as a reference for each of the 64-electrodes and the nose served as ground. Consistent with previous research (Ellis et al. 2006; Silberstein et al. 1995) participants were fitted with half mirrored goggles which presented a 13 Hz white flicker over the visual field to elicit the SSVEP. The visual flicker subtended a horizontal angle of 160° with a vertical angle of 90° and when viewed against a background had a modulation depth of 45%. Brain electrical activity was band pass filtered from 0.74 to 74 Hz and digitized at a rate of 500 Hz with a 16-bit accuracy (i.e. Silberstein et al. 2001; Silberstein et al. 2004).

Participants were seated approximately 1.5 m away from a 16-inch computer screen and were asked to complete the SCR task, the FACE task, the GEND task and the EMOT task. For each task, participants were to view the displayed image and were required to wait for a white screen displaying a “?” indicative of the 2-s response period, before responding to the image. Participants were required to respond by a button press, according to the specific instructions for that particular randomly assigned task. During both the SCR and FACE tasks, participants were asked to first only respond with one hand for half of the stimuli set and then the other hand for the remainder of the stimulus set. During the EMOT task participants were asked to indicate by means of a button press after the 1-s stimulus presentation period if they thought the image presented displayed fear or surprise. During the GEND task participants were asked to indicate by way of a button press after the 1-s stimulus presentation period; whether they thought the individual was male or female. The presentation of stimuli was randomised and the order of presentation of tasks and button press was counterbalanced throughout the tasks to prevent order effects.

Steady State Topography (SST) Signal Processing and Artefact Detection

Extensive SST details have been described previously (Silberstein et al. 1990, 1995), therefore for the purposes of concision only the procedures used in the current project’s analysis will be outlined. The 13 Hz SSVEP sine and cosine Fourier Coefficients (FC) were analysed for each stimulus cycle to produce a time series of FC, a process that was conducted for each electrode site and for each task block. To increase the signal to noise ratio, FC were smoothed using an overlapping cosine window, with a width of 8 stimulus cycles (8/13 s = 615 ms). This equated to an equivalent temporal resolution of approximately 308 ms (N.B. while the width of the cosine window is 615 ms, the equivalent temporal resolution due to the cosine window is half of this period, that is, 308 ms). Epochs of FC were then averaged for each of the conditions being investigated, centred on stimulus presentation and including data 1 s prior to stimulus presentation, 1 s during stimulus presentation and 2 s following stimulus presentation. The averaged FCs were used to calculate amplitude and latency measures. Data containing excessive artefact was replaced with the weighted average of the data recorded from the four adjacent electrodes. Prior to group averaging, normalisation of the SSVEP amplitude for each individual was conducted to account for large inter-subject variations in the SSVEP amplitude that can occur (Silberstein et al. 1990, 1995). Each of the individual’s epochs were then averaged together to form group averages for the both static and dynamic conditions for the SCR task, the FACE task, the EMOT task and the GEND task.

Specific regions of interest (frontal and temporal regions) were chosen a priori. Time series for visual inspection of SSVEP latency and amplitude responses for the SCR task with respect to the static and dynamic FACE task were assessed for electrode F4 and T6 (Fig. 2) and then represented in subsequent topographical maps (Fig. 3).
https://static-content.springer.com/image/art%3A10.1007%2Fs10548-009-0106-5/MediaObjects/10548_2009_106_Fig2_HTML.gif
Fig. 2

Group average SSVEP amplitude (a) and latency (b) time series for electrodes F4 and T6. Blue line indicates SSVEP amplitude and latency variations to SCR task images, pink line indicates SSVEP amplitude and latency variations to static FACE task images and the yellow line represents SSVEP amplitude and latency variations in the dynamic FACE task. The area between the two vertical lines represents the time during which the stimulus was presented

https://static-content.springer.com/image/art%3A10.1007%2Fs10548-009-0106-5/MediaObjects/10548_2009_106_Fig3_HTML.gif
Fig. 3

Topographic maps illustrating static (a) and dynamic (b) face processing SSVEP changes, together with the Hotelling’s T statistic for the time windows centred on 170, 400, and 700 ms after stimulus presentation of the FACE task, relative to the SCR task

Mapping and Statistical Consideration

Topographic maps were produced by subtracting the SSVEP epochs for the static and dynamic FACE task from the mean of the SCR task to assess the neural activity associated with static and dynamic face processing. Secondly, SSVEP amplitude and latency epoch data for both static and dynamic GEND discrimination task was subtracted from the corresponding static and dynamic EMOT task SSVEP amplitude and latency epoch data, yielding the neural response to emotional face processing.

Specific time windows centred on previous ERP time points of interest (170, 400 and 700 ms) were selected a-priori based on previous ERP face processing research (Johnston et al. 2005; Krolak-Salmon et al. 2001). A time window refers to a cosine averaged time period equivalent to 308 ms, or ±154 ms. Changes in phase (radians) and changes in latency (ms) may be used interchangeably, since they are directly related, however for the purposes for the topographical difference maps produced latency measures (ms) will be reported (change in phase/2 × π) × (1000/13) (Gray et al. 2003).

A multivariate statistic utilising the Hotelling’s T2 score was used to illustrate the topographical statistical significance between FACE and SCR conditions and the EMOT and GEND conditions. The square root of the Hotelling’s T2 was mapped. On the basis of 64-electrodes, there is a greater probability of observing a statistically significant event due to chance. To correct for these multiple comparisons Bonferroni correction would normally be applied (.05/64), however this conservative correction has previously been considered to be too rigorous due to the spatial dimensionality of the data not corresponding to 64, rather 5 factors, which reportedly account for 95% of the variance over the 64 electrodes (Silberstein and Cadusch 1992; Silberstein et al. 1995). For a single set of comparisons based on 64 channel scalp recording, a Bonferroni adjusted P value of 0.05/5 (1%) was applied, based on the five factor model proposed by Silberstein and Cadusch (1992).

Based on previous research, three time windows of analysis were chosen, 170, 400 and 700 ms after stimulus presentation (Johnston et al. 2005; Krolak-Salmon et al. 2001). If we consider that for each condition there are three time points that were chosen a-priori, Hotelling’s T maps presented in Results do not require further Bonferroni correction. Therefore a significance threshold of P < .01 will be utilised to determine statistical significance across all tasks. Contours correspond to Hotelling’s T values that represent P values 0.05, 0.01 and 0.001 for N = 26.

Results

Behavioural Results

To investigate response accuracy for the static and dynamic (motion) emotion and gender (face) discrimination tasks, percentage of correct response was calculated. A 2(motion) by 2(face) repeated measures ANOVA was conducted. Means and standard deviations are presented in Table 1.
Table 1

Means and standard deviations of the percentage of correct responses for static and dynamic emotion and gender stimuli

Stimuli

Mean

Standard deviation

Emotion static

74.28

6.74

Emotion dynamic

66.83

8.06

Gender static

83.56

12.27

Gender dynamic

80.70

10.21

N = 26

Results revealed a significant main effect of face, with individuals responding more accurately to static and dynamic gender stimuli than the static and dynamic emotion stimuli, as shown in Table 1 (F(1,25) = 21.58, P < .001). However the interaction between motion and face was not significant (F(1,25) = 3.01, P = .095).

SSVEP Time Series: Static and Dynamic Faces Processing

Visual inspection of the 64-electrode time series’ for the FACE task relative to the SCR task, revealed a predominately SSVEP latency driven changes at specific electrode sites. Investigation of electrodes F4 and T6 revealed predominately SSVEP latency driven difference between static and dynamic FACE task and the SCR task over both electrodes, whilst little difference in SSVEP amplitude can be seen (Fig. 2).

Time series data allows assessment of each electrode individually over time. To investigate variations in SSVEP topography (all electrode sites simultaneously), topographic difference maps of SSVEP amplitude and latency were produced for every time point, however specific time windows centred on 170, 400 and 700 ms post-stimulus onset are displayed (Fig. 3).

SSVEP Topographic Maps of Passive Face Processing, Time Windows Centred on 170, 400 and 700 ms Post-Stimulus Onset

A series of topographical difference maps were produced to assess the neural correlates of face processing. The SSVEP amplitude and latency for both static and dynamic FACE task was compared to the mean SSVEP of the SCR task (Fig. 3). This comparison provides information regarding spatiotemporal variations in brain activity during the passive viewing of facial images (i.e. with no explicit decision making). Consistent with previous research (Johnston et al. 2005; Krolak-Salmon et al. 2001) time windows centred on time points 170, 400 and 700 ms post-stimulus onset were selected and displayed in Fig. 3, to assess the SSVEP amplitude and latency differences for the static (Fig. 3a) and dynamic (Fig. 3b) FACE task when the SCR task was controlled. Warmer colours indicate reduced SSVEP amplitude and latency for the FACE task compared to the SCR task. Conversely, cooler colours represent increased SSVEP amplitude and latency for the FACE task compared to the SCR task.

Figure 2 shows that for the three time windows presented, both the static and dynamic FACE task evoke similar SSVEP amplitude and latency responses, with small SSVEP amplitude differences at central, temporal and parietal regions. However, consistent with the time series shown above (Fig. 2), the SSVEP differences for both the static and dynamic FACE task images compared to the SCR task images was predominately driven by changes in SSVEP latency (Fig. 3). Results for static and dynamic topographical maps will be discussed separately.

Figure 3a indicates that the static FACE task compared to the SCR task images produced a reduction in SSVEP latency for electrodes sites corresponding to temporal and parietal regions, at the time window centred on 170 ms after stimulus presentation. Furthermore at 400 ms time window post-stimulus onset, a reduced left hemisphere SSVEP latency and an increase in right hemisphere SSVEP latency was evident at electrodes corresponding to the temporal and parietal regions and the 700 ms time window after stimulus presentation an increase in SSVEP latency was evident over electrodes consistent with the right temporal region and a larger left parietal SSVEP latency was also evident for the static FACE task images compared to the SCR task images. As shown in Fig. 3a, Hotelling’s T maps showed that SSVEP differences between the static FACE task and the SCR task images reported were statistically significant at electrodes corresponding to the frontal, temporal and parietal regions at the above stated time windows.

Topographic difference maps for dynamic FACE task stimuli with respect to SCR task images are presented in Fig. 3b. A reduced SSVEP latency for electrodes corresponding to bilateral temporal and parietal regions was evident for the dynamic FACE task compared to the SCR task images at the time window centred on 170 ms after stimulus presentation. At the time window centred on 400 ms after stimulus presentation, a reduced SSVEP latency for the dynamic FACE task compared to the SCR task images was evident for electrodes corresponding to the left temporal and parietal regions, extending to the electrodes corresponding to the frontal region. A reduced SSVEP latency was also evident at the time window centred on 700 ms after stimulus presentation in bilateral frontal electrodes and an increased SSVEP latency at the time window centred on 700 ms after stimulus presentation was also evident for the FACE task compared to the SCR task images for electrodes consistent with the left temporal and parietal region. As shown in Fig. 3b, Hotelling’s T maps indicate that SSVEP differences between the dynamic FACE task and the SCR task images were statistically significant within the frontal, temporal and parietal regions for the above stated time windows.

SSVEP Topographical Maps of Emotion Face Processing Compared to Gender face Processing, time Windows Centred on 170, 400, 700 ms Post-Stimulus Onset

The second difference analysis conducted assessed the neural correlates of emotion face processing for time windows centred on 170, 400 and 700 ms post-stimulus onset. The SSVEP amplitude and latency for the static and dynamic GEND discrimination task was subtracted from the SSVEP amplitude and latency response for the static and dynamic EMOT face discrimination task for the relevant time points (Fig. 4).
https://static-content.springer.com/image/art%3A10.1007%2Fs10548-009-0106-5/MediaObjects/10548_2009_106_Fig4_HTML.gif
Fig. 4

Topographic maps illustrating static (a) and dynamic (b) face processing SSVEP changes, together with the Hotelling’s T statistic of those changes for the time windows centred on 170, 400, and 700 ms after stimulus presentation of EMOT discrimination task, relative to GEND discrimination task

Topographical maps of differences between static and dynamic EMOT discrimination tasks in respect to GEND discrimination tasks are shown in Fig. 4a (static) and b (dynamic). Overall topographical maps suggest that changes at electrode sites associated with the investigation of the static and dynamic EMOT task compared to the static and dynamic GEND task appear to be predominately driven by latency changes, this is investigated for static and dynamic comparisons independently.

Figure 4a indicates that the differences in the neural processing of the static EMOT task with respect to static GEND discrimination task appear to be driven predominately by latency changes, with very little amplitude changes evident. At the time window centred on 400 ms post-stimulus onset, an increased SSVEP latency for the static EMOT task relative to the static GEND task for electrodes corresponding to the frontal regions was evident. At the time window centred on 700 ms after stimulus presentation time point, an increased SSVEP latency was evident for electrodes located more centrally, including the frontal region and a reduced SSVEP latency was evident at temporal electrodes for the static EMOT task relative to the static GEND task.

Hotelling’s T maps indicate that the frontal differences at the time windows centred on both 400 and 700 ms for the static comparison were statistically significant.

In contrast, topographical and Hotelling’s T maps displayed in Fig. 4b corresponding to dynamic images for the time windows chosen (170, 400 and 700 ms), revealed no consistent SSVEP amplitude or latency differences between the EMOT and GEND processing tasks, across participants.

As the chosen time windows were based on prior studies employing the use of static images only, the extended processing time associated with dynamic images may not be considered within these time windows. With this in mind, an additional time window centred on 900 ms after stimulus presentation was investigated for both the static and dynamic comparisons of EMOT and GEND tasks (Fig. 5). Results revealed a no consistent SSVEP amplitude or latency changes for the static comparison (Fig. 5a), however a reduction in SSVEP latency for the dynamic EMOT task compared to the dynamic GEND task was evident. This difference was localised to electrodes corresponding to the right temporal lobe.
https://static-content.springer.com/image/art%3A10.1007%2Fs10548-009-0106-5/MediaObjects/10548_2009_106_Fig5_HTML.gif
Fig. 5

Topographic maps illustrating static (a) and dynamic (b) face processing SSVEP changes, together with the Hotelling’s T statistic of those changes for the time window centred on 900 ms after stimulus presentation of the EMOT discrimination task, relative to GEND discrimination task

Figure 5b’s Hotelling’s T map indicates that the temporal differences at the time window centred on 900 ms for the dynamic comparison were statistically significant.

Discussion

The aims of the present study were twofold: to identify neural areas with the use of SST associated with the perception of faces compared to non-faces, and to assess the neural correlates associated with static and dynamic perception of facial emotion. Behavioural data in the current study showed that participants were more accurate in gender discrimination than in emotion discrimination. In some circumstances in neuroimaging research this differential in task difficulty might be considered sub-optimal for allowing robust comparisons. However, in the current study this difference in behavioural performance does not substantially impinge upon our ability to make meaningful interpretation of the results for the following reasons. The primary focus of interest of the study is not upon the differences between emotion processing and gender processing, but upon the differences between the processing of static and dynamic emotion stimuli. Since the increased accuracy in gender discrimination over emotion discrimination was stable across static and dynamic stimuli (i.e. there was no interaction), this overall differential in task difficulty should not impact substantially upon our results. Previous electrophysiological research has identified the right posterior temporal region to be associated with coding of structural components of a face, approximately 170 ms after stimulus presentation (Allison et al. 1999; Bentin et al. 1996; Eimer 2000). Neuroimaging research has also confirmed the involvement of this region in face perception (Haxby et al. 2000). Consistent with current predictions and previous research regarding face processing within the temporal region, results revealed a significant bilateral change in SSVEP latency for electrodes consistent with the temporal and parietal regions for faces. Based on the interpretation of Silberstein, et al. (2000) this may indicate the involvement of increased inhibitory processes. This finding was consistent for both static and dynamic displays, which further suggests that faces are initially processed for structural information, whereby viewpoint or dynamic components do not interfere with facial processing (Bruce and Young 1986). Similarly, this result is also consistent with fMRI studies that have recognised the involvement of the temporal region inclusive of the fusiform gyrus in the processing of faces for invariant information (Review see: Haxby et al. 2000). Furthermore, face processing occurred implicitly, as participants were not told to identify any component of the image, suggesting that face processing does not require attentional modulation and occurs as an automatic process. This provides further support that SST is a useful measure for studying the neural substrates of face processing, revealing activation patterns that are generally consistent with previous studies employing the use of other electrophysiological and neuroimaging techniques (e.g. Wheaton et al. 2004).

Emotion Processing of Static and Dynamic Facial Images

Consistent with the predictions of the current study, SSVEP changes were different for both the static and dynamic comparisons of the emotion task with respect to the gender task. Interestingly the results revealed a different time course of neural processing of static and dynamic emotion processing. Each will be discussed separately.

SST phase or latency is suggested to represent the degree of cortical connections (Silberstein et al. 2001). Latency increases are suggested to be indicative of inefficient processing due to an increase in inhibition within cortico-cortico loops or in other words, a slowing down of processing within these regions, to account for the inhibition of adjacent neurons (Ellis et al. 2006; Gray et al. 2003; Kemp et al. 2004). On the other hand latency decreases are suggested to be indicative of efficient processing due to the excitation or a reduction of inhibition. With this in mind, the results from the current study revealed a latency increase—or inefficient processing within the frontal region at the time window centred on 400 ms after stimulus presentation for the static emotion and gender comparison. Conversely the later temporal response for this comparison (time window centred on 700 ms) is suggestive of increased excitatory processes. These results are partially consistent with Kemp et al. (2004) who identified significant excitatory processes within the temporal region for emotional contexts in females.

Inconsistent with previous emotion face research, for the dynamic comparison there were no significant SSVEP changes for the emotion task with respect to the gender task within the first time window centred on 700 ms after stimulus response. Considering in the past, EEG research has primarily employed only static images of emotions, it may not be surprising that differences have been identified. However, further investigation of the SSVEP data was considered necessary due to the dynamic image taking the entirety of the 1 s presentation time to reveal the emotion, compared to the static stimuli, where the static emotion or gender image was presented for the entirety of the 1 s. Results revealed a right temporal decrease predominately driven by changes in latency for the dynamic emotion condition when dynamic gender was controlled. This decrease in latency indicates efficient processing within this region. A change SSVEP response over the temporal region was also evident for the static comparison; however this result was evident 200 ms earlier in the static comparison than the dynamic comparison. These results are consistent with the concept that the temporal region is involved in the processing of movement in humans, including facial emotions. Furthermore these results illustrate that the role of the temporal region in processing human motion is enhanced by dynamic social stimuli, such as naturalistic facial emotions, rather than more general aspects of biological motion.

In general, whilst the specific underlying cortical regions cannot be identified due to the reduction in spatial resolution in EEG recordings, the broad regions identified for both the static and dynamic comparisons are consistent with previous research. For example, Kilts et al. (2003) identified that static emotion images were processed within the frontal regions, specifically the premotor regions, however dynamic images were primarily processed within the temporal regions, specifically the STS. The recent discovery of mirror neurons within monkeys and later suggested in humans within the frontal regions provides some evidence for the processing of static images within the frontal regions (e.g. Di Pellegrino et al. 1992; Iacoboni et al. 2005). Within this theory, facial emotions are thought to provide an internal simulation of the facial emotion within the premotor region of the cortex, whereby the same neurons used to view the facial emotion of another individual are active when the individual conducts that facial emotion themselves, thereby providing a greater understanding of that emotion (e.g. Iacoboni 2009). Whilst it is suggested that both static and dynamic facial emotions elicit this frontal response (Iacoboni 2009), the decrease in information processing speed within the frontal region for static emotions, identified in the current study is consistent with the concept that greater cortical resources are required to elicit an internal representation of the static facial emotion, as apposed to when viewing dynamic images of emotion. However, further investigation into the role the frontal region plays in static and dynamic emotional face processing is required.

Significance and Limitations of the Study

The current data provides evidence of the distributed neural system involved in the perception of invariant and variant aspects of a face. It also suggests that variant aspects of a face may be further divided into two distributed neural systems of static and dynamic perception of social judgements. This division of neural correlates of the processing of variant aspects of the face in terms of static and dynamic stimuli, has important implications for the interpretation of previous studies that have employed the use of static images for emotion discrimination tasks (Allison et al. 1999; Bentin et al. 1996; Blau et al. 2007; Narumoto et al. 2001). Studies that have assessed neural deficits associated with the processing of static emotion faces in disorders of affect such as Autism and Schizophrenia, may be underestimating these individuals ability to process emotions (e.g. Baron-Cohen 1995; Hoekert et al. 2007). As evidence from patients with brain lesions (Aldophs et al. 2003) and prosopagnosia (Humphreys et al. 1993) has shown these individuals are able to identify others by their gait and also discriminate emotions with the aid of dynamic images, further investigation as to whether this is the case in disorders of affect such as Autism should be conducted.

Like all studies the current study is not without its own limitations. The data must be taken in consideration that the topographical findings are suggestive of cortical regions not specific locations associated with the particular task. Electrophysiological studies are limited in spatial resolution by the summation of several neural populations; therefore precise locations are not suggested. Notwithstanding, the current data provides important time course evidence that is consistent with previous accounts of invariant coding of faces associated with the N170 and gross neural areas consistent with fMRI studies of static and dynamic images (Allison et al. 1999; Bentin et al. 1996; Eimer and Holmes 2007; Haxby et al. 2000).

Neurophysiological significance of changes in SSVEP latency is still speculative. However, previous studies, utilizing spatial working memory tasks and emotional valence tasks, have consistently suggested that changes in latency is a function of changes in processing speed between distributed neural networks (Macpherson et al. 2009; Silberstein and Cadusch 1992; Silberstein et al. 2000). The current study is consistent with these predictions in terms of changes in processing speed for static and dynamic emotion faces within the frontal and temporal region respectively, after the neural processing associated with static and dynamic gender images has been subtracted.

A potential shortcoming of the current study centres on the issue of the inclusion of a explicit discrimination task. The inclusion of an explicit discrimination task is common practice in the neuroimaging literature in relation to facial emotion processing (Adolphs et al. 2001; Ambadar et al. 2005; Johnston et al. 2008; Krolak-Salmon et al. 2001; Lahaie et al. 2006; Miyahara et al. 2007; Pourtois et al. 2004; Schultz and Pilz 2009), since it is important to ensure that the participants remain focussed on the stimuli, and that attentional resources are similarly allocated across active and baseline conditions. However, it is entirely plausible that explicit facial emotion discrimination tasks do not invoke the recruitment of an identical set of structures to those excited by the spontaneous appearance of emotional face stimuli in the visual field in the absence of explicit task demands. Therefore the current results may only be generalised to explicit emotional face processing, as different spatio-temporal changes may be involved in implicit emotional face processing. A future study may assess both static and dynamic implicit versus explicit emotional face processing.

In conclusion the current study identified that static facial emotion discrimination tends to be processed with an increase in recruitment of neural resources within the frontal region and a reduction in neural processing within in the temporal region; however a late temporal lobe reduction in neural processing was found in dynamic images. The findings from this study suggest that when assessing emotional face processing, especially in individuals with disorders of affect, such as Autism, studies should consider whether individuals are deficit on the ability to perceive either or both static and dynamic facial emotions (e.g. Celani et al. 1999). Furthermore static stimuli which are suggested to be processed non-canonically should be compared to more ecologically valid dynamic facial emotions.

Copyright information

© Springer Science+Business Media, LLC 2009