Advertisement

Culture and Brain

, Volume 2, Issue 1, pp 27–51 | Cite as

Neural correlates of emotion perception depending on culture and gaze direction

  • Katharina KrämerEmail author
  • Gary Bente
  • Bojana Kuzmanovic
  • Iva Barisic
  • Ulrich J. Pfeiffer
  • Alexandra L. Georgescu
  • Kai Vogeley
Original Research

Abstract

A person’s cultural background as well as their gaze direction have been identified as relevant factors that influence the behavioural and neural processing of emotional expressions. However, research on their combined influence is sparse. Here, we manipulated the cultural background and gaze direction of emotion-encoders to investigate the interaction of both factors during the neural processing of emotions. Stimuli consisted of short video sequences showing faces that displayed either direct or averted gaze, expressed either anger or happiness, and represented either cultural in-group (European faces) or cultural out-group members (Asian faces). While undergoing functional magnetic resonance imaging, a group of German participants rated the stimuli with respect to their valence. Results revealed that when anger was expressed with direct gaze, more activation was found in the dorsomedial and dorsolateral prefrontal cortices in response to cultural out-group compared to in-group members. However, when anger was expressed with averted gaze, activity increased in the amygdala and the striatum in response to cultural in-group compared to out-group members. With respect to happiness, enhanced neural activation in medial and lateral prefrontal cortical areas was associated with the processing of cultural in-group compared to out-group members expressing happiness with direct gaze. These findings indicate a complex interplay between culture, gaze direction and the valence of emotions.

Keywords

Culture Gaze direction Emotion fMRI 

Introduction

Emotional facial expressions provide a strong communicative and social function in human interactions (Bavelas et al. 1986; Frith 2009). Over the last decade, a wealth of neuroimaging studies have investigated the neural correlates of emotion perception (Barrett et al. 2007; Murphy et al. 2003; Phan et al. 2002). A recent meta-analysis by Lindquist et al. (2012) argues that a network of brain regions (hereinafter referred to as the emotion perception network: EPN) seems to be consistently active during the perception and interpretation of a variety of emotional expressions. Central components of this network are the amygdala, which has repeatedly been shown to respond to perceptual stimuli that are novel or salient to participants (Adolphs 2009; Herry et al. 2007), and the anterior insula, which is involved in the processing of bodily sensations and affective states (Craig 2002, 2009; Critchley et al. 2004). During emotion perception tasks, neural activation is also found in the dorsomedial prefrontal cortex (dmPFC), the ventromedial prefrontal cortex (vmPFC), and the posterior cingulate cortex (PCC). These regions are typically associated with high-level cognitive reasoning (Mitchell 2009; Schacter et al. 2007; Vincent et al. 2006) and are thought to be involved in using stored representations of prior experiences to interpret perceived emotional expressions (Lindquist et al. 2012). As facial expressions are often used as stimuli in emotion research, the visual cortex (Kober et al. 2008) and the so-called face processing network are also active during the perception and decoding of emotions (Johnson 2005; Wieser and Brosch 2012). The latter network substantially overlaps with the EPN and additionally includes the inferior frontal gyrus (IFG), the dorsolateral prefrontal cortex (dlPFC), the thalamus, the fusiform gyrus and the superior temporal sulcus (STS).

The way we perceive and process emotions depends on the situational context in which they are expressed. This situational context is constituted by a variety of different factors including characteristics of the emotion-encoder (e.g. gaze direction and cultural background), the emotion-decoder, and the physical environment (Wieser and Brosch 2012). According to Wieser and Brosch (2012), these factors are interdependent and should therefore be investigated in a combined manner in order to enhance our understanding of emotion processing. Against this background, we decided to focus on the investigation of two contextual factors known to influence emotion perception, namely the cultural background and gaze direction of an emotion-encoder, which can be both presented nonverbally in a combined stimulus set. Although emotions can be accurately and consistently decoded across different cultures (Ekman and Friesen 1971; Mesquita and Frijda 1992), there is also a lot of variation related to both encoding and decoding of emotions. For example, research has shown that nonverbal dialects and accents in facial expressions of emotions exist (Elfenbein 2013; Jack et al. 2012; Marsh et al. 2003) and that people are better at decoding emotions expressed by members of their cultural in-group as compared to members of their cultural out-group (Elfenbein and Ambady 2002; Marsh et al. 2003). Moreover, the perception of intentions conveyed by emotional expressions (Hess et al. 2000) and the valence appraisal of emotions (Krämer et al. 2013) were shown to be influenced by the cultural background of the emotion-encoder and the emotion-decoder. Over the past years, behavioural evidence for cultural influences on emotion perception has been complemented by a variety of findings from studies investigating how culture affects the neural substrates of emotion processing (for an overview see Ambady and Bharucha 2009; Han and Northoff 2008; Han et al. 2013; Kitayama and Uskul 2011; Rule et al. 2013). For instance, Chiao et al. (2008) showed Japanese and Caucasian participants pictures of emotional facial expressions of Japanese and Caucasians. While undergoing fMRI, participants were asked to categorise these emotional expressions. Results revealed greater activation in the amygdala in response to viewing same-culture compared to other-culture fear expressions for both cultural groups. The authors argued that fear displayed by cultural in-group members was interpreted as a more salient signal of potential danger in comparison to fear displayed by cultural out-group members.

An additional contextual factor which influences emotion perception is gaze direction (Wieser and Brosch 2012). For example, direct gaze is an important social cue, as it indicates social engagement between the emotion-encoder and the emotion-decoder. Schilbach et al. (2006) demonstrated that participants felt more engaged with a virtual agent that displayed direct gaze compared to averted gaze. This correlated with greater activation of the dmPFC for direct compared to averted gaze, which has been interpreted as evidence for the involvement of this region in the detection of self-relevance. Moreover, research with Caucasian (Adams et al. 2003; Hadjikhani et al. 2008) as well as Asian participants (Sato et al. 2004) demonstrated that gaze direction influences the processing of emotional facial expressions on a neural level as well. The amygdala showed more activation in response to anger expressed towards the emotion-decoder as compared to anger expressed away from the emotion-decoder (Hadjikhani et al. 2008; Sato et al. 2004). In addition, increased activity in response to fear expressed with direct compared to averted gaze was found in areas related to gaze processing (STS, intraparietal sulcus), face processing (fusiform gyrus, STS), fear processing (amygdala, hypothalamus), and motor preparation (premotor and motor cortices, superior parietal lobule) (Hadjikhani et al. 2008). In sum, these results indicate that a distributed neural network integrates information provided by emotional expressions and gaze direction in order to compute behavioural implications for the emotion-decoder and adjust her/his behaviour accordingly.

While a number of studies investigated the influences of culture and gaze direction on emotion perception separately, research on their combined influence remains limited. We recently investigated the interaction of both factors in a behavioural study (Krämer et al. 2013). In this study, the emotion-encoder either appeared Asian or European in order to manipulate participants’ sense of belonging to the cultural in- or out-group. While watching cultural in-group and out-group members expressing happy and angry emotions with direct and averted gaze, Chinese and German participants assessed the valence of the emotional expressions. Results indicated that the perceived valence of positive and negative emotions expressed by cultural in-group members was independent of gaze direction. Interestingly, gaze direction had an influence on the valence perception of emotions expressed by cultural out-group members. In the latter case, participants from both cultural groups perceived happiness more positively and anger more negatively when cultural out-group members expressed it with direct compared to averted gaze. Due to the fact, however, that our study only investigated emotion perception at a behavioural level, no conclusions can be drawn for the neural correlates of emotion processing depending on culture and gaze direction. Richeson et al. (2008) investigated how gaze direction modulates the neural correlates of culture-related face processing. European-American participants viewed European-American and African-American faces showing direct and averted gaze. Increased neural activation was found in the amygdala, the insula, the hippocampus, and the middle frontal gyrus in response to African-American faces compared to European-American faces displaying direct gaze compared to averted gaze. However, this study used neutral face stimuli and thus did not target the investigation of emotion perception.

To the best of our knowledge, there is only one neuroimaging study which directly assessed the combined influence of culture and gaze direction on emotion processing (Adams et al. 2010). In this study, American and Japanese participants viewed same-culture and other-culture rapid presentations of fear expressions shown with direct or averted gaze, while undergoing fMRI. Results showed differential activation in brain regions known to be involved in emotion and gaze processing in both American and Japanese participants: Fear expressed with direct gaze evoked greater responses for other-culture faces, whereas fear expressed with averted gaze evoked greater responses for same-culture faces. However, the study of Adams et al. (2010) did not consider that depending on the valence of an emotion, culture and gaze direction can exert a differential influence on emotion perception (Krämer et al. 2013). Based on this, one could expect that also the influence of culture and gaze direction on the neural correlates of emotion processing might vary depending on the emotions’ valence. Due to the fact that Adams et al. (2010) investigated the impact of culture and gaze direction only on one emotion that contains a negative valence (fear), no conclusions can be drawn for the influence of culture and gaze direction on the neural correlates of the perception of other emotions with different valences. Also, Adams et al. (2010) used static stimulus material which was shown to participants for only a short amount of time. Previous studies, however, indicated that dynamic nonverbal cues are ecologically more valid and potentially more informative than static ones (Georgescu et al. 2013; Kuzmanovic et al. 2009; Schilbach et al. 2006; Wieser and Brosch 2012). In addition, compared to static emotional facial stimuli, dynamic facial stimuli offer an advantage in the intensity evaluation and recognition of emotions (Biele and Grabowska 2006; Recio et al. 2011) and increase participants’ facial reactions (measured with electromyography) to emotional expressions (Rymarczyk et al. 2011; Weyers et al. 2006). Furthermore, fMRI findings indicated enhanced EPN activation in response to dynamic compared to static facial expressions: Dynamic expressions of fear, disgust, and happiness increased neural activation in the EPN which was related to emotional, perceptual and motor processing of facial expressions (Sato et al. 2004; Trautmann et al. 2009).

The aim of the present study was to extend prior research by using an experimental paradigm that would allow investigating the influence of culture and gaze direction on the neural processing of emotions with different valences. In this regard, we (i) used dynamic stimulus material, (ii) varied participants’ perception of belonging either to the cultural in- or out-group of emotion-encoders, (iii) systematically manipulated emotion-encoders’ gaze direction, and (iv) included two distinct emotions differing in valence (Frijda 1987): Participants watched short video sequences showing facial expressions of happiness and anger, displayed by cultural in-group (Europeans) and out-group (Asians) members, and expressed with direct and averted gaze (see Fig. 1a), while undergoing fMRI. After each video sequence participants had to assess the perceived valence of the observed facial expressions (see Fig. 1c). We expected EPN activation to depend on both the cultural background as well as the gaze direction of the emotion-encoder (Chiao et al. 2008; Schilbach et al. 2006). In addition, the influence of culture and gaze direction was expected to differ for emotions with positive and negative valence (Krämer et al. 2013). With regard to anger, based on the findings of Adams et al. (2010) who investigated the interactive influence of culture and gaze direction on the neural processing of a negative emotion, we hypothesized that anger expressed with averted gaze would enhance EPN activation in response to cultural in-group members, whereas anger expressed with direct gaze would elicit stronger EPN activation in response to cultural out-group members. With regard to happiness, we expected the greatest neural response in the EPN for happiness expressed with direct gaze, because this combination of nonverbal signals conveys the greatest communicative intention and self-relevance (Schilbach et al. 2006). Moreover, its appetitive impact may be greater for cultural in-group than cultural out-group members.
Fig. 1

Experimental design, stimuli and trial structure a Experimental design with factors culture, emotion, and gaze direction, and sample frames of stimulus video clips. The experimental design resulted in the following factor combinations: Ahd Asian happiness direct, Aha Asian happiness averted, Aad Asian anger direct, Aaa Asian anger averted, Ehd European happiness direct, Eha European happiness averted, Ead European anger direct, Eaa European anger averted. b Sample frames extracted out of blurred videos served as high-level base line. c Example of an experimental trial: The participants’ task was to watch each video and rate the perceived valence of the expressed emotions on a four-point scale

Methods

Participants

A group of 22 (11 female; mean age 24.5 ± 2.721 years) participants with no reported history of neurological or psychiatric illness participated in the study. All were born and raised in Germany, had normal or corrected-to-normal vision, and were right-handed. Handedness was confirmed by the Edinburgh Inventory for Handedness (Oldfield 1971). Prior to the scanning session, participants were informed about the necessary safety precautions involving fMRI experiments. They gave written informed consent and were naïve with respect to the experimental task and the purpose of the study. The study was approved by the local ethics committee of the Medical Faculty of the University of Cologne, Germany.

Stimuli

In order to generate adequate dynamic stimulus material, we used computer-animated virtual agents (hereinafter referred to as ‘agents’). Research was able to show that nonverbal behaviour displayed by agents is perceived and processed comparably to that performed by real actors (Bailenson and Yee 2005; Bente et al. 2001; Blascovich et al. 2002; Dyck et al. 2008). Additionally, agents can be precisely and fully controlled while still offering the advantage of physical presence and ecological validity (Vogeley and Bente 2010).

We generated 20 faces of Asian-looking agents (10 female, 10 male) and 20 faces of European-looking agents (10 female, 10 male) based on photographs of Chinese and German people using the software package Face-Gen (Singular Inversions Inc., Toronto, Canada, 2012) in order to manipulate participants’ sense of belonging either to the cultural in- or out-group. Three-second long animations of emotional expressions were created with the virtual reality software Vizard (WorldVizInc, Santa Barbara, USA, 2012). Each agent was used to generate four animations: happiness expressed with direct gaze, anger expressed with direct gaze, happiness expressed with averted gaze, and anger expressed with averted gaze (see Fig. 1a). Accordingly, participants saw a total of 160 animations. The stimulus material was previously used in a cross-cultural study of our group (Krämer et al. 2013). The results of this study and the results of a pilot study validated the stimulus material by showing that the agents’ cultural background as well as their gender and the expressed emotions were accurately and consistently identified. In addition, participants of the present study had to evaluate the cultural background and the perceived likeability of all agents immediately after the fMRI experiment. Results indicated that all agents were identified correctly concerning their cultural background and that Asian and European agents did not differ in likeability ratings (t(1,21) = 0.31, p > .05). For the purpose of generating material for the high-level baseline condition, 20 of the original animations were randomly chosen (5 animations of female Asians; 5 animations of female Europeans; 5 animations of male Asians; 5 animations of male Europeans) and converted into blurred videos via lowpass-filtering using a Matlab based algorithm (The Mathworks Inc., Natick, USA; see Fig. 1b). In these videos, only the shape and texture of the head and the face are recognisable, but no longer the expressed emotions or the agents’ cultural background.

Study design

In the present study, participants saw agents belonging either to their cultural in-group (European) or out-group (Asian) expressing two emotions, happiness and anger. Prior to the experiment, participants were instructed that the agent would either express engagement with them, or that they would merely observe an agent engaging with another person. In the former case, the agent would gaze directly at the participants while expressing the emotions. In the latter case, the agent would be rotated to the left or the right side at an angle of 20°, thereby averting its gaze towards another person while expressing one of the two emotions. Participants were instructed to imagine the other (invisible) person standing on the left or the right side behind them. The three-factorial design including the factors (i) emotion (anger versus happiness), (ii) culture (Asian versus European), and (iii) gaze direction (direct versus averted) is depicted in Fig. 1a.

Experimental procedure

The experiment was conducted in an event-related fashion. Each trial began with a 3 s long stimulus presentation followed by a fixation cross, which was presented in a jittered manner ranging from 1.5 to 2.25 s (mean duration = 1.88 s). Participants then rated the perceived valence of the emotional expression ranging on one dimension from positive to negative. This means, for each trial they had to indicate how positive (positive = 1 or rather positive = 2) or how negative (rather negative = 3 or negative = 4) they perceived the expressed emotion to be, by pressing one of four response buttons within a time window of 3 s (see Fig. 1c). Stimulus presentation and rating scale were separated by a fixation cross in order to enable temporal isolation and independent analyses of stimulus perception and behavioural response. Trials were separated by a fixation cross, which was presented in a jittered manner ranging from 6.5 to 9.5 s (mean duration = 8 s), to increase condition-specific BOLD signal discriminability. The study incorporated two functional runs with 90 trials each (80 sequences of emotional expressions, 10 sequences of blurred videos), which were separated by a 2-min break. Stimuli order was randomized across functional runs and across participants. Each run lasted for about 20 min. Participants alternately used the right or the left hand across runs to balance for lateralized motor-related activations. Stimuli were presented on a screen located behind the participants’ head and reflected to their field of vision via a mirror mounted on the head coil. The stimulus presentation and response recording were performed by the software Vizard (WorldVizInc, Santa Barbara, USA, 2012). Responses were assessed using four buttons of a MR-compatible handheld response device (LUMItouch™, Photon Control Inc., BC, Canada). Prior to the fMRI experiment, all participants received a standardized instruction. They were told that they would see 3 s long animations of emotional expressions and that they would be asked to assess the perceived valence of the emotional expressions on a four-point scale. Participants were further instructed to focus on the fixation cross between trials, on the agents during the presented animations, and to respond as intuitively and quickly as possible after the display of the scale. Following the fMRI experiment, participants had to categorise the culture of each agent presented in the experiment and rate its likeability. In addition, the cultural constructs of individualism and collectivism were assessed using Singelis’ Self-Construal Scale (Singelis 1994) which measures participants’ agreement on individualistic and collectivistic items on two separate subscales.

fMRI data acquisition

A Magnetom Trio 3.0 T whole-body scanner (Siemens Medical Solutions, Erlangen, Germany) was used to acquire structural and functional magnetic resonance images. For the structural images a high-resolution T1-weighted magnetization-prepared rapid acquisition gradient echo (MPRAGE) sequence (TR = 2,250 ms; TE = 3.93 ms, field of view = 256 × 256 mm2, 176 sagittal slices, slice thickness = 1.0 mm, in-plane resolution = 1.0 × 1.0 mm2) and for the fMRI scans a T2*-weighted gradient echo planar imaging (EPI) sequence (TR = 2,200 ms, TE = 30 ms, field of view = 200 × 200 mm2, 36 axial slices, slice thickness 3.0 mm, in-plane resolution = 3.1 × 3.1 mm2) was used. Five additional volumes were collected at the beginning of each EPI session to allow for magnetic saturation. These volumes were discarded prior to further image processing.

Behavioural data analysis

Behavioural data were analyzed using IBM SPSS Statistics 20 (SPSS Inc., Chicago, IL, 2011). A three-way repeated measures analysis of variance (ANOVA) with culture, emotion, and gaze direction as independent variables and the valence ratings as the dependent variable was conducted. Trials with blurred videos were excluded from this analysis as their primary purpose was to provide a high-level baseline for the fMRI paradigm. Bonferroni corrected planned simple comparisons were computed to break down interaction effects. For the main effects, interaction effects, and planned simple comparisons, Pearson’s correlation coefficient (r) is reported as a measure of effect size. The following conventions for interpreting r are suggested: Small effects: r < .10; Moderate effects: r > .10 and r < .30; Large Effects: r > .50 (Field 2009).

fMRI data preprocessing and analysis

Image preprocessing and analysis of fMRI data were performed using SPM8 (The Wellcome Trust Centre for Neuroimaging) implemented in Matlab 7.1 (The MathWorks, Inc.). All functional images of each subject were corrected for head movements using realignment and unwarping and corrected for slice-timing differences (Sladky et al. 2011). The mean EPI image of each participant was computed and coregistered to the corresponding T1 image. All functional images were then normalized to the Montreal Neurological Institute (MNI) reference space using the unified segmentation function in SMP8 and were re-sampled to a voxel size of 2 × 2 × 2 mm3. The transformation was also applied to each participant’s structural image. The normalized functional images were then spatially smoothed with an isotopic Gaussian filter (8 mm full-width at half maximum) to meet the statistical requirements of further analysis and to account for macroanatomical variations across participants.

The data were analyzed using a General Linear Model as implemented in SPM8. In all single subject analyses, effects of interest were modelled separately using a boxcar reference vector convolved with the canonical hemodynamic response function and its time derivative. Further, all six realignment parameters obtained from preprocessing were included in the design matrix to reduce residual motion effects. Trials were classified according to nine event types: (1) Asian anger direct (Aad), (2) Asian anger averted (Aaa), (3) Asian happiness direct (Ahd), (4) Asian happiness averted (Aha), (5) European anger direct (Ead), (6) European anger averted (Eaa), (7) European happiness direct (Ehd), (8) European happiness averted (Eha), (9) blurred videos. Durations for events of interest were set at 3 s, which corresponded to the duration of each animation. Low-frequency signal drifts were filtered using a cut-off of 128 s. For each subject and each condition, a comparison with the implicit baseline was implemented as an individual contrast image, by weighting only the regressor corresponding to that particular condition with 1 and all other regressors with 0. The single subject contrasts were fed into the second level analysis using a flexible factorial ANOVA (factors: condition and subject), employing a random-effects model (Penny et al. 2003).

After performing the second level analysis, different sets of contrasts were conducted. Due to the fact that the design of the present study is complex and involves numerous contrasts, we decided to report and discuss only the results of those contrasts that contribute to our research questions. Therefore, only the results of the following two sets of contrasts are reported: Firstly, the results of the contrasts that aimed to investigate the influence of culture on the perception of anger and happiness separately for each gaze direction ((i) Aad > Ead; (ii) Aaa > Eaa; (iii) Ahd > Ehd; (iv) Aha > Eha; (v) Ead > Aad; (vi) Eaa > Aaa; (vii) Ehd > Ahd; (viii) Eha > Aha). Secondly, the results of the contrasts that aimed to investigate the interaction of culture and gaze direction on the perception of anger and happiness ((i) (Aad > Aaa) > (Ead > Eaa) (anger direct compared to anger averted for Asian compared to European); (ii) (Ahd > Aha) > (Ehd > Eha) (happiness direct compared to happiness averted for Asian compared to European); (iii) (Ead > Eaa) > (Aad > Aaa) (anger direct compared to anger averted for European compared to Asian); (iv) (Ehd > Eha) > (Ahd > Aha) (happiness direct compared to happiness averted for European compared to Asian)). The results of the other sets of contrasts are summarized in the supplementary material.

A significance threshold of p < .05, FWE-corrected for multiple comparisons at the cluster level, and p < .005 at the voxel level, uncorrected for multiple comparisons was used. Significant activations were anatomically localized by using the brain atlas by Duvernoy (Duvernoy and Bourgouin 1999) and the SPM anatomy toolbox, version 1.8 (Eickhoff et al. 2005, 2007). Group activation maps were superimposed on a mean T1 image that was constructed from the individual normalized T1 images of the participants.

Results

Behavioural data

The three-way repeated measures ANOVA revealed a significant interaction effect of culture and emotion, F(1, 20) = 8.687, p = .008, r = .304 (see Fig. 2). Simple comparisons revealed that participants perceived European anger (M = 1.394; SD = .285) more negative (F(1, 20) = 11.291, p = .003, r = .273) than Asian anger (M = 1.503; SD = .242). However, participants did not differ in their perception of Asian happiness compared to European happiness, F(1,20) = 0.007, p > .05. Furthermore, there was a significant interaction effect of gaze direction and emotion, F(1, 20) = 8.537, p = .008, r = .324. Simple comparisons showed that happiness direct (M = 3.573; SD = .253) was rated more positively (F(1, 20) = 12.13, p = .002, r = .44) than happiness averted (M = 3.406; SD = .261). Interestingly, there was no difference in the perception of anger direct compared to anger averted, F (1,20) = 2.807, p > .05. Finally, neither the two-way interaction between culture and gaze direction (F(1,20) = 0.027, p > .05), nor the three-way interaction between culture, emotion and gaze direction (F(1,20) = 0.014, p > .05) were significant.
Fig. 2

Behavioural results showing interaction effect of culture*emotion The scales on the y-axis indicate the mean of valence ratings of stimuli on a four-point rating scale (ranging from 1 = negative to 4 = positive). Error bars indicate the 95 % confidence interval; ** p < .01; ns not significant

Results of participants’ ratings on the Self-Construal Scale show that they scored higher on the subscale measuring individualism (M = 5.386; SD = .405) than on the subscale measuring collectivism (M = 4.19; SD = .592), t(1,21) = 62.443, p = .000, r = .866, indicating they agreed more strongly to individualistic than to collectivistic items.

fMRI data

The set of contrasts that investigated the influence of culture on the perception of anger and happiness separately for each gaze direction yielded the following results: The comparison between European anger averted and Asian anger averted (Eaa > Aaa) revealed activations in the left vmPFC extending into the left middle cingulate cortex, and the left IFG. Activation patterns were also found bilaterally in the hippocampus, the pallidum, the putamen and the caudate nucleus, as well as in the right amygdala and right thalamus (see Fig. 3a; Table 1). Regions identified as differentially responsive to Asian anger direct compared to European anger direct (Aad > Ead) were the left dmPFC extending into the left dlPFC (see Fig. 3b; Table 1). The comparison between European happiness direct and Asian happiness direct (Ehd > Ahd) revealed significant neural responses in the left IFG extending into the left ventrolateral prefrontal cortex (vlPFC), the left vmPFC, and bilaterally in the visual cortex (see Fig. 4a; Table 1). All other comparisons computed in this set of contrasts (Ead > Aad; Ahd > Ehd; Aaa > Eaa; Eaa > Aaa; Aha > Eha; Eha > Aha) did not reveal any differential neural responses.
Fig. 3

Effects of culture and gaze direction on the neural processing of anger a Differential neural activity for observing European anger averted compared to Asian anger averted (Eaa > Aaa). Plots illustrate corresponding contrast estimates obtained for the two stimulus categories for two different local maxima: right Amygdala (28, −8, −16), right Putamen (30, −2, −8). b Differential neural activity for observing Asian anger direct compared to European anger direct (Aad > Ead). Plots illustrate corresponding contrast estimates obtained for the two stimulus categories for two different local maxima: left dmPFC (–12, 34, 34), left dlPFC (−32, 18, 44). c Differential neural activity for observing anger direct compared to anger averted expressed by Asian compared to European agents ((Aad > Aaa) > (Ead > Eaa)). Plots illustrate corresponding contrast estimates obtained for the four stimulus categories for local maxima of left PCC (−3, −50, 28); Error bars indicate the 90 % confidence interval. The principally activated voxels are overlaid on the mean structural anatomic image of the 22 participants. p < .05, FWE-corrected for multiple comparisons at the cluster level; L left hemisphere, R right hemisphere, dmPFC dorsomedial prefrontal cortex, dlPFC dorsolateral prefrontal cortex, PCC posterior cingulate cortex

Table 1

Differential neural processing of anger and happiness depending on culture and gaze direction

Region

Cluster-level

T

Side

MNI coordinates

Size

p FWE-corr

  

x

y

z

1. Eaa > Aaa

 vmPFC

1,151

.000

4.83

L

−30

42

0

 Middle cingulate cortex

  

3.94

L

−14

−18

44

 IFG (p. triang.)

  

3.70

L

−46

38

14

 Putamen

1,237

.000

4.36

R

30

−2

−8

 Hippocampus

  

4.08

R

36

−14

−14

 Thalamus

  

3.93

R

18

−24

2

 Pallidum

  

3.67

R

26

0

−6

 Amygdala

  

3.45

R

28

−8

−16

 Pallidum

925

.001

4.13

L

−10

0

0

 Hippocampus

  

3.87

L

−32

−14

−16

 Putamen

  

3.86

L

−28

2

8

 Nucleus caudatus

  

3.55

L

−12

14

2

 Nucleus caudatus

  

3.08

R

8

14

2

 Amygdala

  

2.93

L

−30

−10

16

2. Aad > Ead

 dmPFC

512

.019

3.78

L

−12

34

34

 dlPFC

  

3.51

L

−32

18

44

3. (Aad > Aaa) > (Ead > Eaa)

 Putamen

2,178

.000

4.35

L

−28

−2

8

 dlPFC

  

4.14

L

−24

26

30

 vlPFC

  

4.05

L

−30

46

0

 dmPFC

  

3.94

L

−14

34

34

 Postcentral gyrus

  

3.93

L

−48

−8

16

 Thalamus

2,205

.000

4.35

R

10

−6

8

 PCC

  

3.97

L

−3

−50

28

 Putamen

  

3.80

R

30

−4

10

4. Ehd > Ahd

 IFG (p. triang.)

799

.002

5.10

L

−36

40

0

 vlPFC

  

4.36

L

−36

50

2

 vmPFC

  

3.52

L

−10

46

−4

 IFG (p. orbit.)

  

3.32

L

−40

30

−12

 Superior occipital gyrus

1,462

.000

4.24

R

24

−92

18

 Superior occipital gyrus

  

4.08

L

−16

−92

6

5. (Ehd > Eha) > (Ahd > Aha)

 vlPFC

956

.000

4.86

L

−36

46

2

 IFG (p. triang.)

  

4.67

L

−36

36

2

 vmPFC

  

3.47

L

−14

46

−6

 Anterior insula

  

2.92

L

−36

24

−2

1. Regions more responsive to European than Asian agents expressing anger with averted gaze; 2. Regions more responsive to Asian than European agents expressing anger with direct gaze; 3. Regions more responsive to happiness expressed with direct gaze compared to happiness expressed with averted gaze for European compared to Asian agents; 4. Regions more responsive to European than Asian agents expressing happiness with direct gaze; 5. Regions more responsive to anger expressed with direct gaze compared to anger expressed with averted gaze for Asian compared to European agents

L left hemisphere, R right hemisphere, IFG inferior frontal gyrus, p. orbit. pars orbitalis, p. triang. pars triangularis, vlPFC ventrolateral prefrontal cortex, vmPFC ventromedial prefrontal cortex, dmPFC dorsomedial prefrontal cortex, dlPFC dorsolateral prefrontal cortex, PCC posterior cingulate cortex

Fig. 4

Effects of culture and gaze direction on the neural processing of happiness a Differential neural activity for observing European happiness direct compared to Asian happiness direct (Ehd > Ahd). Plots illustrate corresponding contrast estimates obtained for the two stimulus categories for two different local maxima: left vmPFC (−18, 46, −4), left IFG (−36, 40, 0). b Differential neural activity for observing happiness direct compared to happiness averted expressed by European compared to Asian agents ((Ehd > Eha) > (Ahd > Aha)). Plots illustrate corresponding contrast estimates obtained for the four stimulus categories for local maxima of left anterior insula (−36, 24, −2). Error bars indicate the 90 % confidence interval. The principally activated voxels are overlaid on the mean structural anatomic image of the 22 participants: p < .05, cluster-level corrected; L left hemisphere, R right hemisphere, vmPFC ventromedial prefrontal cortex, IFG inferior frontal gyrus

The set of contrasts that evaluated the combined influence of gaze direction and culture on the perception of anger and happiness yielded the following results: The interaction investigating the perception of anger direct compared to anger averted expressed by Asian compared to European agents ((Aad > Aaa) > (Ead > Eaa)) resulted in activations in the left dlPFC, the left dmPFC, the left vlPFC, and the left postcentral gyrus. Significant neural activity was also found in the right thalamus and the right PCC, as well as bilaterally in the putamen (see Fig. 3c; Table 1). Brain regions more responsive to happiness direct than happiness averted expressed by European compared to Asian agents ((Eha > Eha) > (Ahd > Aha)) were the left vlPFC extending into the left IFG, the left vmPFC, and the left anterior insula (see Fig. 4b; Table 1). The other interactions computed in the final analyses ((Ead > Eaa) > (Aad > Aaa); (Ahd > Aha) > (Ehd > Eha)) did not reveal any differential neural activations. The results of the other sets of contrasts can be found in the supplementary material.

Discussion

Behavioural findings

At the behavioural level, participants rated European anger as more negative than Asian anger. This is in line with previous findings which suggested that cultural stereotypes, such as the perceived likelihood for an expression to be shown by a member of a specific cultural group, have an influence on the perception of emotions (Brown et al. 2006). According to this, Caucasians were rated as more likely to show anger expressions than Asians which resulted in higher dominance ratings for Caucasians expressing anger compared to Asians (Hess et al. 2000).

Furthermore, happiness direct was perceived as more positive than happiness averted. This corresponds to findings from our own group: Participants rated happiness expressed with direct gaze as compared to averted gaze as more positive (Krämer et al. 2013), and also felt more engaged with a person that showed happiness with direct as compared to averted gaze (Schilbach et al. 2006).

fMRI findings for anger processing

Based on findings of Adams et al. (2010), we hypothesized that the perception of anger expressed with averted gaze would elicit stronger EPN activation in response to cultural in-group members, whereas the perception of anger expressed with direct gaze would enhance EPN activation in response to cultural out-group members. Indeed, this is what we found in the present study: When anger was expressed with averted gaze more activation was found in the EPN in response to cultural in-group than cultural out-group members, whereas when anger was expressed with direct gaze more activation was found in the EPN in response to cultural out-group than cultural in-group members.

Effect of culture on anger expressed with averted gaze

Greater neural activation was observed in areas of the EPN in response to European anger averted compared to Asian anger averted (Eaa > Aaa). Activation was found in the amygdala, which has been shown to respond to perceptual stimuli that are novel or salient to participants (Adolphs 2009; Herry et al. 2007). Chiao et al. (2008), for example, found greater amygdala activation in response to viewing fear expressed by cultural in-group compared to cultural out-group members in participants of two cultural groups. Furthermore, Adams et al. (2003) investigated the role of the amygdala in processing threat-related ambiguity by testing whether amygdala sensitivity to anger would differentially vary as a function of gaze direction. They found stronger amygdala responses for anger expressed with averted gaze (ambiguous threat) compared to anger expressed with direct gaze (clear threat). We conclude that the results of our study corroborate the findings of Chiao et al. (2008) and Adams et al. (2003) as our participants showed enhanced amygdala activation in response to what could be perceived as an ambiguous threat cue (anger averted) only when it was expressed by cultural in-group members. For cultural out-group members no such effect could be observed. This might be the case because our participants perceived an ambiguous threat expressed by cultural in-group members as a more salient cue than an ambiguous threat expressed by cultural out-group members.

Furthermore, we observed stronger activation for European anger averted compared to Asian anger averted bilaterally in the putamen and the caudate nucleus, two striatal structures associated with reward processing (Balleine et al. 2007; Delgado et al. 2003; Delgado 2007; Haruno et al. 2004). Bavel et al. (2008) were able to show that the motivational consequence of belonging to a group (as indicated by perceiving in-group compared to out-group faces) enhanced neural activity in the striatum. These findings are corroborated by results that show striatal activation during viewing pictures of loved ones (Bartels and Zeki 2000) and acts of mutual cooperation (Rilling et al. 2002). One could speculate that our participants perceived the observation of anger expressed not directly towards them, but towards someone else as more rewarding when the emotion-encoder was a cultural in-group member compared to when she/he was a cultural out-group member.

Effect of culture on anger expressed with direct gaze

The comparison of Asian anger direct compared to European anger direct (Aad > Ead) yielded enhanced neural activation in two brain areas that have previously been linked to emotion processing, the dmPFC and the left dlPFC. Apart from the role the dmPFC plays in emotion evaluation and emotion processing (Heinzel et al. 2005; Lane et al. 2001; Northoff et al. 2004), this region has also been associated with mentalizing (also termed “Theory of Mind”, ToM) which is defined as the ability to ascribe mental states to others in order to understand and predict their behaviour (Frith and Frith 1999; Premack and Woodruff 1978; Vogeley et al. 2001). Results of a study by Mitchell et al. (2005) suggested that while a more ventral part of the mPFC was engaged when participants mentalized about similar others, mentalizing about dissimilar others yielded the highest neural response in the dmPFC.

Also, neural activation was found in the left dlPFC in the contrast of Asian anger direct compared to European anger direct. Previous research suggested that this region plays an important role in emotion reappraisal and regulation (Golkar et al. 2012). This is further supported by findings that show dlPFC engagement in a regulatory mechanism that controlled implicit and potentially unwanted racial associations and racially biased responses in previous cross cultural studies (Ito and Bartholow 2009; Kubota et al. 2012). In order to investigate the neural correlates of automatic and controlled social evaluation, Cunningham et al. (2004) presented European-American participants who reported a strong motivation to control prejudice with either subliminal (30 ms) or 525 ms presentations of pictures of European-American and African-American faces. When the faces were shown too briefly to be consciously detected (automatic social evaluation), participants showed greater amygdala activation for African-American compared to European-American faces. Interestingly, this difference in amygdala activation disappeared when the faces were presented for a longer time frame (controlled social evaluation). In this condition, the presentation of African-American faces elicited greater activity in cognitive brain structures including the dlPFC. The authors speculated that the dlPFC activation in the latter condition reflected enhanced control over implicit negative evaluations, contrary to the subliminal face presentations that elicited amygdala activation. They concluded that a neural distinction between automatic and controlled processing of social groups could thereby be observed and that controlled processes were able to modulate automatic evaluations.

Taken together, our results suggest that the processing of Asian anger direct compared to European anger direct resulted in more dmPFC activation possibly because the evaluation of anger expressed by cultural out-group members (dissimilar others) induced increased high-level inferential cognitive processing such as mentalizing compared to the evaluation of anger expressed by cultural in-group members (similar others). Also, our results might indicate that the evaluation of a negative emotion and the regulation of intergroup behaviour elicited more dlPFC activation when participants observed cultural out-group compared to cultural in-group members express a negative emotion with direct gaze.

Interaction effect of culture and gaze direction

Our analyses revealed similar activation patterns in the dmPFC and the left dlPFC in response to anger direct compared to anger averted expressed by Asian relative to European agents ((Aad > Aaa) > (Ead > Eaa)), as compared to those activation patterns discussed above (Aad > Ead). This indicates that the involvement of these regions in the processing of out-group compared to in-group members displaying anger with direct gaze was increased also when computed relative to faces displaying anger with averted gaze, thereby emphasizing the specific effect of direct gaze in this context.

The comparison of anger direct compared to anger averted expressed by Asian relative to European agents also activated the PCC. In the context of emotion perception, previous studies have shown that the PCC is involved in using prior experiences from the self and from observing others to decode and evaluate emotional expressions (Johnson et al. 2006; Ochsner et al. 2004). Furthermore, research on impression formation indicates that this brain region is important for forming first impressions based on verbal information about unknown persons during initial encounters (Kuzmanovic et al. 2012; Schiller et al. 2009). Taken together, these findings suggest that the greater PCC activation may be associated with increased emotion evaluation and impression formation in response to cultural out-group compared to cultural in-group members that expressed a negative emotion with direct relative to averted gaze.

fMRI findings for happiness processing

As hypothesized, the neural findings of the present study indicate a complex interplay between culture, gaze direction and the valence of the expressed emotions. Results revealed a different interactive influence of culture and gaze direction on the neural processing of happiness (an emotion with positive valence) compared to the neural processing of anger (an emotion with negative valence): Enhanced activation was found in a number of brain areas of the EPN when cultural in-group compared to out-group members expressed happiness with direct gaze.

Effect of culture on happiness expressed with direct gaze

The comparison of European happiness direct compared to Asian happiness direct (Ehd > Ahd) revealed neural activation in several areas of the EPN. One of these areas was the left IFG, which has been associated with a number of different tasks such as empathy perception (Schulte-Rüther et al. 2007; Shamay-Tsoory et al. 2009) and a variety of cognitive functions (Press et al. 2012). It has also been repeatedly involved in the perception of emotional expressions. In concordance with our findings, a greater recruitment of the IFG was found by Greer et al. (2012) in response to happiness expressed by cultural in-group members in European-American as well as African-American participants. Based on these results, the findings of the present study might indicate that although the IFG is generally relevant for a variety of emotional and cognitive processing, it may play a particularly important role during the perception of a positive emotion expressed by cultural in-group compared to cultural out-group members with direct gaze.

Additionally, activation was found in the vmPFC in response to European happiness direct compared to Asian happiness direct. Former studies indicated that the vmPFC encodes emotional value. In particular, the experience of positive emotions has been demonstrated to activate this structure. Winecoff et al. (2013) could show that participants’ positive valence ratings of emotional stimuli correlated with activation in the vmPFC, and that vmPFC activation furthermore predicted participants’ valence ratings. Additionally, greater vmPFC activity was previously observed when participants mentalized about faces they rated as similar to themselves compared to faces they rated as dissimilar to themselves (Mitchell et al. 2005). This corroborates findings that indicate the vmPFC’s involvement in empathy perception (Shamay-Tsoory et al. 2009), as both abilities, mentalizing and empathy, are critically involved in the understanding of emotions. Overall, there is substantial evidence that the vmPFC is of crucial relevance for cognitive and affective evaluations as it encodes emotional value, evaluates similarities with other persons, and supports mentalizing and empathy perception. The present study further yields evidence that the vmPFC shows particular involvement in response to stimuli that are evaluated as positive and faces that are perceived as similar to oneself.

Interestingly, the opposite contrast where we compared cultural out-group to cultural in-group members expressing happiness with direct gaze, revealed no significant activation patterns. This indicates that the perception of a positive emotion expressed with direct gaze results in enhanced activation in the EPN only in response to cultural in-group compared to out-group members but not in the opposite contrast.

Interaction effect of culture and gaze direction

Not surprisingly, the activation patterns in response to happiness direct compared to happiness averted expressed by European relative to Asian agents ((Ehd > Eha) > (Ahd > Aha)) were similar compared to those of the contrast Ehd > Ahd. These activation patterns were located in the left IFG and the vmPFC. This indicates that the involvement of these regions in the processing of in-group compared to out-group members displaying happiness with direct gaze was increased also when computed relative to faces displaying happiness with averted gaze, thereby emphasizing the specific effect of direct gaze in this context.

Interestingly, the combined influence of culture and gaze direction on emotion processing revealed activation in the left anterior insula in response to happiness direct compared to happiness averted expressed by European relative to Asian agents. Earlier studies have linked insular activation to gaze processing (Schilbach et al. 2006) and the evaluative processing of emotions (Ito and Bartholow 2009; Lindquist et al. 2012). Also, the perception of cultural group membership has been shown to influence insular activation. Compared to our study, activation of a very similar area of the anterior insula was found in a study that investigated the influence of cultural group-membership on face perception (Beer et al. 2008): Caucasian participants had to complete a ‘Black-White Implicit Association Test’ (Greenwald et al. 1998) in order to measure their automatic and implicit attitudes towards in-group members (pictures of white faces) and out-group members (pictures of black faces). Results indicated that enhanced neural activity in the anterior insula correlated with positive associations in response to in-group members.

We did not find anterior insula activation in the contrast that investigated the effects of culture on the perception of happiness expressed with direct gaze (Ehd > Ahd). However, the contrast that additionally investigated the influence of direct gaze compared to averted gaze on the perception of happiness expressed by cultural in-group compared to out-group members revealed anterior insula activation. This indicates that the interaction of culture and gaze direction further amplifies neural activation patterns during the processing of a positive emotion expressed by cultural in-group members. Additionally, this emphasizes that the investigation of such interactions effects is crucial for our understanding of emotion processing across cultures.

Limitations

It is important to emphasize that the neural correlates of emotion perception depending on culture and gaze direction were only studied in one culture in the present study: German participants had to evaluate emotions expressed by cultural in-group (European) compared to out-group (Asian) members while undergoing fMRI. Therefore, no conclusions can be drawn with respect to neural response patterns of members of other cultures. Findings from a cross-cultural study of our group (Krämer et al. 2013), however, indicated similar behavioural response patterns for Chinese and German participants in the assessment of emotions expressed by cultural in-group compared to out-group members with direct compared to averted gaze. Based on these results we would expect that the neural responses of Chinese participants to cultural in-group compared to out-group members expressing anger and happiness with direct compared to averted gaze might also be similar compared to those of German participants. In particular, we hypothesize that Chinese participants would show enhanced involvement of the EPN when cultural in-group members express happiness with direct gaze and cultural out-group members express anger with direct gaze. Due to the fact, however, that we investigated solely German participants in the present study, we can only speculate on the neural response patterns of Chinese participants. Future research should therefore investigate the neural correlates of emotion perception depending on culture and gaze direction by comparing participants from both cultural groups.

Furthermore, alongside our argumentation that the ratings of European anger reflected a difference in the perception of anger expressed by cultural in-group compared to out-group members, we cannot fully rule out the possibility that these stimuli were not well-matched, as our German participants rated European agents expressing anger as more negative than Asian agents expressing anger. Although we evaluated the stimulus material in previous studies and matched it accordingly, only comparing the ratings of participants from both cultures in one study would entirely address this issue.

Conclusion

In the present study we investigated the combined influence of culture and gaze direction on the neural mechanisms underlying the processing of happiness and anger, two emotions with differing valence. First, our results indicate a differential influence of gaze direction (direct versus averted) on the processing of anger depending on the cultural background of the emotion-encoder. When anger was expressed with direct gaze, more activation was found in the EPN in response to cultural out-group than in-group members, whereas when anger was expressed with averted gaze, more activation was found in the EPN in response to cultural in-group than out-group members. We suggest that a direct threat (anger direct) was perceived as more salient when expressed by cultural out-group compared to in-group members, whereas an ambiguous threat (anger averted) was perceived as more salient when expressed by cultural in-group compared to out-group members. Second, our results show that the combined influence of culture and gaze direction on the neural processing of happiness (an emotion with positive valence) differs from the neural processing of anger (an emotion with negative valence). Enhanced activation was found in a number of brain areas of the EPN when cultural in-group compared to out-group members expressed a positive emotion with direct gaze. This indicates that participants perceived a positive emotion expressed by cultural in-group members with direct gaze as a more salient cue than a positive emotion expressed by cultural out-group members with direct gaze.

In sum, we delivered new evidence that not only the complex interplay of culture and gaze direction modulates the processing of a perceived emotion, but that the emotion’s valence is also relevant for the neural correlates of emotion processing. Future research needs to further investigate these combined influences on emotion processing in order to enhance our understanding of the neural processes that underlie emotion perception and to uncover insights into how people navigate their social worlds.

Notes

Acknowledgments

The authors would like to thank the members of the Neuroimaging Group of the Department of Psychiatry, University Hospital Cologne, for their support and valuable comments on this article. Kliment Yanev deserves much appreciation for his assistance with stimulus generation and technical support, and Barbara Elghahwagi and Dorothe Krug deserve our gratitude for their assistance with the fMRI scanning. Finally, we would like to thank all the participants that took part in the study.

This research has been funded by a grant of the Volkswagen Foundation (KV, GB) and a doctoral fellowship of the Studienstiftung des deutschenVolkes to KK.

Supplementary material

40167_2014_13_MOESM1_ESM.pdf (42 kb)
Supplementary material 1 (PDF 41 kb)

References

  1. Adams, R. B., Franklin, R. G., Rule, N. O., Freeman, J. B., Kveraga, K., Hadjikhani, N., et al. (2010). Culture, gaze and the neural processing of fear expressions. Social Cognitive and Affective Neuroscience, 5(2–3), 340–348. doi: 10.1093/scan/nsp047.PubMedCentralPubMedCrossRefGoogle Scholar
  2. Adams, R. B., Gordon, H. L., Baird, A. A., Ambady, N., & Kleck, R. E. (2003). Effects of gaze on amygdala sensitivity to anger and fear faces. Science, 300(5625), 1536. doi: 10.1126/science.1082244.PubMedCrossRefGoogle Scholar
  3. Adolphs, R. (2009). The social brain: Neural basis of social knowledge. Annual Review of Psychology, 60, 693–716. doi: 10.1146/annurev.psych.60.110707.163514.PubMedCentralPubMedCrossRefGoogle Scholar
  4. Ambady, N., & Bharucha, J. (2009). Culture and the brain. Current Directions in Psychological Science, 18(6), 342–345. doi: 10.1111/j.1467-8721.2009.01664.x.CrossRefGoogle Scholar
  5. Bailenson, J. N., & Yee, N. (2005). Digital chameleons automatic assimilation of nonverbal gestures in immersive virtual environments. Psychological Science, 16(10), 814–819. doi: 10.1111/j.1467-9280.2005.01619.x.PubMedCrossRefGoogle Scholar
  6. Balleine, B. W., Delgado, M. R., & Hikosaka, O. (2007). The role of the dorsal striatum in reward and decision-making. The Journal of Neuroscience, 27(31), 8161–8165. doi: 10.1523/JNEUROSCI.1554-07.2007.PubMedCrossRefGoogle Scholar
  7. Barrett, L. F., Mesquita, B., Ochsner, K. N., & Gross, J. J. (2007). The experience of emotion. Annual Review of Psychology, 58(1), 373–403. doi: 10.1146/annurev.psych.58.110405.085709.PubMedCentralPubMedCrossRefGoogle Scholar
  8. Bartels, A., & Zeki, S. (2000). The neural basis of romantic love [miscellaneous article]. Neuroreport, 11(17), 3829–3834.PubMedCrossRefGoogle Scholar
  9. Bavel, J. J. V., Packer, D. J., & Cunningham, W. A. (2008). The neural substrates of in-group bias a functional magnetic resonance imaging investigation. Psychological Science, 19(11), 1131–1139. doi: 10.1111/j.1467-9280.2008.02214.x.PubMedCrossRefGoogle Scholar
  10. Bavelas, J. B., Black, A., Lemery, C. R., & Mullett, J. (1986). I show how you feel: Motor mimicry as a communicative act. Journal of Personality and Social Psychology, 50(2), 322–329.CrossRefGoogle Scholar
  11. Beer, J. S., Stallen, M., Lombardo, M. V., Gonsalkorale, K., Cunningham, W. A., & Sherman, J. W. (2008). The Quadruple Process model approach to examining the neural underpinnings of prejudice. NeuroImage, 43(4), 775–783. doi: 10.1016/j.neuroimage.2008.08.033.PubMedCrossRefGoogle Scholar
  12. Bente, G., Petersen, A., Krämer, N. C., & De Ruiter, J. P. (2001). Transcript-based computer animation of movement: Evaluating a new tool for nonverbal behavior research. Behavior Research Methods, Instruments, & Computers, 33(3), 303–310.CrossRefGoogle Scholar
  13. Biele, C., & Grabowska, A. (2006). Sex differences in perception of emotion intensity in dynamic and static facial expressions. Experimental Brain Research, 171(1), 1–6. doi: 10.1007/s00221-005-0254-0.PubMedCrossRefGoogle Scholar
  14. Blascovich, J., Loomis, J., Beall, A. C., Swinth, K. R., Hoyt, C. L., & Bailenson, J. N. (2002). Immersive virtual environment technology as a methodological tool for social psychology. Psychological Inquiry, 13(2), 103–124.CrossRefGoogle Scholar
  15. Brown, L. M., Bradley, M. M., & Lang, P. J. (2006). Affective reactions to pictures of ingroup and outgroup members. Biological Psychology, 71(3), 303–311. doi: 10.1016/j.biopsycho.2005.06.003.PubMedCrossRefGoogle Scholar
  16. Chiao, J. Y., Iidaka, T., Gordon, H. L., Nogawa, J., Bar, M., Aminoff, E., et al. (2008). Cultural specificity in amygdala response to fear faces. Journal of Cognitive Neuroscience, 20(12), 2167–2174.PubMedCrossRefGoogle Scholar
  17. Craig, A. D. (2002). How do you feel? Interoception: The sense of the physiological condition of the body. Nature Reviews Neuroscience, 3(8), 655–666. doi: 10.1038/nrn894.PubMedGoogle Scholar
  18. Craig, A. D. (2009). How do you feel—now? The anterior insula and human awareness. Nature Reviews Neuroscience, 10(1), 59–70. doi: 10.1038/nrn2555.PubMedCrossRefGoogle Scholar
  19. Critchley, H. D., Wiens, S., Rotshtein, P., Ohman, A., & Dolan, R. J. (2004). Neural systems supporting interoceptive awareness. Nature Neuroscience, 7(2), 189–195. doi: 10.1038/nn1176.PubMedCrossRefGoogle Scholar
  20. Cunningham, W. A., Johnson, M. K., Raye, C. L., Gatenby, J. C., Gore, J. C., & Banaji, M. R. (2004). Separable neural components in the processing of black and white faces. Psychological Science, 15(12), 806–813. doi: 10.1111/j.0956-7976.2004.00760.x.PubMedCrossRefGoogle Scholar
  21. Delgado, M. R. (2007). Reward-related responses in the human striatum. Annals of the New York Academy of Sciences, 1104(1), 70–88.PubMedCrossRefGoogle Scholar
  22. Delgado, M. R., Locke, H. M., Stenger, V. A., & Fiez, J. A. (2003). Dorsal striatum responses to reward and punishment: Effects of valence and magnitude manipulations. Cognitive, Affective, & Behavioral Neuroscience, 3(1), 27–38.CrossRefGoogle Scholar
  23. Duvernoy, H. M., & Bourgouin, P. (1999). The human brain: Surface, three-dimensional sectional anatomy with mri, and blood supply. New York: Springer.CrossRefGoogle Scholar
  24. Dyck, M., Winbeck, M., Leiberg, S., Chen, Y., Gur, R. C., & Mathiak, K. (2008). Recognition profile of emotions in natural and virtual faces. PLoS ONE, 3(11), 1–8. doi: 10.1371/journal.pone.0003628.CrossRefGoogle Scholar
  25. Eickhoff, S. B., Paus, T., Caspers, S., Grosbras, M.-H., Evans, A. C., Zilles, K., et al. (2007). Assignment of functional activations to probabilistic cytoarchitectonic areas revisited. NeuroImage, 36(3), 511–521. doi: 10.1016/j.neuroimage.2007.03.060.PubMedCrossRefGoogle Scholar
  26. Eickhoff, S. B., Stephan, K. E., Mohlberg, H., Grefkes, C., Fink, G. R., Amunts, K., et al. (2005). A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data. NeuroImage, 25(4), 1325–1335. doi: 10.1016/j.neuroimage.2004.12.034.PubMedCrossRefGoogle Scholar
  27. Ekman, P., & Friesen, W. V. (1971). Constants across cultures in the face and emotion. Journal of Personality and Social Psychology, 17(2), 124–129.PubMedCrossRefGoogle Scholar
  28. Elfenbein, H. A. (2013). Nonverbal dialects and accents in facial expressions of emotion. Emotion Review, 5(1), 90–96. doi: 10.1177/1754073912451332.CrossRefGoogle Scholar
  29. Elfenbein, H. A., & Ambady, N. (2002). On the universality and cultural specificity of emotion recognition: A meta-analysis. Psychological Bulletin, 128(2), 203–235.PubMedCrossRefGoogle Scholar
  30. Field, A. (2009). Discovering statistics using SPSS. London: Sage Publications Limited.Google Scholar
  31. Frijda, N. H. (1987). The emotions. Cambridge: Cambridge University Press.Google Scholar
  32. Frith, C. D. (2009). Role of facial expressions in social interactions. Philosophical Transactions of the Royal Society of London. Series B, Biological sciences, 364(1535), 3453–3458. doi: 10.1098/rstb.2009.0142.PubMedCentralPubMedCrossRefGoogle Scholar
  33. Frith, C. D., & Frith, U. (1999). Interacting minds: A biological basis. Science, 286(5445), 1692–1695.PubMedCrossRefGoogle Scholar
  34. Georgescu, A. L., Kuzmanovic, B., Santos, N. S., Tepest, R., Bente, G., Tittgemeyer, M., et al. (2013). Perceiving nonverbal behavior: Neural correlates of processing movement fluency and contingency in dyadic interactions. Human Brain Mapping,. doi: 10.1002/hbm.22259.PubMedGoogle Scholar
  35. Golkar, A., Lonsdorf, T. B., Olsson, A., Lindstrom, K. M., Berrebi, J., Fransson, P., et al. (2012). Distinct contributions of the dorsolateral prefrontal and orbitofrontal cortex during emotion regulation. PLoS ONE, 7(11), e48107. doi: 10.1371/journal.pone.0048107.PubMedCentralPubMedCrossRefGoogle Scholar
  36. Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. (1998). Measuring individual differences in implicit cognition: The implicit association test. Journal of Personality and Social Psychology, 74(6), 1464.PubMedCrossRefGoogle Scholar
  37. Greer, T. M., Vendemia, J. M. C., & Stancil, M. (2012). Neural correlates of race-related social evaluations for African Americans and White Americans. Neuropsychology, 26(6), 704–712. doi: 10.1037/a0030035.PubMedCrossRefGoogle Scholar
  38. Hadjikhani, N., Hoge, R., Snyder, J., & de Gelder, B. (2008). Pointing with the eyes: The role of gaze in communicating danger. Brain and Cognition, 68(1), 1–8. doi: 10.1016/j.bandc.2008.01.008.PubMedCentralPubMedCrossRefGoogle Scholar
  39. Han, S., & Northoff, G. (2008). Culture-sensitive neural substrates of human cognition: A transcultural neuroimaging approach. Nature Reviews Neuroscience, 9(8), 646–654.PubMedCrossRefGoogle Scholar
  40. Han, S., Northoff, G., Vogeley, K., Wexler, B. E., Kitayama, S., & Varnum, M. E. W. (2013). A cultural neuroscience approach to the biosocial nature of the human brain. Annual Review of Psychology, 64(1), 335–359. doi: 10.1146/annurev-psych-071112-054629.PubMedCrossRefGoogle Scholar
  41. Haruno, M., Kuroda, T., Doya, K., Toyama, K., Kimura, M., Samejima, K., et al. (2004). A neural correlate of reward-based behavioral learning in caudate nucleus: A functional magnetic resonance imaging study of a stochastic decision task. The Journal of Neuroscience, 24(7), 1660–1665. doi: 10.1523/JNEUROSCI.3417-03.2004.PubMedCrossRefGoogle Scholar
  42. Heinzel, A., Bermpohl, F., Niese, R., Pfennig, A., Pascual-Leone, A., Schlaug, G., et al. (2005). How do we modulate our emotions? Parametric fMRI reveals cortical midline structures as regions specifically involved in the processing of emotional valences. Cognitive Brain Research, 25(1), 348–358. doi: 10.1016/j.cogbrainres.2005.06.009.PubMedCrossRefGoogle Scholar
  43. Herry, C., Bach, D. R., Esposito, F., Di Salle, F., Perrig, W. J., Scheffler, K., et al. (2007). Processing of temporal unpredictability in human and animal amygdala. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 27(22), 5958–5966. doi: 10.1523/JNEUROSCI.5218-06.2007.CrossRefGoogle Scholar
  44. Hess, U., Blairy, S., & Kleck, R. E. (2000). The influence of facial emotion displays, gender, and ethnicity on judgments of dominance and affiliation. Journal of Nonverbal Behavior, 24(4), 265–283. doi: 10.1023/A:1006623213355.CrossRefGoogle Scholar
  45. Ito, T. A., & Bartholow, B. D. (2009). The neural correlates of race. Trends in Cognitive Sciences, 13(12), 524–531. doi: 10.1016/j.tics.2009.10.002.PubMedCentralPubMedCrossRefGoogle Scholar
  46. Jack, R. E., Garrod, O. G. B., Yu, H., Caldara, R., & Schyns, P. G. (2012). Facial expressions of emotion are not culturally universal. Proceedings of the National Academy of Sciences of the United States of America, 109(19), 7241–7244. doi: 10.1073/pnas.1200155109.PubMedCentralPubMedCrossRefGoogle Scholar
  47. Johnson, M. H. (2005). Subcortical face processing. Nature Reviews Neuroscience, 6(10), 766–774. doi: 10.1038/nrn1766.PubMedCrossRefGoogle Scholar
  48. Johnson, M. K., Raye, C. L., Mitchell, K. J., Touryan, S. R., Greene, E. J., & Nolen-Hoeksema, S. (2006). Dissociating medial frontal and posterior cingulate activity during self-reflection. Social Cognitive and Affective Neuroscience, 1(1), 56–64. doi: 10.1093/scan/nsl004.PubMedCentralPubMedCrossRefGoogle Scholar
  49. Kitayama, S., & Uskul, A. K. (2011). Culture, mind, and the brain: Current evidence and future directions. Annual Review of Psychology, 62(1), 419–449. doi: 10.1146/annurev-psych-120709-145357.PubMedCrossRefGoogle Scholar
  50. Kober, H., Barrett, L. F., Joseph, J., Bliss-Moreau, E., Lindquist, K., & Wager, T. D. (2008). Functional grouping and cortical–subcortical interactions in emotion: A meta-analysis of neuroimaging studies. NeuroImage, 42(2), 998–1031. doi: 10.1016/j.neuroimage.2008.03.059.PubMedCentralPubMedCrossRefGoogle Scholar
  51. Krämer, K., Bente, G., Luo, S., Pfeiffer, U. J., Han, S., & Vogeley, K. (2013). Influence of ethnic group-membership and gaze direction on the perception of emotions. A Cross-cultural study between Germany and China. plos one, 8(6), e66335. doi: 10.1371/journal.pone.0066335.PubMedCentralPubMedCrossRefGoogle Scholar
  52. Kubota, J. T., Banaji, M. R., & Phelps, E. A. (2012). The neuroscience of race. Nature Neuroscience, 15(7), 940–948.PubMedCrossRefGoogle Scholar
  53. Kuzmanovic, B., Bente, G., von Cramon, D. Y., Schilbach, L., Tittgemeyer, M., & Vogeley, K. (2012). Imaging first impressions: Distinct neural processing of verbal and nonverbal social information. NeuroImage, 60(1), 179–188. doi: 10.1016/j.neuroimage.2011.12.046.PubMedCrossRefGoogle Scholar
  54. Kuzmanovic, B., Georgescu, A. L., Eickhoff, S. B., Shah, N. J., Bente, G., Fink, G. R., et al. (2009). Duration matters: dissociating neural correlates of detection and evaluation of social gaze. NeuroImage, 46(4), 1154–1163. doi: 10.1016/j.neuroimage.2009.03.037.PubMedCrossRefGoogle Scholar
  55. Lane, R., Fort, C., Johnson, S., Ryan, L., & Trouard, T. (2001). Dissociable representations of emotional state in dorsal and ventral medial prefrontal cortex. NeuroImage, 13(6 Supplement), 437. doi: 10.1016/S1053-8119(01)91780-X.CrossRefGoogle Scholar
  56. Lindquist, K. A., Wager, T. D., Kober, H., Bliss-Moreau, E., & Barrett, L. F. (2012). The brain basis of emotion: A meta-analytic review. The Behavioral and Brain Sciences, 35(3), 121–143. doi: 10.1017/S0140525X11000446.PubMedCrossRefGoogle Scholar
  57. Marsh, A. A., Elfenbein, H. A., & Ambady, N. (2003). Nonverbal accents: Cultural differences in facial expressions of emotion. Psychological Science, 14(4), 373–376.PubMedCrossRefGoogle Scholar
  58. Mesquita, B., & Frijda, N. H. (1992). Cultural variations in emotions: a review. Psychological Bulletin, 112(2), 179–204.PubMedCrossRefGoogle Scholar
  59. Mitchell, J. P. (2009). Inferences about mental states. Philosophical Transactions of the Royal Society of London. Series B, Biological sciences, 364(1521), 1309–1316. doi: 10.1098/rstb.2008.0318.PubMedCentralPubMedCrossRefGoogle Scholar
  60. Mitchell, J. P., Banaji, M. R., & Macrae, C. N. (2005). The link between social cognition and self-referential thought in the medial prefrontal cortex. Journal of Cognitive Neuroscience, 17(8), 1306–1315. doi: 10.1162/0898929055002418.PubMedCrossRefGoogle Scholar
  61. Murphy, F. C., Nimmo-Smith, I., & Lawrence, A. D. (2003). Functional neuroanatomy of emotions: A meta-analysis. Cognitive, Affective & Behavioral Neuroscience, 3(3), 207–233.CrossRefGoogle Scholar
  62. Northoff, G., Heinzel, A., Bermpohl, F., Niese, R., Pfennig, A., Pascual-Leone, A., et al. (2004). Reciprocal modulation and attenuation in the prefrontal cortex: An fMRI study on emotional–cognitive interaction. Human Brain Mapping, 21(3), 202–212. doi: 10.1002/hbm.20002.PubMedCrossRefGoogle Scholar
  63. Ochsner, K. N., Knierim, K., Ludlow, D. H., Hanelin, J., Ramachandran, T., Glover, G., et al. (2004). Reflecting upon feelings: An fMRI study of neural systems supporting the attribution of emotion to self and other. Journal of Cognitive Neuroscience, 16(10), 1746–1772.PubMedCrossRefGoogle Scholar
  64. Oldfield, R. C. (1971). The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia, 9(1), 97–113.PubMedCrossRefGoogle Scholar
  65. Penny, W. D., Holmes, A. P., & Friston, K. J. (2003). Random effects analysis. In R. S. J. Frackowiak, et al. (Eds.), Human brain function (pp. 843–850). San Diego: Academic Press.Google Scholar
  66. Phan, K. L., Wager, T., Taylor, S. F., & Liberzon, I. (2002). Functional neuroanatomy of emotion: A meta-analysis of emotion activation studies in PET and fMRI. NeuroImage, 16(2), 331–348. doi: 10.1006/nimg.2002.1087.PubMedCrossRefGoogle Scholar
  67. Premack, D., & Woodruff, G. (1978). Does the chimpanzee have a theory of mind? Behavioral and Brain Sciences, 1(04), 515–526. doi: 10.1017/S0140525X00076512.CrossRefGoogle Scholar
  68. Press, C., Weiskopf, N., & Kilner, J. M. (2012). Dissociable roles of human inferior frontal gyrus during action execution and observation. NeuroImage, 60(3), 1671–1677. doi: 10.1016/j.neuroimage.2012.01.118.PubMedCentralPubMedCrossRefGoogle Scholar
  69. Recio, G., Sommer, W., & Schacht, A. (2011). Electrophysiological correlates of perceiving and evaluating static and dynamic facial emotional expressions. Brain Research, 1376, 66–75. doi: 10.1016/j.brainres.2010.12.041.PubMedCrossRefGoogle Scholar
  70. Richeson, J. A., Todd, A. R., Trawalter, S., & Baird, A. A. (2008). Eye-gaze direction modulates race-related amygdala activity. Group Processes & Intergroup Relations, 11(2), 233–246.CrossRefGoogle Scholar
  71. Rilling, J. K., Gutman, D. A., Zeh, T. R., Pagnoni, G., Berns, G. S., & Kilts, C. D. (2002). A neural basis for social cooperation. Neuron, 35(2), 395–405. doi: 10.1016/S0896-6273(02)00755-9.PubMedCrossRefGoogle Scholar
  72. Rule, N. O., Freeman, J. B., & Ambady, N. (2013). Culture in social neuroscience: A review. Social Neuroscience, 8(1), 3–10. doi: 10.1080/17470919.2012.695293.PubMedCrossRefGoogle Scholar
  73. Rymarczyk, K., Biele, C., Grabowska, A., & Majczynski, H. (2011). EMG activity in response to static and dynamic facial expressions. International Journal of Psychophysiology: Official Journal of the International Organization of Psychophysiology, 79(2), 330–333. doi: 10.1016/j.ijpsycho.2010.11.001.CrossRefGoogle Scholar
  74. Sato, W., Kochiyama, T., Yoshikawa, S., Naito, E., & Matsumura, M. (2004a). Enhanced neural activity in response to dynamic facial expressions of emotion: An fMRI study. Cognitive Brain Research, 20(1), 81–91. doi: 10.1016/j.cogbrainres.2004.01.008.PubMedCrossRefGoogle Scholar
  75. Sato, W., Yoshikawa, S., Kochiyama, T., & Matsumura, M. (2004b). The amygdala processes the emotional significance of facial expressions: An fMRI investigation using the interaction between expression and face direction. NeuroImage, 22(2), 1006–1013. doi: 10.1016/j.neuroimage.2004.02.030.PubMedCrossRefGoogle Scholar
  76. Schacter, D. L., Addis, D. R., & Buckner, R. L. (2007). Remembering the past to imagine the future: The prospective brain. Nature Reviews Neuroscience, 8(9), 657–661. doi: 10.1038/nrn2213.PubMedCrossRefGoogle Scholar
  77. Schilbach, L., Wohlschlaeger, A. M., Kraemer, N. C., Newen, A., Shah, N. J., Fink, G. R., et al. (2006). Being with virtual others: Neural correlates of social interaction. Neuropsychologia, 44(5), 718–730.PubMedCrossRefGoogle Scholar
  78. Schiller, D., Freeman, J. B., Mitchell, J. P., Uleman, J. S., & Phelps, E. A. (2009). A neural mechanism of first impressions. Nature Neuroscience, 12(4), 508–514. doi: 10.1038/nn.2278.PubMedCrossRefGoogle Scholar
  79. Schulte-Rüther, M., Markowitsch, H. J., Fink, G. R., & Piefke, M. (2007). Mirror neuron and theory of mind mechanisms involved in face-to-face interactions: A functional magnetic resonance imaging approach to empathy. Journal of Cognitive Neuroscience, 19(8), 1354–1372. doi: 10.1162/jocn.2007.19.8.1354.PubMedCrossRefGoogle Scholar
  80. Shamay-Tsoory, S. G., Aharon-Peretz, J., & Perry, D. (2009). Two systems for empathy: a double dissociation between emotional and cognitive empathy in inferior frontal gyrus versus ventromedial prefrontal lesions. Brain, 132(3), 617–627. doi: 10.1093/brain/awn279.PubMedCrossRefGoogle Scholar
  81. Singelis, T. M. (1994). The measurement of independent and interdependent self-construals. Personality and Social Psychology Bulletin, 20(5), 580–591.CrossRefGoogle Scholar
  82. Sladky, R., Friston, K. J., Trostl, J., Cunnington, R., Moser, E., & Windischberger, C. (2011). Slice-timing effects and their correction in functional MRI. Neuroimage, 58(2–2), 588–594. doi: 10.1016/j.neuroimage.2011.06.078.PubMedCentralPubMedCrossRefGoogle Scholar
  83. Trautmann, S. A., Fehr, T., & Herrmann, M. (2009). Emotions in motion: Dynamic compared to static facial expressions of disgust and happiness reveal more widespread emotion-specific activations. Brain Research, 1284, 100–115.PubMedCrossRefGoogle Scholar
  84. Vincent, J. L., Snyder, A. Z., Fox, M. D., Shannon, B. J., Andrews, J. R., Raichle, M. E., et al. (2006). Coherent spontaneous activity identifies a hippocampal-parietal memory network. Journal of Neurophysiology, 96(6), 3517–3531. doi: 10.1152/jn.00048.2006.PubMedCrossRefGoogle Scholar
  85. Vogeley, K., & Bente, G. (2010). “Artificial humans”: Psychology and neuroscience perspectives on embodiment and nonverbal communication. Neural Networks: The Official Journal of the International Neural Network Society, 23(8–9), 1077–1090. doi: 10.1016/j.neunet.2010.06.003.CrossRefGoogle Scholar
  86. Vogeley, K., Bussfeld, P., Newen, A., Herrmann, S., Happe, F., Falkai, P., et al. (2001). Mind reading: Neural mechanisms of theory of mind and self-perspective. Neuroimage, 14(1), 170–181.PubMedCrossRefGoogle Scholar
  87. Weyers, P., Mühlberger, A., Hefele, C., & Pauli, P. (2006). Electromyographic responses to static and dynamic avatar emotional facial expressions. Psychophysiology, 43(5), 450–453. doi: 10.1111/j.1469-8986.2006.00451.x.PubMedCrossRefGoogle Scholar
  88. Wieser, M. J., & Brosch, T. (2012). Faces in context: A review and systematization of contextual influences on affective face processing. Frontiers in Psychology, 3, 471. doi: 10.3389/fpsyg.2012.00471.PubMedCentralPubMedCrossRefGoogle Scholar
  89. Winecoff, A., Clithero, J. A., Carter, R. M., Bergman, S. R., Wang, L., & Huettel, S. A. (2013). Ventromedial prefrontal cortex encodes emotional value. The Journal of Neuroscience, 33(27), 11032–11039. doi: 10.1523/JNEUROSCI.4317-12.2013.PubMedCentralPubMedCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Katharina Krämer
    • 1
    Email author
  • Gary Bente
    • 2
  • Bojana Kuzmanovic
    • 3
  • Iva Barisic
    • 2
    • 4
  • Ulrich J. Pfeiffer
    • 1
  • Alexandra L. Georgescu
    • 1
  • Kai Vogeley
    • 1
    • 5
  1. 1.Neuroimaging Group, Department of Psychiatry and PsychotherapyUniversity Hospital CologneCologneGermany
  2. 2.Department of PsychologyUniversity of CologneCologneGermany
  3. 3.Institute for Neuroscience and MedicineEthics in the Neuroscience (INM8), Research Center JuelichJülichGermany
  4. 4.Cognitive Science, Department of HumanitiesSocial and Political Sciences, ETH ZurichZurichSwitzerland
  5. 5.Institute for Neuroscience and MedicineCognitive Neuroscience (INM3), Research Center JuelichJülichGermany

Personalised recommendations