The Neurocognitive Mechanisms of Unconscious Emotional Responses

Abstract

The neurocognitive mechanism of emotion without conscious awareness has long been a subject of great interest (Pribram KH, Gill MM, Freud’s “project” re-assessed: preface to contemporary cognitive theory and neuropsychology. Basic Books, 1976). Several pervious psychological studies have used subliminal presentations of emotional facial expressions in the context of the affective priming paradigm to investigate unconscious emotional processing (e.g., Murphy ST, Zajonc RB, J Person Soc Psychol 64:723–739, 1993; for a review, see Eastwood JD, Smilek D, Conscious Cognit 14:565–584, 2005). In a typical application of this paradigm, a facial expression depicting a negative or positive emotion is flashed briefly as a prime, then an emotionally neutral target (e.g., an ideograph) is presented. Participants are asked to make emotion-related judgments about the target. The studies reported that evaluations of the target were negatively biased by unconscious negative primes, compared to positive primes. This effect has been interpreted as evidence that unconscious emotion can be elicited and that it affects the evaluation of unrelated targets.

Introduction

The neurocognitive mechanisms for emotion without conscious awareness have been a long-standing topic of research (Pribram & Gill, 1976). Several pervious psychological studies have investigated unconscious emotional processing by means of the paradigm of subliminal affective priming (e.g., Murphy & Zajonc, 1993; for a review, see Eastwood & Smilek, 2005). In these studies, a facial expression displaying negative or positive emotion is presented subliminally as a prime, then an emotionally neutral target, such as an ideogram, is presented supraliminally. Participants are instructed to evaluate the target. The studies showed that unconscious negative primes bias evaluations of the target more negatively than positive primes. This effect has been discussed as evidence that emotion can be unconsciously evoked, and that it modulates the evaluation of subsequent targets.

The subliminal affective priming paradigm, however, does not always produce clear effects, and several previous studies have failed to find the effects (e.g., Kemps et al., 1996). While Murphy and Zajonc (1993) found that the priming effect is stronger with subliminal than supraliminal emotional primes, the use of a very short presentation duration for stimuli may prevent even unconscious processing of the stimuli.

Furthermore, the neural mechanisms for unconscious emotional processing remain unclear. Although several neuroimaging (e.g., Morris et al., 1998) and neuropsychological (e.g., Kubota et al., 2000) studies have suggested that the amygdala plays an indispensable role in this process, previous findings are not consistent and debate remains in the literature (Pessoa & Adolphs, 2010). Additionally, the neural pathways underlying unconscious emotional processing remain unexplored. While some studies provided correlational data suggesting that the subcortical visual pathway sends information to the amygdala to implement unconscious emotional processing (e.g., Morris et al., 1999), there was no causal evidence. Besides, the accurate timing data of amygdala emotional processing was scarce.

In this chapter, I present the findings of our psychological and neuroscientific studies that investigated these issues. Our psychological experiments revealed that emotion arises rapidly and unconsciously. We identified the neural mechanisms for this process using functional magnetic resonance imaging (fMRI) and intracranial electroencephalography (EEG).

Psychological Study of the Unconscious Processing of Emotional Expressions

First, we tried to clearly demonstrate that emotional responses arise prior to the conscious awareness of the stimuli evoking such responses using the subliminal presentation of dynamic facial expressions. Dynamic facial expressions may be relevant in this regard because these are more ecologically valid than static expressions. Some previous psychological research has shown that dynamic facial expressions induce more obvious behavioral responses, such as subjective emotion elicitation (Sato & Yoshikawa, 2007b) and facial mimicry (Sato & Yoshikawa, 2007a), than static expressions. These data imply that it is advantageous to use dynamic rather than static facial expressions when attempting to elicit unconscious emotions.

We tested 22 healthy participants. As prime stimuli, we presented dynamic and static facial expressions of fear and happiness during 30 ms. The raw materials of the primes were grayscale photographs of facial expressions depicting fearful, happy, and neutral emotions, and they were used to create dynamic facial expressions by a morphing method. First, facial expressions with 34% and 66% intensities were created, and then 34%, 66%, and 100% facial expressions were displayed in succession to create a dynamic clip. The presentation duration for each image was 10 ms; therefore, the duration of each clip was 30 ms. The photographs of 100% facial expressions were presented as static expressions during 30 ms. A randomized mosaic image was made using a neutral face photograph by splitting the photo into pieces and randomly reordering them. The target stimuli were emotionally neutral ideograms. In each trial (Fig. 2.1), after a fixation cross, a prime stimulus was presented to either the left or right hemi-visual field; this was immediately replaced by a mask in the same place during 300 ms. Directly afterward, the target ideogram was displayed at the same location during 1000 ms. Finally, the rating scale was displayed and participants rated their preference for the target ideogram. After the subliminal priming task, we conducted a forced-choice recognition session and confirmed that no participant had consciously perceived the prime stimuli.

Fig. 2.1

Sato et al.’s (2014b) study. (Upper) An illustration of the trial sequence. The prime stimuli of dynamic and static facial expressions of fear and happiness were presented subliminally. (Lower) Mean (± SE) preference ratings. The asterisks indicate a significant difference between the fear and happiness conditions

For the preference ratings (Fig. 2.1), the results of our analysis of variance (ANOVA) with presentation condition (dynamic or static) and emotion (fear or happiness) as factors indicated that the interaction was significant. Follow-up simple effect analyses revealed that the effect of emotion was significant under the dynamic, but not static, presentation condition, indicating that the subliminal presentation of dynamic fearful versus happy facial expressions reduced preferences for targets.

The results demonstrated that dynamic facial expressions induce evident subliminal affective priming effects. These results extend our understanding of unconscious emotional processing and the boosting effect of dynamic facial expressions. No clear subliminal effects were detected under the static condition. The presentation duration may not have been sufficient to activate the emotional processing with static stimuli.

These results provide hints about the neural mechanism for the unconscious processing of facial expressions, implying that the mechanism is sensitive to dynamic information. This notion is in agreement with the neuroimaging finding that the unconscious emotional processing of facial expressions is performed via the subcortical visual pathway into the amygdala comprising the pulvinar and superior colliculus (Morris et al., 1999). Studies on anatomical connections in animals (Day-Brown et al., 2010) and humans (Tamietto et al., 2012) have revealed that the amygdala receives visual information through the subcortical pathway. Regarding the effect of visual motion information on these brain structures, a neuroimaging study in humans (Schneider & Kastner, 2005) and numerous physiological studies in animals (for a review, see Waleszczyk et al., 2004) indicated that the superior colliculus is more sensitive to dynamic than static information. Together with these data, our results suggest the possibility that unconscious emotional processing is implemented by the activation of the amygdala via the subcortical pathway.

Psychological Study of the Unconscious Emotional Processing of Food

Next, we tried to test the generalizability of unconscious emotional responses and their impact on daily behaviors using food stimuli. Emotional responses to food have important consequences for humans, both positively (e.g., facilitating wellbeing) and negatively (e.g., triggering overeating and lifestyle-related diseases). Previous psychological studies have shown that both the observation and ingestion of food evoke positive emotional reactions (Rodríguez et al., 2005), which in turn stimulate food intake (for a review, see Sørensen et al. 2003).

However, whether emotional responses to food could be unconsciously elicited remained unknown. As we discussed above, several psychological studies using the subliminal affective priming paradigm have shown that non-food emotional stimuli (e.g., facial expressions) induced unconscious emotional processing. On the basis of this evidence, we hypothesized that emotional responses to food would also be unconsciously activated.

In addition, we expected that unconscious food processing would have an influence on daily eating habits. Previous psychological studies have reported that eating habits can be assessed using self-reported questionnaires such as the Dutch Eating Behavior Questionnaire (DEBQ) (van Strien et al., 1986). The DEBQ assesses some eating habits related to overeating. Among the DEBQ sub-scales, several previous studies have shown that the external eating tendency, defined as eating behaviors in response to external (e.g., visual and olfactory) food stimuli, modulates automatic food processing (e.g., attentional shift to food; Brignell et al., 2009). Based on these data, we hypothesized that unconscious emotional reactions to food could be associated with external eating tendency.

To examine these hypotheses, we examined unconscious and conscious emotional responses to food and non-food stimuli and the relationships between these responses and eating habits (Sato et al., 2016). We tested 34 healthy participants. All participants fasted for more than 3 h prior to the experiment. Unconscious emotional responses were tested using the subliminal affective priming paradigm (Murphy & Zajonc, 1993). Food stimuli were color photographs of fast food (e.g., hamburgers) and Japanese diet (e.g., grilled teriyaki fish) (Fig. 2.2). Randomized mosaic stimuli were made from the food stimuli; all food stimuli were split into small squares and randomly sorted. A mask stimulus was also prepared by creating a randomized mosaic pattern. The photographs of neutral faces were used as targets under the subliminal condition. The target stimuli were randomly assigned to the experimental conditions. We used the Japanese version of the DEBQ (van Strien et al., 1986) to assess eating habits related to overeating. In each trial under the subliminal condition, a food or mosaic prime was displayed during 33 ms in the left or right hemi-visual field after a fixation cross; this was immediately replaced by a mask image during 167 ms. The target face was then displayed in the center during 1000 ms. Finally, the response panel was displayed and participants rated their preferences for the target faces. In each trial under the supraliminal condition, after the presentation of the fixation cross, a food or mosaic target was displayed during 200 ms in the left or right hemi-visual field. Participants rated their preferences for the target images. A following forced-choice recognition task was conducted to ensure that none of the participants had consciously recognized the primes.

Fig. 2.2

Sato et al.’s (2016) study. (Upper) The illustrations of food and mosaic stimuli. (Lower left) Mean (± SE) preference ratings in the subliminal condition. The asterisk indicates a significant difference between food and mosaic prime conditions. (Lower right) A scatter plot with a regression line showing a relationship between food preference scores under the subliminal condition and external eating tendency. The asterisk indicates a significant association

Under the subliminal condition, the ANOVA with stimulus type (food or mosaic) as a factor revealed a significant main effect of stimulus type, demonstrating higher preference ratings for faces primed by food images than those for faces primed by mosaics (Fig. 2.2). Similarly, under the supraliminal condition, the main effect of stimulus type was significant, showing higher preference ratings for food images than for mosaics. Correlation analysis showed a significant positive correlation between food preference scores under the subliminal condition and external eating tendency (Fig. 2.2).

These results revealed that unconscious emotional responses are elicited by food stimuli. The data, together with other evidence, suggest that unconscious emotional responses can be triggered by various types of stimuli, including emotional expressions and food. Moreover, the results demonstrated that the unconscious emotional responses to food are positively associated with the tendency for external eating. This suggests that unconscious emotional reactions play a key role in behaviors in daily life.

fMRI Study of the Neural Mechanisms for Unconscious Emotional Processing of Food

Next, we explored the neural mechanisms for unconscious emotional processing using visual food stimuli. Several prior fMRI studies have investigated neural activity in response to supraliminally presented food images. These studies consistently reported that some brain regions, including the neocortical visual areas (e.g., the fusiform gyrus) and limbic regions (e.g., the amygdala), are activated more strongly in response to food images than non-food images (e.g., Holsen et al., 2005; for a review, see van Meer et al., 2015). Accordingly, some scholars proposed that the neocortical visual areas are involved in the visual recognition of food images, which, in turn, activates the amygdala and other related regions for emotional processing (Chen et al., 2016).

However, the neural mechanisms underpinning the unconscious emotional responses to food remained unknown. In other literatures, several previous neuroimaging studies have reported that the unconscious processing of emotional expressions activates the amygdala (e.g., Morris et al., 1998). A few neuropsychological studies also found an indispensable role of the amygdala in the unconscious processing of emotional scenes (e.g., Kubota et al., 2000). Based on these findings, we hypothesized that the amygdala could be activated during both conscious and unconscious emotional processing of food.

Furthermore, prior neuroimaging studies investigating facial expression processing have found that neural pathways are different between conscious and unconscious emotional processing. Some studies provided evidence, though correlational and non-causal results, that emotional information in facial expressions is transmitted unconsciously through the subcortical pathway to the amygdala, such as the superior colliculus and pulvinar (e.g., Morris et al., 1999). It was also reported that the visual pathways involved in conscious and unconscious processing of emotional facial expressions differ (e.g., Vuilleumier et al., 2001). On the basis of such evidence, we hypothesized that the visual pathways to the amygdala for conscious and unconscious processing of food would differ and that subcortical structures would be involved in unconscious food processing.

In this study (Sato et al., 2019), we tested these hypotheses by measuring fMRI while participants viewed supraliminally or subliminally presented food images. We examined the commonalities and differences in neural responses to food versus mosaic images across presentation conditions. Furthermore, we conducted dynamic causal modeling and compared models with the subcortical, cortical, and dual visual pathways to the amygdala. We tested 22 healthy participants, all of whom had fasted for more than 3 h before the experiment. The stimuli presented were identical to those used in the above psychological experiment (Sato et al., 2016). Color photographs of fast food and Japanese diet and their corresponding randomized mosaic images were used. The participants completed two runs of 128 trials using a block design. Each run included one of the presentation conditions, and the order was fixed to the first subliminal and second supraliminal conditions. In each trial, a food or mosaic image was displayed in the center after a fixation cross. Under the subliminal conditions, the stimulus was displayed during 17 ms, immediately replaced by a mask for 1483 ms. Under the supraliminal condition, the stimulus was displayed during 1500 ms without mask presentation. In eight trials pseudo-randomly placed throughout the task blocks, a red cross was displayed during 1500 ms as the target, instead of the food or mosaic images. Participants were instructed to perform a dummy task to detect the red cross.

We performed a conjunction analysis to determine commonalities in neural responses to food versus mosaic images across presentation conditions. The results showed significantly stronger activation in the bilateral amygdala in response to food than mosaic images under both the subliminal and supraliminal conditions (Fig. 2.3). We conducted the interaction contrast between stimulus type and presentation condition to analyze differences in neural responses to food versus mosaic images across presentation conditions. The results showed significantly stronger activation for food versus mosaic images under the supraliminal than subliminal condition in the broad bilateral posterior regions, including the fusiform gyrus (Fig. 2.3).

Fig. 2.3

Sato et al.’s (2019) study. (Upper left) Statistical parametric maps showing significant neural activation in response to food versus mosaic images under both the subliminal and supraliminal conditions and mean (± SE) effect size differences between the food and mosaic conditions. The blue cross indicates the activation focus at the right amygdala. (Upper right) Statistical parametric maps showing significantly stronger neural responses to food versus mosaic images under the supraliminal than subliminal condition and mean (± SE) effect size differences between the food and mosaic conditions. The blue cross indicates the activation focus at the right fusiform gyrus. (Lower) Models (left) and model comparison results (right) of dynamic causal modeling. The solid and dashed arrows indicate modulatory connections in the subcortical and cortical pathway models, respectively. The dual pathways model contains both pathways. The model comparison results in the right hemisphere are shown. AMY amygdala, FG fusiform gyrus, PUL pulvinar, V1 primary visual cortex

We performed dynamic causal modelling to determine the visual pathway to the amygdala in each hemisphere. We compared the models in which the subcortical (pulvinar–amygdala), cortical (primary visual cortex–fusiform gyrus–amygdala), and dual visual pathways were functionally coupled with the amygdala specifically during food processing (Fig. 2.3). In both hemispheres, the model comparison indicated that the subcortical pathway model was the most likely under the subliminal condition, while the dual pathways model was optimal under the supraliminal condition (Fig. 2.3).

Our results demonstrated that the amygdala is active in response to food images in both the subliminal and supraliminal conditions. These results imply that the amygdala is commonly involved in the unconscious and conscious emotional processing of food. These results are consistent with, and extend the substantial neuroimaging and neuropsychological evidence indicating, the involvement of the amygdala in the processing of stimuli with emotional significance (e.g., Sato et al., 2004). The visual areas were activated in response to supraliminally versus subliminally presented food. The neocortical visual areas may be related to the conscious perception of food.

Furthermore, our dynamic causal modeling analyses provide causal evidence that the amygdala is activated by visual food stimuli through the subcortical visual pathway before the conscious recognition of food occurs. Subsequently, the amygdala receives the processed visual information of food through the neocortical pathway. In addition to the aforementioned anatomical (Day-Brown et al., 2010; Tamietto et al., 2012) and neuroimaging (Morris et al., 1999) findings, our model of the subcortical visual input to the amygdala under the subliminal condition is consistent with data showing that a patient with damage in the neocortical visual areas showed amygdala activity, which was functionally coupled with pulvinar activity, in response to unseen emotional expressions (Morris et al., 2001).

Intracranial EEG Study of the Neural Processing of Emotional Expressions

Here, we tried to demonstrate rapid amygdala activation during emotional processing using facial expression stimuli. As described above, a number of neuroimaging studies have shown that the amygdala is active during the visual processing of emotional stimuli, such as emotional facial expressions and palatable food, even in the absence of conscious awareness of the stimuli (e.g., Morris et al., 1999). Some researchers proposed that the amygdala may be activated during an early stage of emotional processing because the amygdala receives sensory input from the subcortical pathway.

However, the temporal profile of the amygdala activation, specifically during the processing of emotional facial expressions, remained unclear. Some studies have examined this issue by recording magnetoencephalography while participants observed emotional facial expressions and found that stronger amygdala responses to fearful/threatening than neutral expressions occurred rapidly, approximately 100 ms after the stimulus onset (e.g., Luo et al., 2007). However, the results were inconsistent and there remains debate over whether the activity of such a deep and complex brain structure as the amygdala can be appropriately estimated from scalp-recorded electromagnetic signals (Papadelis et al., 2009).

Intracranial EEG recordings can offer direct evidence of electric neural activity with high temporal resolution. In this regard, a previous study examined amygdala activity while participants viewed negative, positive, and neutral scenes by employing intracranial EEG recordings and time–frequency analyses (Oya et al., 2002). The results showed stronger gamma-band (around 40 Hz) oscillations in the amygdala in response to negative scenes, as compared with both positive and neutral scenes, as early as 50–150 ms after stimulus onset. Based on these data, we hypothesized that the amygdala could reveal similar rapid gamma-band oscillations while viewing other emotional stimuli, i.e., fearful facial expressions. In this study (Sato et al., 2011), to test this hypothesis, we recorded the intracranial EEG from the amygdala while participants observed fearful, happy, and neutral facial expressions.

We tested six patients. All patients suffered from pharmacologically resistant epilepsy and their intracranial EEG was recorded in a presurgical evaluation. Surgical and electrophysiological assessments suggested that the main epileptic foci were outside the amygdala. Pre- and post-implantation anatomical assessments showed no structural abnormalities in any patient’s bilateral amygdala. Implantation of intracranial electrodes was performed according to a stereotactic method (Mihara & Baba, 2001). Post-implantation anatomical MRI assessments ensured that the target electrodes were located in the amygdala (Fig. 2.4). The stimuli consisted of grayscale photographs of seven individuals’ faces depicting fearful, happy, and neutral expressions. In each trial, after a fixation cross, the stimulus was displayed during 1000 ms in the center of the screen. The response panel was then displayed and the participants performed a dummy task to specify the gender of the displayed faces.

Fig. 2.4

Sato et al.’s (2011) study. (Upper) Representative anatomical magnetic resonance image. The red cross indicates the location of the amygdala electrode. (Lower) Statistical parametric maps for amygdala gamma-band activation for fearful compared with neutral facial expressions (left) and mean (± SE) effect size at the peak activation focus (right)

Time–frequency statistical parametric mapping analyses for the comparison between fearful and neutral expressions revealed significant gamma-band activity between 50 and 150 ms (starting before 100 ms; Fig. 2.4).

What are the implications of rapid amygdala activity triggered before 100 ms for our understanding of emotional processing? Numerous previous scalp- (e.g., Bentin et al., 1996) and subdurally-recorded (e.g., Sato et al., 2014a) EEG studies have reported that the first face-specific visual analysis in the neocortical visual areas occurs after 100 ms. Together with such findings, our data imply that the emotional processing of facial expressions in the amygdala is faster than the first visual analysis of faces in the neocortex. Furthermore, another line of scalp-recorded EEG research that investigated conscious awareness of visual stimuli has shown that the negative deflection at the posterior cortices from 200 to 400 ms was greater in response to seen than unseen stimuli (e.g., Genetti et al., 2009; for a review, see Koivisto & Revonsuo, 2010). Together with these findings, our results suggest that amygdala activity at about 100 ms reflects the emotional processing that takes place prior to the conscious perception of the stimuli.

Conclusion

In summary, our psychological data demonstrate that humans have psychological mechanisms for unconscious emotional processing. The findings presented in section “Psychological study of the unconscious emotional processing of food” suggest that such unconscious emotional responses are general and play important roles in daily life. The fMRI data presented reveal that the amygdala is involved in emotional processing via the subcortical pathway prior to conscious awareness of the stimuli. The intracranial EEG data described demonstrate that the amygdala is rapidly activated in response to emotional stimuli, specifically after approximately 100 ms. Taken together, these findings suggest that the amygdala implements rapid and unconscious emotional processing via the subcortical pathways at approximately 100 ms.

These findings have implications for human behavior. For example, first, the model suggests that rapidly evaluating stimulus emotional significance (via the subcortical visual pathway and unconscious and rapid amygdala activity) is mandatory, and difficult to consciously prevent. Therefore, people should acknowledge such psychological mechanisms and take precautions or slowly adjust their behaviors to mitigate rapid emotional responses. For example, when someone wants to control their eating behaviors, they should not visit food-abundant environments, such as supermarkets and convenience stores. Second, the model suggests that subjective emotional states could provide valuable information about the rapid and unconscious evaluative processes that take place in the amygdala. For example, if one feels slightly negative or positive feelings during social interaction, this subjective information may indicate that our amygdala has automatically detected subtle biologically or socially significant messages.