Introduction

The ability to understand other people’s emotions and to judge their facial expressions is essential for social interaction (Kappas et al., 2013). Although prototypical emotion expressions can be easily recognized using simple pattern matching, nuanced displays are more difficult to discern due to their subtlety and complexity (Hess et al., 1997). Distinguishing between different types of smiles is an example of such a complex recognition task. Even though as many as 18 types of smiles have been described in the literature (Ekman, 2009), the most commonly used smile typology is the distinction between enjoyment (genuine, Duchenne) and non-enjoyment (posed, non-Duchenne) smiles. Accordingly, smiles that involve a contraction of both the zygomaticus major muscle (Action Unit (AU) 12; Facial Action Coding System (FACS), Ekman et al., 2002) and orbicularis oculi muscle (AU 6) tend to be classified as genuine, whereas non-enjoyment smiles lack activation of the orbicularis oculi. Besides this distinction, smiles can be categorized based on their social function (e.g., Ambadar et al., 2009; Niedenthal et al., 2010). For example, Niedenthal and colleagues (2010) proposed a classification of smiles related to three main areas of social interaction: reward smiles, which communicate positive affective states; affiliation smiles, which communicate prosocial motives; and dominance smiles, associated with negotiating and maintaining one’s social status. Recent studies reveal that these three smile types are morphologically different (Rychlowska et al., 2017), and that people are able to identify and distinguish them (Orlowska et al., 2018).

Despite perceptual differences between different smile types, accurately judging smiles may still be challenging as these expressions may involve complex facial movements (Krumhuber & Kappas, 2005; Krumhuber & Manstead, 2009) and facial actions co-occurring with the contraction of the zygomaticus major muscle (Rychlowska et al., 2017). In addition, the multitude of messages that smiles convey suggests that their interpretation is influenced by the affective state of the perceiver or characteristics of the situation. In other words, although the analysis of the morphology of smiles is necessary to judge these facial expressions, it may not be sufficient for an accurate interpretation of their meaning (Hess & Fischer, 2013; Niedenthal et al., 2010). Instead inferences may be guided by other mechanisms, including sensorimotor simulation (Niedenthal et al., 2010; Wood et al., 2016) and contextual influences (e.g., Hess and Hareli, 2015).

The term “sensorimotor simulation” describes an active representation of the other person’s state, generated in the motor, somatosensory, and reward brain circuits (Ferrari & Coudé, 2018; Pitcher et al., 2008). This complex process is often indexed by facial mimicry (Wood et al., 2016a, b) a phenomenon defined as a spontaneous, unconscious imitation of others facial expressions (Dimberg & Thunberg, 1998). Interestingly, the empirical evidence on the role of mimicry in emotion processing has been mixed so far. Moreover, the effects of mimicry can be confused with the effects of other constructs such as empathy (Kosonogov et al., 2015) or mood (Forgas & East, 2008; Niedenthal et al., 2001). Specifically, highly empathetic observers can be more accurate in recognizing facial expressions, and being in a happy emotional state can increase a person’s sensitivity to subtle changes in smiles (Niedenthal et al., 2001). Although several studies suggest that disrupting facial movements decreases observers’ ability to judge emotion expressions (e.g., Maringer et al., 2011; Oberman et al., 2007; Ponari et al., 2012), others do not show associations between the magnitude of facial mimicry and the accuracy of emotion recognition (e.g., Blairy et al., 1999; Korb et al., 2014; Orlowska et al., 2018). One potential explanation for such discrepancies is that mimicry and sensorimotor processes act as a guide for complex facial processing tasks; for example, when observers judge non-prototypical facial expressions (Hess & Fischer, 2013; Niedenthal et al., 2010; Wood et al., 2016b). Interpreting subtle variations between different smile types is an example of such a task, which is likely to be informed by the perceiver’s facial mimicry. In line with this notion, studies show that when people are unable to freely move their facial muscles, the ability to infer other’s emotions, in particular expression authenticity, is impaired (Maringer et al., 2011; Oberman et al., 2007; Ponari et al., 2012; Rychlowska et al., 2014).

Beyond sensorimotor processes, another important factor in smile interpretation is the observer’s belief about what the smile means in a particular situation (Fischer & Hess, 2017; Greenaway et al., 2018; Hess & Hareli, 2015). Facial expressions do not appear in isolation but are embedded within a given context (Barrett et al., 2011). As a result, the interpretation of facial expressions is often guided by extraneous factors such as body posture (e.g., Aviezer et al., 2008), the presence of other people (e.g., Gray et al., 2017), group status or cultural background of the expresser (e.g., Crivelli et al., 2017). One particularly important source of such influences is the presence of verbal information about the meaning of facial expressions (Barrett et al., 2007). Psychophysiological research suggests that learned categories and prior knowledge guide the extraction of information from visual objects such as faces (Sowden & Schyns, 2006). Consistently, facial expressions are easier to recognize when observers are provided with labels that reduce ambiguity and boost categorical perception (Barrett et al., 2007; Fugate et al., 2010; Nook et al., 2015). When emotionally ambiguous faces (e.g., blends of anger and sadness) are labelled as “angry”, they are further remembered as angrier than the same faces labelled as “sad” (Halberstadt & Niedenthal, 2001). A study by Krumhuber and colleagues (2021) found that such effects extended to posed smiles, which observers misremembered as genuine smiles after seeing them paired with a happy context scenario. Another study by the same team (Namba et al., 2020) showed that verbal descriptions affected perceivers’ judgments of posed smiles, such that these smiles appeared more genuine when paired with descriptions of happy situations than when paired with descriptions of situations implying politeness. Facial expression processing can also be impaired by providing verbal information that is incongruent with the visual percept (Goodenough & Tinker, 1931; Knudsen & Muzekari, 1983; Woll & Martinez, 1982). Beyond simply labelling facial expressions, verbal information can refer to events and situations that observers associate with specific feeling states based on their experiential knowledge (Barsalou, 2008).

Importantly, the influences of sensorimotor simulation and contextual information on the processing of facial expressions may interact with each other. According to the Accessibility Model of Emotion Self-Reports (Robinson & Clore, 2002), people are more likely to interpret feelings using generalized beliefs and stereotypes when the direct emotion experience is not accessible. Applying this reasoning to facial expression processing, when judging emotion displays such as smiles, observers may use two sources of information: their direct bodily experience resulting from seeing and simulating the expression; and their conceptual knowledge about what this expression means in a given context. However, alterations of facial mimicry, the resulting disruption of sensorimotor simulation, and the reduced emotion experience may increase observers’ reliance on their conceptual knowledge about the meaning of a given facial expression. Research on emotion expression in groups indirectly supports this prediction. As such, people tend to mimic facial displays of their in-group members to a greater extent than facial displays of members of other groups (Bourgeois & Hess, 2008; Hess et al., 2022). Conversely, observers’ interpretation of facial expressions of out-group members is influenced by stereotypic beliefs about the target group (Hess & Kirouac, 2000).

Surprisingly, there exists only one empirical study to date that provides a direct test of the joint influences of facial mimicry and social context on the interpretation of facial expressions. By asking participants to hold a pencil in their mouth, Maringer et al. (2011, Study 2) showed that restricting mimicry increased their reliance on contextual information about the meaning of genuine smiles. Specifically, participants presented with positive contextual information rated smiles as more genuine than those given ambiguous information. Although this research provides an interesting initial demonstration of how both facial mimicry and observers’ semantic knowledge about the context contribute to smile interpretation, several limitations should be noted. First, the results are based on very small sample sizes (12 participants per condition), thereby risking type 1 error, an issue particularly pertinent given recent failures of reproducibility in the facial feedback literature (Coles et al., 2019; Wagenmakers et al., 2016). Second, the study used computer-generated faces as stimuli, which raises objections regarding the ecological validity and generalization of the findings to real human expressions (Kätsyri et al., 2019).

The Present Research

The aim of the present research is to test the joint effects of mimicry restriction and verbal context information on the interpretation of smiles. For this purpose, we conducted two highly powered studies employing photographs of smiling people as stimuli. Given that posed smiles convey more ambiguous messages and are harder to classify than genuine enjoyment smiles (Johnston et al., 2010; Orlowska et al., 2018; Rychlowska et al., 2017), the study’s focus was on non-enjoyment smiles derived from two different databases.

In Study 1, participants watched images of non-enjoyment smiles presented either in a happy context, in which genuine expressions would be expected, or in a polite context in which posed expressions would be expected (between-subjects). For half the participants, facial mimicry was restricted and the remaining participants could freely move their faces. We predicted that, among participants whose mimicry was restricted, judgments of the extent to which the smiles were affiliative (versus happy) would be strongly influenced by the context. Such influence should be weaker for participants who could mimic the observed smiles.

Study 2 used a different set of real enjoyment and non-enjoyment smiles, and manipulated social context as a within-subjects variable. In addition to participants’ judgments of the smile meaning, we examined the extent to which participants were able to correctly remember the smiles that they had previously seen. Using these two dependent measures allowed us to examine the extent to which the effects of context and mimicry restriction vary with task demands (Barrett et al., 2007). In light of previous research showing that facial mimicry interacts with empathy (Jospe et al., 2018; Kosonogov et al., 2015; Hsu et al., 2018), and affects the perception of facial expression in a way similar to mood (Niedenthal et al., 2001), we further included empathy and mood as control variables. Similar to Study 1, we predicted that smile judgments would be more strongly influenced by social context when mimicry is inhibited than free.

Study 1

Method

Participants and Design

A total of 176 participants were recruited face to face at the university campus or via the departmental subject pool. Responses from three participants who correctly guessed the purpose of the study were excluded from analysis, leaving a final sample of 173 participants (59 male, Mage = 21.20 years, SD = 4.76). All participants identified themselves as White. Sensitivity analysis using G*Power 3.1 (Faul, Erdfelder, Lang, & Buchner, 2007) indicated that this sample size was sufficient to detect a medium-sized interaction effect (f = 0.25) with 90% statistical power (α = 0.05).

The study manipulated facial mimicry (free vs. restricted) and the context in which smiles were displayed (polite vs. happy) in a between-subjects experimental design. Participants were randomly assigned to one of the four conditions, resulting in approximately 43 persons per group. Ethical approval was granted by the departmental ethics committee. Subjects provided written informed consent prior to participation.

Materials

The facial stimuli consisted of full colour, frontal images (size: 960 × 540 pixels) of eight (four female) White adult faces with direct gaze and neutral background. Images were retrieved from a stimulus set developed by Martin and colleagues (Martin et al., 2018) and displayed a non-enjoyment smile corresponding to the description of affiliation smiles (Rychlowska et al., 2017). In morphological terms (FACS, Ekman et al., 2002), the smile expressions involved the Lip Corner Puller (Action Unit (AU) 12) and Chin Raiser (AU17), with or without the Brow Raiser (AU1 + 2).

We conducted a pilot study to select the verbal context labels for the main experiment. For this, a separate group of participants (N = 33; 27 women) rated the likely occurrence of a reward, affiliation, and dominance smile (Niedenthal et al., 2010; Rychlowska et al., 2017) in 15 different situations related to the job market. In order to choose situations most conducive to enjoyment (reward) and non-enjoyment (affiliation) smiles, we opted for two verbal descriptions that participants most frequently associated with affiliation smiles (“smiling during a job interview”, 97%) and reward smiles (“smiling after receiving a dream job offer”, 91%).

Procedure

Participants were tested individually on computers running Qualtrics, a web-based software (Provo, UT). Upon arrival, they were informed that the study aimed to develop training materials for improving job interview skills and that their task was to rate smiles, as making appropriate facial expressions can contribute to one’s success at a job interview. Participants were then randomly assigned to one of the four experimental conditions. Half the participants learned that people’s judgments of others’ faces are more objective if their own facial movement is restrained. Hence, they would hold a pencil in their mouth while performing the smile evaluation task. The experimenter explained that the pencil should be held sideways, using lips and teeth, without exerting any pressure, and they then demonstrated the correct way of holding it. This version of the pen-in-mouth procedure was identical to techniques previously used to examine the effects of restricting mimicry on judgments of facial expressions (Maringer et al., 2011; Niedenthal et al., 2001). Single-use disinfection wipes were provided to clean the pencil before using it. The other half of participants, who were assigned to the free mimicry condition, did not receive such instructions and were able to freely mimic the smile expressions.

For the manipulation of social context, half the participants were informed that the images they would see featured people who were “smiling during a job interview” (polite condition). The other half was told that the persons were “smiling after receiving a job offer” (happy condition). Prior to the smile evaluation task, we tested the validity of the two scenarios, examining whether the polite context would be more conducive to non-enjoyment affiliation smiles than the happy context, which in turn should encourage enjoyment reward smiles. To that end, subjects were asked to imagine a person smiling “during a job interview” or “after receiving a job offer”, consistent with the condition to which they were assigned. They then rated the extent to which the imagined smile would communicate that the person wanted “to be nice and to express positive intentions” (affiliation rating) or was “feeling happy and content” (happiness rating), using scales ranging from 0 to 100%. Participants’ ratings of these two dimensions were non-independent and had to add up to 100%.

The main smile evaluation task involved the presentation of eight smile expressions in combination with one of the two social context scenarios (happy or polite). The context information was shown at the top of the screen together with each stimulus. Presentation order was randomized, with each smile being displayed for 3 s. For each face-context pair, participants rated the extent to which the smile would communicate that the person wanted “to be nice and to express positive intentions” (ratings of affiliation, from 0 to 100%) or was “feeling happy and content” (ratings of happiness, from 0 to 100%). Again, both ratings were complementary and added up to 100%. At the end of the procedure, participants were probed for suspicion, thanked and debriefed.

Results

Manipulation Check

To examine participants’ a priori beliefs about the type of smile expected in the two social context scenarios, we conducted a 2 (context: polite, happy) by 2 (mimicry: free, restricted) between-subjects ANOVA on ratings of affiliation (averaged across the stimulus faces) as the dependent variable. Consistent with predictions, participants expected smiles paired with the polite context to communicate more affiliation (M = 73.55, SD = 16.81), and thus less happiness than smiles paired with the happy context (M = 29.08, SD = 24.31), F(1, 169) = 194.31, p < .001, ηp² = 0.53. In other words, the polite context was more conducive to the non-enjoyment affiliation smiles than the happy context. Participants in the free mimicry condition expected smiles to be less affiliative (M = 46.90, SD = 30.41) than those in the muscle-restricted condition (M = 56.05, SD = 30.08), but the main effect of mimicry did not reach conventional levels of significance, F(1, 169) = 4.04, p = .05. The interaction between mimicry and context was not significant, F(1, 169) = 0.19, p = .660, ηp² = 0.001.

Effects of Context and Mimicry on Judgments of Smiles

Ratings of affiliation were averaged across the stimulus faces and analyzed in a between-subjects ANOVA as a function of context and mimicry. The main effect of context was significant, F(1, 169) = 55.58, p < .001, ηp² = 0.25, such that smiles were perceived as more affiliative when they were presented together with a polite (M = 67.42, SD = 13.92) than with a happy context (M = 49.28, SD = 18.14). Mimicry had no effect on perceived affiliation, F(1, 169) = 0.27, p = .606, ηp² = 0.002. However, there was a significant interaction between context and mimicry, F(1, 169) = 7.60, p < .006, ηp² = 0.04 (see Fig. 1). As expected, the impact of context on smile ratings was stronger when facial mimicry was restricted (polite context: M = 71.17, SD = 2.35; happy context: M = 46.38, SD = 2.52, F(1, 169) = 51.81, p < .001, ηp² = 0.24) than free (polite context: M = 63.22, SD = 2.49; happy context: M = 51.81, SD = 2.35, F(1, 169) = 11.11, p = .001, ηp² = 0.06).

Fig. 1
figure 1

Ratings of affiliation as a function of social context and mimicry in Study 1. Error bars represent standard errors of the mean (SEM)

Full size image

Discussion

In Study 1, we tested whether restricting the ability to mimic would moderate the influence of verbal context on participants’ judgments of affiliation smiles. Consistent with our predictions, participants held specific beliefs about the meaning of smiles in different situations. Specifically, they believed that the context of a job interview (polite context) is more conducive to non-enjoyment affiliation smiles than the context of receiving a job offer (happy context), which in turn was perceived as conducive to enjoyment or reward smiles. In our study, verbal information about such contexts significantly affected participants’ evaluations of smiles. Specifically, these expressions were judged as conveying more niceness and positive intentions and less happiness and contentment when they occurred in the context of a polite than happy situation. As predicted, such contextual influence was found to be stronger for participants whose mimicry was restrained, suggesting that they relied more on conceptual knowledge in order to judge the genuineness of the smile. In Study 2, we aimed to replicate the present findings and control for two potential moderating variables—empathy and mood.

Study 2

The procedure for this study was similar to the one used in Study 1. Participants judged smile expressions presented in a polite and happy context under conditions of free and restricted mimicry. This time, however, we used a different set of facial stimuli. We also manipulated the social context as a within-subjects variable to account for possible individual differences. As in Study 1, participants rated the extent to which each smile communicated affiliation and happiness. Since the memory of facial expressions is also likely to be affected by both verbal information (Krumhuber et al., 2021; Woll & Martinez, 1982) and mimicry restriction (Wood et al., 2016), in addition to smile judgments, we assessed how accurately participants remembered the smiles they saw. Specifically, we examined whether, similarly to participants’ judgments of smiles, their memory of smiles previously seen would be more strongly influenced by context under conditions of restricted mimicry. Furthermore, we included control measures of empathy and mood to test whether the interaction of context and mimicry would persist after including these variables in the analysis. As in Study 1, we predicted that judgments of smiles would be more strongly influenced by social context when input from mimicry is inhibited rather than free.

Method

Participants and Design

A total of 66 participants (50 women, Mage = 22.71 years, SD = 5.35) were recruited at the university campus and completed the study without remuneration. Sixty-five participants identified themselves as White and one reported “other ethnical background” (Turkish nationality). Sensitivity analysis using G*Power indicated that this sample size was sufficient to detect a medium-sized interaction effect (f = 0.24) with 90% statistical power (α = 0.05, rcor = 0.3).

The study used a mixed experimental design, with facial mimicry (free vs. restricted) as a between-subjects variable and context information (polite vs. happy) as a within-subjects variable. Participants were randomly assigned to one of the two mimicry conditions (free: 34 participants, restricted: 32 participants). Ethical approval was granted by the departmental ethics committee, and subjects provided written informed consent prior to participation.

Materials

The facial stimuli consisted of full colour, frontal images of 12 White female faces displaying a posed smile and a genuine smile for a total of 24 photographs. All models were presented with direct gaze and on a neutral background. Images were retrieved from stimuli sets developed by Johnston et al. (2010) and McLellan et al. (2010). All smiles involved the Lip Corner Puller (AU12). Genuine smiles corresponded to the description of a Duchenne smile (Frank et al., 1993) in that they involved both the Lip Corner Puller (AU12) and Cheek Raiser (AU6). In contrast, posed smiles did not involve AU6.

To measure participants’ empathy levels, we used the Empathy Quotient questionnaire (Eq. 40, Baron-Cohen and Wheelwright, 2004), which consists of 40 items presented as statements such as “I find it easy to put myself in somebody else’s shoes” or “I am good at predicting how someone will feel.” Participants have to use a 4-point Likert scale to rate how strongly they agree with the statements. Scores for each item range from 0 to 2 points, and the maximum overall score is 80, with higher values indicating higher levels of empathy. For the current sample, internal consistency was good at α = 0.856.

Procedure

Similar to Study 1, participants were tested individually and completed the study on Qualtrics. Upon arrival, they were informed that the aim of the study was to develop educational materials for job candidates wanting to improve their job interview skills. They were then provided with brief definitions of each smile type (genuine: “a smile displayed when someone is happy or amused and is truly feeling the emotion”; posed: “a smile that is intentional in the sense that someone wants to be nice and express positive intentions but does not feel the respective emotion”).

Prior to the smile evaluation task, we examined participants’ perceptions of the polite and happy contextual vignettes. Subjects were asked to imagine a person smiling during a job interview and after receiving a job offer, and to rate the extent to which both smiles would communicate that the person wanted “to be nice and to express positive intentions” (affiliation rating) or was “feeling happy and content” (happiness rating), using scales ranging from 0 to 100%. As in Study 1, participants’ responses across the two rating dimensions had to add up to 100%. After rating their beliefs about the meaning of smiles displayed in the polite and happy contexts, participants were randomly assigned to one of the two mimicry conditions. The restriction procedure was identical to the one used in Study 1.

The smile evaluation task involved the presentation of the 12 posed smiles (size: 360 × 480 pixels) in two blocks (each involving six photographs). The social context information was shown at the top of the screen together with each smile stimulus. To mirror the time course of the two scenarios, the first block always started with a description of the polite context (“This person is smiling during a job interview”), followed by the happy context description in the second block (“This person is smiling after receiving a job offer”). The selection of images in each block was counterbalanced across participants. We randomized presentation order within blocks, with every smile expression being displayed for 3 s. For each face-context pair, participants rated the extent to which the smile would communicate that the person wanted “to be nice and to express positive intentions” (ratings of affiliation, from 0 to 100%) or was “feeling happy and content” (ratings of happiness, from 0 to 100%). Again, both ratings were complementary and added up to 100%.

After completing the smile evaluation part, participants in the mimicry-restricted condition removed the pencil from their mouth. All participants then rated their mood on a scale ranging from 1 (“very negative”) to 7 (“very positive), and completed the EQ questionnaire (Baron-Cohen & Wheelwright, 2004). Both tasks lasted approximately 5 minutes.

Next, participants engaged in an unexpected two-alternative forced-choice smile discrimination task that examined how well subjects remembered the smiles they saw earlier in the experiment. On each trial, participants saw a posed smile from the first part of the experiment alongside a genuine smile (distractor stimulus) displayed by the same actor (sizes: 250 × 310 pixels). Importantly, each of these image pairs was accompanied by the context information, identical to the one provided with the posed smile earlier in the experiment. This information was presented at the bottom of the screen. The participants’ task was to indicate which of the two facial stimuli they had seen earlier. Thus, posed smiles were always the correct response, whereas genuine smiles were foils. Participants were informed that the context description was intended as a memory aid; but they were also instructed to base their judgments on their memories of the facial expression seen, rather than the social context information. Both the target and distractor image remained on the screen until participants submitted their response. We randomized the order of trials and counterbalanced the sides on which each type of smile appeared. Finally, participants were thanked and debriefed.

Results

Manipulation Check

As in Study 1, we examined participants’ a priori beliefs about the type of smile expected in the two social context scenarios. A 2 (context: polite, happy) by 2 (mimicry: free, restricted) mixed ANOVA showed that, consistent with predictions, participants expected smiles paired with the polite context to communicate more affiliation (M = 76.48, SD = 17.84) than smiles paired with the happy context (M = 21.41, SD = 20.99), F(1, 64) = 283.95, p < .001, ηp² = 0.82. The main effect of mimicry and the interaction between mimicry and context were not significant, F(1, 64) < 0.001, p = .990, ηp² < 0.001 and F(1, 64) = 3.14, p = .081, ηp² = 0.05, respectively. Adding mood and empathy as covariates in an ANCOVA did not change the pattern of statistical results and revealed no main effects of or interactions with the covariates (Fs < 3.42, ps > 0.068).

Effects of Context and Mimicry on Judgments of Smiles

Ratings of affiliation were averaged across the stimulus faces and analyzed in a mixed ANOVA as a function of context and mimicry. The analysis revealed a main effect of context, F(1, 64) = 98.20, p < .001, ηp² = 0.60, such that smiles were perceived as more affiliative when they appeared together with a polite (M = 67.72, SD = 11.40) than with a happy context (M = 44.60, SD = 13.15). The main effect of mimicry on perceived affiliation did not reach significance, F(1, 64) = 3.74, p = .06, ηp² = 0.05. However, there was a significant interaction between context and mimicry, F(1, 64) = 5.29, p = .025, ηp² = 0.08 (see Fig. 2). As expected, the impact of social context on smile ratings was stronger when facial mimicry was restricted (polite context: M = 72.75, SD = 11.40; happy context: M = 44.05, SD = 12.02, F(1, 64) = 72.35, p < .001, ηp² = 0.53) than free (polite context: M = 63.00, SD = 14.58; happy context: M = 45.12, SD = 14.30, F(1, 64) = 29.86, p < .001, ηp² = 0.32).

Fig. 2
figure 2

Ratings of affiliation as a function of social context and mimicry in Study 2. Error bars represent SEM

Full size image

When controlling for mood and empathy as covariates in an ANCOVA, the main effect of context, F(1, 62) = 6.77, p = .012, ηp² = 0.10, as well as the context by mimicry interaction, F(1, 62) = 4.70, p = .034, ηp² = 0.07, remained significant. There were no main effects of or interactions with the covariates (Fs < 1.43, ps > 0.235).

Effects of Context and Mimicry on Smile Recognition Memory

Percentages of correctly identified posed smiles were averaged across the stimulus faces and analyzed in a mixed ANOVA as a function of context and mimicry. The main effect of context was significant, F(1, 64) = 5.18, p = .026, ηp² = 0.08, such that a higher percentage of smiles was correctly identified as posed when they were presented together with a polite (M = 85.35%, SD = 18.83%) than with a happy context (M = 77.27%, SD = 26.98%). Mimicry had no effect on smile recall, F(1, 64) = 0.35, p = .555, ηp² < 0.01, and the interaction between context and mimicry was not significant, F(1, 64) = 0.05, p = .297, ηp² = 0.02.

An ANCOVA revealed no main effects of or interactions with mood and empathy as covariates (Fs < 1.40, ps > 0.263). However, the main effect of context was no longer significant, F(1, 62) = 1.39, p = .243, ηp² = 0.02.

Discussion

In Study 2, we investigated the joint influences of social context information and the ability to mimic on participants’ judgments and recognition memory of posed smiles. The findings replicate and extend the results of Study 1. As predicted, participants held specific beliefs about the meaning of smiles produced in a polite and happy context, and thought that the former were likely to communicate more affiliation. Furthermore, providing participants with labels describing such contexts affected the way they interpreted the posed smiles. Specifically, smiles were perceived as conveying more niceness and positive intentions and less happiness and contentment when they were paired with a polite rather than happy situation. Importantly, this influence of social context on smile judgments was stronger for participants whose mimicry was restrained, suggesting that they relied more on their conceptual knowledge. None of the above effects was modulated by empathy and mood, which points toward the critical role of mimicry beyond internal factors. It is also worth noting that restricting facial mimicry did not worsen recognition memory for smiles, suggesting that it is unlikely that the mimicry manipulation distracted participants from paying attention to the task. Interestingly, social context influenced the way participants remembered smiles, such that, when presented with a happy situation, subjects were more likely to erroneously report that they had earlier seen an enjoyment, not a posed smile. Although this result indicates that context information can affect the way information is retrieved, the effect became non-significant when controlling for empathy and mood in the analysis. This change, together with the generally high accuracy of participants’ responses in the memory task, suggest that more difficult questions are a promising avenue for future research examining how altering mimicry influences the memory of facial expressions.

General Discussion

The present research examined the influences of facial mimicry and social context information in the interpretation of non-enjoyment smiles. We tested these effects in two studies in which we altered participants’ facial activity using a standard pen-in-mouth manipulation and exposed them to employment-related descriptions of situations depicting happy and polite context. Participants judged the extent to which the smiles conveyed affiliation versus happiness and, in a task assessing recognition memory, indicated which smile they had seen earlier in the experiment. As predicted, our analyses revealed that in both studies participants perceived smiles as more affiliative when they were presented in a context associated with politeness than in a context judged as happy. In addition, context influenced smile recognition memory, such that participants were more likely to report seeing posed rather than genuine smiles when the recall task was accompanied by the polite (rather than happy) context information. These results are in line with previous research documenting the importance of verbal information on the processing of facial expressions (Barrett et al., 2007; Halberstadt & Niedenthal, 2001; Krumhuber et al., 2021; Namba et al., 2020). Our findings also confirm that people have specific beliefs about the types of smile appropriate in specific social situations and that this conceptual knowledge can shape the way they interpret and perceive smiles.

Furthermore, consistent with existing literature on sensorimotor simulation (e.g. Niedenthal et al., 2010; Wood et al., 2016b), the two studies show that experimentally restricting facial mimicry can increase participants’ reliance on contextual information. This result is in line with the studies of Maringer and colleagues (2011), which, to our knowledge, were the only evidence of joint effects of sensorimotor simulation and social context on smile processing. Here, we extend these findings using more naturalistic stimuli presenting human actors rather than animated characters. We also tested the effects of mimicry restriction and contextual information on the perception of non-enjoyment posed smiles, therefore extending the research of Maringer and colleagues (2011) which focused on genuine smiles. In addition, both studies are confirmatory and adequately powered, which is particularly needed given the recent evidence on low reproducibility of research on facial feedback effects (Coles et al., 2019).

It is worth noting that, when exploring the discrimination between reward, affiliation, and dominance smiles, Orlowska et al. (2018) did not find a significant effect of restricting participants’ facial movements. One potential explanation of these seemingly inconsistent findings is that the present studies assessed the perceived meaning of a smile within a given situation, whereas in the study by Orlowska and colleagues (2018), participants were asked to categorize perceived expressions as instances of reward, affiliation, or dominance smiles. It is possible that facial mimicry, as well as other sensorimotor processes, may not be necessary to label the type of smile but plays an important role in different tasks, such as interpreting the meaning of a smile in a social situation. Another important factor could be the response format, with continuous slider scales allowing for more variability than categorical responses.

It is also worth highlighting that Maringer et al. (2011, Study 2) did not observe a significant main effect of context. This could be due to the relatively low statistical power of this prior study, involving only 12 participants per condition. Another plausible explanation is the type of smile used. In the present studies, participants were asked to evaluate the meaning of non-enjoyment smiles, whereas genuine smiles served as target stimuli in the study by Maringer and colleagues. As non-enjoyment smiles are more ambiguous and challenging to classify than enjoyment smiles (Orlowska et al., 2018; Rychlowska et al., 2017), we believe that their judgment is more susceptible to the influence of contextual information. Such a tendency may be particularly marked for static stimuli which are more difficult to discern than dynamic expressions (Krumhuber et al., 2007, 2013).

Both studies revealed consistent effects of context information on smile processing. In addition to showing that such information affects smile judgments, Study 2 reveals that verbal labels describing social situation may also affect participants’ ability to remember smiles. Specifically, compared with a happy situation, a polite situation made participants more likely to think that they have previously seen posed, not genuine smiles. Such results are in line with previous research (Woll & Martinez, 1982), which suggests that memorization of positive facial expressions is particularly vulnerable to verbal contextual information. Indeed, accurate memory for facial features presumably informs face perception (Tong & Nakayama, 1999), but it has been shown that both face perception and memory are malleable and subject to biases (Hugenberg & Sacco, 2008). For instance, social information about a target face can induce assimilation of the face to a categorical prototype, which influences how a face is perceived and remembered (Corneille et al., 2004; Hugenberg et al., 2011). Given that emotion expressions are often represented as categorical prototypes (i.e. prototypical emotion terms like “anger” or “fear” reflect cognitive representations in memory and language; Shaver et al., 1987), it is reasonable to expect that such assimilation processes also occur in the case of social context for facial displays. In other words, emotion expressions may be misremembered as having occurred in a social context more prototypical of the category (in this case: smile type).

We did not observe any effects of mimicry restriction in the recognition memory task. This finding suggests that verbal information affects smile processing to a greater extent than sensorimotor processes. In addition, the correct recollection of smiles, as required in the memory task, may be less sensitive to disruptions of sensorimotor processing than online judgments of these facial expressions. This result is in line with the findings from Kirkham and colleagues (2015), who showed that facial mimicry is reduced by inconsistent verbal information in the initial stages of encoding an expression but not when the same expression is viewed later.

Unlike the study of Maringer and colleagues (2011), the present experiments controlled for a number of potential confounds. Specifically, Study 2 included measures of participants’ empathy levels and mood and accounted for their joint effects (Jospe et al., 2018; Niedenthal et al., 2001; Sessa et al., 2018). The effects of restricting facial mimicry on participants’ smile judgments remained similar after controlling for these internal factors, suggesting that our findings cannot be explained by participants’ lower mood or empathy levels. However, the observed effect of context on recognition memory of smiles was no longer significant after adding empathy and mood as covariates. Thus, the influence of verbal information may vary depending on task demands and be less marked in remembering previously seen facial expressions rather than in interpreting the meaning of facial expressions when seeing them for the first time. Importantly, we did not find main effects of mimicry condition on participants’ ratings in any of the two studies, making it unlikely that altering facial movements distracted participants from paying attention to the task. Such an interpretation was also discarded by the study of Rychlowska and colleagues (2014), who showed that judgments of smile genuineness can be disrupted by restricting mimicry but not by other distracting tasks (i.e., holding a squeeze ball or wearing a finger-cuff heart rate monitor). One limitation of the present studies, as well as much of the existing literature on facial mimicry, concerns the limited information about the extent to which the pen-in-mouth paradigm actually impacts participants’ facial activity. Measuring those effects more explicitly and establishing guidelines for blocking facial mimicry is an important challenge for future research. Another challenge is examining how the influence of mimicry manipulations on the accuracy of smile judgments varies depending on different smiles and contexts. For example, which contexts are most influential? How would contextual vignettes affect real social interactions when mimicry is not possible?

To our knowledge, our studies are the first to experimentally test the joint influences of social context and restricting facial mimicry on the processing of smiles displayed by human actors. Our findings reveal that people hold specific beliefs about the likelihood of observing non-enjoyment and enjoyment smiles in different social contexts and that this knowledge is used to guide the interpretation and the memory of smiles. Importantly, this influence is particularly strong when participants’ facial mimicry is inhibited by holding a pen in the mouth. The present results have important implications for everyday life, offering insights into mechanisms guiding facial expression processing in clinical populations, such as patients with facial palsy or persons whose facial mimicry is altered due to reduced social motivation (Bourgeois & Hess, 2008). By investigating the interplay of social context and sensorimotor processes, the present research is an important first step in this direction and provides promising avenues for future research.