Abstract
There is ongoing debate as to whether emotion perception is determined by facial expressions or context (i.e., non-facial cues). The present investigation examined the independent and interactive effects of six emotions (anger, disgust, fear, joy, sadness, neutral) conveyed by combinations of facial expressions, bodily postures, and background scenes in a fully crossed design. Participants viewed each face-posture-scene (FPS) combination for 5 s and were then asked to categorize the emotion depicted in the image. Four key findings emerged from the analyses: (1) For fully incongruent FPS combinations, participants categorized images using the face in 61% of instances and the posture and scene in 18% and 11% of instances, respectively; (2) postures (with neutral scenes) and scenes (with neutral postures) exerted differential influences on emotion categorizations when combined with incongruent facial expressions; (3) contextual asymmetries were observed for some incongruent face-posture pairings and their inverse (e.g., anger-fear vs. fear-anger), but not for face-scene pairings; (4) finally, scenes exhibited a boosting effect of posture when combined with a congruent posture and attenuated the effect of posture when combined with a congruent face. Overall, these findings highlight independent and interactional roles of posture and scene in emotion face perception. Theoretical implications for the study of emotions in context are discussed.
This is a preview of subscription content, access via your institution.
References
Abramson, L., Marom, I., Petranker, R., & Aviezer, H. (2017). Is fear in your head? A comparison of instructed and real-life expressions of emotion in the face and body. Emotion, 17(3), 557–565. https://doi.org/10.1037/emo0000252
Aviezer, H., Ensenberg, N., & Hassin, R. R. (2017). The inherently contextualized nature of facial emotion perception. Current Opinion in Psychology, 17, 47–54. https://doi.org/10.1016/j.copsyc.2017.06.006
Aviezer, H., Hassin, R. R., Ryan, J., Grady, C., Susskind, J., Anderson, A., & …Bentin, S. . (2008). Angry, disgusted, or afraid? Studies on the malleability of emotion perception. Psychological Science, 19, 724–732. https://doi.org/10.1111/j.1467-9280.2008.02148.x
Aviezer, H., Trope, Y., & Todorov, A. (2012a). Body cues, not facial expressions, discriminate between intense positive and negative emotions. Science, 338, 1225–1229.
Aviezer, H., Trope, Y., & Todorov, A. (2012b). Holistic person processing: Faces with bodies tell the whole story. Journal of Personality and Social Psychology, 103, 20–37. https://doi.org/10.1037/a0027411
Barrett, L. F., & Kensinger, E. A. (2010). Context is routinely encoded during emotion perception. Psychological Science, 21, 595–599. https://doi.org/10.1177/0956797610363547
Burton, A. (2013). Why has research in face recognition progressed so slowly? The importance of variability. The Quarterly Journal of Experimental Psychology, 66, 1467–1485.
Carroll, J. M., & Russell, J. A. (1996). Do facial expressions signal specific emotions? Judging emotion from the face in context. Journal of Personality and Social Psychology, 70, 205.
Carroll, J. M., & Russell, J. A. (1997). Facial expressions in Hollywood’s portrayal of emotion. Journal of Personality and Social Psychology, 72, 164–176.
Chen, Z., & Whitney, D. (2019). Tracking the affective state of unseen persons. Proceedings of the National Academy of Sciences, 116(15), 7559–7564.
Civile, C., & Obhi, S. S. (2015). Towards a mechanistic understanding of the effects of body posture on facial emotion categorization. The American Journal of Psychology, 128, 367–377.
Conley, M. I., Dellarco, D. V., Rubien-Thomas, E., Cohen, A. O., Cervera, A., Tottenham, N., & Casey, B. J. (2018). The racially diverse affective expression (RADIATE) face stimulus set. Psychiatry Research, 270, 1059–1067.
de Gelder, B. (2006). Towards the neurobiology of emotional body language. Nature Reviews. Neuroscience, 7, 242–249. https://doi.org/10.1038/nrn1872
de Gelder, B., Böcker, K. B., Tuomainen, J., Hensen, M., & Vroomen, J. (1999). The combined perception of emotion from voice and face: Early interaction revealed by human electric brain responses. Neuroscience Letters, 260, 133–136. https://doi.org/10.1016/S0304-3940(98)00963-X
de Gelder, B., & Vroomen, J. (2000). The perception of emotions by ear and by eye. Cognition & Emotion, 14, 289–311. https://doi.org/10.1080/026999300378824
DiGirolamo, M. A., & Russell, J. A. (2017). The emotion seen in a face can be a methodological artifact: The process of elimination hypothesis. Emotion, 17, 538–546. https://doi.org/10.1037/emo0000247
Ekman, P. (1993). Facial expression and emotion. American Psychologist, 48, 384–392. https://doi.org/10.1037/0003-066x.48.4.384
Ekman, P., & Friesen, W. V. (1967). Head and body cues in the judgment of emotion: A reformulation. Perceptual and Motor Skills, 24(3), 711–724.
Ekman, P., Friesen, W., & Ellsworth, P. (1972). The face and emotion. Pergamon.
Ethofer, T., Anders, S., Erb, M., Droll, C., Royen, L., Saur, R., ... & Wildgruber, D. (2006). Impact of voice on emotional judgment of faces: An event‐related fMRI study. Human Brain Mapping, 27, 707-714. https://doi.org/10.1002/hbm.20212
Gendron, M., Roberson, D., van der Vyver, J. M., & Barrett, L. F. (2014). Cultural relativity in perceiving emotion from vocalizations. Psychological Science, 25(4), 911–920.
Hassin, R. R., Aviezer, H., & Bentin, S. (2013). Inherently ambiguous: Facial expressions of emotions, in context. Emotion Review, 5, 60–65. https://doi.org/10.1177/1754073912451331
Hess, U., Dietrich, J., Kafetsios, K., Elkabetz, S., & Hareli, S. (2020). The bidirectional influence of emotion expressions and context: Emotion expressions, situational information and real-world knowledge combine to inform observers’ judgments of both the emotion expressions and the situation. Cognition and Emotion, 34(3), 539–552.
Israelashvili, J., Hassin, R. R., & Aviezer, H. (2019). When emotions run high: A critical role for context in the unfolding of dynamic, real-life facial affect. Emotion, 19(3), 558–562.
Izard, C. E. (1994). Innate and universal facial expressions: Evidence from developmental and cross-cultural research. Psychological Bulletin, 115, 288–299. https://doi.org/10.1037/0033-2909.115.2.288
Kret, M. E., & de Gelder, B. (2010). Social context influence recognition of bodily expressions. Experimental Brain Research, 203, 169–180. https://doi.org/10.1007/s00221-010-2220-8
Lazarus, R. S. (1991). Emotion and adaptation. Oxford University Press.
Lecker, M., Dotsch, R., Bijlstra, G., & Aviezer, H. (2020) Bidirectional contextual influence between faces and bodies in emotion perception. Emotion, 20(7), 1154–1164. https://doi.org/10.1037/emo0000619
Lindquist, K. A., & Gendron, M. (2013). What’s in a word? Language constructs emotion perception. Emotion Review, 5, 66–71.
Lopez, L. D., Reschke, P. J., Knothe, J. M., & Walle, E. A. (2017). Postural communication of emotion: Perception of distinct poses of five discrete emotions. Frontiers in Psychology, 8, 710. https://doi.org/10.3389/fpsyg.2017.00710
Masuda, T., Ellsworth, P. C., Mesquita, B., Leu, J., Tanida, S., & Van de Veerdonk, E. (2008). Placing the face in context: Cultural differences in the perception of facial emotion. Journal of Personality and Social Psychology, 94, 365–381. https://doi.org/10.1037/0022-3514.94.3.365
Masuda, T., & Nisbett, R. E. (2001). Attending holistically versus analytically: Comparing the context sensitivity of Japanese and Americans. Journal of Personality and Social Psychology, 81, 922–934. https://doi.org/10.1037/0022-3514.81.5.922
Matsumoto, D., Yoo, S. H., Hirayama, S., & Petrova, G. (2005). Development and validation of a measure of display rule knowledge: The display rule assessment inventory. Emotion, 5, 23–40. https://doi.org/10.1037/1528-3542.5.1.23
Matsumoto, D., & Hwang, H. (2010). Judging faces in context. Social and Personality Psychology Compass, 4, 393–402. https://doi.org/10.1111/j.1751-9004.2010.00271.x
Matsumoto, D., Hwang, H. S., & Yamada, H. (2012). Cultural differences in the relative contributions of face and context to judgments of emotions. Journal of Cross-Cultural Psychology, 43, 198–218.
Meeren, H. K., van Heijnsbergen, C. C., & de Gelder, B. (2005). Rapid perceptual integration of facial expression and emotional body language. Proceedings of the National Academy of Sciences of the United States of America, 102, 16518–16523. https://doi.org/10.1073/pnas.0507650102
Mehrabian, A. (1969). Significance of posture and position in the communication of attitude and status relationships. Psychological Bulletin, 71(5), 359–372.
Mondloch, C. J., Nelson, N. L., & Horner, M. (2013). Asymmetries of influence: Differential effects of body postures on perceptions of emotional facial expressions. PLoS ONE, 8, e73605. https://doi.org/10.1371/journal.pone.0073605
Ngo, N., & Isaacowitz, D. M. (2015). Use of context in emotion perception: The role of top-down control, cue type, and perceiver’s age. Emotion, 15, 292–302.
Reschke, P. J., Knothe, J. M., Lopez, L. D., & Walle, E. A. (2018). Putting “context” in context: The effects of body posture and emotion scene on adult categorizations of disgust facial expressions. Emotion, 18, 153–158. https://doi.org/10.1037/emo0000350
Righart, R., & de Gelder, B. (2008). Recognition of facial expressions is influenced by emotional scene gist. Cognitive, Affective, & Behavioral Neuroscience, 8, 264–272. https://doi.org/10.3758/CABN.8.3.264
Russell, J. A. (1993). Forced-choice response format in the study facial expression. Motivation and Emotion, 17, 41–51. https://doi.org/10.1007/BF00995206
Russell, J. A., Weiss, A., & Mendelsohn, G. A. (1989). Affect grid: A single-item scale of pleasure and arousal. Journal of Personality and Social Psychology, 57, 493–502. https://doi.org/10.1037/0022-3514.57.3.493
Schneider, B. A., Avivi-Reich, M., & Mozuraitis, M. (2015). A cautionary note on the use of the Analysis of Covariance (ANCOVA) in classification designs with and without within-subject factors. Frontiers in Psychology, 6, 474.
Tottenham, N., Tanaka, J. W., Leon, A. C., McCarry, T., Nurse, M., Hare, T. A., … Nelson, C. (2009). The NimStim set of facial expressions: judgments from untrained research participants. Psychiatry Research, 168, 242-249. https://doi.org/10.1016/j.psychres.2008.05.006
Van den Stock, J. V., Righart, R., & de Gelder, B. (2007). Body expressions influence recognition of emotions in the face and voice. Emotion, 7, 487–494. https://doi.org/10.10337/1528-3542.7.3.487
Van den Stock, J. V., Vandenbulcke, M., Sinke, C., & de Gelder, B. (2014). Affective scenes influence fear perception of individual body expressions. Human Brain Mapping, 35, 492–502. https://doi.org/10.1002/hbm.22195
Wilcox, R. R. (1987). New designs in analysis of variance. Annual Review of Psychology, 38, 29–60. https://doi.org/10.1146/annurev.ps.38.020187.000333
Acknowledgements
The authors would like to thank Ryan McLean for assisting with the stimuli creation and McKay Morgan for helpful insight into an early version of the manuscript.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Data Availability
The data for these studies are available on OSF: https://osf.io/vqjpc/?view_only=c2c7d08cac9549ba9f4d5b6cdd0c5112.
Ethical Approval
The study was conducted following APA ethical standards and with approval of the Institutional Review Board (IRB) at Brigham Young University (Approval Number X18379, Protocol Title: “Perceiving Emotions in Context”).
Conflicts of Interest
The authors declare that there is no conflict of interest.
Informed Consent
Informed consent was obtained from all research participants.
Additional information
Handling Editor: Kristen Lindquist
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Reschke, P.J., Walle, E.A. The Unique and Interactive Effects of Faces, Postures, and Scenes on Emotion Categorization. Affec Sci 2, 468–483 (2021). https://doi.org/10.1007/s42761-021-00061-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42761-021-00061-x
Keywords
- Emotion categorization
- Facial expressions
- Body postures
- Background scenes
- Emotion perception