Sensorimotor simulation and emotion processing: Impairing facial action increases semantic retrieval demands

  • Joshua D. Davis
  • Piotr Winkielman
  • Seana Coulson
Article

Abstract

Sensorimotor models suggest that understanding the emotional content of a face recruits a simulation process in which a viewer partially reproduces the facial expression in their own sensorimotor system. An important prediction of these models is that disrupting simulation should make emotion recognition more difficult. Here we used electroencephalogram (EEG) and facial electromyogram (EMG) to investigate how interfering with sensorimotor signals from the face influences the real-time processing of emotional faces. EEG and EMG were recorded as healthy adults viewed emotional faces and rated their valence. During control blocks, participants held a conjoined pair of chopsticks loosely between their lips. During interference blocks, participants held the chopsticks horizontally between their teeth and lips to generate motor noise on the lower part of the face. This noise was confirmed by EMG at the zygomaticus. Analysis of EEG indicated that faces expressing happiness or disgust—lower face expressions—elicited larger amplitude N400 when they were presented during the interference than the control blocks, suggesting interference led to greater semantic retrieval demands. The selective impact of facial motor interference on the brain response to lower face expressions supports sensorimotor models of emotion understanding.

Keywords

Emotion ERP Embodied cognition 

References

  1. Achaibou, A., Pourtois, G., Schwartz, S., & Vuilleumier, P. (2008). Simultaneous recording of EEG and facial muscle reactions during spontaneous emotional mimicry. Neuropsychologia, 46(4), 1104–1113.CrossRefPubMedGoogle Scholar
  2. Adolphs, R. (2002). Recognizing emotion from facial expressions: Psychological and neurological mechanisms. Behavioral and Cognitive Neuroscience Reviews, 1(21), 21–62.CrossRefPubMedGoogle Scholar
  3. Atkinson, A. P., & Adolphs, R. (2011). The neuropsychology of face perception: Beyond simple dissociations and functional selectivity. Philosophical Transactions of the Royal Society B-Biological Sciences, 366(1571), 1726–1738.CrossRefPubMedCentralGoogle Scholar
  4. Barrett, L. F. (2006). Variety is the spice of life: A psychological construction approach to understanding variability in emotion. Cognition and Emotion, 23(7), 1284–1306.CrossRefGoogle Scholar
  5. Barrett, L. F. (2009). Are emotions natural kinds? Perspectives on Psychological Science, 1(1), 28–58.CrossRefGoogle Scholar
  6. Barrett, L. F. (2013). Psychological construction: The Darwinian approach to the science of emotion. Emotion Review, 5(4), 379–389.CrossRefGoogle Scholar
  7. Barrett, L. F., Wilson-Mendenhall, C. D., & Barsalou, L. W. (2015). The conceptual act theory. In L. F. Barrett & J. A. Russell (Eds.), The psychological construction of emotion (pp. 83–110). New York, NY: Guilford Press.Google Scholar
  8. Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–645.CrossRefPubMedGoogle Scholar
  9. Bartlett, M. S., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., & Movellan, J. (2005). Recognizing facial expression: Machine learning and application to spontaneous behavior. IEEE International Conference on Computer Vision and Pattern Recognition (pp. 568–573). doi:10.1109/CVPR.2005.297
  10. Bartlett, M., Littlewort, G., Frank, M., & Lee, K. (2014). Automated detection of deceptive facial expressions of pain. Current Biology, 24(7), 738–743.CrossRefPubMedPubMedCentralGoogle Scholar
  11. Bruce, V., & Young, A. (1986). Understanding face recognition. British Journal of Psychology, 77, 305–327.CrossRefPubMedGoogle Scholar
  12. Burton, A. M., Bruce, V., & Hancock, P. J. B. (1999). From pixels to people: A model of familiar face recognition. Cognitive Science, 23(1), 1–31.CrossRefGoogle Scholar
  13. Calder, A. J., Keane, J., Cole, J., Campbell, R., & Young, A. W. (2010). Facial expression recognition by people with Möbius syndrome. Cognitive Neuropsychology, 17(1/2/3), 73–87.Google Scholar
  14. Davis, J. I., Senghas, A., Brandt, F., & Ochsner, K. N. (2010). The effects of BOTOX injections on emotional experience. Emotion, 10(3), 433.CrossRefPubMedPubMedCentralGoogle Scholar
  15. Davis, J. D., Winkielman, P., & Coulson, S. (2015). Facial action and emotional language: ERP evidence that blocking facial feedback selectively impairs sentence comprehension. Journal of Cognitive Neuroscience, 27(11), 2269–2280.CrossRefPubMedGoogle Scholar
  16. Dimberg, U. (1982). Facial reactions to facial expressions. Psychophysiology, 19(6), 643–647.CrossRefPubMedGoogle Scholar
  17. Dimberg, U., & Öhman, A. (1996). Behold the wrath: Psychophysiological responses to facial stimuli. Motivation and Emotion, 20(2), 149–182.CrossRefGoogle Scholar
  18. Dimberg, U., Thunberg, M., & Elmehed, K. (2000). Unconscious facial reactions to emotional facial expressions. Psychological Science, 11(1), 86–89.CrossRefPubMedGoogle Scholar
  19. Dimberg, U., Thunberg, M., & Grunedal, S. (2002). Facial reactions to emotional stimuli: Automatically controlled emotional responses. Cognition and Emotion, 16(4), 449–471.CrossRefGoogle Scholar
  20. Eimer, M. (2000). Event-related brain potentials distinguish processing stages involved in face perception and recognition. Clinical Neurophysiology, 111, 694–705.CrossRefPubMedGoogle Scholar
  21. Foroni, F., & Semin, G. R. (2009). Language that puts you in touch with your bodily feelings. Psychological Science, 20, 974–980.CrossRefPubMedGoogle Scholar
  22. Fridlund, A. J., & Cacioppo, J. T. (1986). Guidelines for human electromyographic research. Psychophysiology, 23(5), 567–589.CrossRefPubMedGoogle Scholar
  23. Goldman, A. I., & Sripada, C. S. (2005). Simulationist models of face-based emotion recognition. Cognition, 94(3), 193–213.CrossRefPubMedGoogle Scholar
  24. Gordon, I., Pierce, M. D., Bartlett, M. S., & Tanaka, J. W. (2014). Training facial expression production in children on the autism spectrum. Journal of Autism and Developmental Disorders, 44(10), 2486–2498.CrossRefPubMedGoogle Scholar
  25. Halberstadt, J., Winkielman, P., Niedenthal, P. M., & Dalle, N. (2009). Emotional conception: How embodied emotion concepts guide perception and facial action. Psychological Science, 20, 1254–1261.CrossRefPubMedGoogle Scholar
  26. Hyniewska, S., & Sato, W. (2015). Facial feedback affects valence judgments of dynamic and static emotional expressions. Frontiers in Psychology, 6(291). doi:10.3389/fpsyg.2015.00291
  27. Ipser, A., & Cook, R. (2015). Inducing a concurrent motor load reduces categorization precision for facial expressions. Journal of Experimental Psychology: Human Perception and Performance. Advance online publication. doi:10.1037/xhp0000177
  28. Korb, S., With, S., Niedenthal, P., Kaiser, S., & Grandjean, D. (2014). The perception and mimicry of facial movements predict judgments of smile authenticity. PLoS ONE, 9(6), e99194.CrossRefPubMedPubMedCentralGoogle Scholar
  29. Korb, S., Malsert, J., Rochas, V., Rihs, T. A., Rieger, S. W., Schwab, S., … Grandjean, D. (2015). Gender differences in the neural network of facial mimicry of smiles–An rTMS study. Cortex, 70, 101–114Google Scholar
  30. Künecke, J., Hildebrandt, A., Recio, G., Sommer, W., & Wilhelm, O. (2014). Facial EMG responses to emotional expressions are related to emotion perception ability. PLoS ONE, 9(1), e84053.CrossRefPubMedPubMedCentralGoogle Scholar
  31. Kutas, M., & Federmeier, K. D. (2011). Thirty years and counting: Finding meaning in the N400 component of the event-related brain potential (ERP). Annual Review of Psychology, 62, 621–647.CrossRefPubMedPubMedCentralGoogle Scholar
  32. Larsen, J. T., Norris, C. J., & Cacioppo, J. T. (2003). Effects of positive and negative affect on electromyographic activity over zygomaticus major and corrugator supercilii. Psychophysiology, 40(5), 776–785.CrossRefPubMedGoogle Scholar
  33. Lindquist, K. A., & Gendron, M. (2013). What’s in a word? Language constructs emotion perception. Emotion Review, 5(1), 66–71.CrossRefGoogle Scholar
  34. Lindquist, K. A., Gendron, M., Barrett, L. F., & Dickerson, B. C. (2014). Emotion perception, but not affect perception, is impaired with semantic memory loss. Emotion, 14(2), 375–387.CrossRefPubMedPubMedCentralGoogle Scholar
  35. Lobmaier, J. S., & Fischer, M. H. (2015). Facial feedback affects perceived intensity but not quality of emotional expressions. Brain Science, 5, 357–368.CrossRefGoogle Scholar
  36. Luck, S. J. (2005). An introduction in the event-related potential technique. Cambridge, MA: MIT Press.Google Scholar
  37. Mangels, J. A., Picton, T. W., & Craik, F. I. (2001). Attention and successful episodic encoding: An event-related potential study. Cognitive Brain Research, 11, 77–95.CrossRefPubMedGoogle Scholar
  38. Neal, D. T., & Chartrand, T. L. (2011). Embodied emotion perception amplifying and dampening facial feedback modulates emotion perception accuracy. Social Psychological and Personality Science, 2(6), 673–678.CrossRefGoogle Scholar
  39. Niedenthal, P. M. (2007). Embodying emotion. Science, 316(1002), 1002–1005.CrossRefPubMedGoogle Scholar
  40. Niedenthal, P. M., Brauer, M., Halberstadt, J. B., & Innes-Ker, Å. H. (2001). When did her smile drop? Facial mimicry and the influences of emotional state on the detection of change in emotional expression. Cognition & Emotion, 15(6), 853–864.CrossRefGoogle Scholar
  41. Niedenthal, P. M., Barsalou, L. W., Winkielman, P., Krauth-Gruber, S., & Ric, F. (2005). Embodiment in attitudes, social perception, and emotion. Personality and Social Psychology Review, 9(3), 184–211.CrossRefPubMedGoogle Scholar
  42. Niedenthal, P. M., Winkielman, P., Mondillon, L., & Vermeulen, N. (2009). Embodiment of emotional concepts: Evidence from EMG measures. Journal of Personality and Social Psychology, 96, 1120–1136.CrossRefPubMedGoogle Scholar
  43. Niedenthal, P. M., Mermillod, M., Maringer, M., & Hess, U. (2010). The simulation of smiles (SIMS) model: Embodied simulation and the meaning of facial expression. Behavioral and Brain Sciences, 33(06), 417–433.CrossRefPubMedGoogle Scholar
  44. Oberman, L. M., Winkielman, P., & Ramachandran, V. S. (2007). Face to face: Blocking facial mimicry can selectively impair recognition of emotional expressions. Social Neuroscience, 2, 167–178.CrossRefPubMedGoogle Scholar
  45. Paller, K. A., Gonsalves, B., Grabowecky, M., Bozic, V. S., & Yamada, S. (2000). Electrophysiological correlates of recollecting faces of known and unknown individuals. NeuroImage, 11, 98–110.CrossRefPubMedGoogle Scholar
  46. Paracampo, R., Tidoni, E., Borgomaneri, S., di Pellegrino, G., & Avenanti, A. (2016). Sensorimotor network crucial for inferring amusement from smiles. Cerebral Cortex. doi:10.1093/cercor/bhw294 PubMedGoogle Scholar
  47. Paulmann, S., & Pell, M. D. (2009). Facial expression decoding as a function of emotional meaning status: ERP evidence. NeuroReport, 20(18), 1603–1608.CrossRefPubMedGoogle Scholar
  48. Paulmann, S., & Pell, M. D. (2010). Contextual influences of emotional speech prosody on face processing: How much is enough? Cognitive, Affective, & Behavioral Neuroscience, 10(2), 230–242.CrossRefGoogle Scholar
  49. Peterson, D., Littlewort, G., Bartlett, M., Macerollo, A., Perlmutter, J. S., Jinnah, H. A., … Sejnowski, T. (2016). Objective, computerized video-based rating of blepharospasm severity. Neurology, 87(20), 2146–2153.Google Scholar
  50. Pitcher, D., Garrido, L., Walsh, V., & Duchaine, B. (2008). TMS disrupts the perception and embodiment of facial expressions. Journal of Neuroscience, 28(36), 8929–8933.CrossRefPubMedGoogle Scholar
  51. Ponari, M., Conson, M., D’Amico, N. P., Grossi, D., & Trojano, L. (2012). Mapping correspondence between facial mimicry and emotion recognition in healthy subjects. Emotion, 12(6), 1398–1403.CrossRefPubMedGoogle Scholar
  52. Price, T. F., Dieckman, L. W., & Harmon-Jones, E. (2012). Embodying approach motivation: Body posture influences startle eyeblink and event-related potential responses to appetitive stimuli. Biological Psychology, 90(3), 211–217.CrossRefPubMedGoogle Scholar
  53. Russell, J. A., Bachorowski, J. A., & Fernández-Dols, J. M. (2003). Facial and vocal expressions of emotion. Annual Review of Psychology, 54(1), 329–349.CrossRefPubMedGoogle Scholar
  54. Rychlowska, M., Cañadas, E., Wood, A., Krumhuber, E. G., Fischer, A., & Niedenthal, P. (2014). Blocking mimicry makes true and false smiles look the same. PLoS ONE, 9(3), e90876.CrossRefPubMedPubMedCentralGoogle Scholar
  55. Saxe, R. (2005). Against simulation: The argument from error. Trends in Cognitive Sciences, 9, 174–179.CrossRefPubMedGoogle Scholar
  56. Schweinberger, S. R. (1996). How Gorbachev primed Yeltsin: Analysis of associative priming in person recognition by means of reaction times and event-related brain potentials. Journal of Experimental Psychology: Learning, Memory& Cognition, 22, 1383–1407.Google Scholar
  57. Schweinberger, S. R., & Burton, A. M. (2003). Covert recognition and the neural system for face processing. Cortex, 39, 9–30.CrossRefPubMedGoogle Scholar
  58. Schweinberger, S. R., Pickering, E. C., Burton, A. M., & Kaufmann, J. M. (2002). Human brain potential correlates of repetition priming in face and name recognition. Neuropsychologia, 40, 2057–2073.CrossRefPubMedGoogle Scholar
  59. Schweinberger, S. R., Pickering, E. C., Jentzsch, I., Burton, A. M., & Kaufmann, J. M. (2002). Event-related brain potential evidence for a response of inferior temporal cortex to familiar face repetitions. Cognitive Brain Research, 14, 398–409.CrossRefPubMedGoogle Scholar
  60. Schyns, P. G., Petro, L. S., & Smith, M. L. (2009). Transmission of facial expressions of emotion co-evolved with their efficient decoding in the brain: Behavioral and brain evidence. PLoS ONE, 4(5), e5625.CrossRefPubMedPubMedCentralGoogle Scholar
  61. Sikka, K., Ahmed, A., Diaz, D., Goodwin, M., Craig, K., Bartlett, M., & Huang, J. (2015). Automated assessment of children’s post-operative pain using computer vision. Journal of Pediatrics, 136(1), e124–e131.CrossRefGoogle Scholar
  62. Smith, M. L., Cottrell, G. W., Gosselin, F., & Schyns, P. G. (2005). Transmitting and decoding facial expressions. Psychological Science, 16(3), 184–189.CrossRefPubMedGoogle Scholar
  63. Strack, F., Martin, L. L., & Stepper, S. (1988). Inhibiting and facilitating conditions of the human smile: A nonobtrusive test of the facial feedback hypothesis. Journal of Personality and Social Psychology, 54(5), 768–777.CrossRefPubMedGoogle Scholar
  64. Tottenham, N., Tanaka, J. W., Leon, A. C., McCarry, T., Nurse, M., Hare, T. A., … Nelson, C. (2009). The NimStim set of facial expressions: Judgments from untrained research participants. Psychiatry Research, 168(3), 242–249.Google Scholar
  65. Wagenmakers, E.-J., Beek, T., Dijkhoff, L., Gronau, Q. F., Acosta, A., Adams, R. B., … Zwaan, R. A. (2016). Registered replication report Strack, Martin, & Stepper (1988). Perspectives on Psychological Science. doi:10.1177/1745691616674458
  66. Winkielman, P., Niedenthal, P., Wielgosz, J., Eelen, J., & Kavanagh, L. C. (2015). Embodiment of cognition and emotion. In M. Mikulincer, P. R. Shaver, E. Borgida, & J. A. Bargh (Eds.), APA handbook of personality and social psychology, Vol. 1. Attitudes and social cognition (Vol. 1, pp. 151–175). Washington, DC: American Psychological Association.CrossRefGoogle Scholar
  67. Wood, A., Lupyan, G., Sherrin, S., & Niednthal, P. (2015). Altering sensorimotor feedback disrupts visual discrimination of facial expressions. Psychonomic Bulletin and Review. doi:10.3758/s13423-015-0974-5 Google Scholar
  68. Wood, A., Rychlowska, M., Korb, S., & Niedenthal, P. (2016). Fashioning the face: Sensorimotor simulation contributes to facial expression recognition. Trends in Cognitive Science, 20(3), 227–240.CrossRefGoogle Scholar
  69. Zaki, J. (2013). Cue integration a common framework for social cognition and physical perception. Perspectives on Psychological Science, 8(3), 296–312.CrossRefPubMedGoogle Scholar
  70. Zanette, S., Gao, X., Brunet, M., Bartlett, M. S., & Lee, K. (2016). Automated decoding of facial expressions reveals marked differences in children when telling antisocial versus prosocial lies. Journal of Experimental Child Psychology, 150, 165–179.CrossRefPubMedGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2017

Authors and Affiliations

  • Joshua D. Davis
    • 1
  • Piotr Winkielman
    • 2
    • 3
    • 4
  • Seana Coulson
    • 1
  1. 1.Cognitive ScienceUniversity of California, San DiegoSan DiegoUSA
  2. 2.Department of PsychologyUniversity of California, San DiegoSan DiegoUSA
  3. 3.Behavioural Science Group, Warwick Business SchoolUniversity of WarwickCoventryUK
  4. 4.SWPS University of Social Sciences and HumanitiesWarsawPoland

Personalised recommendations