Journal of Nonverbal Behavior

, Volume 43, Issue 4, pp 493–511 | Cite as

The Role of Physiology and Voice in Emotion Perception During Social Stress

  • Nathaniel S. EcklandEmail author
  • Teresa M. Leyro
  • Wendy Berry Mendes
  • Renee J. Thompson
Original Paper


Deciphering others’ affect is ubiquitous in daily life and is important for navigating social interactions and relationships. Research has found that behavioral components, such as facial expressions or body language, are critical channels by which people understand other people’s affect. In the current research, we examined how people’s perceptions of targets’ positive affect (PA) and negative affect (NA) are associated with targets’ physiological reactivity, and whether behavioral indices mediate these associations. A total of 94 participants (i.e., observers) watched videos of targets completing a social stress task during which targets’ physiological reactivity [i.e., changes in respiratory sinus arrhythmia (RSA), cardiac output (CO), and ventricular contractility (VC)] was assessed. We predicted (1) targets’ RSA reactivity would be negatively associated with observers’ perceptions of PA and NA (to a lesser magnitude than PA); (2) targets’ CO reactivity would be positively associated with observers’ perceptions of PA and unrelated to perceptions of NA; and (3) targets’ VC would be positively associated perceptions of PA or NA (VC was an exploratory hypothesis). Our hypotheses were largely supported. Mediational analyses revealed that vocal prosody was a significant mediator of the association between perceptions of targets’ affect and their physiological reactivity. The findings suggest that observers can reliably detect targets’ emotional experiences as they manifest at a physiological level and that voice is an especially useful marker of how people perceive others’ affective experience. The findings have implications for aspects of relationships involving emotion perception, including affect contagion and interpersonal emotion regulation.


Emotion Emotion perception Social stress Autonomic nervous system 



This research was supported by the Sarlo-Ekman endowment to Wendy Berry Mendes. We thank the research assistants who worked on this project through the Emotion, Health, and Psychophysiology Lab at UCSF for their help in the collection of experimental stimuli

Compliance with Ethical Standards

Conflicts of interest

The authors declare that they have no conflicts of interest.

Human and Animal Rights

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.


  1. Akinola, M., & Mendes, W. B. (2008). The dark side of creativity: Biological vulnerability and negative emotions lead to greater artistic creativity. Personality and Social Psychology Bulletin, 34, 1677–1686.PubMedGoogle Scholar
  2. Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59(4), 390–412.Google Scholar
  3. Bagby, R. M., Parker, J. D., & Taylor, G. J. (1994). The twenty-item Toronto Alexithymia Scale. I. Item selection and cross-validation of the factor structure. Journal of Psychosomatic Research, 38(1), 23–32.PubMedGoogle Scholar
  4. Bänziger, T., Grandjean, D., & Scherer, K. R. (2009). Emotion recognition from expressions in face, voice, and body: The multimodal emotion recognition test (MERT). Emotion, 9(5), 691–704.PubMedGoogle Scholar
  5. Barrett, L. F., & Kensinger, E. A. (2010). Context is routinely encoded during emotion perception. Psychological Science, 21(4), 595–599.PubMedPubMedCentralGoogle Scholar
  6. Barrett, L. F., & Russell, J. A. (1999). The structure of current affect: Controversies and emerging consensus. Current Directions in Psychological Science, 8, 10–14.Google Scholar
  7. Baucom, B. R., Atkins, D. C., Eldridge, K., McFarland, P., Sevier, M., & Christensen, A. (2011). The language of demand/withdraw: Verbal and vocal expression in dyadic interactions. Journal of Family Psychology, 25(4), 570–580.PubMedPubMedCentralGoogle Scholar
  8. Bell, B. A., Ferron, J. M., & Kromrey, J. D. (2008). Cluster size in multilevel models: The impact of sparse data structures on point and interval estimates in two-level models. JSM Proceedings, Section on Survey Research Methods, pp 1122–1129. Google Scholar
  9. Bell, B. A., Morgan, G. B., Schoeneberger, J. A., Kromrey, J. D., & Ferron, J. M. (2014). How low can you go? An investigation of the influence of sample size and model complexity on point and interval estimates in two-level linear models. Methodology, 10, 1–11.Google Scholar
  10. Berntson, G. G., Cacioppo, J. T., & Quigley, K. S. (1993). Respiratory sinus arrhythmia: Autonomic origins, physiological mechanisms, and psychophysiological implications. Psychophysiology, 30(2), 183–196.PubMedGoogle Scholar
  11. Berntson, G. G., Thomas Bigger, J., Eckberg, D. L., Grossman, P., Kaufmann, P. G., Malik, M., et al. (1997). Heart rate variability: Origins, methods, and interpretive caveats. Psychophysiology, 34(6), 623–648.PubMedPubMedCentralGoogle Scholar
  12. Blascovich, J., & Mendes, W. B. (2010). Social psychophysiology and embodiment. In S. T. Fiske, D. T. Gilbert, & G. Lindzey (Eds.), The handbook of social psychology (5th ed., pp. 194–227). New York, NY: Wiley.Google Scholar
  13. Bliss-Moreau, E., Machado, C. J., & Amaral, D. G. (2013). Macaque cardiac physiology is sensitive to the valence of passively viewed sensory stimuli. PLoS ONE, 8(8), e71170.PubMedPubMedCentralGoogle Scholar
  14. Boersma, P., & Weenink, D. (2017). Praat: doing phonetics by computer [Computer program]. Version 6.0.21, retrieved 25 July 2017 from
  15. Campbell, J., & Ehlert, U. (2012). Acute psychosocial stress: Does the emotional stress response correspond with physiological responses? Psychoneuroendocrinology, 37(8), 1111–1134.PubMedGoogle Scholar
  16. Clore, G. L., Gasper, K., & Garvin, E. (2001). Affect as information. In J. P. Forgas (Ed.), Handbook of affect and social cognition (pp. 121–144). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  17. Cordaro, D. T., Sun, R., Keltner, D., Kamble, S., Huddar, N., & McNeil, G. (2018). Universals and cultural variations in 22 emotional expressions across five cultures. Emotion, 18(1), 75–93.PubMedGoogle Scholar
  18. Demaree, H. A., & Everhart, D. E. (2004). Healthy high-hostiles: Reduced parasympathetic activity and decreased sympathovagal flexibility during negative emotional processing. Personality and Individual Differences, 36(2), 457–469.Google Scholar
  19. Eckland, N. S., Leyro, T. M., Mendes, W. B., & Thompson, R. J. (2018). A multi-method investigation of the association between emotional clarity and empathy. Emotion, 18(5), 638–645.PubMedGoogle Scholar
  20. Edwards, L. J., Muller, K. E., Wolfinger, R. D., Qaqish, B. F., & Schabenberger, O. (2008). An R2 statistic for fixed effects in the linear mixed model. Statistics in Medicine, 27(29), 6137–6157.PubMedPubMedCentralGoogle Scholar
  21. Eisinga, R., Te Grotenhuis, M., & Pelzer, B. (2013). The reliability of a two-item scale: Pearson, cronbach, or spearman-brown? International Journal of Public Health, 58(4), 637–642.PubMedGoogle Scholar
  22. Ekman, P., Freisen, W. V., & Ancoli, S. (1980). Facial signs of emotional experience. Journal of Personality and Social Psychology, 39(6), 1125–1134.Google Scholar
  23. Ekman, P., Friesen, W. V., & Hager, J. C. (2002). Facial action coding system. Salt Lake City, UT: Network Information Research.Google Scholar
  24. Elfenbein, H. A., & Ambady, N. (2002). On the universality and cultural specificity of emotion recognition: A meta-analysis. Psychological Bulletin, 128, 203–235.PubMedGoogle Scholar
  25. Global Workplace Analytics & Flexjobs. (2017). The 2017 state of telecommuting in the U.S. employee workforce. Retrieved from
  26. Gros, D. F., Morland, L. A., Greene, C. J., Acierno, R., Strachan, M., Egede, L. E., et al. (2013). Delivery of evidence-based psychotherapy via video telehealth. Journal of Psychopathology and Behavioral Assessment, 35(4), 506–521.Google Scholar
  27. Gross, J. J., & John, O. P. (2003). Individual differences in two emotion regulation processes: Implications for affect, relationships, and well-being. Journal of Personality and Social Psychology, 85, 348–362.PubMedGoogle Scholar
  28. Hagan, M. J., Bush, N., Mendes, W. B., Arenander, J., Cohodes, E., Epel, E. S., et al. (2017). Childhood adversity is related to daily coping strategies among those who are sensitive to context. Anxiety Stress and Coping, 30, 163–175.Google Scholar
  29. Hayes, A. F. (2013). An introduction to mediation, moderation, and conditional process analysis. New York: The Guilford Press.Google Scholar
  30. Herrald, M. M., & Tomaka, J. (2002). Patterns of emotion-specific appraisal, coping, and cardiovascular reactivity during an ongoing emotional episode. Journal of Personality and Social Psychology, 83(2), 434–450.PubMedGoogle Scholar
  31. Hess, U. (2017). Body language. Encyclopedia of personality and individual differences. Berlin: Springer.Google Scholar
  32. Hess, U., Kafetsios, K., Mauersberger, H., Blaison, C., & Kessler, C. L. (2016). Signal and noise in the perception of facial emotion expressions: From labs to life. Personality and Social Psychology Bulletin, 42(8), 1092–1110.PubMedGoogle Scholar
  33. Human, L. J., & Mendes, W. B. (2018). Cardiac vagal flexibility and accurate personality impressions: Examining a physiological correlate of the good judge. Journal of Personality, 86, 1065–1077.PubMedGoogle Scholar
  34. Jameson, J. P., & Blank, M. B. (2007). The role of clinical psychology in rural mental health services: Defining problems and developing solutions. Clinical Psychology: Science and Practice, 14(3), 283–298.Google Scholar
  35. Jefferson, A. L. (2010). Cardiac output as a potential risk factor for abnormal brain aging. Journal of Alzheimer’s Disease, 20(3), 813–821.PubMedGoogle Scholar
  36. Jolliffe, D., & Farrington, D. P. (2006). Development and validation of the Basic Empathy Scale. Journal of Adolescence, 29(4), 589–611.PubMedGoogle Scholar
  37. Juslin, P. N., & Scherer, K. R. (2005). Vocal expression of affect. In J. Harrigan, R. Rosenthal, & K. Scherer (Eds.), The new handbook of methods in nonverbal behaviour research (pp. 65–135). Oxford: New York, NY.Google Scholar
  38. Kappas, A., Hess, U., & Scherer, K. (1991). Voice and emotion. In R. S. Feldman & B. Rimé (Eds.), Fundamentals of nonverbal behavior (pp. 200–238). New York, NY: Cambridge University Press.Google Scholar
  39. Kassam, K. S., Koslov, K., & Mendes, W. B. (2009). Decisions under distress: Stress profiles influence anchoring and adjustment. Psychological Science, 20(11), 1394–1399.PubMedPubMedCentralGoogle Scholar
  40. Keltner, D., & Cordaro, D. T. (2017). Understanding multimodal emotional expressions: Recent advances in basic emotion theory. In J. Russell & J. M. Fernandez-Dols (Eds.), Facial expression. New York: Oxford University Press.Google Scholar
  41. Keltner, D., & Gross, J. J. (1999). Functional accounts of emotions. Cognition and Emotion, 13(5), 467–480.Google Scholar
  42. Keltner, D., Tracy, J., Sauter, D., Cordaro, D., & McNeil, G. (2016). Emotional expression. In L. E. Barrett, M. Lewis, & J. Haviland-Jones (Eds.), Handbook of emotions (4th ed.). New York, NY: Guilford Press.Google Scholar
  43. Kirschbaum, C., Pirke, K. M., & Hellhammer, D. H. (1993). The ‘Trier Social Stress Test’—A tool for investigating psychobiological stress responses in a laboratory setting. Neuropsychobiology, 28(1–2), 76–81.PubMedGoogle Scholar
  44. Koslov, K., Mendes, W. B., Pajtas, P. E., & Pizzagalli, D. A. (2011). Asymmetry in resting intracortical activity as a buffer to social threat. Psychological Science, 22(5), 641–649.PubMedPubMedCentralGoogle Scholar
  45. Kubzansky, L. D., Mendes, W. B., Appleton, A. A., Block, J., & Adler, G. K. (2012). A heartfelt response: Oxytocin effects on response to social stress in men and women. Biological Psychology, 90(1), 1–9.PubMedPubMedCentralGoogle Scholar
  46. Larsen, R. J., & Diener, E. (1992). Promises and problems with the circumplex model of emotion. In M. S. Clark (Ed.), Emotion: The review of personality and social psychology (Vol. 13, pp. 25–59). Thousand Oaks, CA: Sage.Google Scholar
  47. Luo, W., & Kwok, O. M. (2009). The impacts of ignoring a crossed factor in analyzing cross-classified data. Multivariate Behavioral Research, 44(2), 182–212.PubMedGoogle Scholar
  48. Markus, H. R., & Kitayama, S. (1991). Culture and the self: Implications for cognition, emotion, and motivation. Psychological Review, 98, 224–253.Google Scholar
  49. Mauss, I. B., Cook, C. L., Cheng, J. Y., & Gross, J. J. (2007). Individual differences in cognitive reappraisal: Experiential and physiological responses to an anger provocation. International Journal of Psychophysiology, 66(2), 116–124.PubMedGoogle Scholar
  50. Mendes, W. B. (2016). Emotion and the autonomic nervous system. In L. E. Barrett, M. Lewis, & J. Haviland-Jones (Eds.), Handbook of emotions (4th ed.). New York, NY: Guilford Press.Google Scholar
  51. Mendes, W. B., Major, B., McCoy, S., & Blascovich, J. (2008). How attributional ambiguity shapes physiological and emotional responses to social rejection and acceptance. Journal of Personality and Social Psychology, 94(2), 278–291.PubMedPubMedCentralGoogle Scholar
  52. Meyers, J. L., & Beretvas, S. N. (2006). The impact of inappropriate modeling of cross-classified data structures. Multivariate Behavioral Research, 41(4), 473–497.PubMedGoogle Scholar
  53. Morland, L. A., Hynes, A. K., Mackintosh, M. A., Resick, P. A., & Chard, K. M. (2011). Group cognitive processing therapy delivered to veterans via telehealth: A pilot cohort. Journal of Traumatic Stress, 24(4), 465–469.PubMedGoogle Scholar
  54. Muhtadie, L., Koslov, K., Akinola, M., & Mendes, W. B. (2015). Vagal flexibility: A physiological predictor of social sensitivity. Journal of Personality and Social Psychology, 109(1), 106–120.PubMedGoogle Scholar
  55. Obradović, J., Bush, N. R., Stamperdahl, J., Adler, N. E., & Boyce, W. T. (2010). Biological sensitivity to context: The interactive effects of stress reactivity and family adversity on socioemotional behavior and school readiness. Child Development, 81(1), 270–289.PubMedPubMedCentralGoogle Scholar
  56. Porges, S. W. (2001). The polyvagal theory: Phylogenetic substrates of a social nervous system. International Journal of Psychophysiology, 42(2), 123–146.PubMedGoogle Scholar
  57. Porges, S. W., Doussard-Roosevelt, J. A., & Maiti, A. K. (1994). Vagal tone and the physiological regulation of emotion. In N. A. Fox (Ed.), Monographs of the society for research in child development (Vol. 59, pp. 167–186). Boston: Blackwell Publishing.Google Scholar
  58. Porges, S. W., Doussard-Roosevelt, J. A., Portales, A. L., & Greenspan, S. I. (1996). Infant regulation of the vagal “brake” predicts child behavior problems: A psychobiological model of social behavior. Developmental Psychobiology, 29(8), 697–712.PubMedGoogle Scholar
  59. Psychology Software Tools, Inc. [E-Prime 2.0]. (2012). Retrieved from
  60. Russell, J. A., Bachorowski, J. A., & Fernández-Dols, J. M. (2003). Facial and vocal expressions of emotion. Annual Review of Psychology, 54(1), 329–349.PubMedGoogle Scholar
  61. Salovey, P., Mayer, J. D., Goldman, S. L., Turvey, C., & Palfai, T. P. (1995). Emotional attention, clarity, and repair: Exploring emotional intelligence using the Trait Meta-Mood Scale. In J. W. Pennebaker (Ed.), Emotion, disclosure and health (pp. 125–154). Washington, DC: American Psychological Association.Google Scholar
  62. Seery, M. D. (2013). The biopsychosocial model of challenge and threat: Using the heart to measure the mind. Social and Personality Psychology Compass, 7, 637–653.Google Scholar
  63. Smith, A. (2006). Cognitive empathy and emotional empathy in human behavior and evolution. The Psychological Record, 56(3), 3–21.Google Scholar
  64. Storbeck, J., & Clore, G. L. (2008). Affective arousal as information: How affective arousal influences judgments, learning, and memory. Social and Personality Psychology Compass, 2(5), 1824–1843.PubMedPubMedCentralGoogle Scholar
  65. Thayer, J. F., Åhs, F., Fredrikson, M., Sollers, J. J., III, & Wager, T. D. (2012). A meta-analysis of heart rate variability and neuroimaging studies: Implications for heart rate variability as a marker of stress and health. Neuroscience and Biobehavioral Reviews, 36(2), 747–756.PubMedGoogle Scholar
  66. Waters, S. F., West, T. V., & Mendes, W. B. (2014). Stress contagion: Physiological covariation between mothers and infants. Psychological Science, 25(4), 934–942.PubMedPubMedCentralGoogle Scholar
  67. Watson, D., Weber, K., Assenheimer, J. S., & Clark, L. A. (1995). Testing a tripartite model: I. Evaluating the convergent and discriminant validity of anxiety and depression scales. Journal of Abnormal Psychology, 104, 3–14.PubMedGoogle Scholar
  68. Weisbuch, M., Seery, M. D., Ambady, N., & Blascovich, J. (2009). On the correspondence between physiological and nonverbal responses: Nonverbal behavior accompanying challenge and threat. Journal of Nonverbal Behavior, 33(2), 141–148.Google Scholar
  69. Weusthoff, S., Baucom, B. R., & Hahlweg, K. (2013). The siren song of vocal fundamental frequency for romantic relationships. Frontiers in Psychology, 4, 439.PubMedPubMedCentralGoogle Scholar
  70. Zaki, J., Bolger, N., & Ochsner, K. (2008). It takes two the interpersonal nature of empathic accuracy. Psychological Science, 19(4), 399–404.PubMedGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of PsychologyUniversity of Illinois at Urbana-ChampaignChampaignUSA
  2. 2.Department of PsychologyRutgers University, The State University of New JerseyNew BrunswickUSA
  3. 3.Department of PsychiatryUniversity of California, San FranciscoSan FranciscoUSA
  4. 4.Department of Psychological and Brain SciencesWashington University in St. LouisSt. LouisUSA

Personalised recommendations