Abstract
The aim of the current study was to examine facial and body posture emotion recognition among deaf and hard-of-hearing (DHH) and hearing young adults. Participants were (N = 126) DHH (n = 48) and hearing (n = 78) college students who completed two emotion recognition tasks in which they were shown photographs of faces and body postures displaying different emotions of both high and low intensities and had to infer the emotion being displayed. Compared to hearing participants, DHH participants performed worse on the body postures emotion task for both high and low intensities. They also performed more poorly on the facial emotion task, but only for low-intensity emotional facial expressions. On both tasks, DHH participants whose primary mode of communication was Signed English performed significantly more poorly than those whose primary mode was American Sign Language (ASL) or spoken English. Moreover, DHH participants who communicated using ASL performed similarly to hearing participants. This suggests that difficulties in affect recognition among DHH individuals occur when processing both facial and body postures that are more subtle and reflective of real-life displays of emotion. Importantly, this also suggests that ASL as a primary form of communication in this population may serve as a protective factor against emotion recognition difficulties, which could, in part, be due to the complex nature of this language and its requirement to perceive meaning through facial and postural expressions with a wide visual lens.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
The data that support the findings of this study are available from the corresponding author upon reasonable request.
Notes
The distribution of the scores was negatively skewed. This is not surprising, as the majority of scores fell within the higher range. To correct for this, we performed a log10 transformation that included a reflection of the data prior to the log10 analyses. We calculated the ANOVAs prior to transforming and then recalculated the ANOVAs using the log10 values. Results were consistent across the transformed data and the original data, so we have reported ANOVAs with the original data.
References
Adolphs, R., Gosselin, F., Buchanan, T. W., Tranel, D., Schyns, P., & Damasio, A. R. (2005). A mechanism for impaired fear recognition after amygdala damage. Nature, 433(7021), 68–72. https://doi.org/10.1038/nature03086.
Aleman, A., & Kahn, R. S. (2005). Strange feelings: Do amygdala abnormalities dysregulate the emotional brain in schizophrenia? Progress in Neurobiology, 77(5), 283–298. https://doi.org/10.1016/j.pneurobio.2005.11.005.
Ambadar, Z., Schooler, J. W., & Cohn, J. F. (2005). Deciphering the enigmatic face the importance of facial dynamics in interpreting subtle facial expressions. Psychological Science, 16(5), 403–410. https://doi.org/10.1111/j.0956-7976.2005.01548.x.
Atkinson, A., Dittrich, W. H., Gemmell, A. J., & Young, A. W. (2004). Emotion perception from dynamic and static body expressions in point-light and full-light displays. Perception, 33(6), 717–746. https://doi.org/10.1068/p5096.
Aviezer, H., Hassin, R. R., Ryan, J., Grady, C., Susskind, J., Anderson, A., Moscovitch, M., & Bentin, S. (2008). Angry, disgusted, or afraid? Studies on the malleability of emotion perception. Psychological Science, 19(7), 724–732. https://doi.org/10.1111/j.1467-9280.2008.02148.x.
Aviezer, H., Trope, Y., & Todorov, A. (2012). Body cues, not facial expressions, discriminate between intense positive and negative emotions. Science, 338(6111), 1225–1229. https://doi.org/10.1126/science.1224313.
Blose, B. A., & Schenkel, L. S. (2022). Theory of mind and alexithymia in deaf and hard-of-hearing young adults. Journal of Deaf Studies and Deaf Education, 27(2), 179–192. https://doi.org/10.1093/deafed/enac001.
Booth, A. J., Rodgers, J. D., Volker, M. A., Lopata, C., & Thomeer, M. L. (2019). Psychometric characteristics of the DANVA-2 in high-functioning children with ASD. Journal of Autism and Developmental Disorders, 49(10), 4147–4158. https://doi.org/10.1007/s10803-019-04130-w.
Bornstein, H., Saulnier, K. L., Hamilton, L. B., & Miller, R. R. (1983). The comprehensive Signed English dictionary. Gallaudet University Press.
Bould, E., & Morris, N. (2008). Role of motion signals in recognizing subtle facial expressions of emotion. British Journal of Psychology, 99(2), 167–189. https://doi.org/10.1348/000712607X206702.
Brackett, M. A., Rivers, S. E., Shiffman, S., Lerner, N., & Salovey, P. (2006). Relating emotional abilities to social functioning: A comparison of self-report and performance measures of emotional intelligence. Journal of Personality and Social Psychology, 91(4), 780–795. https://doi.org/10.1037/0022-3514.91.4.780.
Calder, A. J., Rowland, D., Young, A. W., Nimmo-Smith, I., Keane, J., & Perrett, D. I. (2000). Caricaturing facial expressions. Cognition, 76(2), 105–146. https://doi.org/10.1016/S0010-0277(00)00074-3.
Calder, A. J., Young, A. W., Perrett, D. I., Etcoff, N. L., & Rowland, D. (1996). Categorical perception of morphed facial expressions. Visual Cognition, 3(2), 81–118. https://doi.org/10.1080/713756735.
Claudino, R. G., da e, I. F., Nascimento, I. N., d., A., & Torro, N. (2020). Use of sign language does not favor recognition of static and dynamic emotional faces in deaf people. Psychology & Neuroscience, 13(4), 531–538. https://doi.org/10.1037/pne0000238.
Corina, D. P., Bellugi, U., & Reilly, J. (1999). Neuropsychological studies of linguistic and affective facial expressions in deaf signers. Language and Speech, 42(2–3), 307–331. https://doi.org/10.1177/00238309990420020801.
Coulson, M. (2004). Attributing emotion to static body postures: Recognition accuracy, confusions, and viewpoint dependence. Journal of Nonverbal Behavior, 28(2), 117–139. https://doi.org/10.1023/B:JONB.0000023655.255550.be.
Dael, N., Mortillaro, M., & Scherer, K. R. (2012). Emotion expression in body action and posture. Emotion, 12(5), 1085–1101. https://doi.org/10.1037/a0025737.
Daros, A. R., Zakzanis, K. K., & Ruocco, A. C. (2013). Facial emotion recognition in borderline personality disorder. Psychological Medicine, 43(9), 1953–1963. https://doi.org/10.1017/S0033291712002607.
de Gelder, B., & Vroomen, J. (2000). The perception of emotions by ear and by eye. Cognition and Emotion, 14(3), 289–311. https://doi.org/10.1080/026999300378824.
Denham, S. A., Blair, K. A., DeMulder, E., Levitas, J., Sawyer, K., Auerbach-Major, S., & Queenan, P. (2003). Preschool emotional competence: Pathway to social competence. Child Development, 74(1), 238–256. https://doi.org/10.1111/1467-8624.00533.
Dobel, C., Nestler-Collatz, B., Guntinas-Lichius, O., Schweinberger, S. R., & Zäske, R. (2020). Deaf signers outperform hearing non-signers in recognizing happy facial expressions. Psychological Research Psychologische Forschung, 84(6), 1485–1494. https://doi.org/10.1007/s00426-019-01160-y.
Dyck, M. J., & Denver, E. (2003). Can the emotion recognition ability of deaf children be enhanced? A pilot study. Journal of Deaf Studies and Deaf Education, 8(3), 348–356. https://doi.org/10.1093/deafed/eng019.
Dyck, M. J., Farrugia, C., Shochet, I. M., & Holmes-Brown, M. (2004). Emotion recognition/understanding ability in hearing or vision-impaired children: Do sounds, sights, or words make the difference? Journal of Child Psychology and Psychiatry, 45(4), 789–800. https://doi.org/10.1111/j.1469-7610.2004.00272.x.
Eisenberg, N., Fabes, R. A., Guthrie, I. K., & Reiser, M. (2000). Dispositional emotionality and regulation: Their role in predicting quality of social functioning. Journal of Personality and Social Psychology, 78(1), 136–157. https://doi.org/10.1037//0022-3514.78.1.136.
Fellinger, J., Holzinger, D., & Pollard, R. (2012). Mental health of deaf people. The Lancet, 379(9820), 1037–1044. https://doi.org/10.1016/S0140-6736(11)61143-4.
Ferrari, C., Papagno, C., Todorov, A., & Cattaneo, Z. (2019). Differences in emotion recognition from body and face cues between deaf and hearing individuals. Multisensory Research, 32(6), 499–519. https://doi.org/10.1163/22134808-20191353.
Gao, X., & Maurer, D. (2010). A happy story: Developmental changes in children’s sensitivity to facial expressions of varying intensities. Journal of Experimental Child Psychology, 107(2), 67–86. https://doi.org/10.1016/j.jecp.2010.05.003.
Gross, J. J. (1998). The emerging field of emotion regulation: An integrative review. Review of General Psychology, 2(3), 271–299. https://doi.org/10.1037/1089-2680.2.3.271.
Gu, H., Chen, Q., Xing, X., Zhao, J., & Li, X. (2019). Facial emotion recognition in deaf children: Evidence from event-related potentials and event-related spectral perturbation analysis. Neuroscience Letters, 703, 198–204. https://doi.org/10.1016/j.neulet.2019.01.032.
Hall, W. C., Smith, S. R., Sutter, E. J., DeWindt, L. A., & Dye, T. (2018). Considering parental hearing status as a social determinant of deaf population health: Insights from experiences of the dinner table syndrome. PLOS One, 13(9), e0202169. https://doi.org/10.1371/journal.pone.0202169.
Hauser, P., O’Hearn, A., McKee, M., Steider, A., & Thew, D. (2010). Deaf epistemology: Deafhood and deafness. American Annals of the Deaf, 154(5), 486–492. https://doi.org/10.1353/aad.0.0120.
He, H., Xu, B., & Tanaka, J. (2016). Investigating the face inversion effect in a deaf population using the dimensions tasks. Visual Cognition, 24(3), 201–211. https://doi.org/10.1080/13506285.2016.1221488.
Herba, C. M., Landau, S., Russell, T., Ecker, C., & Phillips, M. L. (2006). The development of emotion-processing in children: Effects of age, emotion, and intensity. Journal of Child Psychology and Psychiatry, 47(11), 1098–1106. https://doi.org/10.1111/j.1469-7610.2006.01652.x.
Jones, A. C., Gutierrez, R., & Ludlow, A. K. (2018). The role of motion and intensity in deaf children’s recognition of real human facial expressions of emotion. Cognition and Emotion, 32(1), 102–115. https://doi.org/10.1080/02699931.2017.1289894.
Klasen, M., Kreifelts, B., Chen, Y., Seubert, J., & Mathiak, K. (2014). Neural processing of emotion in multimodal settings. Frontiers in Human Neuroscience, 8(822), 1–4. https://doi.org/10.3389/fnhum.2014.00822.
Krejtz, I., Krejtz, K., Wisiecka, K., Abramczyk, M., Olszanowski, M., & Duchowski, A. T. (2020). Attention dynamics during emotion recognition by deaf and hearing individuals. Journal of Deaf Studies and Deaf Education, 25(10), 10–21. https://doi.org/10.1093/deafed/enz036.
Kuchuk, A., Vibbert, M., & Bornstein, M. H. (1986). The perception of smiling and its experiential correlates in three-month-old infants. Child Development, 57(4), 1054–1061. https://doi.org/10.2307/1130379.
Letourneau, S. M., & Mitchell, T. V. (2011). Gaze patterns during identity and emotion judgments in hearing adults and deaf users of American sign Language. Perception, 40(5), 563–575. https://doi.org/10.1068/p6858.
Marschark, M., Edwards, L., Peterson, C., Crowe, K., & Walton, D. (2019). Understanding theory of mind in deaf and hearing college students. Journal of Deaf Studies and Deaf Education, 24(2), 104–118. https://doi.org/10.1093/deafed/eny039.
Martins, A. T., Faísca, L., Vieira, & Gonçalves, G. (2019). Emotional recognition and empathy in both deaf and blind adults. Journal of Deaf Studies and Deaf Education, 24(2), 119–127. https://doi.org/10.1093/deafed/eny046.
McCullough, S., Emmorey, K., & Sereno, M. (2005). Neural organization for recognition of grammatical and emotional facial expressions in deaf ASL signers and hearing nonsigners. Cognitive Brain Research, 22(2), 193–203. https://doi.org/10.1016/j.cogbrainres.2004.08.012.
Montagne, B., Kessels, R. P., de Haan, E. H., & Perrett, D. I. (2007). The emotion recognition task: A paradigm to measure the perception of facial emotional expressions at different intensities. Perceptual and Motor Skills, 104(2), 589–598. https://doi.org/10.2466/pms.104.2.589-598.
Montepare, J., Koff, E., Zaitchik, D., & Albert, M. (1999). The use of body movements and gestures as cues to emotions in younger and older adults. Journal of Nonverbal Behavior, 23(2), 133–152. https://doi.org/10.1023/A:1021435526134.
Montirosso, R., Peverelli, M., Frigerio, E., Crespi, M., & Borgatti, R. (2010). The development of dynamic facial expression recognition at different intensities in 4- to 18-year-olds. Social Development, 19(1), 71–92. https://doi.org/10.1111/j.1467-9567-9507.2008.00527.x.
Most, T., & Aviner, C. (2009). Auditory, visual, and auditory-visual perception of emotions by individuals with cochlear implants, hearing aids, and normal hearing. Journal of Deaf Studies and Deaf Education, 14(4), 449–464. https://doi.org/10.1093/deafed/enp007.
Motley, M. T., & Camden, C. T. (1988). Facial expression of emotion: A comparison of posed expressions versus spontaneous expressions in an interpersonal communication setting. Western Journal of Speech Communication, 52(1), 1–22. https://doi.org/10.1080/10570318809389622.
Nelson, C. A. (1987). The recognition of facial expressions in the first two years of life: Mechanisms of development. Child Development, 58(4), 889–909. https://doi.org/10.2307/1130530.
Nowicki, S., & Carton, J. (1993). The measurement of emotional intensity from facial expressions. The Journal of Social Psychology, 133(5), 749–750. https://doi.org/10.1080/00224545.1993.9713934.
Nowicki, S., & Duke, M. (1994). Individual differences in the nonverbal communication of affect: The diagnostic analysis of nonverbal accuracy scale. Journal of Nonverbal Behavior, 18(1), 9–35. https://doi.org/10.1007/BF02169077.
Peñacoba, C., Garvi, D., Gómez, L., & Álvarez, A. (2020). Emotional functioning, positive relationships, and language use in deaf adults. Journal of Deaf Studies and Deaf Education, 25(1), 22–32. https://doi.org/10.1093/deafed/enz034.
Peterson, C., & Siegal, M. (1995). Deafness, conversation, and theory of mind. Journal of Child Psychology and Psychiatry, 36(3), 459–474. https://doi.org/10.1111/j.1469-7610.1995.tb01303.x.
Peterson, C., & Siegal, M. (1998). Changing focus on the representational mind: Deaf, autistic, and normal children’s concepts of false drawings and false beliefs. British Journal of Developmental Psychology, 16(3), 301–320. https://doi.org/10.1111/j.2044-835X.1998.tb00754.x.
Peterson, C., & Siegal, M. (1999). Representing inner worlds: Theory of mind in autistic, deaf, and normal hearing children. Psychological Science, 10(2), 126–129. https://doi.org/10.1111/1467-9280.00119.
Pitterman, H., & Nowicki, S. (2004). A test of the ability to identify emotion in human standing and sitting postures: The diagnostic analysis of Nonverbal Accuracy-2 posture test (DANVA2-POS). Genetic Social and General Psychology Monographs, 130(2), 146–162. https://doi.org/10.3200/MONO.130.2.146-162.
Pollick, F. E., Paterson, H. M., Bruderlin, A., & Sanford, A. J. (2001). Perceiving affect from arm movement. Cognition, 82(2), 51–61. https://doi.org/10.1016/S0010-0277(01)00147-0.
Rodger, H., Lao, J., Stoll, C., Richoz, A. R., Pascalis, O., Dye, M., & Caldara, R. (2021). The recognition of facial expressions of emotion in deaf and hearing individuals. Heliyon, 7(5), e07018. https://doi.org/10.1016/j.heliyon.2021.e07018.
Schenkel, L. S., & Towne, T. (2020). Errors in identifying emotion in body postures and facial expressions among pediatric patients with bipolar disorder. Journal of Clinical and Experimental Neuropsychology, 42(7), 735–746. https://doi.org/10.1080/13803395.2020.1799946.
Scherer, K. R., & Ellgring, H. (2007). Multimodal expression of emotion: Affect programs or componential appraisal patterns? Emotion, 7(1), 158–171. https://doi.org/10.1037/1528-3542.7.1.158.
Schirmer, A., & Adolphs, R. (2017). Emotion perception from face, voice, and touch: Comparisons and convergence. Trends in Cognitive Science, 21(3), 216–228. https://doi.org/10.1016/j.tics.2017.01.001.
Shipley, W. C., Gruber, C. P., Martin, T. A., & Klein, A. M. (2009). Shipley-2 manual. Western Psychological Services.
Sidera, F., Amadó, A., & Martínez, L. (2017). Influences on facial emotion recognition in deaf children. Journal of Deaf Studies and Deaf Education, 22(2), 164–177. https://doi.org/10.1093/deafed/enw072.
Sifneos, P. E. (1996). Alexithymia: Past and present. The American Journal of Psychiatry, 153(7), 137–142. https://doi.org/10.1176/ajp.153.7.137.
Stoll, C., Rodger, H., Lao, J., Richoz, A. R., Pascalis, O., Dye, M., & Caldara, R. (2019). Quantifying facial expression intensity and signal use in deaf signers. Journal of Deaf Studies and Deaf Education, 24(4), 346–355. https://doi.org/10.1093/deafed/enz023.
Sundqvist, A., Lyxell, B., Jönsson, R., & Heimann, M. (2014). Understanding minds: Early cochlear implantation and the development of theory of mind in children with profound hearing impairment. International Journal of Pediatric Otorhinolaryngology, 78(3), 537–543. https://doi.org/10.1016/j.ijporl.2013.12.039.
Takagi, S., Hiramatsu, S., Tabei, K., & Tanaka, A. (2015). Multisensory perception of the six basic emotions is modulated by attentional instruction and unattended modality. Frontiers in Integrative Neuroscience, 9(1), 1–10. https://doi.org/10.3389/fnint.2015.00001.
Uljarevic, M., & Hamilton, A. (2013). Recognition of emotions in autism: A formal meta-analysis. Journal of Autism and Developmental Disorders, 43(7), 1517–1526. https://doi.org/10.1007/s10803-012-1695-5.
Wehrle, T., Kaiser, S., Schmidt, S., & Scherer, K. R. (2000). Studying the dynamics of emotional expressions using synthesized facial muscle movements. Journal of Personality and Social Psychology, 78(1), 105–119. https://doi.org/10.1037//0022-3514.78.1.105.
Widen, S. C., & Russell, J. A. (2015). Do dynamic facial expressions convey emotions to children better than do static ones? Journal of Cognition and Development, 16(5), 802–811. https://doi.org/10.1080/15248372.2014.916295.
Funding
The authors did not receive funding from any organization for the submitted work.
Author information
Authors and Affiliations
Contributions
L.S. contributed to the study conception, design, data collection, analysis, and manuscript editing. B.B. contributed to the data analysis and manuscript writing. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethical Approval
The authors confirm that the study was approved by the Institutional Review Board and certify that it was performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.
Consent to Participate
Informed consent was obtained from all individual participants included in the study.
Competing Interests
The authors have no relevant financial or non-financial interests to disclose.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Blose, B.A., Schenkel, L.S. Facial and Body Posture Emotion Identification in Deaf and Hard-of-Hearing Young Adults. J Nonverbal Behav (2024). https://doi.org/10.1007/s10919-024-00458-9
Accepted:
Published:
DOI: https://doi.org/10.1007/s10919-024-00458-9