Journal of Nonverbal Behavior

, Volume 29, Issue 4, pp 193–215 | Cite as

Prosody–face Interactions in Emotional Processing as Revealed by the Facial Affect Decision Task

  • Marc D. Pell


Previous research employing the facial affect decision task (FADT) indicates that when listeners are exposed to semantically anomalous utterances produced in different emotional tones (prosody), the emotional meaning of the prosody primes decisions about an emotionally congruent rather than incongruent facial expression (Pell, M. D., Journal of Nonverbal Behavior, 29, 45–73). This study undertook further development of the FADT by investigating the approximate timecourse of prosody–face interactions in nonverbal emotion processing. Participants executed facial affect decisions about happy and sad face targets after listening to utterance fragments produced in an emotionally related, unrelated, or neutral prosody, cut to 300, 600, or 1000 ms in duration. Results underscored that prosodic information enduring at least 600 ms was necessary to presumably activate shared emotion knowledge responsible for prosody–face congruity effects.


emotion processing vocal expression facial expression speech communication affective priming social cognition 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Adolphs, R., Damasio, H., Tranel, D., Damasio, A. R. 1996Cortical systems for the recognition of emotion in facial expressionsThe Journal of Neuroscience1676787687PubMedGoogle Scholar
  2. Attardo, S., Eisterhold, J., Hay, J., Poggi, I. 2003Multimodal markers of irony and sarcasmHumor16243260CrossRefGoogle Scholar
  3. Banse, R., Scherer, K. R. 1996Acoustic profiles in vocal emotion expressionJournal of Personality and Social Psychology70614636CrossRefPubMedGoogle Scholar
  4. Bargh, J., Chaiken, S., Govender, R., Pratto, F. 1992The generality of the automatic attitude activation effectJournal of Personality and Social Psychology62893912CrossRefPubMedGoogle Scholar
  5. Borod, J. 2000Relationships among facial, prosodic, and lexical channels of emotional perceptual processingCognition and Emotion14193211CrossRefGoogle Scholar
  6. Bower, G. H. 1981Mood and memoryAmerican Psychologist36129148PubMedGoogle Scholar
  7. Bowers, D., Bower, R., Heilman, K. 1993The nonverbal affect lexicon: Theoretical perspectives from neuropsychological studies of affect perceptionNeuropsychology7433444CrossRefGoogle Scholar
  8. Calder, A., Keane, J., Manly, T., Sprengelmeyer, R., Scott, S., Nimmo-Smith, I., Young, A. 2003Facial expression recognition across the adult life spanNeuropsychologia41195202CrossRefPubMedGoogle Scholar
  9. Gelder, B., Bocker, K. B. E., Tuomainen, J., Hensen, M., Vroomen, J. 1999The combined perception of emotion from voice and face: Early interaction revealed by human brain responsesNeuroscience Letters260133136PubMedGoogle Scholar
  10. Gelder, B., Vroomen, J. 2000The perception of emotions by ear and by eyeCognition and Emotion14289311Google Scholar
  11. Dimberg, U. 1991Emotional reactions to facial expressions: A case of automatic responding?Psychophysiology28S19Google Scholar
  12. Dimberg, U., Öhman, A. 1996Beyond the wrath: Psychophysiological responses to facial stimuliMotivation and Emotion20149182CrossRefGoogle Scholar
  13. Etcoff, N. L., Magee, J. L. 1992Categorical perception of facial expressionsCognition44227240CrossRefPubMedGoogle Scholar
  14. Fazio, R. H., Sanbonmatsu, D. M., Powell, M. C., Kardes, F. R. 1986On the automatic activation of attitudesJournal of Personality and Social Psychology50229238CrossRefPubMedGoogle Scholar
  15. Halberstadt, J., Niedenthal, P. 2001Effects of emotion concepts on perceptual memory for emotional expressionsJournal of Personality and Social Psychology81587598CrossRefPubMedGoogle Scholar
  16. Hansen, C. H., Shantz, C. A. 1995Emotion-specific priming: Congruence effects on affect and recognition across negative emotionsJournal of Personality and Social Psychology21548557Google Scholar
  17. Hermans, D., Houwer, J., Eelen, P. 1994The affective priming effect: Automatic activation of evaluative information in memoryCognition and Emotion8515533Google Scholar
  18. Innes-Ker, A., Niedenthal, P. 2002Emotion concepts and emotional states in social judgment and categorizationJournal of Personality and Social Psychology83804816CrossRefPubMedGoogle Scholar
  19. Juslin, P., Laukka, P. 2003Communication of emotions in vocal expression and music performance: Different channels, same code?Psychological Bulletin129770814CrossRefPubMedGoogle Scholar
  20. Kirouac, G., Dore, F. 1984Accuracy and latency of judgment of facial expressions of emotionPerceptual Motor Skills57683686Google Scholar
  21. Kitayama, S., Ishii, K. 2002Word and voice: Spontaneous attention to emotional utterances in two languagesCognition and Emotion162959CrossRefGoogle Scholar
  22. Klauer, K. C., Musch, J. 2002Goal dependent and goal independent effects of irrelevant evaluationsPersonality and Social Psychology Bulletin28802814Google Scholar
  23. Klauer, K., Musch, J. 2003Affective priming: Findings and theoriesMusch, J.Klauer, K. eds. The psychology of evaluation: Affective processes in cognition and emotionErlbaumLondon749Google Scholar
  24. Kutas, M., Federmeier, K. D. 2000Electrophysiology reveals semantic memory use in language comprehensionTrends in Cognitive Sciences4463470CrossRefPubMedGoogle Scholar
  25. Ladd, D. R., Silverman, K., Tolkmitt, F., Bergmann, G., Scherer, K. 1985Evidence for the independent function of intonation contour type, voice quality, and F0 range in signaling speaker affectJournal of the Acoustical Society of America78435444CrossRefGoogle Scholar
  26. Massaro, D., Egan, P. 1996Perceiving affect from the voice and the facePsychonomic Bulletin and Review3215221Google Scholar
  27. Mathews, A., Mackintosh, B. 1998A cognitive model of selective processing in anxietyCognitive Therapy and Research22539560CrossRefGoogle Scholar
  28. Mogg, K., Bradley, B. P. 1999Orienting of attention to threatening facial expressions presented under conditions of restricted awarenessCognition and Emotion13713740CrossRefGoogle Scholar
  29. Musch, J., Klauer, K. C. 2001Locational uncertainty moderates affective congruency effects in the evaluative decision taskCognition and Emotion15167188CrossRefGoogle Scholar
  30. Niedenthal, P. M., Halberstadt, J. B. 1995The acquisition and structure of emotional response categoriesThe Psychology of Learning and Motivation332364Google Scholar
  31. Niedenthal, P., Halberstadt, J. 1999Emotional response categorizationPsychological Review106337361CrossRefGoogle Scholar
  32. Niedenthal, P. M., Halberstadt, J. B., Setterlund, M. B. 1997Being happy and seeing “happy”: Emotional state mediates visual word recognitionCognition and Emotion11403432CrossRefGoogle Scholar
  33. Öhman, A. 1987The psychophysiology of emotion: An evolutionary-cognitive perspectiveAdvances in Psychophysiology279127Google Scholar
  34. Pell, M. D. 1998Recognition of prosody following unilateral brain lesion: Influence of functional and structural attributes of prosodic contoursNeuropsychologia36701715CrossRefPubMedGoogle Scholar
  35. Pell, M. D. 2001Influence of emotion and focus location on prosody in matched statements and questionsJournal of the Acoustical Society of America10916681680CrossRefPubMedGoogle Scholar
  36. Pell, M. D. 2002Evaluation of nonverbal emotion in face and voice: Some preliminary findings on a new battery of testsBrain and Cognition48499504PubMedGoogle Scholar
  37. Pell, M. D. 2004A method for on-line evaluation of emotional prosody in healthy and disordered populationsBrain and Language912526CrossRefGoogle Scholar
  38. Pell, M. D. 2005Nonverbal emotion priming: Evidence from the ‘Facial Affect Decision Task’Journal of Nonverbal Behavior294573CrossRefGoogle Scholar
  39. Pell, M. D., Leonard, C. 2003Processing emotional tone from speech in Parkinson’s disease: A role for the basal gangliaCognitive, Affective, & Behavioral Neuroscience3275288Google Scholar
  40. Pell, M. D., Leonard, C. L. 2005Facial expression decoding in early Parkinson’s diseaseCognitive Brain Research23327340CrossRefPubMedGoogle Scholar
  41. Pourtois, G., Gelder, B., Vroomen, J., Rossion, B., Crommelinck, M. 2000The time-course of intermodal binding between seeing and hearing affective informationNeuroreport1113291333PubMedGoogle Scholar
  42. Pourtois, G., Debatisse, D., Despland, P.-A., Gelder, B. 2002Facial expressions modulate the time course of long latency auditory brain potentialsCognitive Brain Research1499105CrossRefPubMedGoogle Scholar
  43. Pratto, F., John, O. P. 1991Automatic vigilance: The attention-grabbing power of negative social informationJournal of Personality and Social Psychology61380391CrossRefPubMedGoogle Scholar
  44. Rosnow, R., Rosenthal, R. 2003Effect sizes for experimenting psychologistsCanadian Journal of Experimental Psychology57221237PubMedGoogle Scholar
  45. Scherer, K. 1986Vocal affect expression: A review and a model for future researchPsychological Bulletin99143165CrossRefPubMedGoogle Scholar
  46. Schirmer, A., Kotz, S. 2003ERP evidence for a sex-specific stroop effect in emotional speechJournal of Cognitive Neuroscience1511351148CrossRefPubMedGoogle Scholar
  47. Schirmer, A., Zysset, S., Kotz, S., Cramon, D. Y. 2004Gender differences in the activation of inferior frontal cortex during emotional speech perceptionNeuroImage2111141123CrossRefPubMedGoogle Scholar
  48. Swinney, D. 1979Lexical access during sentence comprehension: (Re)consideration of context effectsJournal of Verbal Learning and Verbal Behavior18645659Google Scholar
  49. Honk, J., Tuiten, A., Haan, E., Hout, M., Stam, H. 2001Attentional biases for angry faces: Relationships to trait anger and anxietyCognition and Emotion15279297Google Scholar
  50. Vroomen, J., Driver, J., Gelder, B. 2001Is cross-modal integration of emotional expressions independent of attentional resourcesCognitive, Affective, & Behavioral Neuroscience1382387CrossRefGoogle Scholar
  51. Wallbott, H. G. 1988Big girls don’t frown, big boys don’t cry: Gender differences of professional actors in communicating emotion via facial expressionJournal of Nonverbal Behavior1298106CrossRefGoogle Scholar
  52. Wentura, D. 2000Dissociative affective and associative priming effects in the lexical decision task: Yes versus no responses to word targets reveal evaluative judgement tendenciesJournal of Experimental Psychology: Learning, Memory, and Cognition26456469CrossRefPubMedGoogle Scholar
  53. Williams, C., Stevens, K. 1981Vocal correlates of emotion statesDarby, J. K. eds. Speech evaluation in psychiatryGrune & StrattonNew York221240Google Scholar
  54. Wurm, L. H., Vakoch, D. A., Strasser, M. R., Calin-Jageman, R., Ross, S. E. 2001Speech perception and vocal expression of emotionCognition and Emotion15831852CrossRefGoogle Scholar
  55. Young, A., Rowland, D., Calder, A., Etcoff, N., Seth, A., Perrett, D. 1997Facial expression megamix: Tests of dimensional and category accounts of emotion recognitionCognition63271313CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media, Inc. 2005

Authors and Affiliations

  1. 1.School of Communication Sciences and DisordersMcGill UniversityMontréalCanada

Personalised recommendations