Skip to main content
SpringerLink
Log in

Prosody–face Interactions in Emotional Processing as Revealed by the Facial Affect Decision Task

  • Published:
Journal of Nonverbal Behavior Aims and scope Submit manuscript

Abstract

Previous research employing the facial affect decision task (FADT) indicates that when listeners are exposed to semantically anomalous utterances produced in different emotional tones (prosody), the emotional meaning of the prosody primes decisions about an emotionally congruent rather than incongruent facial expression (Pell, M. D., Journal of Nonverbal Behavior, 29, 45–73). This study undertook further development of the FADT by investigating the approximate timecourse of prosody–face interactions in nonverbal emotion processing. Participants executed facial affect decisions about happy and sad face targets after listening to utterance fragments produced in an emotionally related, unrelated, or neutral prosody, cut to 300, 600, or 1000 ms in duration. Results underscored that prosodic information enduring at least 600 ms was necessary to presumably activate shared emotion knowledge responsible for prosody–face congruity effects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • R. Adolphs H. Damasio D. Tranel A. R. Damasio (1996) ArticleTitleCortical systems for the recognition of emotion in facial expressions The Journal of Neuroscience 16 7678–7687 Occurrence Handle8922424

    PubMed  Google Scholar 

  • S. Attardo J. Eisterhold J. Hay I. Poggi (2003) ArticleTitleMultimodal markers of irony and sarcasm Humor 16 243–260 Occurrence Handle000184668000005 Occurrence Handle10.1515/humr.2003.012

    Article  ISI  Google Scholar 

  • R. Banse K. R. Scherer (1996) ArticleTitleAcoustic profiles in vocal emotion expression Journal of Personality and Social Psychology 70 614–636 Occurrence Handle10.1037/0022-3514.70.3.614 Occurrence Handle8851745 Occurrence HandleA1996TZ88300016

    Article  PubMed  ISI  Google Scholar 

  • J. Bargh S. Chaiken R. Govender F. Pratto (1992) ArticleTitleThe generality of the automatic attitude activation effect Journal of Personality and Social Psychology 62 893–912 Occurrence Handle10.1037/0022-3514.62.6.893 Occurrence Handle1619549 Occurrence HandleA1992HY08800002

    Article  PubMed  ISI  Google Scholar 

  • J. Borod (2000) ArticleTitleRelationships among facial, prosodic, and lexical channels of emotional perceptual processing Cognition and Emotion 14 193–211 Occurrence Handle10.1080/026999300378932

    Article  Google Scholar 

  • G. H. Bower (1981) ArticleTitleMood and memory American Psychologist 36 129–148 Occurrence Handle7224324 Occurrence HandleA1981LE97800003

    PubMed  ISI  Google Scholar 

  • D. Bowers R. Bower K. Heilman (1993) ArticleTitleThe nonverbal affect lexicon: Theoretical perspectives from neuropsychological studies of affect perception Neuropsychology 7 433–444 Occurrence Handle10.1037/0894-4105.7.4.433

    Article  Google Scholar 

  • A. Calder J. Keane T. Manly R. Sprengelmeyer S. Scott I. Nimmo-Smith A. Young (2003) ArticleTitleFacial expression recognition across the adult life span Neuropsychologia 41 195–202 Occurrence Handle10.1016/S0028-3932(02)00149-5 Occurrence Handle12459217 Occurrence Handle000180131900009

    Article  PubMed  ISI  Google Scholar 

  • B. Gelder Particlede K. B. E. Bocker J. Tuomainen M. Hensen J. Vroomen (1999) ArticleTitleThe combined perception of emotion from voice and face: Early interaction revealed by human brain responses Neuroscience Letters 260 133–136 Occurrence Handle10025717

    PubMed  Google Scholar 

  • B. Gelder Particlede J. Vroomen (2000) ArticleTitleThe perception of emotions by ear and by eye Cognition and Emotion 14 289–311

    Google Scholar 

  • U. Dimberg (1991) ArticleTitleEmotional reactions to facial expressions: A case of automatic responding? Psychophysiology 28 S19

    Google Scholar 

  • U. Dimberg A. Öhman (1996) ArticleTitleBeyond the wrath: Psychophysiological responses to facial stimuli Motivation and Emotion 20 149–182 Occurrence Handle10.1007/BF02253869 Occurrence HandleA1996VV41600004

    Article  ISI  Google Scholar 

  • N. L. Etcoff J. L. Magee (1992) ArticleTitleCategorical perception of facial expressions Cognition 44 227–240 Occurrence Handle10.1016/0010-0277(92)90002-Y Occurrence Handle1424493 Occurrence HandleA1992JN71500002

    Article  PubMed  ISI  Google Scholar 

  • R. H. Fazio D. M. Sanbonmatsu M. C. Powell F. R. Kardes (1986) ArticleTitleOn the automatic activation of attitudes Journal of Personality and Social Psychology 50 229–238 Occurrence Handle10.1037/0022-3514.50.2.229 Occurrence Handle3701576 Occurrence HandleA1986C109000001

    Article  PubMed  ISI  Google Scholar 

  • J. Halberstadt P. Niedenthal (2001) ArticleTitleEffects of emotion concepts on perceptual memory for emotional expressions Journal of Personality and Social Psychology 81 587–598 Occurrence Handle10.1037/0022-3514.81.4.587 Occurrence Handle11642347 Occurrence Handle000171577400003

    Article  PubMed  ISI  Google Scholar 

  • C. H. Hansen C. A. Shantz (1995) ArticleTitleEmotion-specific priming: Congruence effects on affect and recognition across negative emotions Journal of Personality and Social Psychology 21 548–557

    Google Scholar 

  • D. Hermans J. Houwer ParticleDe P. Eelen (1994) ArticleTitleThe affective priming effect: Automatic activation of evaluative information in memory Cognition and Emotion 8 515–533

    Google Scholar 

  • A. Innes-Ker P. Niedenthal (2002) ArticleTitleEmotion concepts and emotional states in social judgment and categorization Journal of Personality and Social Psychology 83 804–816 Occurrence Handle10.1037/0022-3514.83.4.804 Occurrence Handle12374436 Occurrence Handle000178156600002

    Article  PubMed  ISI  Google Scholar 

  • P. Juslin P. Laukka (2003) ArticleTitleCommunication of emotions in vocal expression and music performance: Different channels, same code? Psychological Bulletin 129 770–814 Occurrence Handle10.1037/0033-2909.129.5.770 Occurrence Handle12956543 Occurrence Handle000184890400006

    Article  PubMed  ISI  Google Scholar 

  • G. Kirouac F. Dore (1984) ArticleTitleAccuracy and latency of judgment of facial expressions of emotion Perceptual Motor Skills 57 683–686

    Google Scholar 

  • S. Kitayama K. Ishii (2002) ArticleTitleWord and voice: Spontaneous attention to emotional utterances in two languages Cognition and Emotion 16 29–59 Occurrence Handle10.1080/0269993943000121

    Article  Google Scholar 

  • K. C. Klauer J. Musch (2002) ArticleTitleGoal dependent and goal independent effects of irrelevant evaluations Personality and Social Psychology Bulletin 28 802–814 Occurrence Handle000175641500009

    ISI  Google Scholar 

  • K. Klauer J. Musch (2003) Affective priming: Findings and theories J. Musch K. Klauer (Eds) The psychology of evaluation: Affective processes in cognition and emotion Erlbaum London 7–49

    Google Scholar 

  • M. Kutas K. D. Federmeier (2000) ArticleTitleElectrophysiology reveals semantic memory use in language comprehension Trends in Cognitive Sciences 4 463–470 Occurrence Handle10.1016/S1364-6613(00)01560-6 Occurrence Handle11115760 Occurrence Handle000165861200006

    Article  PubMed  ISI  Google Scholar 

  • D. R. Ladd K. Silverman F. Tolkmitt G. Bergmann K. Scherer (1985) ArticleTitleEvidence for the independent function of intonation contour type, voice quality, and F0 range in signaling speaker affect Journal of the Acoustical Society of America 78 435–444 Occurrence Handle10.1121/1.392466 Occurrence HandleA1985ANV3200008

    Article  ISI  Google Scholar 

  • D. Massaro P. Egan (1996) ArticleTitlePerceiving affect from the voice and the face Psychonomic Bulletin and Review 3 215–221

    Google Scholar 

  • A. Mathews B. Mackintosh (1998) ArticleTitleA cognitive model of selective processing in anxiety Cognitive Therapy and Research 22 539–560 Occurrence Handle10.1023/A:1018738019346 Occurrence Handle000077736100002

    Article  ISI  Google Scholar 

  • K. Mogg B. P. Bradley (1999) ArticleTitleOrienting of attention to threatening facial expressions presented under conditions of restricted awareness Cognition and Emotion 13 713–740 Occurrence Handle10.1080/026999399379050

    Article  Google Scholar 

  • J. Musch K. C. Klauer (2001) ArticleTitleLocational uncertainty moderates affective congruency effects in the evaluative decision task Cognition and Emotion 15 167–188 Occurrence Handle10.1080/0269993004200042

    Article  Google Scholar 

  • P. M. Niedenthal J. B. Halberstadt (1995) ArticleTitleThe acquisition and structure of emotional response categories The Psychology of Learning and Motivation 33 23–64

    Google Scholar 

  • P. Niedenthal J. Halberstadt (1999) ArticleTitleEmotional response categorization Psychological Review 106 337–361 Occurrence Handle10.1037/0033-295X.106.2.337 Occurrence Handle000080682100004

    Article  ISI  Google Scholar 

  • P. M. Niedenthal J. B. Halberstadt M. B. Setterlund (1997) ArticleTitleBeing happy and seeing “happy”: Emotional state mediates visual word recognition Cognition and Emotion 11 403–432 Occurrence Handle10.1080/026999397379863

    Article  Google Scholar 

  • A. Öhman (1987) ArticleTitleThe psychophysiology of emotion: An evolutionary-cognitive perspective Advances in Psychophysiology 2 79–127

    Google Scholar 

  • M. D. Pell (1998) ArticleTitleRecognition of prosody following unilateral brain lesion: Influence of functional and structural attributes of prosodic contours Neuropsychologia 36 701–715 Occurrence Handle10.1016/S0028-3932(98)00008-6 Occurrence Handle9751436 Occurrence Handle000075500900001

    Article  PubMed  ISI  Google Scholar 

  • M. D. Pell (2001) ArticleTitleInfluence of emotion and focus location on prosody in matched statements and questions Journal of the Acoustical Society of America 109 1668–1680 Occurrence Handle10.1121/1.1352088 Occurrence Handle11325135 Occurrence Handle000167932600037

    Article  PubMed  ISI  Google Scholar 

  • M. D. Pell (2002) ArticleTitleEvaluation of nonverbal emotion in face and voice: Some preliminary findings on a new battery of tests Brain and Cognition 48 499–504 Occurrence Handle12030496 Occurrence Handle000174687000103

    PubMed  ISI  Google Scholar 

  • M. D. Pell (2004) ArticleTitleA method for on-line evaluation of emotional prosody in healthy and disordered populations Brain and Language 91 25–26 Occurrence Handle10.1016/j.bandl.2004.06.017 Occurrence Handle000224442800011

    Article  ISI  Google Scholar 

  • M. D. Pell (2005) ArticleTitleNonverbal emotion priming: Evidence from the ‘Facial Affect Decision Task’ Journal of Nonverbal Behavior 29 45–73 Occurrence Handle10.1007/s10919-004-0889-8 Occurrence Handle000228505100003

    Article  ISI  Google Scholar 

  • M. D. Pell C. Leonard (2003) ArticleTitleProcessing emotional tone from speech in Parkinson’s disease: A role for the basal ganglia Cognitive, Affective, & Behavioral Neuroscience 3 275–288

    Google Scholar 

  • M. D. Pell C. L. Leonard (2005) ArticleTitleFacial expression decoding in early Parkinson’s disease Cognitive Brain Research 23 327–340 Occurrence Handle10.1016/j.cogbrainres.2004.11.004 Occurrence Handle15820640 Occurrence Handle000228883500017

    Article  PubMed  ISI  Google Scholar 

  • G. Pourtois B. Gelder Particlede J. Vroomen B. Rossion M. Crommelinck (2000) ArticleTitleThe time-course of intermodal binding between seeing and hearing affective information Neuroreport 11 1329–1333 Occurrence Handle10817616 Occurrence Handle000086931600037

    PubMed  ISI  Google Scholar 

  • G. Pourtois D. Debatisse P.-A. Despland B. Gelder Particle de (2002) ArticleTitleFacial expressions modulate the time course of long latency auditory brain potentials Cognitive Brain Research 14 99–105 Occurrence Handle10.1016/S0926-6410(02)00064-2 Occurrence Handle12063133 Occurrence Handle000178630600010

    Article  PubMed  ISI  Google Scholar 

  • F. Pratto O. P. John (1991) ArticleTitleAutomatic vigilance: The attention-grabbing power of negative social information Journal of Personality and Social Psychology 61 380–391 Occurrence Handle10.1037/0022-3514.61.3.380 Occurrence Handle1941510 Occurrence HandleA1991GD39200003

    Article  PubMed  ISI  Google Scholar 

  • R. Rosnow R. Rosenthal (2003) ArticleTitleEffect sizes for experimenting psychologists Canadian Journal of Experimental Psychology 57 221–237 Occurrence Handle14596479 Occurrence Handle000185549300008

    PubMed  ISI  Google Scholar 

  • K. Scherer (1986) ArticleTitleVocal affect expression: A review and a model for future research Psychological Bulletin 99 143–165 Occurrence Handle10.1037/0033-2909.99.2.143 Occurrence Handle3515381 Occurrence HandleA1986A747600001

    Article  PubMed  ISI  Google Scholar 

  • A. Schirmer S. Kotz (2003) ArticleTitleERP evidence for a sex-specific stroop effect in emotional speech Journal of Cognitive Neuroscience 15 1135–1148 Occurrence Handle10.1162/089892903322598102 Occurrence Handle14709232 Occurrence Handle000186944400006

    Article  PubMed  ISI  Google Scholar 

  • A. Schirmer S. Zysset S. Kotz D. Y. Cramon Particlevon (2004) ArticleTitleGender differences in the activation of inferior frontal cortex during emotional speech perception NeuroImage 21 1114–1123 Occurrence Handle10.1016/j.neuroimage.2003.10.048 Occurrence Handle15006679 Occurrence Handle000220148900033

    Article  PubMed  ISI  Google Scholar 

  • D. Swinney (1979) ArticleTitleLexical access during sentence comprehension: (Re)consideration of context effects Journal of Verbal Learning and Verbal Behavior 18 645–659 Occurrence HandleA1979JB25100001

    ISI  Google Scholar 

  • J. Honk Particlevan A. Tuiten E. Haan Particlede M. Hout Particlevan den H. Stam (2001) ArticleTitleAttentional biases for angry faces: Relationships to trait anger and anxiety Cognition and Emotion 15 279–297

    Google Scholar 

  • J. Vroomen J. Driver B. Gelder Particlede (2001) ArticleTitleIs cross-modal integration of emotional expressions independent of attentional resources Cognitive, Affective, & Behavioral Neuroscience 1 382–387 Occurrence Handle10.3758/CABN.1.4.382

    Article  Google Scholar 

  • H. G. Wallbott (1988) ArticleTitleBig girls don’t frown, big boys don’t cry: Gender differences of professional actors in communicating emotion via facial expression Journal of Nonverbal Behavior 12 98–106 Occurrence Handle10.1007/BF00986928 Occurrence HandleA1988U555800002

    Article  ISI  Google Scholar 

  • D. Wentura (2000) ArticleTitleDissociative affective and associative priming effects in the lexical decision task: Yes versus no responses to word targets reveal evaluative judgement tendencies Journal of Experimental Psychology: Learning, Memory, and Cognition 26 456–469 Occurrence Handle10.1037/0278-7393.26.2.456 Occurrence Handle000087485200010 Occurrence Handle10764106

    Article  ISI  PubMed  Google Scholar 

  • C. Williams K. Stevens (1981) Vocal correlates of emotion states J. K. Darby (Eds) Speech evaluation in psychiatry Grune & Stratton New York 221–240

    Google Scholar 

  • L. H. Wurm D. A. Vakoch M. R. Strasser R. Calin-Jageman S. E. Ross (2001) ArticleTitleSpeech perception and vocal expression of emotion Cognition and Emotion 15 831–852 Occurrence Handle10.1080/02699930143000086

    Article  Google Scholar 

  • A. Young D. Rowland A. Calder N. Etcoff A. Seth D. Perrett (1997) ArticleTitleFacial expression megamix: Tests of dimensional and category accounts of emotion recognition Cognition 63 271–313 Occurrence Handle10.1016/S0010-0277(97)00003-6 Occurrence Handle9265872 Occurrence HandleA1997XQ19900003

    Article  PubMed  ISI  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Communication Sciences and Disorders, McGill University, 1266 ave. des Pins ouest, Montréal, Que, H3G 1A8, Canada

    Marc D. Pell

Authors
  1. Marc D. Pell
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marc D. Pell.

Additional information

Marc D. Pell is affiliated with the School of Communication Sciences and Disorders, McGill University, Montréal, Canada.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pell, M.D. Prosody–face Interactions in Emotional Processing as Revealed by the Facial Affect Decision Task. J Nonverbal Behav 29, 193–215 (2005). https://doi.org/10.1007/s10919-005-7720-z

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10919-005-7720-z

Keywords

Search

Navigation