Abstract
Reports of sex differences in language processing are inconsistent and are thought to vary by task type and difficulty. In two experiments, we investigated a sex difference in visual influence on heard speech (the McGurk effect). First, incongruent consonant-vowel stimuli were presented where the visual portion of the signal was brief (100 msec) or full (temporally equivalent to the auditory). Second, to determine whether men and women differed in their ability to extract visual speech information from these brief stimuli, the same stimuli were presented to new participants with an additional visual-only (lipread) condition. In both experiments, women showed a significantly greater visual influence on heard speech than did men for the brief visual stimuli. No sex differences for the full stimuli or in the ability to lipread were found. These findings indicate that the more challenging brief visual stimuli elicit sex differences in the processing of audiovisual speech.
Article PDF
Similar content being viewed by others
References
Adobe Systems (1995). Adobe Premiere Version 5.1 [Computer software]. San Jose, CA: Author.
Aloufy, S., Lapidot, M., &Myslobodsky, M. (1996). Differences in susceptibility to the “blending illusion” among native Hebrew and English speakers.Brain & Language,53, 51–57.
Alsius, A., Navarra, J., Campbell, R., &Soto-Faraco, S. (2005). Audiovisual integration of speech falters under high attention demands.Current Biology,15, 839–843.
Baxter, L. C., Saykin, A. J., Flashman, L. A., Johnson, S. C., Guerin, S. J., Babcock, D. R., &Wishart, H. A. (2003). Sex differences in semantic language processing: A functional MRI study.Brain & Language,84, 264–272.
Baynes, K., Funnell, M. G., &Fowler, C. A. (1994). Hemispheric contributions to the integration of visual and auditory information in speech perception.Perception & Psychophysics,55, 633–641.
Brancazio, L. (2004). Lexical influences in audiovisual speech perception.Journal of Experimental Psychology: Human Perception & Performance,30, 445–463.
Brancazio, L., &Miller, J. (2005). Use of visual information in speech perception: Evidence for a visual rate effect both with and without a McGurk effect.Perception & Psychophysics,67, 759–769.
Calvert, G. A. (2001). Crossmodal processing in the human brain: Insights from functional neuroimaging studies.Cerebral Cortex,11, 1110–1123.
Calvert, G. A., Brammer, M. J., Bullmore, E. T., Campbell, R., Iversen, S. D., &David, A. S. (1999). Response amplification in sensory-specific cortices during cross-modal binding.NeuroReport,10, 2619–2623.
Calvert, G. A., &Campbell, R. (2003). Reading speech from still and moving faces: The neural substrates of visible speech.Journal of Cognitive Neuroscience,15, 57–70.
Calvert, G. A., Campbell, R., &Brammer, M. J. (2000). Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex.Current Biology,10, 649–657.
Campbell, R. (1994). Audiovisual speech: Where, what, when, how?Current Psychology of Cognition,13, 76–80.
Coney, J. (2002). Lateral asymmetry in phonological processing: Relating behavioral measures to neuroimaged structures.Brain & Language,80, 355–365.
Conrey, B. L. (2004). Multimodal sentence intelligibility and the detection of auditory-visual asynchrony in speech and nonspeech signals: A first report.Research on Spoken Language Processing: Progress Report No. 26 (pp. 345–355). Bloomington: Indiana University, Department of Psychology, Speech Research Laboratory.
Conrey, B. L., &Pisoni, D. B. (2004). Detection of auditory-visual asynchrony in speech and nonspeech signals.Research on Spoken Language Processing: Progress Report No. 26 (pp. 71–94). Bloomington: Indiana University, Department of Psychology, Speech Research Laboratory.
Daly, N., Bench, R. J., &Chappell, H. (1996). Gender differences in speech readability.Journal of the Academy of Rehabilitative Audiology,29, 1–14.
Daly, N., Bench, [R.] J., &Chappell, H. (1997). Gender differences in visual speech variables.Journal of the Academy of Rehabilitative Audiology,30, 63–76.
Desjardins, R. N., Rogers, J., &Werker, J. F. (1997). An exploration of why preschoolers perform differently than do adults in audiovisual speech perception tasks.Journal of Experimental Child Psychology,66, 85–110.
Desjardins, R. N., &Werker, J. F. (2004). Is the integration of heard and seen speech mandatory for infants?Developmental Psychobiology,45, 187–203.
Diesch, E. (1995). Left and right hemifield advantages of fusions and combinations in audiovisual speech perception.Quarterly Journal of Experimental Psychology,48A, 320–333.
Fowler, C. A., Brown, J. M., &Mann, V. A. (2000). Contrast effects do not underlie effects of preceding liquids on stop-consonant identification by humans.Journal of Experimental Psychology: Human Perception & Performance,26, 1–12.
Fowler, C. A., &Dekle, D. J. (1991). Listening with eye and hand: Cross-modal contributions to speech perception.Journal of Experimental Psychology: Human Perception & Performance,17, 816–828.
Freyd, J. J. (1983a). The mental representation of movement when static stimuli are viewed.Perception & Psychophysics,33, 575–581.
Freyd, J. J. (1983b). Representing the dynamics of a static form.Memory & Cognition,11, 342–346.
Frost, J. A., Binder, J. R., Springer, J. A., Hammeke, T. A., Bellgowan, P. S. F., Rao, S. M., &Cox, R. W. (1999). Language processing is strongly left-lateralized in both sexes: Evidence from functional MRI.Brain,122, 199–208.
Green, K. P. (1998). The use of auditory and visual information during phonetic processing: Implications for theories of speech perception. In R. Campbell, B. Dodd, & D. K. Burnham (Eds.),Hearing by eye: II. Advances in the psychology of speechreading and auditory-visual speech (pp. 3–25). Hove, U.K.: Psychology Press.
Green, K. P., Kuhl, P. K., Meltzoff, A. N., &Stevens, E. B. (1991). Integrating speech information across talkers, gender, and sensory modality: Female faces and male voices in the McGurk effect.Perception & Psychophysics,50, 524–536.
Green, K. P., &Norrix, L. W. (1997). Acoustic cues to place of articulation and the McGurk effect: The role of release bursts, aspiration, and formant transitions.Journal of Speech & Hearing Research,40, 646–665.
Gur, R. C., Alsop, D., Glahn, D., Petty, R., Swanson, C. L., Maldjian, J. A., et al. (2000). An fMRI study of sex differences in regional activation to a verbal and spatial task.Brain & Language,74, 157–170.
Jaeger, J. J., Lockwood, A. H., Van Valin, R. D., Kemmerer, D. L., Murphy, B. W., &Wack, D. S. (1998). Sex differences in brain regions activated by grammatical and reading tasks.NeuroReport,9, 2803–2807.
Johnson, F. M., Hicks, L. H., Goldberg, T., &Myslobodsky, M. S. (1988). Sex differences in lipreading.Bulletin of the Psychonomic Society,26, 106–108.
Jones, J. A., &Callan, D. E. (2003). Brain activity during audiovisual speech perception: An fMRI study of the McGurk effect.NeuroReport,14, 1129–1133.
Josse, G., &Tzourio-Mazoyer, N. (2004). Hemispheric specialization for language.Brain Research Reviews,44, 1–12.
Kansaku, K., &Kitazawa, S. (2001). Imaging studies on sex differences in lateralization of language.Neuroscience Research,41, 333–337.
Knecht, S., Drager, B., Deppe, M., Bobe, L., Lohmann, H., Floel, A., et al. (2000). Handedness and hemispheric dominance in healthy humans.Brain,123, 2512–2518.
MacDonald, J., Andersen, S., &Bachmann, T. (2000). Hearing by eye: How much spatial degradation can be tolerated?Perception,29, 1155–1168.
Majeres, R. L. (1999). Sex differences in phonological processes: Speeded matching and word reading.Memory & Cognition,27, 246–253.
Massaro, D. W. (1987).Speech perception by ear and eye: A paradigm for psychological enquiry. Hillsdale, NJ: Erlbaum.
Massaro, D. W., &Cohen, M. M. (1983). Integration of visual and auditory information in speech perception.Journal of Experimental Psychology: Human Perception & Performance,9, 753–771.
McGurk, H., &MacDonald, J. (1976). Hearing lips and seeing voices.Nature,264, 746–748.
Munhall, K. G., &Tokhura, Y. (1998). Audiovisual gating and the time course of speech perception.Journal of the Acoustical Society of America,104, 530–539.
Öhrström, N., &Traunmüller, H. (2004). Audiovisual perception of Swedish vowels with and without conflicting cues. InProceedings, Fonetik 2004 (pp. 40–43). Stockholm: Stockholm University, Department of Linguistics.
Olson, I. R., Gatenby, J., &Gore, J. C. (2002). A comparison of bound and unbound audio-visual information processing in the human cerebral cortex.Cognitive Brain Research,14, 129–138.
Paré, M., Richler, R. C., Ten Hove, M., &Munhall, K. G. (2003). Gaze behavior in audiovisual speech perception: The influence of ocular fixations on the McGurk effect.Perception & Psychophysics,65, 553–567.
Pugh, K. R., Shaywitz, B. A., Shaywitz, S. E., Constable, R. T., Skudlarski, P., Fulbright, R. K., et al. (1996). Cerebral organization of component processes in reading.Brain,119, 1221–1238.
Pugh, K. R., Shaywitz, B. A., Shaywitz, S. E., Fulbright, R. K., Byrd, D., Skudlarski, P., et al. (1996). Auditory selective attention: An fMRI investigation.NeuroImage,4, 159–173.
Riecker, A., Wildgruber, D., Dogil, G., Grodd, W., &Ackermann, H. (2002). Hemispheric lateralization effects of rhythm implementation during syllable repetitions: An fMRI study.NeuroImage,16, 169–176.
Rosenblum, L. D., &Saldaña, H. M. (1996). An audiovisual test of kinematic primitives for visual speech perception.Journal of Experimental Psychology: Human Perception & Performance,22, 318–331.
Rosenblum, L. D., &Saldaña, H. M. (1998). Time-varying information for visual speech perception. In R. Campbell, B. Dodd, & D. K. Burnham (Eds.),Hearing by eye: II. Advances in the psychology of speechreading and auditory-visual speech (pp. 61–81). Hove, U.K.: Psychology Press.
Rosenblum, L. D., Schmuckler, M. A., &Johnson, J. A. (1997). The McGurk effect in infants.Perception & Psychophysics,59, 347–357.
Rossell, S. L., Bullmore, E. T., Williams, S. C. R., &David, A. S. (2002). Sex differences in functional brain activation during a lexical visual field task.Brain & Language,80, 97–105.
Sekiyama, K., Kanno, I., Miura, S., &Sugita, Y. (2003). Auditory-visual speech perception examined by fMRI and PET.Neuroscience Research,47, 277–287.
Shaywitz, B. A., Shaywitz, S. E., Pugh, K. R., Constable, R. T., Skudlarski, P., Fulbright, R. K., et al. (1995). Sex differences in the functional organization of the brain for language.Nature,373, 607–609.
Sumby, W. H., &Pollack, I. (1954). Visual contribution to speech intelligibility in noise.Journal of the Acoustical Society of America,26, 212–215.
Tiippana, K., Andersen, T. S., &Sams, M. (2004). Visual attention modulates audiovisual speech perception.European Journal of Cognitive Psychology,16, 457–472.
Van Wassenhove, V., Grant, K. W., &Poeppel, D. (2005). Visual speech speeds up the neural processing of auditory speech.Proceedings of the National Academy of Sciences,102, 1181–1186.
Watson, C. S., Qiu, W. W., Chamberlain, M. M., &Li, X. (1996). Auditory and visual speech perception: Confirmation of a modality-independent source of individual differences in speech recognition.Journal of the Acoustical Society of America,100, 1153–1162.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Irwin, J.R., Whalen, D.H. & Fowler, C.A. A sex difference in visual influence on heard speech. Perception & Psychophysics 68, 582–592 (2006). https://doi.org/10.3758/BF03208760
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03208760