Perception & Psychophysics

, Volume 68, Issue 4, pp 582–592 | Cite as

A sex difference in visual influence on heard speech



Reports of sex differences in language processing are inconsistent and are thought to vary by task type and difficulty. In two experiments, we investigated a sex difference in visual influence on heard speech (the McGurk effect). First, incongruent consonant-vowel stimuli were presented where the visual portion of the signal was brief (100 msec) or full (temporally equivalent to the auditory). Second, to determine whether men and women differed in their ability to extract visual speech information from these brief stimuli, the same stimuli were presented to new participants with an additional visual-only (lipread) condition. In both experiments, women showed a significantly greater visual influence on heard speech than did men for the brief visual stimuli. No sex differences for the full stimuli or in the ability to lipread were found. These findings indicate that the more challenging brief visual stimuli elicit sex differences in the processing of audiovisual speech.


  1. Adobe Systems (1995). Adobe Premiere Version 5.1 [Computer software]. San Jose, CA: Author.Google Scholar
  2. Aloufy, S., Lapidot, M., &Myslobodsky, M. (1996). Differences in susceptibility to the “blending illusion” among native Hebrew and English speakers.Brain & Language,53, 51–57.CrossRefGoogle Scholar
  3. Alsius, A., Navarra, J., Campbell, R., &Soto-Faraco, S. (2005). Audiovisual integration of speech falters under high attention demands.Current Biology,15, 839–843.CrossRefPubMedGoogle Scholar
  4. Baxter, L. C., Saykin, A. J., Flashman, L. A., Johnson, S. C., Guerin, S. J., Babcock, D. R., &Wishart, H. A. (2003). Sex differences in semantic language processing: A functional MRI study.Brain & Language,84, 264–272.CrossRefGoogle Scholar
  5. Baynes, K., Funnell, M. G., &Fowler, C. A. (1994). Hemispheric contributions to the integration of visual and auditory information in speech perception.Perception & Psychophysics,55, 633–641.CrossRefGoogle Scholar
  6. Brancazio, L. (2004). Lexical influences in audiovisual speech perception.Journal of Experimental Psychology: Human Perception & Performance,30, 445–463.CrossRefGoogle Scholar
  7. Brancazio, L., &Miller, J. (2005). Use of visual information in speech perception: Evidence for a visual rate effect both with and without a McGurk effect.Perception & Psychophysics,67, 759–769.CrossRefGoogle Scholar
  8. Calvert, G. A. (2001). Crossmodal processing in the human brain: Insights from functional neuroimaging studies.Cerebral Cortex,11, 1110–1123.CrossRefPubMedGoogle Scholar
  9. Calvert, G. A., Brammer, M. J., Bullmore, E. T., Campbell, R., Iversen, S. D., &David, A. S. (1999). Response amplification in sensory-specific cortices during cross-modal binding.NeuroReport,10, 2619–2623.CrossRefPubMedGoogle Scholar
  10. Calvert, G. A., &Campbell, R. (2003). Reading speech from still and moving faces: The neural substrates of visible speech.Journal of Cognitive Neuroscience,15, 57–70.CrossRefPubMedGoogle Scholar
  11. Calvert, G. A., Campbell, R., &Brammer, M. J. (2000). Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex.Current Biology,10, 649–657.CrossRefPubMedGoogle Scholar
  12. Campbell, R. (1994). Audiovisual speech: Where, what, when, how?Current Psychology of Cognition,13, 76–80.Google Scholar
  13. Coney, J. (2002). Lateral asymmetry in phonological processing: Relating behavioral measures to neuroimaged structures.Brain & Language,80, 355–365.CrossRefGoogle Scholar
  14. Conrey, B. L. (2004). Multimodal sentence intelligibility and the detection of auditory-visual asynchrony in speech and nonspeech signals: A first report.Research on Spoken Language Processing: Progress Report No. 26 (pp. 345–355). Bloomington: Indiana University, Department of Psychology, Speech Research Laboratory.Google Scholar
  15. Conrey, B. L., &Pisoni, D. B. (2004). Detection of auditory-visual asynchrony in speech and nonspeech signals.Research on Spoken Language Processing: Progress Report No. 26 (pp. 71–94). Bloomington: Indiana University, Department of Psychology, Speech Research Laboratory.Google Scholar
  16. Daly, N., Bench, R. J., &Chappell, H. (1996). Gender differences in speech readability.Journal of the Academy of Rehabilitative Audiology,29, 1–14.Google Scholar
  17. Daly, N., Bench, [R.] J., &Chappell, H. (1997). Gender differences in visual speech variables.Journal of the Academy of Rehabilitative Audiology,30, 63–76.Google Scholar
  18. Desjardins, R. N., Rogers, J., &Werker, J. F. (1997). An exploration of why preschoolers perform differently than do adults in audiovisual speech perception tasks.Journal of Experimental Child Psychology,66, 85–110.CrossRefPubMedGoogle Scholar
  19. Desjardins, R. N., &Werker, J. F. (2004). Is the integration of heard and seen speech mandatory for infants?Developmental Psychobiology,45, 187–203.CrossRefPubMedGoogle Scholar
  20. Diesch, E. (1995). Left and right hemifield advantages of fusions and combinations in audiovisual speech perception.Quarterly Journal of Experimental Psychology,48A, 320–333.Google Scholar
  21. Fowler, C. A., Brown, J. M., &Mann, V. A. (2000). Contrast effects do not underlie effects of preceding liquids on stop-consonant identification by humans.Journal of Experimental Psychology: Human Perception & Performance,26, 1–12.CrossRefGoogle Scholar
  22. Fowler, C. A., &Dekle, D. J. (1991). Listening with eye and hand: Cross-modal contributions to speech perception.Journal of Experimental Psychology: Human Perception & Performance,17, 816–828.CrossRefGoogle Scholar
  23. Freyd, J. J. (1983a). The mental representation of movement when static stimuli are viewed.Perception & Psychophysics,33, 575–581.CrossRefGoogle Scholar
  24. Freyd, J. J. (1983b). Representing the dynamics of a static form.Memory & Cognition,11, 342–346.CrossRefGoogle Scholar
  25. Frost, J. A., Binder, J. R., Springer, J. A., Hammeke, T. A., Bellgowan, P. S. F., Rao, S. M., &Cox, R. W. (1999). Language processing is strongly left-lateralized in both sexes: Evidence from functional MRI.Brain,122, 199–208.CrossRefPubMedGoogle Scholar
  26. Green, K. P. (1998). The use of auditory and visual information during phonetic processing: Implications for theories of speech perception. In R. Campbell, B. Dodd, & D. K. Burnham (Eds.),Hearing by eye: II. Advances in the psychology of speechreading and auditory-visual speech (pp. 3–25). Hove, U.K.: Psychology Press.Google Scholar
  27. Green, K. P., Kuhl, P. K., Meltzoff, A. N., &Stevens, E. B. (1991). Integrating speech information across talkers, gender, and sensory modality: Female faces and male voices in the McGurk effect.Perception & Psychophysics,50, 524–536.CrossRefGoogle Scholar
  28. Green, K. P., &Norrix, L. W. (1997). Acoustic cues to place of articulation and the McGurk effect: The role of release bursts, aspiration, and formant transitions.Journal of Speech & Hearing Research,40, 646–665.Google Scholar
  29. Gur, R. C., Alsop, D., Glahn, D., Petty, R., Swanson, C. L., Maldjian, J. A., et al. (2000). An fMRI study of sex differences in regional activation to a verbal and spatial task.Brain & Language,74, 157–170.CrossRefGoogle Scholar
  30. Jaeger, J. J., Lockwood, A. H., Van Valin, R. D., Kemmerer, D. L., Murphy, B. W., &Wack, D. S. (1998). Sex differences in brain regions activated by grammatical and reading tasks.NeuroReport,9, 2803–2807.CrossRefPubMedGoogle Scholar
  31. Johnson, F. M., Hicks, L. H., Goldberg, T., &Myslobodsky, M. S. (1988). Sex differences in lipreading.Bulletin of the Psychonomic Society,26, 106–108.Google Scholar
  32. Jones, J. A., &Callan, D. E. (2003). Brain activity during audiovisual speech perception: An fMRI study of the McGurk effect.NeuroReport,14, 1129–1133.CrossRefPubMedGoogle Scholar
  33. Josse, G., &Tzourio-Mazoyer, N. (2004). Hemispheric specialization for language.Brain Research Reviews,44, 1–12.CrossRefPubMedGoogle Scholar
  34. Kansaku, K., &Kitazawa, S. (2001). Imaging studies on sex differences in lateralization of language.Neuroscience Research,41, 333–337.CrossRefPubMedGoogle Scholar
  35. Knecht, S., Drager, B., Deppe, M., Bobe, L., Lohmann, H., Floel, A., et al. (2000). Handedness and hemispheric dominance in healthy humans.Brain,123, 2512–2518.CrossRefPubMedGoogle Scholar
  36. MacDonald, J., Andersen, S., &Bachmann, T. (2000). Hearing by eye: How much spatial degradation can be tolerated?Perception,29, 1155–1168.CrossRefPubMedGoogle Scholar
  37. Majeres, R. L. (1999). Sex differences in phonological processes: Speeded matching and word reading.Memory & Cognition,27, 246–253.CrossRefGoogle Scholar
  38. Massaro, D. W. (1987).Speech perception by ear and eye: A paradigm for psychological enquiry. Hillsdale, NJ: Erlbaum.Google Scholar
  39. Massaro, D. W., &Cohen, M. M. (1983). Integration of visual and auditory information in speech perception.Journal of Experimental Psychology: Human Perception & Performance,9, 753–771.CrossRefGoogle Scholar
  40. McGurk, H., &MacDonald, J. (1976). Hearing lips and seeing voices.Nature,264, 746–748.CrossRefPubMedGoogle Scholar
  41. Munhall, K. G., &Tokhura, Y. (1998). Audiovisual gating and the time course of speech perception.Journal of the Acoustical Society of America,104, 530–539.CrossRefPubMedGoogle Scholar
  42. Öhrström, N., &Traunmüller, H. (2004). Audiovisual perception of Swedish vowels with and without conflicting cues. InProceedings, Fonetik 2004 (pp. 40–43). Stockholm: Stockholm University, Department of Linguistics.Google Scholar
  43. Olson, I. R., Gatenby, J., &Gore, J. C. (2002). A comparison of bound and unbound audio-visual information processing in the human cerebral cortex.Cognitive Brain Research,14, 129–138.CrossRefPubMedGoogle Scholar
  44. Paré, M., Richler, R. C., Ten Hove, M., &Munhall, K. G. (2003). Gaze behavior in audiovisual speech perception: The influence of ocular fixations on the McGurk effect.Perception & Psychophysics,65, 553–567.CrossRefGoogle Scholar
  45. Pugh, K. R., Shaywitz, B. A., Shaywitz, S. E., Constable, R. T., Skudlarski, P., Fulbright, R. K., et al. (1996). Cerebral organization of component processes in reading.Brain,119, 1221–1238.CrossRefPubMedGoogle Scholar
  46. Pugh, K. R., Shaywitz, B. A., Shaywitz, S. E., Fulbright, R. K., Byrd, D., Skudlarski, P., et al. (1996). Auditory selective attention: An fMRI investigation.NeuroImage,4, 159–173.CrossRefPubMedGoogle Scholar
  47. Riecker, A., Wildgruber, D., Dogil, G., Grodd, W., &Ackermann, H. (2002). Hemispheric lateralization effects of rhythm implementation during syllable repetitions: An fMRI study.NeuroImage,16, 169–176.CrossRefPubMedGoogle Scholar
  48. Rosenblum, L. D., &Saldaña, H. M. (1996). An audiovisual test of kinematic primitives for visual speech perception.Journal of Experimental Psychology: Human Perception & Performance,22, 318–331.CrossRefGoogle Scholar
  49. Rosenblum, L. D., &Saldaña, H. M. (1998). Time-varying information for visual speech perception. In R. Campbell, B. Dodd, & D. K. Burnham (Eds.),Hearing by eye: II. Advances in the psychology of speechreading and auditory-visual speech (pp. 61–81). Hove, U.K.: Psychology Press.Google Scholar
  50. Rosenblum, L. D., Schmuckler, M. A., &Johnson, J. A. (1997). The McGurk effect in infants.Perception & Psychophysics,59, 347–357.CrossRefGoogle Scholar
  51. Rossell, S. L., Bullmore, E. T., Williams, S. C. R., &David, A. S. (2002). Sex differences in functional brain activation during a lexical visual field task.Brain & Language,80, 97–105.CrossRefGoogle Scholar
  52. Sekiyama, K., Kanno, I., Miura, S., &Sugita, Y. (2003). Auditory-visual speech perception examined by fMRI and PET.Neuroscience Research,47, 277–287.CrossRefPubMedGoogle Scholar
  53. Shaywitz, B. A., Shaywitz, S. E., Pugh, K. R., Constable, R. T., Skudlarski, P., Fulbright, R. K., et al. (1995). Sex differences in the functional organization of the brain for language.Nature,373, 607–609.CrossRefPubMedGoogle Scholar
  54. Sumby, W. H., &Pollack, I. (1954). Visual contribution to speech intelligibility in noise.Journal of the Acoustical Society of America,26, 212–215.CrossRefGoogle Scholar
  55. Tiippana, K., Andersen, T. S., &Sams, M. (2004). Visual attention modulates audiovisual speech perception.European Journal of Cognitive Psychology,16, 457–472.CrossRefGoogle Scholar
  56. Van Wassenhove, V., Grant, K. W., &Poeppel, D. (2005). Visual speech speeds up the neural processing of auditory speech.Proceedings of the National Academy of Sciences,102, 1181–1186.CrossRefGoogle Scholar
  57. Watson, C. S., Qiu, W. W., Chamberlain, M. M., &Li, X. (1996). Auditory and visual speech perception: Confirmation of a modality-independent source of individual differences in speech recognition.Journal of the Acoustical Society of America,100, 1153–1162.CrossRefPubMedGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2006

Authors and Affiliations

  • Julia R. Irwin
    • 1
  • D. H. Whalen
    • 1
  • Carol A. Fowler
    • 1
    • 2
    • 3
  1. 1.Haskins LaboratoriesNew Haven
  2. 2.University of ConnecticutStorrs
  3. 3.Yale UniversityNew Haven

Personalised recommendations