Skip to main content
Download PDF

Effects of talker variability on speechreading

Perception & Psychophysics volume 62, pages 1405–1412 (2000)Cite this article

Abstract

The effects of talker variability on visual speech perception were tested by having subjects speechread sentences from either single-talker or mixed-talker sentence lists. Results revealed that changes in talker from trial to trial decreased speechreading performance. To help determine whether this decrement was due to talker change—and not a change in superficial characteristics of the stimuli—Experiment 2 tested speechreading from visual stimuli whose images were tinted by a single color, or mixed colors. Results revealed that the mixed-color lists did not inhibit speechreading performance relative to the single-color lists. These results are analogous to findings in the auditory speech literature and suggest that, like auditory speech, visual speech operations include a resource-demanding component that is influenced by talker variability.

References

  • Bench, J., &Bamford, J. (1979).Speech-hearing tests and the spoken language of hearing impaired children. London: Academic Press.

    Google Scholar 

  • Biederman, I., &Ju, G. (1988). Surface versus edge-based determinants of visual recognition.Cognition,20, 38–64.

    Google Scholar 

  • Bradlow, A. R., Nyoaard, L. C., &Pisoni, D. B. (1999). Effects of talker, rate, and amplitude variation on recognition memory for spoken words.Perception & Psychophysics.61, 206–219.

    Article  Google Scholar 

  • Burt, D. M., &Perrett, D. I. (1995). Perception of age in adult Caucasian male faces: Computer graphic manipulation of shape and colour information.Proceedings of the Royal Society of London: Series B,259, 137–143.

    Article  Google Scholar 

  • Campbell, R. T., Brooks, B., De Haan, E., &Roberts, T. (1996). Dissoicating face processing skills: Decisions about lip-read speech expression and identity.Quarterly Journal of Experimental Psychology,49A, 295–314.

    Article  Google Scholar 

  • Campbell, R. T., Landis, T., &Regard, M. (1986). Face recognition and lip-reading. A neurological dissociation.Brain,109, 509–521.

    Article  PubMed  Google Scholar 

  • Church, B. A., &Schacter, D. L. (1994). Perceptual specificity of auditory priming: Implicit memory for voice intonation and fundamental frequency.Journal of Experimental Psychology: Learning, Memory, & Cognition,20, 521–533.

    Article  Google Scholar 

  • Craik, F. I., &Kirsner.K. (1974). The effect of speaker’s voice on word recognition.Quarterly Journal of Experimental Psychology,26, 274–284.

    Article  Google Scholar 

  • Creelman, C. D. (1957). Case of the unknown talker.Journal of the Acoustical Society of America,29, 655.

    Article  Google Scholar 

  • Demorest, M. E., &Bernstein, L. E. (1992). Sources of variability in speechreading sentences: A generalizability analysis.Journal of Speech & Hearing Research,35, 876–891.

    Google Scholar 

  • Edwards, M., &Badcock, D. (1996). Global motion perception: Interaction of chromatic and luminance signals.Vision Research,36, 2423–2431.

    Article  PubMed  Google Scholar 

  • Ellis, H. D. (1989). Processes underlying face recognition. In R. Bruyer (Ed.),Neuropsychology offace perception and facial expression (pp. 41–49). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Fodor, S. A. (1983).Modularity of mind. Cambridge, MA: MIT Press, Bradford Books.

    Google Scholar 

  • Fowler, C.A. (1986).An event approach to the study of speech perception from a direct-realist perspective.Journal of Phonetics,14, 3–28.

    Google Scholar 

  • Gegenfurtner, K. R., &Hawken, M. J. (1996). Interaction of motion and color in the visual pathways.Trends in Neurosciences,19, 394–401.

    Article  PubMed  Google Scholar 

  • Green, K. P. (1994). The influence of an inverted face on the McGurk effect (abstract).Journal of the Acoustical Society of America,95, 3014.

    Article  Google Scholar 

  • Halle, M. (1985). Speculations about the representation of words in memory. In V. A. Fromkin (Ed.),Phonetic linguistics (pp. 101–104). New York: Academic Press.

    Google Scholar 

  • Hill, H., Bruce, V., &Akamatsu, S. (1995). Perceiving the sex and race of faces: The role of shape and colour.Proceedings of the Royal Society of London: Series B,261, 367–373.

    Article  Google Scholar 

  • Hillger, L. A., &Koenig, O. (1991). Separable mechanisms in face processing: Evidence from hemispheric specialization [Special issue: Face perception].Journal of Cognitive Neuroscience,3, 42–58.

    Article  Google Scholar 

  • Humphreys, G. K., Goodale, M. A., Jakobson, L. S., &Servos, P. (1994). The role of surface information in object recognition: Studies of a visual form agnosic and normal subjects.Perception,23, 1457–1481.

    Article  Google Scholar 

  • Johnson, J. A., &Rosenblum, L. D. (1996). Hemispheric differences in perceiving and integrating dynamic visual speech information.Journal of the Acoustical Society of America,100, 2570.

    Article  Google Scholar 

  • Johnson, K. (1990). The role of perceived speaker identity in F0 normalization of vowels.Journal of the Acoustical Societv of America,88, 642–654.

    Article  Google Scholar 

  • Jordan, T. R., &Bevan, K. (1997). Seeing and hearing rotated faces: Influences of facial orientation on visual and audio-visual speech recognition.Journal of Experimental Psychology: Human Perception & Performance,23, 388–403.

    Article  Google Scholar 

  • Kricos, P. B., &Lesner, S. A. (1982, May). Differences in visual intelligibility across talkers.The Volta Review,84, 219–225.

    Google Scholar 

  • Lee, K. J., &Perrett, D. (1997). Presentation-time measures of the effects of manipulations in colour space on discrimination of famous faces.Perception,26, 733–752.

    Article  PubMed  Google Scholar 

  • Liberman, A. M., &Mattingly, I. G. (1985). The motor theory of speech perception revised.Cognition,21, 1–36.

    Article  PubMed  Google Scholar 

  • Livingstone, M. S., &Hubel, D. H. (1987). Psychophysical evidence for separate channels for the perception of form, color, movement, and depth.Journal of Neuroscience,7, 3416–3468.

    PubMed  Google Scholar 

  • Macleod, A., &Summerfield, Q. (1987). Quantifying the contribution of vision to speech perception in noise.British Journal of Audiology,21, 131–141.

    Article  PubMed  Google Scholar 

  • Massaro, D. W., &Cohen, M. M. (1996). Perceiving speech from inverted faces.Perception & Psychophysics,58, 1047–1065.

    Google Scholar 

  • McClelland, J. L., &Elman, J. L. (1986). The TRACE model of speech perception.Cognitive Psychology,18, 1–86.

    Article  PubMed  Google Scholar 

  • McGurk, H., &Macdonald, J. W. (1976). Hearing lips and seeing voices.Nature,264, 746–748.

    Article  PubMed  Google Scholar 

  • Mills, A. E. (1987). The development of phonology in the blind child. In B. Dodd & R. Campbell (Eds.),Hearing by eye: The psychology of lip-reading (pp. 145–162). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Montgomery, A. A., Walden, B. E., &Prosek, R. A. (1987). Effects of consonantal contest on vowel lipreading.Journal of Speech & Hearing Research,30, 50–59.

    Google Scholar 

  • Mullennix, J. W., &Pisoni, D. B. (1990).Talker variability and processing dependencies between word and voice (Tech. Rep. No. 13). Bloomington: Indiana University.

    Google Scholar 

  • Mullennix, J. W., Pisoni, D. B., &Martin, C. S. (1989). Some effects of talker variability on spoken word recognition.Journal of the Acoustical Society of America,92, 1085–1099.

    Google Scholar 

  • Nygaard, L. C., &Burt, S. A. (1996). Sources of variability as linguistically relevant aspects of speech.Journal of the Acoustical Society of America,100, 2572.

    Article  Google Scholar 

  • Nygaard, L. C., Sommers, M. S., &Pisoni, D. B. (1994). Speech perception as a talker-contingent process.Psychological Science,5, 42–46.

    Article  Google Scholar 

  • Palmeri, T. J., Goldinger, S. D., &Pisoni, D. B. (1993). Episodic encoding of voice attributes and recognition memory for spoken words.Journal of Experimental Psychology: Learning, Memory, & Cognition,19, 309–328.

    Article  Google Scholar 

  • Pollack, I., Pickett, J. M., &Sumby, W. H. (1954). On the identification of speakers by voice.Journal of the Acoustical Society of America,26, 403–406.

    Article  Google Scholar 

  • Price, C. J., &Humphreys, G. W. (1989). The effects of surface detail on object categorization and naming.Quarterly Journal of Experimental Psychology,41A, 797–828.

    Google Scholar 

  • Ramachandran, V. S., &Gregory, R. L. (1978). Does colour provide an input to human motion perception?Nature,275, 55–56.

    Article  PubMed  Google Scholar 

  • Reisberg, D., McLean, J., &Goldfield, A. (1987). Easy to hear but hard to understand: A lipreading advantage with intact auditory stimuli. In B. Dodd & R. Campbell (Eds.),Hearing by ear and eye: The psychology of lipreading (pp. 97–113). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Remez, R. E., Fellowes, J. M., &Rubin, P. E. (1997). Talker identification based on phonetic information.Journal of Experimental Psychology: Human Perception & Performance,23, 651 -666.

    Article  Google Scholar 

  • Rosenblum, L. D., Johnson, J. A., &Saldana, H. M. (1996). Visual kinematic information for embellishing speech in noise.Journal of Speech & Hearing Research,39, 1159–1170.

    Google Scholar 

  • Rosenblum, L. D., &Saldana, H. M. (1998). Time-varying information for visual speech perception. In R. Campbell, B. Dodd, & D. Burnham (Eds.),Hearing by eye II: Advances in the psychology of speechreading and auditory-visual speech. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Rosenblum, L. D., Yakel, D. A., &Green, K. P. (2000). Face and mouth inversion effects on visual and audiovisual speech perception.Journal of Experimental Psychology: Human Perception & Performance,26, 806–819.

    Article  Google Scholar 

  • Saldaña, H. M., Nygaard, L. C., &Pisoni, D. B. (1996). Encoding of visual speaker attributes and recognition memory for spoken words. In D. G. Stork & M. E. Hennecke (Eds.),Speechreading by humans and machines: Models, systems, and applications (NATO ASI Series F: Computers and Systems Sciences, No. 150). New York: Springer-Verlag.

    Google Scholar 

  • Saldana, H. M., &Rosenblum, L. D. (1994). Voice information in auditory form-based priming.Journal of the Acoustical Society of America,95, 2870.

    Article  Google Scholar 

  • Schwein Berger, S. R., &Soukup, G. R. (1998). Asymmetrie relationships among perceptions of facial identity, emotion, and facial speech.Journal of Experimental Psychology: Human Perception & Performance,24, 1748–1765.

    Article  Google Scholar 

  • Sergent, J. (1982). About face: Left hemisphere involvement in processing physiognomies.Journal of Experimental Psychology: Human Perception & Performance,8, 1–14.

    Article  Google Scholar 

  • Sheffert, S. M., &Fowler, C. A. (1995). The effects of voice and visible speaker change on memory for spoken words.Journal of Memory & Language,34, 665–685.

    Article  Google Scholar 

  • Sommers, M. S., Nygaard, L. C., &Pisoni, D. B. (1994). Stimulus variability and spoken word recognition: I. Effects of variability in speaking rate and overall amplitude.Journal of the Acoustical Society of America,96, 1314–1324.

    Article  PubMed  Google Scholar 

  • Strange, W., Verbrugge, R. R., Shankweiler, D. P., &Edman, T. R. (1976). Consonant environment specifies vowel identity.Journal of the Acoustical Society of America,60, 213–224.

    Article  PubMed  Google Scholar 

  • Summerfield, Q., &Haggard, M. P. (1973). Vocal tract normalization and representation.Brain Language,28, 12–23.

    Google Scholar 

  • Thompson, P. (1980). Margaret Thatcher: A new illusion.Perception,9, 483–484.

    Article  PubMed  Google Scholar 

  • Valentine, T. (1988). Upside-down faces: A review of the effect of inversion upon face recognition.British Journal of Psychology,79, 471–491.

    PubMed  Google Scholar 

  • Valentine.T., &Bruce, V. (1985). What’s up? The Margaret Thatcher illusion revisited.Perception,14, 515–516.

    Article  PubMed  Google Scholar 

  • Van Lancker, D. V., Kreiman, J., &Emmorey, K. (1985). Familiar voice recognition: patterns and parameters, Part I: Recognition of backward voices.Journal of Phonetics,13, 19–38.

    Google Scholar 

  • Verbrugge, R. R., Strange, W., Shankweiler, D. P., &Edman, T. R. (1976). What information enables a listener to map a talker’s vowel space?Journal of the Acoustical Society of America,60, 198–212.

    Article  PubMed  Google Scholar 

  • Walker, S., Bruce, V., &O’malley, C. (1995). Facial identity and facial speech processing: Familiar faces and voices in the McGurk effect.Perception & Psychophysics,57, 1124–1133.

    Article  Google Scholar 

  • Yakel, D. A., &Rosenblum, L. D. (1996). Face identification using visual speech information.Journal of the Acoustical Society of America,100, 2570.

    Article  Google Scholar 

  • Young, A. W., Hay, D. C., McWeeney, K. H., Ellis, A. W., &Barry, C. (1985). Familiarity decisions for faces presented to the left and right cerebral hemispheres.Brain & Cognition,4, 439–450.

    Article  Google Scholar 

Download references

Author information

Author notes

  1. Deborah A. Yakel

    Present address: Department of Psychology at Orange Coast College, USA

  2. Michelle A. Fortier

    Present address: Department of Psychology at San Diego State University, USA

Affiliations

  1. Department of Psychology, University of California, 92521, Riverside, CA

    Deborah A. Yakel, Lawrence D. Rosenblum & Michelle A. Fortier

Authors
  1. Deborah A. Yakel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lawrence D. Rosenblum
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michelle A. Fortier
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lawrence D. Rosenblum.

Additional information

This research was supported by NSF Grant SBR-9617047 awarded to L.D.R.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Yakel, D.A., Rosenblum, L.D. & Fortier, M.A. Effects of talker variability on speechreading. Perception & Psychophysics 62, 1405–1412 (2000). https://doi.org/10.3758/BF03212142

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.3758/BF03212142

Keywords

  • Speech Perception
  • Visual Speech
  • Journal Ofthe Acoustical Society ofAmerica
  • Audiovisual Speech
  • Talker Condition