Abstract
The effects of talker variability on visual speech perception were tested by having subjects speechread sentences from either single-talker or mixed-talker sentence lists. Results revealed that changes in talker from trial to trial decreased speechreading performance. To help determine whether this decrement was due to talker change—and not a change in superficial characteristics of the stimuli—Experiment 2 tested speechreading from visual stimuli whose images were tinted by a single color, or mixed colors. Results revealed that the mixed-color lists did not inhibit speechreading performance relative to the single-color lists. These results are analogous to findings in the auditory speech literature and suggest that, like auditory speech, visual speech operations include a resource-demanding component that is influenced by talker variability.
Article PDF
Similar content being viewed by others
References
Bench, J., &Bamford, J. (1979).Speech-hearing tests and the spoken language of hearing impaired children. London: Academic Press.
Biederman, I., &Ju, G. (1988). Surface versus edge-based determinants of visual recognition.Cognition,20, 38–64.
Bradlow, A. R., Nyoaard, L. C., &Pisoni, D. B. (1999). Effects of talker, rate, and amplitude variation on recognition memory for spoken words.Perception & Psychophysics.61, 206–219.
Burt, D. M., &Perrett, D. I. (1995). Perception of age in adult Caucasian male faces: Computer graphic manipulation of shape and colour information.Proceedings of the Royal Society of London: Series B,259, 137–143.
Campbell, R. T., Brooks, B., De Haan, E., &Roberts, T. (1996). Dissoicating face processing skills: Decisions about lip-read speech expression and identity.Quarterly Journal of Experimental Psychology,49A, 295–314.
Campbell, R. T., Landis, T., &Regard, M. (1986). Face recognition and lip-reading. A neurological dissociation.Brain,109, 509–521.
Church, B. A., &Schacter, D. L. (1994). Perceptual specificity of auditory priming: Implicit memory for voice intonation and fundamental frequency.Journal of Experimental Psychology: Learning, Memory, & Cognition,20, 521–533.
Craik, F. I., &Kirsner.K. (1974). The effect of speaker’s voice on word recognition.Quarterly Journal of Experimental Psychology,26, 274–284.
Creelman, C. D. (1957). Case of the unknown talker.Journal of the Acoustical Society of America,29, 655.
Demorest, M. E., &Bernstein, L. E. (1992). Sources of variability in speechreading sentences: A generalizability analysis.Journal of Speech & Hearing Research,35, 876–891.
Edwards, M., &Badcock, D. (1996). Global motion perception: Interaction of chromatic and luminance signals.Vision Research,36, 2423–2431.
Ellis, H. D. (1989). Processes underlying face recognition. In R. Bruyer (Ed.),Neuropsychology offace perception and facial expression (pp. 41–49). Hillsdale, NJ: Erlbaum.
Fodor, S. A. (1983).Modularity of mind. Cambridge, MA: MIT Press, Bradford Books.
Fowler, C.A. (1986).An event approach to the study of speech perception from a direct-realist perspective.Journal of Phonetics,14, 3–28.
Gegenfurtner, K. R., &Hawken, M. J. (1996). Interaction of motion and color in the visual pathways.Trends in Neurosciences,19, 394–401.
Green, K. P. (1994). The influence of an inverted face on the McGurk effect (abstract).Journal of the Acoustical Society of America,95, 3014.
Halle, M. (1985). Speculations about the representation of words in memory. In V. A. Fromkin (Ed.),Phonetic linguistics (pp. 101–104). New York: Academic Press.
Hill, H., Bruce, V., &Akamatsu, S. (1995). Perceiving the sex and race of faces: The role of shape and colour.Proceedings of the Royal Society of London: Series B,261, 367–373.
Hillger, L. A., &Koenig, O. (1991). Separable mechanisms in face processing: Evidence from hemispheric specialization [Special issue: Face perception].Journal of Cognitive Neuroscience,3, 42–58.
Humphreys, G. K., Goodale, M. A., Jakobson, L. S., &Servos, P. (1994). The role of surface information in object recognition: Studies of a visual form agnosic and normal subjects.Perception,23, 1457–1481.
Johnson, J. A., &Rosenblum, L. D. (1996). Hemispheric differences in perceiving and integrating dynamic visual speech information.Journal of the Acoustical Society of America,100, 2570.
Johnson, K. (1990). The role of perceived speaker identity in F0 normalization of vowels.Journal of the Acoustical Societv of America,88, 642–654.
Jordan, T. R., &Bevan, K. (1997). Seeing and hearing rotated faces: Influences of facial orientation on visual and audio-visual speech recognition.Journal of Experimental Psychology: Human Perception & Performance,23, 388–403.
Kricos, P. B., &Lesner, S. A. (1982, May). Differences in visual intelligibility across talkers.The Volta Review,84, 219–225.
Lee, K. J., &Perrett, D. (1997). Presentation-time measures of the effects of manipulations in colour space on discrimination of famous faces.Perception,26, 733–752.
Liberman, A. M., &Mattingly, I. G. (1985). The motor theory of speech perception revised.Cognition,21, 1–36.
Livingstone, M. S., &Hubel, D. H. (1987). Psychophysical evidence for separate channels for the perception of form, color, movement, and depth.Journal of Neuroscience,7, 3416–3468.
Macleod, A., &Summerfield, Q. (1987). Quantifying the contribution of vision to speech perception in noise.British Journal of Audiology,21, 131–141.
Massaro, D. W., &Cohen, M. M. (1996). Perceiving speech from inverted faces.Perception & Psychophysics,58, 1047–1065.
McClelland, J. L., &Elman, J. L. (1986). The TRACE model of speech perception.Cognitive Psychology,18, 1–86.
McGurk, H., &Macdonald, J. W. (1976). Hearing lips and seeing voices.Nature,264, 746–748.
Mills, A. E. (1987). The development of phonology in the blind child. In B. Dodd & R. Campbell (Eds.),Hearing by eye: The psychology of lip-reading (pp. 145–162). Hillsdale, NJ: Erlbaum.
Montgomery, A. A., Walden, B. E., &Prosek, R. A. (1987). Effects of consonantal contest on vowel lipreading.Journal of Speech & Hearing Research,30, 50–59.
Mullennix, J. W., &Pisoni, D. B. (1990).Talker variability and processing dependencies between word and voice (Tech. Rep. No. 13). Bloomington: Indiana University.
Mullennix, J. W., Pisoni, D. B., &Martin, C. S. (1989). Some effects of talker variability on spoken word recognition.Journal of the Acoustical Society of America,92, 1085–1099.
Nygaard, L. C., &Burt, S. A. (1996). Sources of variability as linguistically relevant aspects of speech.Journal of the Acoustical Society of America,100, 2572.
Nygaard, L. C., Sommers, M. S., &Pisoni, D. B. (1994). Speech perception as a talker-contingent process.Psychological Science,5, 42–46.
Palmeri, T. J., Goldinger, S. D., &Pisoni, D. B. (1993). Episodic encoding of voice attributes and recognition memory for spoken words.Journal of Experimental Psychology: Learning, Memory, & Cognition,19, 309–328.
Pollack, I., Pickett, J. M., &Sumby, W. H. (1954). On the identification of speakers by voice.Journal of the Acoustical Society of America,26, 403–406.
Price, C. J., &Humphreys, G. W. (1989). The effects of surface detail on object categorization and naming.Quarterly Journal of Experimental Psychology,41A, 797–828.
Ramachandran, V. S., &Gregory, R. L. (1978). Does colour provide an input to human motion perception?Nature,275, 55–56.
Reisberg, D., McLean, J., &Goldfield, A. (1987). Easy to hear but hard to understand: A lipreading advantage with intact auditory stimuli. In B. Dodd & R. Campbell (Eds.),Hearing by ear and eye: The psychology of lipreading (pp. 97–113). Hillsdale, NJ: Erlbaum.
Remez, R. E., Fellowes, J. M., &Rubin, P. E. (1997). Talker identification based on phonetic information.Journal of Experimental Psychology: Human Perception & Performance,23, 651 -666.
Rosenblum, L. D., Johnson, J. A., &Saldana, H. M. (1996). Visual kinematic information for embellishing speech in noise.Journal of Speech & Hearing Research,39, 1159–1170.
Rosenblum, L. D., &Saldana, H. M. (1998). Time-varying information for visual speech perception. In R. Campbell, B. Dodd, & D. Burnham (Eds.),Hearing by eye II: Advances in the psychology of speechreading and auditory-visual speech. Hillsdale, NJ: Erlbaum.
Rosenblum, L. D., Yakel, D. A., &Green, K. P. (2000). Face and mouth inversion effects on visual and audiovisual speech perception.Journal of Experimental Psychology: Human Perception & Performance,26, 806–819.
Saldaña, H. M., Nygaard, L. C., &Pisoni, D. B. (1996). Encoding of visual speaker attributes and recognition memory for spoken words. In D. G. Stork & M. E. Hennecke (Eds.),Speechreading by humans and machines: Models, systems, and applications (NATO ASI Series F: Computers and Systems Sciences, No. 150). New York: Springer-Verlag.
Saldana, H. M., &Rosenblum, L. D. (1994). Voice information in auditory form-based priming.Journal of the Acoustical Society of America,95, 2870.
Schwein Berger, S. R., &Soukup, G. R. (1998). Asymmetrie relationships among perceptions of facial identity, emotion, and facial speech.Journal of Experimental Psychology: Human Perception & Performance,24, 1748–1765.
Sergent, J. (1982). About face: Left hemisphere involvement in processing physiognomies.Journal of Experimental Psychology: Human Perception & Performance,8, 1–14.
Sheffert, S. M., &Fowler, C. A. (1995). The effects of voice and visible speaker change on memory for spoken words.Journal of Memory & Language,34, 665–685.
Sommers, M. S., Nygaard, L. C., &Pisoni, D. B. (1994). Stimulus variability and spoken word recognition: I. Effects of variability in speaking rate and overall amplitude.Journal of the Acoustical Society of America,96, 1314–1324.
Strange, W., Verbrugge, R. R., Shankweiler, D. P., &Edman, T. R. (1976). Consonant environment specifies vowel identity.Journal of the Acoustical Society of America,60, 213–224.
Summerfield, Q., &Haggard, M. P. (1973). Vocal tract normalization and representation.Brain Language,28, 12–23.
Thompson, P. (1980). Margaret Thatcher: A new illusion.Perception,9, 483–484.
Valentine, T. (1988). Upside-down faces: A review of the effect of inversion upon face recognition.British Journal of Psychology,79, 471–491.
Valentine.T., &Bruce, V. (1985). What’s up? The Margaret Thatcher illusion revisited.Perception,14, 515–516.
Van Lancker, D. V., Kreiman, J., &Emmorey, K. (1985). Familiar voice recognition: patterns and parameters, Part I: Recognition of backward voices.Journal of Phonetics,13, 19–38.
Verbrugge, R. R., Strange, W., Shankweiler, D. P., &Edman, T. R. (1976). What information enables a listener to map a talker’s vowel space?Journal of the Acoustical Society of America,60, 198–212.
Walker, S., Bruce, V., &O’malley, C. (1995). Facial identity and facial speech processing: Familiar faces and voices in the McGurk effect.Perception & Psychophysics,57, 1124–1133.
Yakel, D. A., &Rosenblum, L. D. (1996). Face identification using visual speech information.Journal of the Acoustical Society of America,100, 2570.
Young, A. W., Hay, D. C., McWeeney, K. H., Ellis, A. W., &Barry, C. (1985). Familiarity decisions for faces presented to the left and right cerebral hemispheres.Brain & Cognition,4, 439–450.
Author information
Authors and Affiliations
Corresponding author
Additional information
This research was supported by NSF Grant SBR-9617047 awarded to L.D.R.
Rights and permissions
About this article
Cite this article
Yakel, D.A., Rosenblum, L.D. & Fortier, M.A. Effects of talker variability on speechreading. Perception & Psychophysics 62, 1405–1412 (2000). https://doi.org/10.3758/BF03212142
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03212142