Advertisement

Springer Nature is making Coronavirus research free. View research | View latest news | Sign up for updates

Interleaved lexical and audiovisual information can retune phoneme boundaries

  • 49 Accesses

Abstract

To adapt to situations in which speech perception is difficult, listeners can adjust boundaries between phoneme categories using perceptual learning. Such adjustments can draw on lexical information in surrounding speech, or on visual cues via speech-reading. In the present study, listeners proved they were able to flexibly adjust the boundary between two plosive/stop consonants, /p/-/t/, using both lexical and speech-reading information and given the same experimental design for both cue types. Videos of a speaker pronouncing pseudo-words and audio recordings of Dutch words were presented in alternating blocks of either stimulus type. Listeners were able to switch between cues to adjust phoneme boundaries, and resulting effects were comparable to results from listeners receiving only a single source of information. Overall, audiovisual cues (i.e., the videos) produced the stronger effects, commensurate with their applicability for adapting to noisy environments. Lexical cues were able to induce effects with fewer exposure stimuli and a changing phoneme bias, in a design unlike most prior studies of lexical retuning. While lexical retuning effects were relatively weaker compared to audiovisual recalibration, this discrepancy could reflect how lexical retuning may be more suitable for adapting to speakers than to environments. Nonetheless, the presence of the lexical retuning effects suggests that it may be invoked at a faster rate than previously seen. In general, this technique has further illuminated the robustness of adaptability in speech perception, and offers the potential to enable further comparisons across differing forms of perceptual learning.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

  1. Bertelson, P., Vroomen, J., & De Gelder, B. (2003). Visual recalibration of auditory speech identification: a McGurk aftereffect. Psychological Science, 14(6), 592–597. https://doi.org/10.1046/j.0956-7976.2003.psci_1470.x

  2. Boersma, P., & van Heuven, V. (2001). Speak and unSpeak with PRAAT. Glot International, 5(9/10), 341–347. https://doi.org/10.1097/AUD.0b013e31821473f7

  3. Bruggeman, L. & Cutler, A. (2019). No L1 privilege in talker adaptation. Bilingualism, Language and Cognition. https://doi.org/10.1017/S1366728919000646

  4. Cutler, A. (2012). Native listening: the flexibility dimension. Dutch Journal of Applied Linguistics, 1(2), 169–187. https://doi.org/10.1075/dujal.1.2.02cut

  5. Cutler, A., Eisner, F., McQueen, J. M., & Norris, D. (2010). How abstract phonemic categories are necessary for coping with speaker-related variation. Laboratory Phonology, 10, 91–111. https://doi.org/10.1017/CBO9781107415324.004

  6. Duyck, W., Desmet, T., Verbeke, L. P. C., & Brysbaert, M. (2004). WordGen: a tool for word selection and nonword generation in Dutch, English, German, and French. Behavior Research Methods, Instruments, & Computers: A Journal of the Psychonomic Society, Inc, 36(3), 488–499. https://doi.org/10.3758/BF03195595

  7. Eimas, P. D., & Corbit, J. D. (1973). Selective adaptation of linguistic feature detectors. Cognitive Psychology. https://doi.org/10.1016/0010-0285(73)90006-6

  8. Eisner, F., & McQueen, J. M. (2006). Perceptual learning in speech: stability over time. The Journal of the Acoustical Society of America, 119(4), 1950-1953. https://doi.org/10.1121/1.2178721

  9. Ganong, W. F. (1980). Phonetic categorization in auditory word perception. Journal of Experimental Psychology. Human Perception and Performance, 6(1), 110–125. https://doi.org/10.1037/0096-1523.6.1.110

  10. Keetels, M., Pecoraro, M., & Vroomen, J. (2015). Recalibration of auditory phonemes by lipread speech is ear-specific. Cognition, 141, 121–126. https://doi.org/10.1016/j.cognition.2015.04.019

  11. Keetels, M., Stekelenburg, J. J., & Vroomen, J. (2016). A spatial gradient in phonetic recalibration by lipread speech. Journal of Phonetics, 56, 124–130. https://doi.org/10.1016/j.wocn.2016.02.005

  12. Kleinschmidt, D. F., & Jaeger, T. F. (2016). Robust speech perception: recognize the familiar, generalize to the similar, and adapt to the novel. Psychological Review, 122(2), 148–203. https://doi.org/10.1037/a0038695

  13. Kraljic, T., & Samuel, A. G. (2007). Perceptual adjustments to multiple speakers. Journal of Memory and Language, 56(1), 1–15. https://doi.org/10.1016/j.jml.2006.07.010

  14. Kraljic, T., & Samuel, A. G. (2009). Perceptual learning for speech. Attention, Perception & Psychophysics, 71(3), 481–489. https://doi.org/10.3758/APP

  15. Lüttke, C. S., Pérez-Bellido, A., & de Lange, F. P. (2018). Rapid recalibration of speech perception after experiencing the McGurk illusion. Royal Society Open Science, 5(3), 170909. https://doi.org/10.1098/rsos.170909

  16. Macleod, A., & Summerfield, Q. (1987). Quantifying the contribution of vision to speech perception in noise. British Journal of Audiology, 21(2), 131-141. https://doi.org/10.3109/03005368709077786

  17. Massaro, D. W., & Jesse, A. (2007). Audiovisual speech perception and word recognition. In M. G. Gaskell (Ed.), The Oxford Handbook of Psycholinguistics (pp. 19-35). Oxford: Oxford University Press. https://doi.org/10.1093/oxfordhb/9780198568971.013.0002

  18. McGurk, H., & MacDonald, M. (1976). Hearing lips and seeing voices. Nature, 264(5588), 746.

  19. McQueen, J. M. (1991). The influence of the lexicon on phonetic categorization: stimulus quality in word-final ambiguity. Journal of Experimental Psychology: Human Perception and Performance, 17(2), 433–443. https://doi.org/10.1037/0096-1523.17.2.433

  20. Mitchel, A. D., Gerfen, C., & Weiss, D. J. (2016). Audiovisual perceptual learning with multiple speakers. Journal of Phonetics, 56, 66–74. https://doi.org/10.1016/j.wocn.2016.02.003

  21. Mitterer, H., & Reinisch, E. (2017). Visual speech influences speech perception immediately but not automatically. Attention, Perception & Psychophysics, 79(2), 660–678. https://doi.org/10.3758/s13414-016-1249-6

  22. Norris, D., McQueen, J. M., & Cutler, A. (2003). Perceptual learning in speech. Cognitive Psychology, 47(2), 204–238. https://doi.org/10.1016/S0010-0285(03)00006-9

  23. Repp, B. H. (1981). Perceptual equivalence of two kinds of ambiguous speech stimuli. Bulletin of the Psychonomic Society, 18(1), 12-14. https://doi.org/10.3758/BF03333556

  24. Samuel, A. G. (2001). Knowing a word affects the fundamental perception of the sounds within it. Psychological Science, 12(4), 348–351. https://doi.org/10.1111/1467-9280.00364

  25. Sumby, W. H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. The Journal of the Acoustical Society of America, 26(2), 212-215. https://doi.org/10.1121/1.1907309

  26. Van Linden, S., & Vroomen, J. (2007). Recalibration of phonetic categories by lipread speech versus lexical information. Journal of Experimental Psychology: Human Perception and Performance, 33(6), 1483–1494. https://doi.org/10.1037/0096-1523.33.6.1483

  27. Vroomen, J. & Baart, M. (2009). Recalibration of phonetic categories by lipread speech: measuring aftereffects after a 24-hour delay. Language and Speech, 52(2-3), 341-350. https://doi.org/10.1177/0023830909103178

  28. Vroomen, J., & Baart, M. (2012). Phonetic recalibration in audiovisual speech. In M. M. Murray and M. T. Wallace (Eds.) The Neural Bases of Multisensory Processes. (pp. 363–379). Boca raton (FL): CRC Press.

  29. Vroomen, J., van Linden, S., de Gelder, B., & Bertelson, P. (2007). Visual recalibration and selective adaptation in auditory-visual speech perception: contrasting build-up courses. Neuropsychologia, 45(3), 572–577. https://doi.org/10.1016/j.neuropsychologia.2006.01.031

  30. Vroomen, J., van Linden, S., Keetels, M., de Gelder, B., & Bertelson, P. (2004). Selective adaptation and recalibration of auditory speech by lipread information: dissipation. Speech Communication, 44(1-4), 55–61. https://doi.org/10.1016/j.specom.2004.03.009

Download references

Acknowledgements

We acknowledge financial support from the Netherlands Organization for Scientific Research gravity program Language in Interaction.

Author information

Correspondence to Shruti Ullas.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ullas, S., Formisano, E., Eisner, F. et al. Interleaved lexical and audiovisual information can retune phoneme boundaries. Atten Percept Psychophys (2020). https://doi.org/10.3758/s13414-019-01961-8

Download citation

Keywords

  • Phoneme boundary
  • Recalibration
  • Perceptual retuning
  • Lexical
  • Audiovisual