How can coarticulation models account for speech sensitivity to audio-visual desynchronization?

Abstract

How to relate audio and visual perceptual results on bimodal timing — obtained through desynchronization — to the behavioural analysis of visible oro-facial movements in the frame of competing coarticulation models? The coherence of the perceptual building up of audio, visual, and AV, was explored for vowel-to-vowel gestures through acoustic pauses. Results support the claim that the natural audio delay relative to the anticipated visual signal can be reduced by desynchronization without affecting intelligibility, as long as the configurational visual cues remain in accordance with the sound. A related exploration of anticipatory behaviour on the speech production side led us to develop a new coarticulation model, the MEM (Movement expansion model). In this framework results found from zero to large intervocalic intervals fall fairly well within the range of sensitivities obtained for intelligibility in AV desynchronization experiments, thus offering a production rationale for the perceptual phenomenology of bimodal speech coherence.