Abstract
Research demonstrates that listening to and viewing speech excites tongue and lip motor areas involved in speech production. This perceptual-motor relationship was investigated behaviourally by presenting video clips of a speaker producing vowel-consonant-vowel syllables in three conditions: visual-only, audio-only, and audiovisual. Participants identified target letters that were flashed over the mouth during the video, either manually or verbally as quickly as possible. Verbal responses were fastest when the target matched the speech stimuli in all modality conditions, yet optimal facilitation was observed when participants were presented with visual-only stimuli. Critically, no such facilitation occurred when participants were asked to identify the target manually. Our findings support previous research suggesting a close relationship between speech perception and production by demonstrating that viewing speech can ‘prime’ our motor system for subsequent speech production.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Note, this is a very simplified time course for speech processing, and audiovisual speech perception is indeed much more complex (see Munhall and Tohkura 1998)
References
Alsius A, Navarra J, Campbell R et al (2005) Audiovisual integration of speech falters under high attentional demands. Curr Biol 15:839–843
Anderson TS, Tiipana K, Laarni J, Kojo I, Sams M (2009) The role of visual spatial attention in audiovisual speech perception. Speech Commun 51:184–193
Biccino G, Solodkin A, Small SL (2006) Functions of the mirror neuron system: Implications for neurorehabilitation. Cogn Behav Neurol 19:56–63
Binkofski F, Buccino G (2004) Motor functions of the Broca’s region. Brain Lang 89:362–369
Brass M, Bekkering H, Wohlschlager A, Prinz W (2000) Compatibility between observed and executed finger movements: comparing symbolic, spatial, and imitative cues. Brain Cogn 44:124–143
Brass M, Bekkering H, Prinz W (2001) Movement observation affects movement execution in a simple response task. Acta Psychol 106:3–22
Calvert GA, Campbell R (2003) Reading speech from still and moving faces: The neural substrates of visible speech. J Cogn Neurosci 15:57–70
Di Pellegrino G, Fadiga L, Fogassi L, Gallese V, Rizzolatti G (1992) Understanding motor events: a neurophysiological study. Exp Brain Res 91:176–180
Fadiga L, Craighero L, Buccino G et al (2002) Speech listening specifically modulates the excitability of tongue muscles: a TMS study. Eur J Neurosci 15:399–402
Ferrari PF, Gallese V, Rizzolatti G, Fogassi L (2003) Mirror neurons responding to the observation of ingestive and communicative mouth actions in the monkey ventral premotor cortex. Eur J Neurosci 17:1703–1714
Fowler CA (2004) Speech as a supramodal or amodal phenomenon. In: Calvert GA, Spense C, Stein BE (eds) The handbook of multisensory processes. MIT Press, Cambridge
Heyes C, Bird G, Johnson H, Haggard P (2005) Experience modulates automatic imitation. Cogn Brain Res 22:233–240
Hickok G, Poeppel D (2004) Dorsal and ventral streams: a framework for understanding aspects of the functional anatomy of language. Cognition 92:67–99
Jarick M, Jones JA (2008) Observation of static gestures influences speech perception. Exp Brain Res 189:221–228
Kerzel D (2002) Evidence for effects of phonological correspondence between visible speech and written syllables. Psychol Res 66:195–200
Kerzel D, Bekkering H (2000) Motor activation from visible speech: evidence from stimulus response compatibility. J Exp Psychol Hum Percept Perform 26:634–647
Kohler E, Keysers C, Umilta MA, Fogassi L, Gallese V, Rizzolatti G (2002) Hearing sounds, understanding actions: action representation in mirror neurons. Science 297:846–848
Lahav A, Saltzman E, Schlaug G (2007) Action representation of sound: audiomotor recognition network while listening to newly acquired actions. J Neurosci 27:308–314
Leslie KR, Johnson-Frey SH, Grafton S (2004) Functional imaging of face and hand imitation: towards a motor theory of empathy. Neuroimage 21:601–607
Liberman AM, Mattingly I (1985) The motor theory revised. Cognition 21:1–36
McGurk H, MacDonald J (1976) Hearing lips and seeing voices. Nature 264:746–748
Mottonen R, Krause CM, Tiippana et al (2002) Processing of changes in visual speech in the human auditory cortex. Brain Res Cogn Brain Res 13:417–425
Munhall KG, Tohkura Y (1998) Audiovisual gating and the time course of speech perception. J Acoust Soc Am 104:530–539
Nishitani N, Hari R (2002) Viewing lips forms: cortical dynamics. Neuron 36:1211–1220
Petrides M, Pandya DN (1999) Dorsolateral prefrontal cortex: comparative cytoarchitectonic analysis in the human and the macaque brain and corticocortical connection patterns. Eur J Neurosci 11:1011–1136
Pulvermuller F, Huss M, Kherif F et al (2006) Motor cortex maps articulatory features of speech sounds. Proc Natl Acad Sci USA 103:7865–7870
Rizzolatti G, Arbib MA (1998) Language within our grasp. Trends Neurosci 21:188–194
Rizzolatti G, Craighero L (2004) The mirror-neuron system. Annu Rev Neurosci 27:169–192
Rizzolatti G, Fadiga L, Gallese V et al (1996) Premotor cortex and the recognition of motor actions. Brain Res Cogn Brain Res 3:131–141
Skipper JI, Nusbaum HC, Small SL (2005) Listening to talking faces: motor cortical activation during speech production. Neuroimage 25:76–89
Soto-Faraco S, Navarra J, Alsius A (2004) Assessing automaticity in audiovisual speech integration: evidence from the speeded classification task. Cognition 92:B13–B23
Sturmer B, Aschersleben G, Prinz W (2000) Correspondence effects with manual gestures and postures: a study of imitation. J Exp Psychol Hum Percept Perform 26:1746–1759
Summerfield Q (1987) Some preliminaries to a comprehensive account of audio-visual speech perception. In: Dodd B, Campbell R (eds) Hearing by eye: the psychology of lip-reading. Lawrence Erlbaum Associates, London
Sundara M, Namasivayam AK, Chen R (2001) Observation-execution matching system for speech: a magnetic stimulation study. Neuroreport 12:1341–1344
Tiippana K, Anderson TS, Sams M (2004) Visual attention modulates audiovisual speech perception. Eur J Cogn Psychol 16:457–472
Van Strien JW (1988) Handedness and hemispheric laterality. Dissertation, Vrije Universiteit, Amsterdam
Watkins K, Paus T (2004) Modulation of motor excitability during speech perception: the role of Broca’s area. J Cogn Neurosci 16:978–987
Watkins K, Strafella AP, Paus T (2003) Seeing and hearing speech excites the motor system involved in speech production. Neuropsychologia 41:989–994
Wilson SM, Saygin AP, Sereno MI, Iacoboni M (2004) Listening to speech activates motor areas involved in speech production. Nat Neurosci 7:701–702
Wohlschlager A, Bekkering H (2002) Is human imitation based on a mirror-neurone system? Some behavioural evidence. Exp Brain Res 143:341–355
Acknowledgements
This research was funded by a Natural Sciences and Engineering Research Council of Canada (NSERC) Discovery grant and Research Fellowship from Wilfrid Laurier University awarded to Jeffery A. Jones.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Jarick, M., Jones, J.A. Effects of seeing and hearing speech on speech production: a response time study. Exp Brain Res 195, 175–182 (2009). https://doi.org/10.1007/s00221-009-1765-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00221-009-1765-x