Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Action-Related Speech Modulates Beta Oscillations During Observation of Tool-Use Gestures

Abstract

Language and action have been thought of as closely related. Comprehending words or phrases that are related to actions commonly activates motor and premotor areas, and this comprehension process interacts with action preparation and/or execution. However, it remains unclear whether comprehending action-related language interacts with action observation. In the current study, we examined whether the observation of tool-use gesture subjects to interaction with language. In an electroencephalography (EEG) study (n = 20), participants were presented with video clips of an actor performing tool-use (TU, e.g., hammering with a fist) and emblematic (EM, e.g., the thumb up sign for ‘good job’) gestures accompanied by either comprehensible German (G) or incomprehensible Russian sentences (R). Participants performed a semantic judging task, evaluating whether the co-speech gestures were object- or socially-related. Behavioral results from the semantic task showed faster response for the TU versus EM gestures only in the German condition. For EEG, we found that TU elicited beta power decrease (~ 20 Hz) when compared to EM gestures, however this effect was reduced when gestures were accompanied by German instead of Russian sentences. We concluded that the processing of action-related sentences might facilitate gesture observation, in the sense that motor simulation required for TU gestures, as indexed by reduced beta power, was modulated when accompanied by comprehensible German speech. Our results corroborate the functional role of the beta oscillations during perception of hand gestures, and provide novel evidence concerning language–motor interaction.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3

References

  1. Agnew ZK, Wise RJ, Leech R (2012) Dissociating object directed and non-object directed action in the human mirror system; implications for theories of motor simulation. PLoS ONE 7:e32517

  2. Andric M, Solodkin A, Buccino G, Goldin-Meadow S, Rizzolatti G, Small SL (2013) Brain function overlaps when people observe emblems, speech, and grasping. Neuropsychologia 51:1619–1629

  3. Arnstein D, Cui F, Keysers C, Maurits NM, Gazzola V (2011) µ-suppression during action observation and execution correlates with BOLD in dorsal premotor, inferior parietal, and SI cortices. J Neurosci 31:14243–14249

  4. Avanzini P, Fabbri-Destro M, Volta D, Daprati R, Rizzolatti E, Cantalupo G, G (2012) The dynamics of sensorimotor cortical oscillations during the observation of hand movements: an EEG study. PLoS ONE 7:e37534

  5. Barsalou LW (2008) Grounded cognition. Annu Rev Psychol 59:617–645

  6. Biau E, Soto-Faraco S (2013) Beat gestures modulate auditory integration in speech perception. Brain Lang 124:143–152

  7. Boulenger V, Roy AC, Paulignan Y, Deprez V, Jeannerod M, Nazir TA (2006) Cross-talk between language processes and overt motor behavior in the first 200 msec of processing. J Cogn Neurosci 18:1607–1615

  8. Boulenger V, Silber BY, Roy AC, Paulignan Y, Jeannerod M, Nazir TA (2008) Subliminal display of action words interferes with motor planning: a combined EEG and kinematic study. J Physiol (Paris) 102:130–136

  9. Brinkman L, Stolk A, Dijkerman HC, de Lange FP, Toni I (2014) Distinct roles for alpha-and beta-band oscillations during mental simulation of goal-directed actions. J Neurosci 34:14783–14792

  10. Buccino G, Binkofski F, Fink GR, Fadiga L, Fogassi L, Gallese V, Seitz RJ, Zilles K, Rizzolatti G, Freund HJ (2001) Action observation activates premotor and parietal areas in a somatotopic manner: an fMRI study. Eur J Neurosci 13:400–404

  11. Buccino G, Riggio L, Melli G, Binkofski F, Gallese V, Rizzolatti G (2005) Listening to action-related sentences modulates the activity of the motor system: a combined TMS and behavioral study. Cogn Brain Res 24:355–363

  12. Chersi F, Thill S, Ziemke T, Borghi AM (2010) Sentence processing: linking language to motor chains. Frontiers in neurorobotics 4:4

  13. Gallese V, Fadiga L, Fogassi L, Rizzolatti G (1996) Action recognition in the premotor cortex. Brain 119:593–609

  14. Goldin-Meadow S (2005) Hearing gesture: how our hands help us think. Harvard University Press, Cambridge

  15. Green A, Straube B, Weis S, Jansen A, Willmes K, Konrad K, Kircher T (2009) Neural integration of iconic and unrelated coverbal gestures: a functional MRI study. Hum Brain Mapp 30:3309–3324

  16. Groppe DM, Urbach TP, Kutas M (2011) Mass univariate analysis of event-related brain potentials/fields I: a critical tutorial review. Psychophysiology 48:1711–1725

  17. Hari R, Forss N, Avikainen S, Kirveskari E, Salenius S, Rizzolatti G (1998) Activation of human primary motor cortex during action observation: a neuromagnetic study. Proc Natl Acad Sci 95:15061–15065

  18. Hauk O, Johnsrude I, Pulvermüller F (2004) Somatotopic representation of action words in human motor and premotor cortex. Neuron 41:301–307

  19. He Y, Gebhardt H, Steines M, Sammer G, Kircher T, Nagels A, Straube B (2015) The EEG and fMRI signatures of neural integration: an investigation of meaningful gestures and corresponding speech. Neuropsychologia 72:27–42

  20. Higuchi S, Imamizu H, Kawato M (2007) Cerebellar activity evoked by common tool-use execution and imagery tasks: an fMRI study. Cortex 43:350–358

  21. Holle H, Obermeier C, Schmidt-Kassow M, Friederici AD, Ward J, Gunter TC (2012) Gesture facilitates the syntactic analysis of speech. Front Psychol 3:74

  22. Järveläinen J, Schuermann M, Hari R (2004) Activation of the human primary motor cortex during observation of tool use. Neuroimage 23:187–192

  23. Johnson-Frey SH (2004) The neural bases of complex tool use in humans. Trends Cogn Sci 8:71–78

  24. Johnson-Frey SH, Newman-Norlund R, Grafton ST (2005) A distributed left hemisphere network active during planning of everyday tool use skills. Cereb Cortex 15:681–695

  25. Kendon A (1980) Gesticulation and speech: Two aspects of the process of utterance. Relat Verbal Nonverbal Commun 25:207–227

  26. Kircher T, Straube B, Leube D, Weis S, Sachs O, Willmes K, Konrad K, Green A (2009) Neural interaction of speech and gesture: differential activations of metaphoric co-verbal gestures. Neuropsychologia 47:169–179

  27. Kita S, Özyürek A (2003) What does cross-linguistic variation in semantic coordination of speech and gesture reveal?: Evidence for an interface representation of spatial thinking and speaking. J Mem Lang 48:16–32

  28. Klepp A, Niccolai V, Buccino G, Schnitzler A, Biermann-Ruben K (2015) Language–motor interference reflected in MEG beta oscillations. Neuroimage 109:438–448

  29. Krahmer E, Swerts M (2007) The effects of visual beats on prosodic prominence: acoustic analyses, auditory perception and visual perception. J Mem Lang 57:396–414

  30. Lindenberg R, Uhlig M, Scherfeld D, Schlaug G, Seitz RJ (2012) Communication with emblematic gestures: shared and distinct neural correlates of expression and reception. Hum Brain Mapp 33:812–823

  31. Maris E, Oostenveld R (2007) Nonparametric statistical testing of EEG- and MEG-data. J Neurosci Methods 164:177–190

  32. McNeill D (1992) Hand and mind: What gestures reveal about thought. University of Chicago Press, Chicago

  33. McNeill D (2006) Gesture: a psycholinguistic approach. In: Brown K (ed) The encyclopedia of language linguistics. Elsevier, Boston, pp 58–66

  34. McNeill D (2008) Gesture and thought. University of Chicago Press, Chicago

  35. Moreno I, De Vega M, León I (2013) Understanding action language modulates oscillatory mu and beta rhythms in the same way as observing actions. Brain Cogn 82:236–242

  36. Moreno I, De Vega M, León I, Bastiaansen M, Lewis AG, Magyari L (2015) Brain dynamics in the comprehension of action-related language. A time-frequency analysis of mu rhythms. Neuroimage 109:50–62

  37. Muthukumaraswamy SD, Johnson BW, McNair NA (2004) Mu rhythm modulation during observation of an object-directed grasp. Cogn Brain Res 19:195–201

  38. Newman-Norlund R, van Schie HT, van Hoek ME, Cuijpers RH, Bekkering H (2010) The role of inferior frontal and parietal areas in differentiating meaningful and meaningless object-directed actions. Brain Res 1315:63–74

  39. Oldfield RC (1971) The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9:97–113

  40. Perry A, Bentin S (2009) Mirror activity in the human brain while observing hand movements: a comparison between EEG desynchronization in the µ-range and previous fMRI results. Brain Res 1282:126–132

  41. Pulvermuller F (2005) Brain mechanisms linking language and action. Nat Rev Neurosci 6:576–582

  42. Pulvermüller F, Hauk O, Nikulin VV, Ilmoniemi RJ (2005) Functional links between motor and language systems. Eur J Neurosci 21:793–797

  43. Quandt LC, Marshall PJ, Shipley TF, Beilock SL, Goldin-Meadow S (2012) Sensitivity of alpha and beta oscillations to sensorimotor characteristics of action: An EEG study of action production and gesture observation. Neuropsychologia 50:2745–2751

  44. Rizzolatti G, Fogassi L, Gallese V (2001) Neurophysiological mechanisms underlying the understanding and imitation of action. Nat Rev Neurosci 2:661–670

  45. Salmelin R, Hámáaláinen M, Kajola M, Hari R (1995) Functional segregation of movement-related rhythmic activity in the human brain. Neuroimage 2:237–243

  46. Sato M, Mengarelli M, Riggio L, Gallese V, Buccino G (2008) Task related modulation of the motor system during language processing. Brain Lang 105:83–90

  47. Schaller F, Weiss S, Müller HM (2017) EEG beta-power changes reflect motor involvement in abstract action language processing. Brain Lang 168:95–105

  48. Scorolli C, Borghi AM (2007) Sentence comprehension and action: Effector specific modulation of the motor system. Brain Res 1130:119–124

  49. Skipper JI, Goldin-Meadow S, Nusbaum HC, Small SL (2009) Gestures orchestrate brain networks for language understanding. Curr Biol 19:661–667

  50. Straube B, Green A, Bromberger B, Kircher T (2011) The differentiation of iconic and metaphoric gestures: Common and unique integration processes. Hum Brain Mapp 32:520–533

  51. Straube B, Green A, Weis S, Kircher T (2012) A supramodal neural network for speech and gesture semantics: an fMRI study. PLoS ONE 7:e51207

  52. Villarreal M, Fridman EA, Amengual A, Falasco G, Gerscovich ER, Ulloa ER, Leiguarda RC (2008) The neural substrate of gesture recognition. Neuropsychologia 46:2371–2382

  53. Wang L, Chu M (2013) The role of beat gesture and pitch accent in semantic processing: an ERP study. Neuropsychologia 51:2847–2855

  54. Willems RM, Hagoort P (2007) Neural evidence for the interplay between language, gesture, and action: a review. Brain Lang 101:278–289

  55. Willems RM, Labruna L, D’Esposito M, Ivry R, Casasanto D (2011) A Functional role for the motor system in language understanding evidence from theta-burst transcranial magnetic stimulation. Psychol Sci 22:849–854

  56. Wriessnegger S, Leeb R, Kaiser V, Neuper C, Müller-Putz G (2013) Watching object related movements modulates mirror-like activity in parietal brain regions. Clin Neurophysiol 124:1596–1604

  57. Xu J, Gannon PJ, Emmorey K, Smith JF, Braun AR (2009) Symbolic gestures and spoken language are processed by a common neural system. Proc Natl Acad Sci 106:20664–20669

Download references

Acknowledgements

This research project is supported by a grant from the ‘Von Behring-Röntgen-Stiftung’ (Project No. 59-0002; 64-0001) and by the ‘Deutsche Forschungsgemeinschaft’ (Project No. DFG: STR 1146/9-1 and SFB/TRR135 project A3). M.S. is supported by the DFG (Project No. STR 1146/4-1). B.S. is supported by the DFG (Project No. STR 1146/8-1). Part of the work of this paper has been presented at the Tilburg Gesture Research meeting (TiGER 2013).

Author information

Correspondence to Yifei He.

Additional information

Handling Editor: Micah M. Murray.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

He, Y., Steines, M., Sammer, G. et al. Action-Related Speech Modulates Beta Oscillations During Observation of Tool-Use Gestures. Brain Topogr 31, 838–847 (2018). https://doi.org/10.1007/s10548-018-0641-z

Download citation

Keywords

  • Action observation
  • Beta oscillations
  • Tool-use gesture
  • Language–motor interaction
  • Embodied cognition