Advertisement

Brain Topography

, Volume 31, Issue 5, pp 838–847 | Cite as

Action-Related Speech Modulates Beta Oscillations During Observation of Tool-Use Gestures

  • Yifei HeEmail author
  • Miriam Steines
  • Gebhard Sammer
  • Arne Nagels
  • Tilo Kircher
  • Benjamin Straube
Original Paper

Abstract

Language and action have been thought of as closely related. Comprehending words or phrases that are related to actions commonly activates motor and premotor areas, and this comprehension process interacts with action preparation and/or execution. However, it remains unclear whether comprehending action-related language interacts with action observation. In the current study, we examined whether the observation of tool-use gesture subjects to interaction with language. In an electroencephalography (EEG) study (n = 20), participants were presented with video clips of an actor performing tool-use (TU, e.g., hammering with a fist) and emblematic (EM, e.g., the thumb up sign for ‘good job’) gestures accompanied by either comprehensible German (G) or incomprehensible Russian sentences (R). Participants performed a semantic judging task, evaluating whether the co-speech gestures were object- or socially-related. Behavioral results from the semantic task showed faster response for the TU versus EM gestures only in the German condition. For EEG, we found that TU elicited beta power decrease (~ 20 Hz) when compared to EM gestures, however this effect was reduced when gestures were accompanied by German instead of Russian sentences. We concluded that the processing of action-related sentences might facilitate gesture observation, in the sense that motor simulation required for TU gestures, as indexed by reduced beta power, was modulated when accompanied by comprehensible German speech. Our results corroborate the functional role of the beta oscillations during perception of hand gestures, and provide novel evidence concerning language–motor interaction.

Keywords

Action observation Beta oscillations Tool-use gesture Language–motor interaction Embodied cognition 

Notes

Acknowledgements

This research project is supported by a grant from the ‘Von Behring-Röntgen-Stiftung’ (Project No. 59-0002; 64-0001) and by the ‘Deutsche Forschungsgemeinschaft’ (Project No. DFG: STR 1146/9-1 and SFB/TRR135 project A3). M.S. is supported by the DFG (Project No. STR 1146/4-1). B.S. is supported by the DFG (Project No. STR 1146/8-1). Part of the work of this paper has been presented at the Tilburg Gesture Research meeting (TiGER 2013).

References

  1. Agnew ZK, Wise RJ, Leech R (2012) Dissociating object directed and non-object directed action in the human mirror system; implications for theories of motor simulation. PLoS ONE 7:e32517CrossRefPubMedPubMedCentralGoogle Scholar
  2. Andric M, Solodkin A, Buccino G, Goldin-Meadow S, Rizzolatti G, Small SL (2013) Brain function overlaps when people observe emblems, speech, and grasping. Neuropsychologia 51:1619–1629CrossRefPubMedGoogle Scholar
  3. Arnstein D, Cui F, Keysers C, Maurits NM, Gazzola V (2011) µ-suppression during action observation and execution correlates with BOLD in dorsal premotor, inferior parietal, and SI cortices. J Neurosci 31:14243–14249CrossRefPubMedGoogle Scholar
  4. Avanzini P, Fabbri-Destro M, Volta D, Daprati R, Rizzolatti E, Cantalupo G, G (2012) The dynamics of sensorimotor cortical oscillations during the observation of hand movements: an EEG study. PLoS ONE 7:e37534CrossRefPubMedPubMedCentralGoogle Scholar
  5. Barsalou LW (2008) Grounded cognition. Annu Rev Psychol 59:617–645CrossRefPubMedGoogle Scholar
  6. Biau E, Soto-Faraco S (2013) Beat gestures modulate auditory integration in speech perception. Brain Lang 124:143–152CrossRefPubMedGoogle Scholar
  7. Boulenger V, Roy AC, Paulignan Y, Deprez V, Jeannerod M, Nazir TA (2006) Cross-talk between language processes and overt motor behavior in the first 200 msec of processing. J Cogn Neurosci 18:1607–1615CrossRefPubMedGoogle Scholar
  8. Boulenger V, Silber BY, Roy AC, Paulignan Y, Jeannerod M, Nazir TA (2008) Subliminal display of action words interferes with motor planning: a combined EEG and kinematic study. J Physiol (Paris) 102:130–136CrossRefGoogle Scholar
  9. Brinkman L, Stolk A, Dijkerman HC, de Lange FP, Toni I (2014) Distinct roles for alpha-and beta-band oscillations during mental simulation of goal-directed actions. J Neurosci 34:14783–14792CrossRefPubMedPubMedCentralGoogle Scholar
  10. Buccino G, Binkofski F, Fink GR, Fadiga L, Fogassi L, Gallese V, Seitz RJ, Zilles K, Rizzolatti G, Freund HJ (2001) Action observation activates premotor and parietal areas in a somatotopic manner: an fMRI study. Eur J Neurosci 13:400–404PubMedGoogle Scholar
  11. Buccino G, Riggio L, Melli G, Binkofski F, Gallese V, Rizzolatti G (2005) Listening to action-related sentences modulates the activity of the motor system: a combined TMS and behavioral study. Cogn Brain Res 24:355–363CrossRefGoogle Scholar
  12. Chersi F, Thill S, Ziemke T, Borghi AM (2010) Sentence processing: linking language to motor chains. Frontiers in neurorobotics 4:4CrossRefPubMedPubMedCentralGoogle Scholar
  13. Gallese V, Fadiga L, Fogassi L, Rizzolatti G (1996) Action recognition in the premotor cortex. Brain 119:593–609CrossRefPubMedGoogle Scholar
  14. Goldin-Meadow S (2005) Hearing gesture: how our hands help us think. Harvard University Press, CambridgeGoogle Scholar
  15. Green A, Straube B, Weis S, Jansen A, Willmes K, Konrad K, Kircher T (2009) Neural integration of iconic and unrelated coverbal gestures: a functional MRI study. Hum Brain Mapp 30:3309–3324CrossRefPubMedGoogle Scholar
  16. Groppe DM, Urbach TP, Kutas M (2011) Mass univariate analysis of event-related brain potentials/fields I: a critical tutorial review. Psychophysiology 48:1711–1725CrossRefPubMedPubMedCentralGoogle Scholar
  17. Hari R, Forss N, Avikainen S, Kirveskari E, Salenius S, Rizzolatti G (1998) Activation of human primary motor cortex during action observation: a neuromagnetic study. Proc Natl Acad Sci 95:15061–15065CrossRefPubMedGoogle Scholar
  18. Hauk O, Johnsrude I, Pulvermüller F (2004) Somatotopic representation of action words in human motor and premotor cortex. Neuron 41:301–307CrossRefPubMedGoogle Scholar
  19. He Y, Gebhardt H, Steines M, Sammer G, Kircher T, Nagels A, Straube B (2015) The EEG and fMRI signatures of neural integration: an investigation of meaningful gestures and corresponding speech. Neuropsychologia 72:27–42CrossRefPubMedGoogle Scholar
  20. Higuchi S, Imamizu H, Kawato M (2007) Cerebellar activity evoked by common tool-use execution and imagery tasks: an fMRI study. Cortex 43:350–358CrossRefPubMedGoogle Scholar
  21. Holle H, Obermeier C, Schmidt-Kassow M, Friederici AD, Ward J, Gunter TC (2012) Gesture facilitates the syntactic analysis of speech. Front Psychol 3:74CrossRefPubMedPubMedCentralGoogle Scholar
  22. Järveläinen J, Schuermann M, Hari R (2004) Activation of the human primary motor cortex during observation of tool use. Neuroimage 23:187–192CrossRefPubMedGoogle Scholar
  23. Johnson-Frey SH (2004) The neural bases of complex tool use in humans. Trends Cogn Sci 8:71–78CrossRefPubMedGoogle Scholar
  24. Johnson-Frey SH, Newman-Norlund R, Grafton ST (2005) A distributed left hemisphere network active during planning of everyday tool use skills. Cereb Cortex 15:681–695CrossRefPubMedGoogle Scholar
  25. Kendon A (1980) Gesticulation and speech: Two aspects of the process of utterance. Relat Verbal Nonverbal Commun 25:207–227Google Scholar
  26. Kircher T, Straube B, Leube D, Weis S, Sachs O, Willmes K, Konrad K, Green A (2009) Neural interaction of speech and gesture: differential activations of metaphoric co-verbal gestures. Neuropsychologia 47:169–179CrossRefPubMedGoogle Scholar
  27. Kita S, Özyürek A (2003) What does cross-linguistic variation in semantic coordination of speech and gesture reveal?: Evidence for an interface representation of spatial thinking and speaking. J Mem Lang 48:16–32CrossRefGoogle Scholar
  28. Klepp A, Niccolai V, Buccino G, Schnitzler A, Biermann-Ruben K (2015) Language–motor interference reflected in MEG beta oscillations. Neuroimage 109:438–448CrossRefPubMedGoogle Scholar
  29. Krahmer E, Swerts M (2007) The effects of visual beats on prosodic prominence: acoustic analyses, auditory perception and visual perception. J Mem Lang 57:396–414CrossRefGoogle Scholar
  30. Lindenberg R, Uhlig M, Scherfeld D, Schlaug G, Seitz RJ (2012) Communication with emblematic gestures: shared and distinct neural correlates of expression and reception. Hum Brain Mapp 33:812–823CrossRefPubMedGoogle Scholar
  31. Maris E, Oostenveld R (2007) Nonparametric statistical testing of EEG- and MEG-data. J Neurosci Methods 164:177–190CrossRefPubMedGoogle Scholar
  32. McNeill D (1992) Hand and mind: What gestures reveal about thought. University of Chicago Press, ChicagoGoogle Scholar
  33. McNeill D (2006) Gesture: a psycholinguistic approach. In: Brown K (ed) The encyclopedia of language linguistics. Elsevier, Boston, pp 58–66CrossRefGoogle Scholar
  34. McNeill D (2008) Gesture and thought. University of Chicago Press, ChicagoGoogle Scholar
  35. Moreno I, De Vega M, León I (2013) Understanding action language modulates oscillatory mu and beta rhythms in the same way as observing actions. Brain Cogn 82:236–242CrossRefPubMedGoogle Scholar
  36. Moreno I, De Vega M, León I, Bastiaansen M, Lewis AG, Magyari L (2015) Brain dynamics in the comprehension of action-related language. A time-frequency analysis of mu rhythms. Neuroimage 109:50–62CrossRefPubMedGoogle Scholar
  37. Muthukumaraswamy SD, Johnson BW, McNair NA (2004) Mu rhythm modulation during observation of an object-directed grasp. Cogn Brain Res 19:195–201CrossRefGoogle Scholar
  38. Newman-Norlund R, van Schie HT, van Hoek ME, Cuijpers RH, Bekkering H (2010) The role of inferior frontal and parietal areas in differentiating meaningful and meaningless object-directed actions. Brain Res 1315:63–74CrossRefPubMedGoogle Scholar
  39. Oldfield RC (1971) The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9:97–113CrossRefPubMedGoogle Scholar
  40. Perry A, Bentin S (2009) Mirror activity in the human brain while observing hand movements: a comparison between EEG desynchronization in the µ-range and previous fMRI results. Brain Res 1282:126–132CrossRefPubMedGoogle Scholar
  41. Pulvermuller F (2005) Brain mechanisms linking language and action. Nat Rev Neurosci 6:576–582CrossRefPubMedGoogle Scholar
  42. Pulvermüller F, Hauk O, Nikulin VV, Ilmoniemi RJ (2005) Functional links between motor and language systems. Eur J Neurosci 21:793–797CrossRefPubMedGoogle Scholar
  43. Quandt LC, Marshall PJ, Shipley TF, Beilock SL, Goldin-Meadow S (2012) Sensitivity of alpha and beta oscillations to sensorimotor characteristics of action: An EEG study of action production and gesture observation. Neuropsychologia 50:2745–2751CrossRefPubMedPubMedCentralGoogle Scholar
  44. Rizzolatti G, Fogassi L, Gallese V (2001) Neurophysiological mechanisms underlying the understanding and imitation of action. Nat Rev Neurosci 2:661–670CrossRefPubMedGoogle Scholar
  45. Salmelin R, Hámáaláinen M, Kajola M, Hari R (1995) Functional segregation of movement-related rhythmic activity in the human brain. Neuroimage 2:237–243CrossRefPubMedGoogle Scholar
  46. Sato M, Mengarelli M, Riggio L, Gallese V, Buccino G (2008) Task related modulation of the motor system during language processing. Brain Lang 105:83–90CrossRefPubMedGoogle Scholar
  47. Schaller F, Weiss S, Müller HM (2017) EEG beta-power changes reflect motor involvement in abstract action language processing. Brain Lang 168:95–105CrossRefPubMedGoogle Scholar
  48. Scorolli C, Borghi AM (2007) Sentence comprehension and action: Effector specific modulation of the motor system. Brain Res 1130:119–124CrossRefPubMedGoogle Scholar
  49. Skipper JI, Goldin-Meadow S, Nusbaum HC, Small SL (2009) Gestures orchestrate brain networks for language understanding. Curr Biol 19:661–667CrossRefPubMedPubMedCentralGoogle Scholar
  50. Straube B, Green A, Bromberger B, Kircher T (2011) The differentiation of iconic and metaphoric gestures: Common and unique integration processes. Hum Brain Mapp 32:520–533CrossRefPubMedGoogle Scholar
  51. Straube B, Green A, Weis S, Kircher T (2012) A supramodal neural network for speech and gesture semantics: an fMRI study. PLoS ONE 7:e51207CrossRefPubMedPubMedCentralGoogle Scholar
  52. Villarreal M, Fridman EA, Amengual A, Falasco G, Gerscovich ER, Ulloa ER, Leiguarda RC (2008) The neural substrate of gesture recognition. Neuropsychologia 46:2371–2382CrossRefPubMedGoogle Scholar
  53. Wang L, Chu M (2013) The role of beat gesture and pitch accent in semantic processing: an ERP study. Neuropsychologia 51:2847–2855CrossRefPubMedGoogle Scholar
  54. Willems RM, Hagoort P (2007) Neural evidence for the interplay between language, gesture, and action: a review. Brain Lang 101:278–289CrossRefPubMedGoogle Scholar
  55. Willems RM, Labruna L, D’Esposito M, Ivry R, Casasanto D (2011) A Functional role for the motor system in language understanding evidence from theta-burst transcranial magnetic stimulation. Psychol Sci 22:849–854CrossRefPubMedGoogle Scholar
  56. Wriessnegger S, Leeb R, Kaiser V, Neuper C, Müller-Putz G (2013) Watching object related movements modulates mirror-like activity in parietal brain regions. Clin Neurophysiol 124:1596–1604CrossRefPubMedGoogle Scholar
  57. Xu J, Gannon PJ, Emmorey K, Smith JF, Braun AR (2009) Symbolic gestures and spoken language are processed by a common neural system. Proc Natl Acad Sci 106:20664–20669CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Psychiatry and PsychotherapyPhilipps-University MarburgMarburgGermany
  2. 2.Department of General LinguisticsJohannes-Gutenberg University MainzMainzGermany
  3. 3.Cognitive Neuroscience at Centre for PsychiatryJustus Liebig University GiessenGiessenGermany

Personalised recommendations