Brain Structure and Function

, Volume 220, Issue 2, pp 1109–1125

Effects of congruent and incongruent visual cues on speech perception and brain activity in cochlear implant users

  • Jae-Jin Song
  • Hyo-Jeong Lee
  • Hyejin Kang
  • Dong Soo Lee
  • Sun O. Chang
  • Seung Ha Oh
Original Article

DOI: 10.1007/s00429-013-0704-6

Cite this article as:
Song, JJ., Lee, HJ., Kang, H. et al. Brain Struct Funct (2015) 220: 1109. doi:10.1007/s00429-013-0704-6

Abstract

While deafness-induced plasticity has been investigated in the visual and auditory domains, not much is known about language processing in audiovisual multimodal environments for patients with restored hearing via cochlear implant (CI) devices. Here, we examined the effect of agreeing or conflicting visual inputs on auditory processing in deaf patients equipped with degraded artificial hearing. Ten post-lingually deafened CI users with good performance, along with matched control subjects, underwent H215O-positron emission tomography scans while carrying out a behavioral task requiring the extraction of speech information from unimodal auditory stimuli, bimodal audiovisual congruent stimuli, and incongruent stimuli. Regardless of congruency, the control subjects demonstrated activation of the auditory and visual sensory cortices, as well as the superior temporal sulcus, the classical multisensory integration area, indicating a bottom-up multisensory processing strategy. Compared to CI users, the control subjects exhibited activation of the right ventral premotor-supramarginal pathway. In contrast, CI users activated primarily the visual cortices more in the congruent audiovisual condition than in the null condition. In addition, compared to controls, CI users displayed an activation focus in the right amygdala for congruent audiovisual stimuli. The most notable difference between the two groups was an activation focus in the left inferior frontal gyrus in CI users confronted with incongruent audiovisual stimuli, suggesting top-down cognitive modulation for audiovisual conflict. Correlation analysis revealed that good speech performance was positively correlated with right amygdala activity for the congruent condition, but negatively correlated with bilateral visual cortices regardless of congruency. Taken together these results suggest that for multimodal inputs, cochlear implant users are more vision-reliant when processing congruent stimuli and are disturbed more by visual distractors when confronted with incongruent audiovisual stimuli. To cope with this multimodal conflict, CI users activate the left inferior frontal gyrus to adopt a top-down cognitive modulation pathway, whereas normal hearing individuals primarily adopt a bottom-up strategy.

Keywords

Cochlear implantDeafnessPositron emission tomographyAudiovisualPlasticity

Abbreviations

NH

Normal hearing

AV

Audiovisual

STS

Superior temporal sulcus

IFG

Inferior frontal gyrus

CI

Cochlear implant

PET

Positron emission tomography

MRI

Magnetic resonance imaging

CAP

Categories of auditory performance

MTG

Middle temporal gyrus

ITG

Inferior temporal gyrus

rCBF

Regional cerebral blood flow

vPMC

Ventral premotor cortex

SMG

Supramarginal gyrus

SFG

Superior frontal gyrus

MeFG

Medial frontal gyrus

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Jae-Jin Song
    • 1
  • Hyo-Jeong Lee
    • 2
    • 3
  • Hyejin Kang
    • 4
  • Dong Soo Lee
    • 4
  • Sun O. Chang
    • 3
    • 5
  • Seung Ha Oh
    • 3
    • 5
  1. 1.Department of Otorhinolaryngology-Head and Neck SurgerySeoul National University Bundang HospitalSeongnamKorea
  2. 2.Department of Otorhinolaryngology-Head and Neck SurgeryHallym University College of MedicineChun-CheonKorea
  3. 3.Sensory Organ Research Institute, Seoul National University Medical Research CenterSeoulKorea
  4. 4.Department of Nuclear MedicineSeoul National University HospitalSeoulKorea
  5. 5.Department of Otorhinolaryngology-Head and Neck SurgerySeoul National University HospitalSeoulKorea