HNO

, Volume 61, Issue 3, pp 202–210

Beeinflussung objektbildender Leistungen des Hörsinns durch multimodale Interaktion

Leitthema
  • 144 Downloads

Zusammenfassung

Hintergrund

Die Zuordnung gleichzeitig auftretender Schallreize zu verschiedenen auditorischen Objekten geschieht auf der Basis ihrer zeitlichen und spektralen Eigenschaften. Wenn hohe und tiefe Töne abwechselnd präsentiert werden, können diese als eine zusammenhängende Melodie, oder aber auch als zwei parallele, aus den Tönen der jeweiligen Frequenz bestehende Melodien wahrgenommen werden. In einem Zwischenbereich sind beide Perzepte möglich. Diese Arbeit untersucht, inwieweit diese ambige Wahrnehmungssituation durch synchron zu den Tönen präsentierte visuelle Reize beeinflusst werden kann und damit zu einem eindeutigen auditorischen Perzept führt.

Methoden

Töne aus zwei verschiedenen Frequenzbereichen wurden abwechselnd präsentiert, mit visuellen Stimuli entweder synchron zu dem tieffrequenten 3-Ton-Muster (Tripel) oder dem frequenzübergreifenden Intensitätsmuster. Als Indikator für eine segregierte auditorische Organisation ohne Aufmerksamkeit der Probanden auf die Töne wurde die Mismatch-Negativity (MMN) als Komponente der auditorisch evozierten Potenziale gemessen.

Ergebnisse

Die Ergebnisse zeigen einen modalitätsübergreifenden Effekt auf die auditorische Objektbildung. Die zunächst ambige auditorische Situation wurde durch synchron präsentierte visuelle Stimuli, die entweder eine segregierte oder integrierte Wahrnehmung der Töne verstärken sollten, aufgelöst. Vier neuronale Quellen konnten gefunden werden, die die beobachtete MMN erklären.

Schlüsselwörter

Auditorische Szenenanalyse Mismatch-Negativity Visuelle Wahrnehmung Audiovisuelle Integration Auditorische Wahrnehmung 

Influence of auditory object formation by multimodal interaction

Abstract

Background

The task of assigning concurrent sounds to different auditory objects is known to depend on temporal and spectral cues. When tones of high and low frequencies are presented in alternation, they can be perceived as a single (integrated) melody, or as two parallel (segregated) melodic lines, according to the presentation rate and frequency distance between the sounds. At an intermediate distance or stimulation rate, the percept is ambiguous and alternates between segregated and integrated. This work studied whether an ambiguous sound organization could be modulated towards a robust integrated or a segregated percept by the synchronous presentation of visual cues.

Methods

Two interleaved sets of sounds, one high frequency and one low frequency set were presented with concurrent visual stimuli, synchronized to either a within-set frequency pattern or to the across-set intensity pattern. Elicitation of the mismatch negativity (MMN) component of event-related brain potentials served as indices for the segregated organization, when no task was performed with the sounds. As a result, MMN was elicited only when the visual pattern promoted the segregation of the sounds.

Results

By spatial analysis of the distribution of electromagnetic potentials, four separated neuronal sources underlying the obtained MMN response were identified. One pair was located bilaterally in temporal cortical structures and another pair in occipital areas, representing the auditory and visual origin of the MMN response, evoked by inverted triplets as used in this study. Thus, the results demonstrate cross-modal effects of visual information on auditory object perception.

Keywords

Auditory scene analysis Mismatch negativity Visual perception Audiovisual integration Auditory perception 

Literatur

  1. 1.
    Bregman AS (1990) Auditory scene analysis: the perceptual organisation of sounds. The MIT Press, Cambridge, MassachusettsGoogle Scholar
  2. 2.
    Miller GA (1947) The masking of speech. Psychol Bull 44:105–129PubMedCrossRefGoogle Scholar
  3. 3.
    Bodenmann G, Schaer M (2006) Gestaltpsychologie. Sprache Stimme Gehor 30:21–23CrossRefGoogle Scholar
  4. 4.
    Bozzi P, Vicario G (1960) Due fattori di unificazione fra note musicali: La vicinanza temporale e la vicinanza tonale. Rivista di Psycologia 54:235–258Google Scholar
  5. 5.
    Warren RM (1968) Relation of verbal transformations to other pereceptual phenomena, Institution of Electrical Engineers, London, S 1–8Google Scholar
  6. 6.
    Bregman AS, Campbell KB (1971) Primary auditory stream segregation and perception of order in rapid sequences of tones. J Exp Psychol 89:244–249PubMedCrossRefGoogle Scholar
  7. 7.
    Noorden LPAS van (1977) Minimum differences of level and frequency for perceptual fission of tone sequences ABAB. J Acoust Soc Am 61:1041–1045PubMedCrossRefGoogle Scholar
  8. 8.
    Noorden LPAS van (1975) Temporal coherence in the perception of tone sequences. Dissertation, Eindhoven University of Technology, Eindhoven, NiederlandeGoogle Scholar
  9. 9.
    Fishman Y, Arezzo JC, Steinschneider M (2004) Auditory stream segragtion in monkey auditory cortex: effects of frequency separation, presentation rate, and tone duration. J Acoust Soc Am 116:1656–1670PubMedCrossRefGoogle Scholar
  10. 10.
    Bey C, McAdams S (2002) Schema-based processing in auditory scene analysis. Percept Psychophys 64:844–854PubMedCrossRefGoogle Scholar
  11. 11.
    Sussman E, Ritter W, Vaughan HG Jr (1998) Attention affects the organization of auditory input associated with the mismatch negativity system. Brain Res 789:130–138PubMedCrossRefGoogle Scholar
  12. 12.
    Dodd B (1977) The role of vision in the perception of speech. Perception 6:31–40PubMedCrossRefGoogle Scholar
  13. 13.
    McGurk H, MacDonald J (1976) Hearing lips and seeing voices. Nature 264:746–748PubMedCrossRefGoogle Scholar
  14. 14.
    Nicholls MER, Searle DA, Bradshaw JL (2004) Read My Lips. Res Report 15:138–141Google Scholar
  15. 15.
    Bertelson P, Vroomen J, De Gelder B, Driver J (2000) The ventriloquist effect does not depend on the direction of deliberate visual attention. Percept Psychophys 62:321–332PubMedCrossRefGoogle Scholar
  16. 16.
    Colin C, Radeau M, Soquet A et al (2002) Electrophysiology of spatial scene analysis: the mismatch negativity (MMN) is sensitive to the ventriloquism illusion. Clin Neurophysiol 113:507–518PubMedCrossRefGoogle Scholar
  17. 17.
    Shams L, Kamitani Y, Shimojo S (2000) Illusions: what you see is what you hear. Nature 408:788PubMedCrossRefGoogle Scholar
  18. 18.
    Guttman SE, Gilroy LA, Blake R (2005) Hearing what the eyes see: auditory encoding of visual temporal sequences. Psychol Sci 16:228–235PubMedCrossRefGoogle Scholar
  19. 19.
    Frassinetti F, Bolognini N, Bottari D et al (2005) Audiovisual Integration in Patients with Visual Deficit. J Cogn Neurosci 17:1442–1452PubMedCrossRefGoogle Scholar
  20. 20.
    Sussman ES, Ritter W, Vaughan HG Jr (1999) An investigation of the auditory streaming effect using event-related brain potentials. Psychophysiology 36:22–34PubMedCrossRefGoogle Scholar
  21. 21.
    Näätänen R, Gaillard AW, Mäntysalo S (1978) Early selective-attention effect on evoked potential reinterpreted. Acta Psychol 42:313–329CrossRefGoogle Scholar
  22. 22.
    Sussman E, Wong R, Horváth J et al (2007) The development of the perceptual organization of sound by frequency separation in 5–11-year-old children. Hear Res 225:117–127PubMedCrossRefGoogle Scholar
  23. 23.
    Javitt DC, Doneshka P, Zylberman I et al (1993) Impairment of early cortical processing in schizophrenia: An event-related potential confirmation study. Biol Psychiatry 33:513–519PubMedCrossRefGoogle Scholar
  24. 24.
    Besle J, Fort A, Giard MH (2005) Is the auditory sensory memory sensitive to visual information? Exp Brain Res 166:337–344PubMedCrossRefGoogle Scholar
  25. 25.
    Giard MH, Peronnet F (1999) Auditory-visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. J Cogn Neurosci 11:473–490PubMedCrossRefGoogle Scholar
  26. 26.
    Fort A, Delpuech C, Pernier J, Giard MH (2002) Dynamics of cortico-subcortical cross-modal operations involved in audio-visual object detection in humans. Cereb Cortex 12:1031–1039PubMedCrossRefGoogle Scholar
  27. 27.
    Schröger E, Widmann A (1998) Speeded responses to audiovisual signal changes result from bimodal integration. Psychophysiology 35:755–759PubMedCrossRefGoogle Scholar
  28. 28.
    Remijn GB, Ito H, Nakajima Y (2004) Audiovisual Integration: An Investigation of the ‚Streaming-bouncing‘ Phenomenon. J Physiol Anthropol Appl Human Sci 23:243–247PubMedCrossRefGoogle Scholar
  29. 29.
    Vatakis A, Spence C (2006) Audiovisual synchrony perception for speech and music assessed using a temporal order judgment task. Neurosci Lett 393:40–44PubMedCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  1. 1.Universitätsklinik und Poliklinik für Hals-Nasen-Ohren-Heilkunde, Kopf- und Hals-ChirurgieUniversitätsklinikum Halle (Saale)Halle (Saale)Deutschland

Personalised recommendations