Feedback in Multimodal Self-organizing Networks Enhances Perception of Corrupted Stimuli
It is known from psychology and neuroscience that multimodal integration of sensory information enhances the perception of stimuli that are corrupted in one or more modalities. A prominent example of this is that auditory perception of speech is enhanced when speech is bimodal, i.e. when it also has a visual modality. The function of the cortical network processing speech in auditory and visual cortices and in multimodal association areas, is modeled with a Multimodal Self-Organizing Network (MuSON), consisting of several Kohonen Self-Organizing Maps (SOM) with both feedforward and feedback connections. Simulations with heavily corrupted phonemes and uncorrupted letters as inputs to the MuSON demonstrate a strongly enhanced auditory perception. This is explained by feedback from the bimodal area into the auditory stream, as in cortical processing.
KeywordsAuditory Cortex Auditory Perception Superior Temporal Sulcus Feedback Connection Auditory Stream
Unable to display preview. Download preview PDF.
- 2.Calvert, E.G., Spence, C., Stein, B.E.: The handbook of multisensory processes, 1st edn. MIT Press, Cambridge (2004)Google Scholar
- 14.Möttönen, R., Calvert, G.A., Jääskeläinen, I., Matthews, P.M., Thesen, T., Tuominen, J., Sams, M.: Perceiving identical sounds as speech or non-speech modulates activity in the left posterior superior temporal sulcus. NeuroImage 19 (2005)Google Scholar
- 24.Gustafsson, L., Papliński, A.P.: Bimodal integration of phonemes and letters: an application of multimodal self-organizing networks. In: Proc. Int. Joint Conf. Neural Networks, Vancouver, Canada, pp. 704–710 (2006)Google Scholar
- 26.Kohonen, T.: Self-Organising Maps, 3rd edn. Springer, Berlin (2001)Google Scholar
- 27.Rolls, E.T.: Multisensory neuronal convergence of taste, somatosentory, visual, and auditory inputs. In: Calvert, G., Spencer, C., Stein, B.E. (eds.) The Handbook of multisensory processes, pp. 311–331. MIT Press, Cambridge (2004)Google Scholar
- 28.Kohonen, T.: The neural phonetic typewriter. Computer, 11–22 (1988)Google Scholar
- 29.Gold, B., Morgan, N.: Speech and audio signal processing. John Wiley & Sons, Inc., New York (2000)Google Scholar