Multimodal System Based on Self-organizing Maps

  • Magnus Johnsson
  • Christian Balkenius
  • Germund Hesslow
Part of the Studies in Computational Intelligence book series (SCI, volume 343)

Abstract

We present experiments with a multimodal system based on a novel variant of the Self-Organizing Map (SOM) called the Associative Self-Organizing Map (A-SOM). The A-SOM is similar to the SOM and develops a representation of its input space, but also learns to associate its activity with additional inputs, e.g. the activities of one or several external SOMs. This enables the modelling of expectations in one sensory modality due to the activity elicited in another modality. The paper presents the A-SOM algorithm generalized to an arbitrary number of (possibly delayed) associated activities together with simulation results with a multimodal sensory system and its extension to a system that also includes an action network. The simulation results were very encouraging and confirmed: The ability of the A-SOM to learn to associate the representations of its input space with the representations of the input spaces developed in two connected SOMs; The ability of the extended system to elicit proper activity in the action network; The simulations demonstrated good generalization ability.

Keywords

Self-Organizing Map Neural network Associative Self-Organizing Map A-SOM SOM ANN Expectations Simulation hypothesis Cognitive modelling Cross-modal activation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Magnus Johnsson
    • 1
  • Christian Balkenius
    • 1
  • Germund Hesslow
    • 2
  1. 1.Lund University Cognitive Science, KungshusetLundagårdSweden
  2. 2.Department of Experimental Medical ScienceLund, BMC F10Sweden

Personalised recommendations