Multimodal System Based on Self-organizing Maps
We present experiments with a multimodal system based on a novel variant of the Self-Organizing Map (SOM) called the Associative Self-Organizing Map (A-SOM). The A-SOM is similar to the SOM and develops a representation of its input space, but also learns to associate its activity with additional inputs, e.g. the activities of one or several external SOMs. This enables the modelling of expectations in one sensory modality due to the activity elicited in another modality. The paper presents the A-SOM algorithm generalized to an arbitrary number of (possibly delayed) associated activities together with simulation results with a multimodal sensory system and its extension to a system that also includes an action network. The simulation results were very encouraging and confirmed: The ability of the A-SOM to learn to associate the representations of its input space with the representations of the input spaces developed in two connected SOMs; The ability of the extended system to elicit proper activity in the action network; The simulations demonstrated good generalization ability.
KeywordsSelf-Organizing Map Neural network Associative Self-Organizing Map A-SOM SOM ANN Expectations Simulation hypothesis Cognitive modelling Cross-modal activation
Unable to display preview. Download preview PDF.
- 1.Balkenius, C., Morén, J., Johansson, B., Johnsson, M.: Ikaros: Building cognitive models for robots. In: Hülse, M., Hild, M. (eds.) Workshop on current software frameworks in cognitive robotics integrating different computational paradigms (in conjunction with IROS 2008), Nice, France, pp. 47–54 (2008)Google Scholar
- 3.Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)Google Scholar
- 6.Johnsson, M., Balkenius, C.: Associating SOM representations of haptic submodalities. In: Ramamoorthy, S., Hayes, G.M. (eds.) Towards Autonomous Robotic Systems 2008, pp. 124–129 (2008)Google Scholar
- 7.Johnsson, M., Balkenius, C., Hesslow, G.: Neural network architecture for crossmodal activation and perceptual sequences. In: Papers from the AAAI Fall Symposium Biologically Inspired Cognitive Architectures 2009, pp. 85–86 (2009)Google Scholar
- 12.Nguyen, L.D., Woon, K.Y., Tan, A.H.: A self-organizing neural model for multimedia information fusion. In: International Conference on Information Fusion 2008, pp. 1738–1744 (2008)Google Scholar