Internal Simulation of Perceptions and Actions

  • Magnus Johnsson
  • David Gil
Part of the Advances in Experimental Medicine and Biology book series (AEMB, volume 718)


We present a study of neural network architectures able to internally simulate perceptions and actions. All these architectures employ the novel Associative Self-Organizing Map (A-SOM) as a perceptual neural network. The A-SOM develops a representation of its input space, but in addition also learns to associate its activity with an arbitrary number of additional (possibly delayed) inputs. One architecture is a bimodal perceptual architecture whereas the others include an action neural network adapted by the delta rule. All but one architecture are recurrently connected. We have tested the architectures with very encouraging simulation results. The bimodal perceptual architecture was able to simulate appropriate sequences of activity patterns in the absence of sensory input for several epochs in both modalities. The architecture without recurrent connections correctly classified 100% of the training samples and 80% of the test samples. After ceasing to receive any input the best of the architectures with recurrent connections was able to continue to produce 100% correct output sequences for 28 epochs (280 iterations), and then to continue with 90% correct output sequences until epoch 42.



We want to express our acknowledgement to the Ministry of Science and Innovation (Ministerio de Ciencia e Innovación—MICINN) through the “Jose Castillejo” program from Government of Spain and to the Swedish Research Council through the Swedish Linnaeus project Cognition, Communication and Learning (CCL) as funders of the work exhibited in this chapter.


  1. 1.
    Balkenius, C., Morén, J., Johansson, B., Johnsson, M.: Ikaros: Building cognitive models for robots. Adv. Eng. Inform. (2009). doi: 10.1016/j.aei.2009.08.003 Google Scholar
  2. 2.
    Bartolomeo, P.: The relationship between visual perception and visual mental imagery: a reappraisal of the neuropsychological evidence. Cereb. Cortex 38, 357–378 (2002) Google Scholar
  3. 3.
    Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, London (1995) Google Scholar
  4. 4.
    Carpenter, G.A., Grossberg, S., Markuzon, N., Reynolds, J.H., Rosen, D.B.: Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Trans. Neural Netw. 3, 698–713 (1992) PubMedCrossRefGoogle Scholar
  5. 5.
    Chappell, G.J., Taylor, J.G.: The temporal kohonen map. Neural Netw. 6, 441–445 (1993) CrossRefGoogle Scholar
  6. 6.
    Elman, J.L.: Finding structure in time. Cogn. Sci. 14, 179–211 (1990) CrossRefGoogle Scholar
  7. 7.
    Fritzke, B.: Growing cell structures—a self-organizing network for unsupervised and supervised learning. Neural Netw. 7(9), 1441–1460 (1993) CrossRefGoogle Scholar
  8. 8.
    Fritzke, B.: Growing grid—a self-organizing network with constant neighborhood range and adaptation strength. Neural Process. Lett. 2, 5 (1995) CrossRefGoogle Scholar
  9. 9.
    Hammer, B., Micheli, A., Sperduti, A., Strickert, M.: Recursive self-organizing network models. Neural Netw. 17, 1061–1085 (2004) PubMedCrossRefGoogle Scholar
  10. 10.
    Hesslow, G.: Conscious thought as simulation of behaviour and perception. Trends Cogn. Sci. 6, 242–247 (2002) PubMedCrossRefGoogle Scholar
  11. 11.
    Johnsson, M., Balkenius, C.: Associating som representations of haptic submodalities. In: Ramamoorthy, S., Hayes, G.M. (eds.) Towards Autonomous Robotic Systems 2008, pp. 124–129 (2008) Google Scholar
  12. 12.
    Johnsson, M., Balkenius, C.: Experiments with self-organizing systems for texture and hardness perception. J. Comput. Sci. Technol. 1(2), 53–62 (2009) Google Scholar
  13. 13.
    Johnsson, M., Balkenius, C., Hesslow, G.: Associative self-organizing map. In: International Joint Conference on Computational Intelligence (IJCCI) 2009, pp. 363–370 (2009) Google Scholar
  14. 14.
    Kayser, C., Petkov, C.I., Augath, M., Logothetis, N.K.: Functional imaging reveals visual modification of specific fields in auditory cortex. J. Neurosci. 27, 1824–1835 (2007) PubMedCrossRefGoogle Scholar
  15. 15.
    Kohonen, T.: Self-Organization and Associative Memory. Springer, Berlin (1988) Google Scholar
  16. 16.
    Kosslyn, S.M., Ganis, G., Thompson, W.L.: Neural foundations of imagery. Nat. Rev., Neurosci. 2, 635–642 (2001) CrossRefGoogle Scholar
  17. 17.
    McGurk, H., MacDonald, J.: Hearing lips and seeing voices. Nature 264, 746–748 (1976) PubMedCrossRefGoogle Scholar
  18. 18.
    Nguyen, L.D., Woon, K.Y., Tan, A.H.: A self-organizing neural model for multimedia information fusion. In: International Conference on Information Fusion 2008, pp. 1738–1744 (2008) Google Scholar
  19. 19.
    Strickert, M., Hammer, B.: Merge som for temporal data. Neurocomputing 64, 39–71 (2005) CrossRefGoogle Scholar
  20. 20.
    Tan, A.H.: Adaptive resonance associative map. Neural Netw. 8, 437–446 (1995) CrossRefGoogle Scholar
  21. 21.
    Varsta, M., Millan, J., Heikkonen, J.: A recurrent self-organizing map for temporal sequence processing. In: ICANN 1997 (1997) Google Scholar
  22. 22.
    Voegtlin, T.: Recursive self-organizing maps. Neural Netw. 15, 979–991 (2002) PubMedCrossRefGoogle Scholar
  23. 23.
    Ziemke, T., Jirenhed, D., Hesslow, G.: Internal simulation of perception: a minimal neuro-robotic model. Neurocomputing 68, 85–104 (2005) CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Lund University Cognitive ScienceLundSweden
  2. 2.Computing Technology and Data ProcessingUniversity of AlicanteAlicanteSpain

Personalised recommendations