Temporal Hebbian Self-Organizing Map for Sequences

  • Jan Koutník
  • Miroslav Šnorek
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5163)

Abstract

In this paper we present a new self-organizing neural network called Temporal Hebbian Self-organizing Map (THSOM) suitable for modelling of temporal sequences. The network is based on Kohonen’s Self-organizing Map, which is extended with a layer of full recurrent connections among the neurons. The layer of recurrent connections is trained with Hebb’s rule. The recurrent layer represents temporal order of the input vectors. The THSOM brings a straightforward way of embedding context information in recurrent SOM using neurons with Euclidean metric and scalar product. The recurrent layer can be easily converted into a stochastic automaton (Markov Chain) generating sequences used for previous THSOM training. Finally, two real world examples of THSOM usage are presented. THSOM was applied to extraction of road network from GPS data and to construction of spatio-temporal models of spike train sequences measured in human brain in vivo.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Schmidhuber, J., Bengio, Y.: Evaluating long-term dependency benchmark problems by random guessing (1997)Google Scholar
  2. 2.
    Kohonen, T.: Self-Organizing Maps, 3rd edn. Springer, Heidelberg (2001)MATHGoogle Scholar
  3. 3.
    Hammer, B., Micheli, A., Neubauer, N., Sperduti, A., Strickert, M.: Self organizing maps for time series. In: Proceedings of WSOM 2005, pp. 115–122 (2005)Google Scholar
  4. 4.
    Varsta, M., Heikkonen, J., del R., Millan, J.: Context learning with the self organizing map. In: Proceedings of WSOM 1997, Workshop on Self-Organizing Maps, Espoo, Finland, June 4–6. Helsinki University of Technology, Neural Networks Research Centre, pp. 197–202 (1997)Google Scholar
  5. 5.
    Koskela, T., Varsta, M., Heikkonen, J., Kaski, K.: Temporal sequence processing using recurrent SOM. In: Proceedings of the 2nd International Conference on Knowledge-Based Intelligent Engineering Systems, Adelaide, Australia, vol. 1 (1998)Google Scholar
  6. 6.
    Varsta, M., Heikkonen, J., Lampinen, J.: Analytical comparison of the temporal kohonen map and the recurrent self organizing map. In: Verleysen, M. (ed.) European Symposium on Artificial Neural Networks (ESANN), pp. 273–280. D-facto Publications (2000)Google Scholar
  7. 7.
    Varsta, M.: Self-Organizing Map. In: Sequence Processing. PhD thesis, Helsinki University of Technology (2002)Google Scholar
  8. 8.
    Voegtlin, T.: Recursive self-organizing maps. Neural Netw. 15(8-9), 979–991 (2002)CrossRefGoogle Scholar
  9. 9.
    Voegtlin, T.: Context quantization and contextual self-organizing maps (2000)Google Scholar
  10. 10.
    Strickert, M., Hammer, B.: Neural gas for sequences. In: Yamakawa, T. (ed.) Proceedings of the Workshop on Self-Organizing Networks (WSOM), Kyushu Institute of Technology, pp. 53–58 (2003)Google Scholar
  11. 11.
    Koutník, J.: Inductive modelling of temporal sequences by means of self-organization. In: Proceeding of Internation Workshop on Inductive Modelling (IWIM 2007), Ljubljana, CTU in Prague, pp. 269–277 (2007)Google Scholar
  12. 12.
    Koutník, J.: Self-organizing Maps for Modeling and Recognition of Temporal Sequences. PhD thesis, Czech Technical University in Prague (2008)Google Scholar
  13. 13.
    Koutník, J., Šnorek, M.: Neural network generating Hidden Markov Chain. In: Adaptive and Natural Computing Algorithms - Proceedings of the International Conference in Coimbra, pp. 518–521. Springer, Wien (2005)CrossRefGoogle Scholar
  14. 14.
    Obeso, J.A., Guridi, J., Obeso, J.A., DeLong, M.: Surgery for parkinson’s disease (editorial). Journal of Neurology, Neurosurgery and Psychiatry 62(1), 2–8 (1997)CrossRefGoogle Scholar
  15. 15.
    Rodriguez-Oroz, M., Rodriguez, M., Guridi, J., Mewes, K., Chockkman, V., Vitek, J., DeLong, M.R., Obeso, J.A.: The subthalamic nucleus in Parkinson’s disease: somatotopic organization and physiological characteristics. Brain 124(9), 1777–1790 (2001)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Jan Koutník
    • 1
  • Miroslav Šnorek
    • 1
  1. 1.Computational Intelligence Group, Department of Computer Science and Engineering, Faculty of Electrical EngineeringCzech Technical University in Prague 

Personalised recommendations