Advertisement

Theoretical and Applied Aspects of the Self-Organizing Maps

  • Marie Cottrell
  • Madalina Olteanu
  • Fabrice Rossi
  • Nathalie Villa-Vialaneix
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 428)

Abstract

The Self-Organizing Map (SOM) is widely used, easy to implement, has nice properties for data mining by providing both clustering and visual representation. It acts as an extension of the k-means algorithm that preserves as much as possible the topological structure of the data. However, since its conception, the mathematical study of the SOM remains difficult and has be done only in very special cases. In WSOM 2005, Jean-Claude Fort presented the state of the art, the main remaining difficulties and the mathematical tools that can be used to obtain theoretical results on the SOM outcomes. These tools are mainly Markov chains, the theory of Ordinary Differential Equations, the theory of stability, etc. This article presents theoretical advances made since then. In addition, it reviews some of the many SOM algorithm variants which were defined to overcome the theoretical difficulties and/or adapt the algorithm to the processing of complex data such as time series, missing values in the data, nominal data, textual data, etc.

Keywords

SOM Batch SOM Relational SOM Stability of SOM 

References

  1. 1.
    Aronszajn, N.: Theory of reproducing kernels. Trans. Am. Math. Soc. 68(3), 337–404 (1950)zbMATHMathSciNetCrossRefGoogle Scholar
  2. 2.
    Bishop, C.M., Svensén, M., Williams, C.K.I.: GTM: the generative topographic mapping. Neural Comput. 10(1), 215–234 (1998)CrossRefGoogle Scholar
  3. 3.
    Bourgeois, N., Cottrell, M., Deruelle, B., Lamassé, S., Letrémy, P.: How to improve robustness in kohonen maps and display additional information in factorial analysis: application to text minin. Neurocomputing 147, 120–135 (2015a)CrossRefGoogle Scholar
  4. 4.
    Bourgeois, N., Cottrell, M., Lamassé, S., Olteanu, M.: Search for meaning through the study of co-occurrences in texts. In: Rojas, I., Joya, G., Catala, A. (eds.) Advances in Computational Intelligence, Proceedings of IWANN 2015, Part II. LNCS, vol. 9095, pp. 578–591. Springer, Switzerland (2015b)Google Scholar
  5. 5.
    Cheng, Y.: Convergence and ordering of Kohonen’s batch map. Neural Comput. 9, 1667–1676 (1997)Google Scholar
  6. 6.
    Conan-Guez, B., Rossi, F., El Golli, A.: Fast algorithm and implementation of dissimilarity self-organizing maps. Neural Netw. 19(6–7), 855–863 (2006)zbMATHCrossRefGoogle Scholar
  7. 7.
    Cottrell, M., Fort, J.C.: Étude d’un processus d’auto-organisation. Annales de l’IHP, section B 23(1), 1–20 (1987)zbMATHMathSciNetGoogle Scholar
  8. 8.
    Cottrell, M., Letrémy, P.: How to use the Kohonen algorithm to simultaneously analyse individuals in a survey. Neurocomputing 63, 193–207 (2005)CrossRefGoogle Scholar
  9. 9.
    Cottrell, M., Fort, J.C., Pagès, G.: Theoretical aspects of the SOM algorithm. Neurocomputing 21, 119–138 (1998)zbMATHCrossRefGoogle Scholar
  10. 10.
    Cottrell, M., Ibbou, S., Letrémy, P.: Som-based algorithms for qualitative variables. Neural Netw. 17, 1149–1167 (2004)zbMATHCrossRefGoogle Scholar
  11. 11.
    Cottrell, M., Olteanu, M., Rossi, F., Rynkiewicz, J., Villa-Vialaneix, N.: Neural networks for complex data. Künstliche Intelligenz 26(2), 1–8 (2012). doi: 10.1007/s13218-012-0207-2 Google Scholar
  12. 12.
    de Bodt, E., Cottrell, M., Verleisen, M.: Statistical tools to assess the reliability of self-organizing maps. Neural Netw. 15(8–9), 967–978 (2002). doi: 10.1016/S0893-6080(02)00071-0
  13. 13.
    El Golli, A., Conan-Guez, B., Rossi, F.: Self organizing map and symbolic data. J. Symb. Data Anal. 2(1), 11 (2004a)Google Scholar
  14. 14.
    El Golli, A., Conan-Guez, B., Rossi, F.: A self organizing map for dissimilarity data. In Banks, D., House, L., McMorris, F.R., Arabie, P., Gaul, W. (eds.) Classification, Clustering, and Data Mining Applications (Proceedings of IFCS 2004), pp. 61–68. IFCS, Chicago, Illinois (USA), Springer, 7 (2004b)Google Scholar
  15. 15.
    Erwin, E., Obermayer, K., Schulten, K.: Self-organizing maps: ordering, convergence properties and energy functions. Biol. Cybern. 67(1), 47–55 (1992a)zbMATHCrossRefGoogle Scholar
  16. 16.
    Erwin, E., Obermayer, K., Schulten, K.: Self-organizing maps: stationnary states, metastability and convergence rate. Biol. Cybern. 67(1), 35–45 (1992b)zbMATHCrossRefGoogle Scholar
  17. 17.
    Flanagan, J.A.: Self-organisation in Kohonen’s som. Neural Netw. 6(7), 1185–1197 (1996)Google Scholar
  18. 18.
    Forgy, E.W.: Cluster analysis of multivariate data: efficiency versus interpretability of classifications. Biometrics 21, 768–769 (1965)Google Scholar
  19. 19.
    Fort, J.-C., Cottrell, M., Letrémy, P.: Stochastic on-line algorithm versus batch algorithm for quantization and self organizing maps. In: Neural Networks for Signal Processing XI, 2001, Proceedings of the 2001 IEEE Signal Processing Sociéty Workshop, pp. 43–52. IEEE, North Falmouth, MA, USA (2001)Google Scholar
  20. 20.
    Fort, J.-C., Letrémy, P., Cottrell, M.: Advantages and drawbacks of the batch Kohonen algorithm. In Verleysen, M. (ed.) European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2002), pp. 223–230. d-side publications, Bruges, Belgium (2002)Google Scholar
  21. 21.
    Fort, J.C.: SOM’s mathematics. Neural Netw. 19(6–7), 812–816 (2006)zbMATHCrossRefGoogle Scholar
  22. 22.
    Fort, J.C., Pagès, G.: About the kohonen algorithm: strong or weak self-organisation. Neural Netw. 9(5), 773–785 (1995)CrossRefGoogle Scholar
  23. 23.
    Goldfarb, L.: A unified approach to pattern recognition. Pattern Recognit. 17(5), 575–582 (1984). doi: 10.1016/0031-3203(84)90056-6 zbMATHMathSciNetCrossRefGoogle Scholar
  24. 24.
    Graepel, T., Burger, M., Obermayer, K.: Self-organizing maps: generalizations and new optimization techniques. Neurocomputing 21, 173–190 (1998)zbMATHCrossRefGoogle Scholar
  25. 25.
    Hammer, B., Hasenfuss, A.: Topographic mapping of large dissimilarity data sets. Neural Comput. 22(9), 2229–2284 (2010)zbMATHMathSciNetCrossRefGoogle Scholar
  26. 26.
    Hammer, B., Hasenfuss, A., Rossi, F., Strickert, M.: Topographic processing of relational data. In: Group, Bielefeld University Neuroinformatics (ed.) Proceedings of the 6th Workshop on Self-Organizing Maps (WSOM 07). Bielefeld, Germany, September 2007Google Scholar
  27. 27.
    Heskes, T.: Energy functions for self-organizing maps. In: Oja, E., Kaski, S. (eds.) Kohonen Maps, pp. 303–315. Elsevier, Amsterdam (1999). http://www.snn.ru.nl/reports/Heskes.wsom.ps.gz
  28. 28.
    Heskes, Tom: Self-organizing maps, vector quantization, and mixture modeling. IEEE Trans. Neural Netw. 12(6), 1299–1305 (2001)CrossRefGoogle Scholar
  29. 29.
    Jordan, M.I., Ghahramani, Z., Jaakkola, T.S., Saul, L.K.: An introduction to variational methods for graphical models. Mach. Learn. 37(2), 183–233 (1999)zbMATHCrossRefGoogle Scholar
  30. 30.
    Kaski, S., Honkela, T., Lagus, K., Kohonen, T.: Websom—self-organizing maps of document collections. Neurocomputing 21(1), 101–117 (1998a)zbMATHCrossRefGoogle Scholar
  31. 31.
    Kaski, S., Jari, K., Kohonen, T.: Bibliography of self-organizing map (SOM) papers: 1981–1997. Neural Comput. Surv. 1(3&4), 1–176 (1998b)Google Scholar
  32. 32.
    Kohohen, T., Somervuo, P.J.: Self-organizing maps of symbol strings. Neurocomputing 21, 19–30 (1998)CrossRefGoogle Scholar
  33. 33.
    Kohonen, T.: Self-organized formation of topologically correct feature maps. Biol. Cybern. 43, 59–69 (1982a)zbMATHMathSciNetCrossRefGoogle Scholar
  34. 34.
    Kohonen, T.: Analysis of a simple self-organizing process. Biol. Cybern. 44, 135–140 (1982b)Google Scholar
  35. 35.
    Kohonen, T.: Median strings. Pattern Recognit. Lett. 3, 309–313 (1985)CrossRefGoogle Scholar
  36. 36.
    Kohonen, T.: Self-organization and Associative Memory. Springer, Berlin (1989)CrossRefGoogle Scholar
  37. 37.
    Kohonen, T.: Self-Organizing Maps. Springer Series in Information Science, vol. 30. Springer, Berlin (1995)Google Scholar
  38. 38.
    Kohonen, T.: Self-organizing maps of symbol strings, Technical Report a42, Laboratory of computer and information science, Helsinki University of technoligy, Finland (1996)Google Scholar
  39. 39.
    Kohonen, T.: Comparison of som point densities based on different criteria. Neural Comput. 11, 2081–2095 (1999)CrossRefGoogle Scholar
  40. 40.
    Kohonen, T.: Self-Organizing Maps, vol. 30, 3rd edn. Springer, Berlin (2001)zbMATHCrossRefGoogle Scholar
  41. 41.
    Kohonen, T.: Essentials of self-organizing map. Neural Netw. 37, 52–65 (2013)CrossRefGoogle Scholar
  42. 42.
    Kohonen, T.: MATLAB Implementations and Applications of the Self-Organizing Map. Unigrafia Oy, Helsinki (2014)Google Scholar
  43. 43.
    Kohonen, T., Somervuo, P.J.: How to make large self-organizing maps for nonvectorial data. Neural Netw. 15(8), 945–952 (2002)CrossRefGoogle Scholar
  44. 44.
    Mac Donald, D., Fyfe, C.: The kernel self organising map. In: Proceedings of 4th International Conference on knowledge-based Intelligence Engineering Systems and Applied Technologies, pp. 317–320 (2000)Google Scholar
  45. 45.
    McLachlan, G., Peel. D.: Finite mixture models. Wiley (2004)Google Scholar
  46. 46.
    Oja, M., Kaski, S., Kohonen, T.: Bibliography of self-organizing map (SOM) papers: 1998–2001 addendum. Neural Comput. Surv. 3, 1–156 (2003)Google Scholar
  47. 47.
    Olteanu, M., Villa-Vialaneix, N.: On-line relational and multiple relational SOM. Neurocomputing 147, 15–30 (2015). doi: 10.1016/j.neucom.2013.11.047 CrossRefGoogle Scholar
  48. 48.
    Pollard, D., et al.: Strong consistency of \(k\)-means clustering. Ann. Stat. 9(1), 135–140 (1981)zbMATHMathSciNetCrossRefGoogle Scholar
  49. 49.
    Ritter, H., Schulten, K.: On the stationary state of Kohonen’s self-organizing sensory mapping. Biol. Cybern. 54, 99–106 (1986)Google Scholar
  50. 50.
    Ritter, H., Martinetz, T., Schulten, K.: Neural Computation and Self-Organizing Maps, an Introduction. Addison-Wesley (1992)Google Scholar
  51. 51.
    Rossi, F.: How many dissimilarity/kernel self organizing map variants do we need? In Villmann, T., Schleif, F.M., Kaden, M., Lange, M. (eds.) Advances in Self-Organizing Maps and Learning Vector Quantization (Proceedings of WSOM 2014), Advances in Intelligent Systems and Computing, vol. 295, pages 3–23. Mittweida, Germany, Springer, Berlin (2014). doi: 10.1007/978-3-319-07695-9_1
  52. 52.
    Sadeghi, A.: Convergence in distribution of the multi-dimensional Kohonen algorithm. J. of Appl. Probab. 38(1), 136–151 (2001)Google Scholar
  53. 53.
    Verbeek, J.J., Vlassi, N., Kröse, B.J.A.: Self-organizing mixture models. Neurocomputing 63, 99–123 (2005)CrossRefGoogle Scholar
  54. 54.
    Villa, N., Rossi, F.: A comparison between dissimilarity SOM and kernel SOM for clustering the vertices of a graph. In: 6th International Workshop on Self-Organizing Maps (WSOM: 2007) Bielefield, Germany, 2007. Bielefield University, Neuroinformatics Group (2007). ISBN 978-3-00-022473-7. doi: 10.2390/biecoll-wsom2007-139

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.SAMM - Université Paris 1 Panthéon-SorbonneParisFrance
  2. 2.INRA, UR 0875 MIATCastanet TolosanFrance

Personalised recommendations