How Many Dissimilarity/Kernel Self Organizing Map Variants Do We Need?

Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 295)

Abstract

In numerous applicative contexts, data are too rich and too complex to be represented by numerical vectors. A general approach to extend machine learning and data mining techniques to such data is to really on a dissimilarity or on a kernel that measures how different or similar two objects are.

This approach has been used to define several variants of the Self Organizing Map (SOM). This paper reviews those variants in using a common set of notations in order to outline differences and similarities between them. It discuss the advantages and drawbacks of the variants, as well as the actual relevance of the dissimilarity/kernel SOM for practical applications.

Keywords

Self Organizing Map Dissimilarity data Pairwise data Kernel Deterministic annealing 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Ambroise, C., Govaert, G.: Analyzing dissimilarity matrices via Kohonen maps. In: Proceedings of 5th Conference of the International Federation of Classification Societies (IFCS 1996), Kobe (Japan), vol. 2, pp. 96–99 (March 1996)Google Scholar
  2. 2.
    Andras, P.: Kernel-Kohonen networks. International Journal of Neural Systems 12, 117–135 (2002)CrossRefGoogle Scholar
  3. 3.
    Aronszajn, N.: Theory of reproducing kernels. Transactions of the American Mathematical Society 68(3), 337–404 (1950)MATHMathSciNetCrossRefGoogle Scholar
  4. 4.
    Becker, A., Cleveland, S.: Brushing scatterplots. Technometrics 29(2), 127–142 (1987)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Boulet, R., Jouve, B., Rossi, F., Villa, N.: Batch kernel SOM and related Laplacian methods for social network analysis. Neurocomputing 71(7-9), 1257–1273 (2008)CrossRefGoogle Scholar
  6. 6.
    Buhmann, J.M., Hofmann, T.: A maximum entropy approach to pairwise data clustering. In: Proceedings of the International Conference on Pattern Recognition, Hebrew University, Jerusalem, Israel, vol. II, pp. 207–212. IEEE Computer Society Press (1994)Google Scholar
  7. 7.
    Conan-Guez, B., Rossi, F.: Speeding up the dissimilarity self-organizing maps by branch and bound. In: Sandoval, F., Prieto, A.G., Cabestany, J., Graña, M. (eds.) IWANN 2007. LNCS, vol. 4507, pp. 203–210. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  8. 8.
    Conan-Guez, B., Rossi, F.: Accélération des cartes auto-organisatrices sur tableau de dissimilarités par séparation et évaluation. Revue des Nouvelles Technologies de l’Information, pp. 1–16 (June 2008), RNTI-C-2 Classification: points de vue croisés. Rédacteurs invités : Mohamed Nadif et François-Xavier JolloisGoogle Scholar
  9. 9.
    Conan-Guez, B., Rossi, F.: Dissimilarity clustering by hierarchical multi-level refinement. In: Proceedings of the XXth European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2012), Bruges, Belgique, pp. 483–488 (March 2012)Google Scholar
  10. 10.
    Conan-Guez, B., Rossi, F., El Golli, A.: Fast algorithm and implementation of dissimilarity self-organizing maps. Neural Networks 19(6-7), 855–863 (2006)MATHCrossRefGoogle Scholar
  11. 11.
    Cottrell, M., Hammer, B., Hasenfuß, A., Villmann, T.: Batch and median neural gas. Neural Networks 19(6), 762–771 (2006)MATHCrossRefGoogle Scholar
  12. 12.
    El Golli, A., Conan-Guez, B., Rossi, F.: Self organizing map and symbolic data. Journal of Symbolic Data Analysis 2(1) (November 2004)Google Scholar
  13. 13.
    El Golli, A., Conan-Guez, B., Rossi, F.: A self organizing map for dissimilarity data. In: Banks, D., House, L., McMorris, F.R., Arabie, P., Gaul, W. (eds.) Classification, Clustering, and Data Mining Applications (Proceedings of IFCS 2004), Chicago, Illinois, pp. 61–68. IFCS, Springer (2004)Google Scholar
  14. 14.
    Fort, J.C., Letremy, P., Cottrell, M.: Advantages and drawbacks of the batch kohonen algorithm. In: Proceedings of Xth European Symposium on Artificial Neural Networks (ESANN 2002), vol. 2, pp. 223–230 (2002)Google Scholar
  15. 15.
    Gärtner, T., Lloyd, J.W., Flach, P.A.: Kernels and distances for structured data. Machine Learning 57(3), 205–232 (2004)MATHCrossRefGoogle Scholar
  16. 16.
    Gisbrecht, A., Mokbel, B., Schleif, F.M., Zhu, X., Hammer, B.: Linear time relational prototype based learning. Int. J. Neural Syst. 22(5) (2012)Google Scholar
  17. 17.
    Graepel, T., Burger, M., Obermayer, K.: Self-organizing maps: Generalizations and new optimization techniques. Neurocomputing 21, 173–190 (1998)MATHCrossRefGoogle Scholar
  18. 18.
    Graepel, T., Obermayer, K.: A stochastic self-organizing map for proximity data. Neural Computation 11(1), 139–155 (1999)CrossRefGoogle Scholar
  19. 19.
    Hammer, B., Gisbrecht, A., Hasenfuss, A., Mokbel, B., Schleif, F.-M., Zhu, X.: Topographic mapping of dissimilarity data. In: Laaksonen, J., Honkela, T. (eds.) WSOM 2011. LNCS, vol. 6731, pp. 1–15. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  20. 20.
    Hammer, B., Hasenfuss, A.: Topographic mapping of large dissimilarity data sets. Neural Computation 22(9), 2229–2284 (2010)MATHMathSciNetCrossRefGoogle Scholar
  21. 21.
    Hammer, B., Hasenfuss, A., Rossi, F., Strickert, M.: Topographic processing of relational data. In: Proceedings of the 6th International Workshop on Self-Organizing Maps (WSOM 2007), Bielefeld, Germany (September 2007)Google Scholar
  22. 22.
    Hathaway, R.J., Bezdek, J.C.: Nerf c-means: Non-euclidean relational fuzzy clustering. Pattern Recognition 27(3), 429–437 (1994)CrossRefGoogle Scholar
  23. 23.
    Hathaway, R.J., Davenport, J.W., Bezdek, J.C.: Relational duals of the c-means clustering algorithms. Pattern Recognition 22(2), 205–212 (1989)MATHMathSciNetCrossRefGoogle Scholar
  24. 24.
    Hendrickson, B., Leland, R.: A multilevel algorithm for partitioning graphs. In: Proceedings of the 1995 ACM/IEEE Conference on Supercomputing (CDROM), Supercomputing 1995. ACM, New York (1995), http://doi.acm.org/10.1145/224170.224228 Google Scholar
  25. 25.
    Heskes, T., Kappen, B.: Error potentials for self-organization. In: Proceedings of 1993 IEEE International Conference on Neural Networks (Joint FUZZ-IEEE 1993 and ICNN 1993 [IJCNN 1993]), vol. III, pp. 1219–1223. IEEE/INNS, San Francisco (1993)Google Scholar
  26. 26.
    Hofmann, T., Buhmann, J.M.: Pairwise data clustering by deterministic annealing. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(1), 1–14 (1997)CrossRefGoogle Scholar
  27. 27.
    Kohonen, T.: Self-organizing maps of symbol strings. Technical report A42, Laboratory of computer and information science, Helsinki University of Technology, Finland (1996)Google Scholar
  28. 28.
    Kohonen, T.: Self-Organizing Maps, 3rd edn. Springer Series in Information Sciences, vol. 30. Springer (2001)Google Scholar
  29. 29.
    Kohonen, T., Somervuo, P.J.: Self-organizing maps of symbol strings. Neurocomputing 21, 19–30 (1998)MATHCrossRefGoogle Scholar
  30. 30.
    Kohonen, T., Somervuo, P.J.: How to make large self-organizing maps for nonvectorial data. Neural Networks 15(8), 945–952 (2002)CrossRefGoogle Scholar
  31. 31.
    Mac Donald, D., Fyfe, C.: The kernel self organising map. In: Proceedings of 4th International Conference on Knowledge-Based Intelligence Engineering Systems and Applied Technologies, pp. 317–320 (2000)Google Scholar
  32. 32.
    Martín-Merino, M., Muñoz, A.: Extending the SOM algorithm to non-euclidean distances via the kernel trick. In: Pal, N.R., Kasabov, N., Mudi, R.K., Pal, S., Parui, S.K. (eds.) ICONIP 2004. LNCS, vol. 3316, pp. 150–157. Springer, Heidelberg (2004), http://dx.doi.org/10.1007/978-3-540-30499-9_22 CrossRefGoogle Scholar
  33. 33.
    Müllner, D.: Modern hierarchical, agglomerative clustering algorithms. Lecture Notes in Computer Science 3918(1973), 29 (2011), http://arxiv.org/abs/1109.2378
  34. 34.
    Olteanu, M., Villa-Vialaneix, N., Cottrell, M.: On-line relational SOM for dissimilarity data. In: Estévez, P.A., Príncipe, J.C., Zegers, P. (eds.) Advances in Self-Organizing Maps. AISC, vol. 198, pp. 13–22. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  35. 35.
    Rose, K.: Deterministic annealing for clustering, compression,classification, regression, and related optimization problems. Proceedings of the IEEE 86(11), 2210–2239 (1998)CrossRefGoogle Scholar
  36. 36.
    Rossi, F.: Model collisions in the dissimilarity SOM. In: Proceedings of XVth European Symposium on Artificial Neural Networks (ESANN 2007), Bruges (Belgium), pp. 25–30 (April 2007)Google Scholar
  37. 37.
    Rossi, F., Hasenfuss, A., Hammer, B.: Accelerating relational clustering algorithms with sparse prototype representation. In: Proceedings of the 6th International Workshop on Self-Organizing Maps (WSOM 2007), Bielefeld, Germany (September 2007)Google Scholar
  38. 38.
    Rossi, F., Villa-Vialaneix, N.: Optimizing an organized modularity measure for topographic graph clustering: a deterministic annealing approach. Neurocomputing 73(7-9), 1142–1163 (2010)CrossRefGoogle Scholar
  39. 39.
    Rossi, F., Villa-Vialaneix, N.: Représentation d’un grand réseau à partir d’une classification hiérarchique de ses sommets. Journal de la Société Française de Statistique 152(3), 34–65 (2011)MathSciNetGoogle Scholar
  40. 40.
    Schleif, F.-M., Gisbrecht, A.: Data analysis of (non-)metric proximities at linear costs. In: Hancock, E., Pelillo, M. (eds.) SIMBAD 2013. LNCS, vol. 7953, pp. 59–74. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  41. 41.
    Schölkopf, B., Smola, A., Müller, K.R.: Kernel principal component analysis. In: Gerstner, W., Hasler, M., Germond, A., Nicoud, J.-D. (eds.) ICANN 1997. LNCS, vol. 1327, pp. 583–588. Springer, Heidelberg (1997)Google Scholar
  42. 42.
    Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press (2004)Google Scholar
  43. 43.
    Somervuo, P.J.: Self-organizing map of symbol strings with smooth symbol averaging. In: Workshop on Self-Organizing Maps (WSOM 2003), Hibikino, Kitakyushu, Japan (September 2003)Google Scholar
  44. 44.
    Somervuo, P.J.: Online algorithm for the self-organizing map of symbol strings. Neural Networks 17, 1231–1239 (2004)CrossRefGoogle Scholar
  45. 45.
    Ultsch, A., Siemon, H.P.: Kohonen’s self organizing feature maps for exploratory data analysis. In: Proceedings of International Neural Network Conference (INNC 1990), pp. 305–308 (1990)Google Scholar
  46. 46.
    Ultsch, A., Mörchen, F.: Esom-maps: tools for clustering, visualization, and classification with emergent som. Tech. Rep. 46, Department of Mathematics and Computer Science, University of Marburg, Germarny (2005)Google Scholar
  47. 47.
    Vesanto, J.: Som-based data visualization methods. Intelligent Data Analysis 3(2), 111–126 (1999)MATHCrossRefGoogle Scholar
  48. 48.
    Vesanto, J.: Data Exploration Process Based on the Self–Organizing Map. Ph.D. thesis, Helsinki University of Technology, Espoo (Finland) (May 2002), Acta Polytechnica Scandinavica, Mathematics and Computing Series No. 115Google Scholar
  49. 49.
    Villa, N., Rossi, F.: A comparison between dissimilarity som and kernel som for clustering the vertices of a graph. In: Proceedings of the 6th International Workshop on Self-Organizing Maps (WSOM 2007), Bielefeld, Germany (September 2007)Google Scholar
  50. 50.
    Williams, C., Seeger, M.: Using the nyström method to speed up kernel machines. In: Advances in Neural Information Processing Systems 13 (2001)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.SAMM (EA 4543), Université Paris 1Paris Cedex 13France

Personalised recommendations