Neural Processing Letters

, Volume 38, Issue 2, pp 177–203 | Cite as

Controlling Relations between the Individuality and Collectivity of Neurons and its Application to Self-Organizing Maps

  • Ryotaro Kamimura


In this paper, we consider a society of neurons where different types of neurons interact with each other. For the first approximation to this society, we suppose two types of neurons, namely, individually and collectively treated neurons. Just as individuality must be in harmony with collectivity in actual societies, individually treated neurons must cooperate with collectively treated neurons as much as possible. We here realize this cooperation by making individually treated neurons as similar to collectively treated neurons as possible. The difference between individually and collectively treated neurons is represented by the Kullback–Leibler divergence. This divergence is minimized using free energy minimization. We applied the method to three problems from the well-known machine learning database, namely wine and protein classification, and the image segmentation problem. In all three problems, we succeeded in producing clearer class structures than those obtainable using the conventional SOM. However, we observed that the fidelity to input patterns deteriorated. For this problem, we found that careful treatment of learning processes were needed to keep fidelity to input patterns at an acceptable level.


Input Pattern Class Structure Connection Weight Quantization Error Class Boundary 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Comon P (1994) Independent component analysis: a new concept. Signal Process 36: 287–314CrossRefzbMATHGoogle Scholar
  2. 2.
    Kohonen T (1995) Self-organizing maps. Springer, New YorkCrossRefGoogle Scholar
  3. 3.
    Kohonen T (1990) The self-organization map. Proc IEEE 78(9): 1464–1480CrossRefGoogle Scholar
  4. 4.
    Kohonen T (1982) Self-organized formation of topological correct feature maps. Biol Cybernet 43: 59–69MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Sammon JW (1969) A nonlinear mapping for data structure analysis. IEEE Trans Comput C-18(5): 401–409CrossRefGoogle Scholar
  6. 6.
    Ultsch A, Siemon HP (1990) Kohonen self-organization feature maps for exploratory data analysis. In: Proceedings of international neural network conference. Kulwer Academic, Dordrecht, pp 305–308Google Scholar
  7. 7.
    Ultsch A (2003) U*-matrix: a tool to visualize clusters in high dimensional data. Tech. Rep. 36, Department of Computer Science, University of Marburg, MarburgGoogle Scholar
  8. 8.
    Vesanto J (1999) SOM-based data visualization methods. Intell Data Anal 3: 111–126CrossRefzbMATHGoogle Scholar
  9. 9.
    Kaski S, Nikkila J, Kohonen T (1998) Methods for interpreting a self-organized map in data analysis. In: Proceedings of European symposium on artificial neural networks, BrugesGoogle Scholar
  10. 10.
    Mao I, Jain AK (1995) Artificial neural networks for feature extraction and multivariate data projection. IEEE Trans Neural Netw 6(2): 296–317CrossRefGoogle Scholar
  11. 11.
    De Runz C, Desjardin E, Herbin M (2012) Unsupervised visual data mining using self-organizing maps and a data-driven color mapping. In: 16th International conference information visualisation. IEEE Computer Society, Washington, DC, pp 241–245Google Scholar
  12. 12.
    Shieh S, Liao I (2012) A new approach for data clustering and visualization using self-organizing maps. Expert Syst Appl 39(5): 11924–11933CrossRefGoogle Scholar
  13. 13.
    Yin H (2002) ViSOM: a novel method for multivariate data projection and structure visualization. IEEE Trans Neural Netw 13(1): 237–243CrossRefGoogle Scholar
  14. 14.
    Su M-C, Chang H-T (2001) A new model of self-organizing neural networks and its application in data projection. IEEE Trans Neural Netw 123(1): 153–158Google Scholar
  15. 15.
    Wu S, Chow T (2005) Prsom: a new visualization method by hybridizing multidimensional scaling and self-organizing map. IEEE Trans Neural Netw 16(6): 1362–1380CrossRefGoogle Scholar
  16. 16.
    Xu L, Xu Y, Chow TW (2010) PolSOM: a new method for multidimentional data visualizatio. Pattern Recogn 43: 1668–1675CrossRefzbMATHGoogle Scholar
  17. 17.
    Xu Y, Xu L, Chow T (2011) PPoSOM: a new variant of PoLSOM by using probabilistic assignment for multidimensional data visualization. Neurocomputing 74(11): 2018–2027CrossRefGoogle Scholar
  18. 18.
    Xu L, Chow T (2011) Multivariate data classification using PoLSOM. In: Prognostics and system health management conference (PHM-Shenzhen), IEEE Conference Publications, Shenzhen, pp 1–4Google Scholar
  19. 19.
    Bogdan M, Rosenstiel W (2001) Detection of cluster in self-organizing maps for controlling a prostheses using nerve signals. In: Proceedings of 9th European symposium on artificial neural networks (ESANN’ 2001), D-facto, Evere, pp 131–136Google Scholar
  20. 20.
    Brugger D, Bogdan M, Rosenstiel W (2008) Automatic cluster detection in Kohonen’s som. IEEE Trans Neural Netw 19(3): 442–459CrossRefGoogle Scholar
  21. 21.
    Haraguchi T, Matsushita H, Nishio Y (2009) Community self-organizing map and its application to data extraction. In: International joint conference on neural networks (IJCNN 2009), IEEE, San Jose, pp 1107–1114Google Scholar
  22. 22.
    Li Z, Wang R, Chen L (2009) Extracting community structure of complex networks by self-organizing maps. In: Proceedings of the third international symposium on optimization and systems biology (OSB09), pp 48–56Google Scholar
  23. 23.
    Kamimura R (2010) Information-theoretic enhancement learning and its application to visualization of self-organizing maps. Neurocomputing 73(13–15): 2642–2664CrossRefGoogle Scholar
  24. 24.
    Kamimura R (2011) Selective information enhancement learning for creating interpretable representations in competitive learning. Neural Netw 24(4): 387–405CrossRefGoogle Scholar
  25. 25.
    Kiviluoto K (1996) Topology preservation in self-organizing maps. In: Proceedings of the IEEE international conference on neural networks (ICANN’96), pp 294–299Google Scholar
  26. 26.
    Villmann T, Herrmann RDM, Martinez T (1997) Topology preservation in self-organizing feature maps: exact definition and measurment. IEEE Trans Neural Netw 8(2): 256–266CrossRefGoogle Scholar
  27. 27.
    Bauer H-U, Pawelzik K (1992) Quantifying the neighborhood preservation of self-organizing maps. IEEE Trans Neural Netw 3(4): 570–578CrossRefGoogle Scholar
  28. 28.
    Kaski S, Nikkila J, Oja M, Venna J, Toronen P, Castren E (2003) Trustworthiness and metrics in visualizing similarity of gene expression. BMC Bioinformatics 4: 48CrossRefGoogle Scholar
  29. 29.
    Venna J, Kaski S (2001) Neighborhood preservation in nonlinear projection methods: an experimental study. In: Lecture notes in computer science, vol 2130, Springer, Heidelberg, pp 485–491Google Scholar
  30. 30.
    Polzlbauer G (2004) Survey and comparison of quality measures for self-organizing maps. In: Proceedings of the fifth workshop on data analysis (WDA04), Elfa Academic Press, Slovakia, pp 67–82Google Scholar
  31. 31.
    Lee JA, Verleysen M (2008) Quality assessment of nonlinear dimensionality reduction based on K-ary neighborhoods. In: JMLR: Workshop and conference proceedings, vol 4, pp 21–35Google Scholar
  32. 32.
    Kamimura R, Kamimura T, Shultz TR (2001) Information theoretic competitive learning and linguistic rule acquisition. Trans Jpn Soc Artif Intell 16(2): 287–298CrossRefGoogle Scholar
  33. 33.
    Kamimura R (2003) Information-theoretic competitive learning with inverse Euclidean distance output units. Neural Process Lett 18: 163–184CrossRefGoogle Scholar
  34. 34.
    Rose K, Gurewitz E, Fox GC (1990) Statistical mechanics and phase transition in clustering. Phys Rev Lett 65(8): 945–948CrossRefGoogle Scholar
  35. 35.
    Rose K, Gurewitz E, Fox GC (1992) Vector quantization by deterministic annealing. IEEE Trans Inf Theory 38(4): 1249–1257CrossRefzbMATHGoogle Scholar
  36. 36.
    Ueda N, Nakano R (1995) Deterministic annealing variant of the EM algorithm. In: Advances in neural information processing systems, pp 545–552Google Scholar
  37. 37.
    Ueda N, Nakano R (1998) Deterministic annealing EM algorithm. Neural Netw 11: 271–282CrossRefGoogle Scholar
  38. 38.
    Graepel T, Burger M (1997) Obermayer K Phase transitions in stochastic self-organizing maps. Phys Rev E 56(4): 3876CrossRefGoogle Scholar
  39. 39.
    Heskes T (2001) Self-organizing maps, vector quantization, and mixture modeling. IEEE Trans Neural Netw 12(6): 1299–1305CrossRefGoogle Scholar
  40. 40.
    Kamimura R (2008) Free energy-based competitive learning for mutual information maximization. In: Proceedings of IEEE conference on systems, man, and cybernetics, pp 223–227Google Scholar
  41. 41.
    Kamimura R (2008) Free energy-based competitive learning for self-organizing maps. In: Proceedings of artificial intelligence and applications, pp 414–419Google Scholar
  42. 42.
    Kamimura R (2011) Constrained information maximization by free energy minimization. Int J Gen Syst 40(7): 701–725MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Kamimura R (2011) Self-enhancement learning: target-creating learning and its application to self-organizing maps. Biol Cybernet 104(4–5): 305–338MathSciNetCrossRefzbMATHGoogle Scholar
  44. 44.
    Vesanto J, Himberg J, Alhoniemi E, Parhankangas J (2000) SOM Toolbox for Matlab 5. Tech. Rep. A57, Laboratory of Computer and Information Science, Helsinki University of Technology, HelsinkiGoogle Scholar
  45. 45.
    Venna J (2007) Dimensionality reduction for visual exploration of similarity structures. Dissertation, Helsinki University of TechnologyGoogle Scholar
  46. 46.
    Frank A, Asuncion A (2010) UCI machine learning repository. School of Information and Computer Sciences, University of California, Irvine.
  47. 47.
    Nakai K, Kanehisa M (1991) Expert system for predicting protein localization sites in gram-negative bacteria. Proteins: Struct, Funct, Bioinform 11(2): 95–110CrossRefGoogle Scholar
  48. 48.
    Nakai K, Kanehisa M (1992) A knowledge base for predicting protein localization sites in eukaryotic cells. Genomics 14(4): 897–911CrossRefGoogle Scholar
  49. 49.
    Horton P, Nakai K (1996) A probabilistic classification system for predicting the cellular localization sites of proteins. In: Proceedings of the fourth international conference on intelligent systems for molecular biology, AAAI Press, St. Louis, pp 109–115Google Scholar
  50. 50.
    Merenyi E, Jain A, Villmann T (2007) Explicit magnification control of self-organizing maps for forbidden data. IEEE Trans Neural Netw 18(3): 786–797CrossRefGoogle Scholar
  51. 51.
    Merenyi E, Jain A (2004) Forbidden magnification? II. In: Proceedings of 12th European symposium on artificial neural networks (ESANN’2004), Bruges, pp 57–62Google Scholar
  52. 52.
    Kamimura R (2012) Comprehensibility maximization and humanly comprehensible representations. Int J Gen Syst 41(3): 265–287MathSciNetCrossRefzbMATHGoogle Scholar
  53. 53.
    Van Hulle MM (1999) Faithful representations with topographic maps. Neural Networks 12(6): 803–823CrossRefGoogle Scholar
  54. 54.
    Van Hulle MM (2004) Entropy-based kernel modeling for topographic map formation. IEEE Trans Neural Netw 15(4): 850–858CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2012

Authors and Affiliations

  1. 1.IT Education CenterHiratsukaJapan

Personalised recommendations