Advertisement

Self-Organising Maps for Classification with Metropolis-Hastings Algorithm for Supervision

  • Piotr Płoński
  • Krzysztof Zaremba
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7665)

Abstract

Self-Organising Maps (SOM) provide a method of feature mapping from multi-dimensional space to a usually two-dimensional grid of neurons in an unsupervised way. This way of data analysis has been proved as an efficient tool in many applications. SOM presented by T.Kohonen originally were unsupervised learning algorithm, however it is often used in classification problems. This paper introduces novel method for supervised learning of the SOM. It is based on neuron’s class membership and Metropolis-Hastings algorithm, which control network’s learning process. This approach is illustrated by performing recognition tasks on nine real data sets, such as: faces, written digits or spoken letters. Experimental results show improvements over the state-of-art methods for using SOM as classifier.

Keywords

Self-Organising Maps Classification Supervised learning Metropolis-Hastings algorithm 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Asuncion, A., Newman, D.J.: UCI Mmachine Learning Repository. University of California, Irvine, School of Information and Computer Sciences (2007)Google Scholar
  2. 2.
    Brereton, R.G.: Self Organising Maps for Visualising and Modelling. Chem. Cent. J. 6 (2012)Google Scholar
  3. 3.
    Dozono, H., Tokushima, H., Hara, S., Noguchi, Y.: An Algorithm of SOM using Simulated Annealing in the Batch Update Phase for Sequence Analysis. In: 5th Workshop on Self-Organizing Maps, pp. 171–178 (2005)Google Scholar
  4. 4.
    Fessant, F., Aknin, P., Oukhellou, L., Midenet, S.: Comparison of Supervised Self-Organizing Maps Using Euclidian or Mahalanobis Distance in Classification Context. In: Mira, J., Prieto, A.G. (eds.) IWANN 2001. LNCS, vol. 2084, pp. 637–644. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  5. 5.
    Fiannaca, A., Di Fatta, G., Gaglio, S., Rizzo, R., Urso, A.: Improved SOM Learning Using Simulated Annealing. In: de Sá, J.M., Alexandre, L.A., Duch, W., Mandic, D.P. (eds.) ICANN 2007. LNCS, vol. 4668, pp. 279–288. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  6. 6.
    Hastings, W.K.: Monte Carlo Sampling Methods Using Markov Chains and Their Applications. Biometrika 57, 97–109 (1970)zbMATHCrossRefGoogle Scholar
  7. 7.
    Hinton, G.E., Dayan, P., Revow, M.: Modeling the manifolds of Images of Handwritten Digits. IEEE Trans. Neural Netw. 8, 65–74 (1997)CrossRefGoogle Scholar
  8. 8.
    Hu, W., Xie, D., Tan, T., Maybank, S.: Learning Activity Patterns Using Fuzzy Self-Organizing Neural Network. IEEE T. Syst. Man. Cy. B. 34, 1618–1626 (2004)CrossRefGoogle Scholar
  9. 9.
    Kästner, M., Villmann, T.: Fuzzy Supervised Self-Organizing Map for Semi-supervised Vector Quantization. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2012, Part I. LNCS, vol. 7267, pp. 256–265. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  10. 10.
    Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by Simulated Annealing. Sci. 220, 671–680 (1983)MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Kohonen, T.: Self-organized Formation of Topologically Correct Feature Maps. Biol. Cybern. 43, 59–69 (1982)MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Kohonen, T.: The Self-Organizing Map. Proc. IEEE 78, 1464–1480 (1990)CrossRefGoogle Scholar
  13. 13.
    Kohonen, T., Oja, E., Simula, O., Visa, A., Kangas, J.: Engineering Applications of the Self-organizing Map. Proc. IEEE 84, 1358–1384 (2002)CrossRefGoogle Scholar
  14. 14.
    Melssen, W., Wehrens, R., Buydens, L.: Supervised Kohonen Networks for Classification Problems. Cemometr. Intell. Lab. 83, 99–113 (2006)CrossRefGoogle Scholar
  15. 15.
    Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equations of State Calculations by Fast Computing Machines. J. Chem. Phys. 21, 1087–1092 (1953)CrossRefGoogle Scholar
  16. 16.
    Midenet, S., Grumbach, A.: Learning Associations by Self-Organization: The LASSO model. Neurocomputing 6, 343–361 (1994)CrossRefGoogle Scholar
  17. 17.
    Muruzabal, J.: On the Emulation of Kohonen’s Self-Organization via Single-Map Metropolis-Hastings Algorithms. In: 9th International Conference on Conceptual Structures, pp. 346–355 (2001)Google Scholar
  18. 18.
    Osowski, S., Linh, T.H.: Fuzzy Clustering Neural Network for Classification of ECG Beats. In: International Joint Conference on Neural Networks, pp. 26–32. IEEE Press, New York (2000)Google Scholar
  19. 19.
    Płoński, P., Zaremba, K.: Improving Performance of Self-Organising Maps with Distance Metric Learning Method. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2012, Part I. LNCS, vol. 7267, pp. 169–177. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  20. 20.
    Sohn, S., Dagli, C.H.: Advantages of Using Fuzzy Class Memberships in Self-organizing Map and Support Vector Machines. In: International Joint Conference on Neural Networks, pp. 1886–1890. IEEE Press, New York (2001)Google Scholar
  21. 21.
    Song, X., Hopke, P.K.: Kohonen Neural Network as a Pattern Recognition Method Based on the Weight Interpretation. Anal. Chim. Acta. 334, 57–66 (1996)CrossRefGoogle Scholar
  22. 22.
    Wongravee, K., Lloyd, G.R., Silwood, C.J., Grootveld, M., Brereton, R.G.: Supervised Self Organizing Maps for Classification and Determination of Potentially Discriminatory Variables: Illustrated by Application to Nuclear Magnetic Resonance Metabolomic Profiling. Anal. Chem. 82, 628–638 (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Piotr Płoński
    • 1
  • Krzysztof Zaremba
    • 1
  1. 1.Institute of RadioelectronicsWarsaw University of TechnologyWarsawPoland

Personalised recommendations