Learning Highly Structured Manifolds: Harnessing the Power of SOMs

  • Erzsébet Merényi
  • Kadim Tasdemir
  • Lili Zhang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5400)


In this paper we elaborate on the challenges of learning manifolds that have many relevant clusters, and where the clusters can have widely varying statistics. We call such data manifolds highly structured. We describe approaches to structure identification through self-organized learning, in the context of such data. We present some of our recently developed methods to show that self-organizing neural maps contain a great deal of information that can be unleashed and put to use to achieve detailed and accurate learning of highly structured manifolds, and we also offer some comparisons with existing clustering methods on real data.


Hyperspectral Image Locally Linear Embedding Nonlinear Dimensionality Reduction Well Match Unit Topology Preservation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lee, J., Verleysen, M.: Nonlinear Dimensionality Reduction. Information Science and Statistics. Springer, New York (2007)CrossRefGoogle Scholar
  2. 2.
    Gorban, A., Kégl, B., Wunsch, D., Zinovyev, A. (eds.): Principal Manifolds for data Visualization and Dimension Reduction. Lecture Notes in Computational Science and Engineering. Springer, New York (2008)Google Scholar
  3. 3.
    Cox, T.F., Cox, M.: Multidimensional Scaling. Chapman and Hall/CRC, Boca Raton (2001)Google Scholar
  4. 4.
    Tenenbaum, J.B., de Silva, V., Langford, J.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)CrossRefPubMedGoogle Scholar
  5. 5.
    Roweis, S., Soul, L.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)CrossRefPubMedGoogle Scholar
  6. 6.
    Donoho, D.L., Grimes, C.: Hessian eigenmaps: new locally linear embedding techniques for high-dimensional data. Proc. National Academy of Sciences. 100, 5591–5596 (2003)CrossRefGoogle Scholar
  7. 7.
    Pless, R.: Using Isomap to explore video sequences. In: Proc. International Conference on Computer Vision, pp. 1433–1440 (2003)Google Scholar
  8. 8.
    Yang, M.: Face Recognition Using Extended Isomap. In: Proc. International Conference on Image Processing ICIP 2002, vol. 2, pp. 117–120 (2002)Google Scholar
  9. 9.
    Polito, M., Perona, P.: Grouping and dimensionality reduction by locally linear embedding. In: Proc. Neural Information Processing Systems, NIPS (2001)Google Scholar
  10. 10.
    Vlachos, M., Domeniconi, C., Gunopulos, D., Kollios, G., Koudas, N.: Non-linear dimensionality reduction techniques for classification and visualization. In: Proceedings of 8th SIGKDD, pp. 645–651 (2002)Google Scholar
  11. 11.
    Zhang, J., Li, S.Z., Wang, J.: Manifold learning and applications in recognition. In: Tan, Y.P., Kim Hui Yap, L.W. (eds.) Intelligent Multimedia Processing with Soft Computing. Springer, Heidelberg (2004)Google Scholar
  12. 12.
    Kohonen, T.: Self-Organizing Maps, 2nd edn. Springer, Heidelberg (1997)CrossRefGoogle Scholar
  13. 13.
    Martinetz, T., Berkovich, S., Schulten, K.: Neural Gas network for vector quantization and its application to time-series prediction. IEEE Trans. on Neural Networks 4(4), 558–569 (1993)CrossRefPubMedGoogle Scholar
  14. 14.
    Cottrell, M., Hammer, B., Hasenfuss, A., Villmann, T.: Batch and median neural gas. Neural Networks 19, 762–771 (2006)CrossRefPubMedGoogle Scholar
  15. 15.
    Bishop, C.M., Svensen, M., Williams, C.K.I.: GTM: The Generative Topographic Mapping. Neural Computation 10(1), 215–234 (1998)CrossRefGoogle Scholar
  16. 16.
    Aupetit, M.: Learning topology with the Generative Gaussian Graph and the EM Algorithm. In: Weiss, Y., Schölkopf, B., Platt, J. (eds.) Advances in Neural Information Processing Systems 18, pp. 83–90. MIT Press, Cambridge (2006)Google Scholar
  17. 17.
    Bauer, H.U., Der, R., Herrmann, M.: Controlling the magnification factor of self–organizing feature maps. Neural Computation 8(4), 757–771 (1996)CrossRefGoogle Scholar
  18. 18.
    Merényi, E., Jain, A., Villmann, T.: Explicit magnification control of self-organizing maps for “forbidden” data. IEEE Trans. on Neural Networks 18(3), 786–797 (2007)CrossRefPubMedGoogle Scholar
  19. 19.
    Villmann, T., Claussen, J.: Magnification control in self-organizing maps and neural gas. Neural Computation 18, 446–469 (2006)CrossRefPubMedGoogle Scholar
  20. 20.
    Hammer, B., Hasenfuss, A., Villmann, T.: Magnification control for batch neural gas. Neurocomputing 70, 1125–1234 (2007)CrossRefGoogle Scholar
  21. 21.
    DeSieno, D.: Adding a conscience to competitive learning. In: Proc. IEEE Int’l Conference on Neural Networks (ICNN), New York, July 1988, vol. I, pp. I–117–124 (1988) Google Scholar
  22. 22.
    Cottrell, M., Fort, J., Pages, G.: Theoretical aspects of the SOM algorithm. Neurocomputing 21, 119–138 (1998)CrossRefGoogle Scholar
  23. 23.
    Ritter, H., Schulten, K.: On the stationary state of Kohonen’s self-organizing sensory mapping. Biol. Cyb. 54, 99–106 (1986)CrossRefGoogle Scholar
  24. 24.
    Erwin, E., Obermayer, K., Schulten, K.: Self-organizing maps: ordering, convergence properties and energy functions. Biol. Cyb. 67, 47–55 (1992)CrossRefGoogle Scholar
  25. 25.
    Hammer, B., Villmann, T.: Mathematical aspects of neural networks. In: Proc. Of European Symposium on Artificial Neural Networks (ESANN 2003), Brussels, Belgium. D facto publications (2003)Google Scholar
  26. 26.
    Martinetz, T., Schulten, K.: Topology representing networks. Neural Networks 7(3), 507–522 (1994)CrossRefGoogle Scholar
  27. 27.
    Villmann, T., Der, R., Herrmann, M., Martinetz, T.: Topology Preservation in Self–Organizing Feature Maps: Exact Definition and Measurement. IEEE Transactions on Neural Networks 8(2), 256–266 (1997)CrossRefPubMedGoogle Scholar
  28. 28.
    Bauer, H.U., Pawelzik, K.: Quantifying the neighborhood preservation of Self-Organizing Feature Maps. IEEE Trans. on Neural Networks 3, 570–579 (1992)CrossRefPubMedGoogle Scholar
  29. 29.
    Kiviluoto, K.: Topology preservation in self-organizing maps. In: Proceedings IEEE International Conference on Neural Networks, Bruges, June 3–6, 1996, pp. 294–299 (1996)Google Scholar
  30. 30.
    Zhang, L., Merényi, E.: Weighted Differential Topographic Function: A Refinement of the Topographic Function. In: Proc. 14th European Symposium on Artificial Neural Networks (ESANN 2006), Brussels, Belgium, pp. 13–18. D facto publications (2006)Google Scholar
  31. 31.
    Csathó, B., Krabill, W., Lucas, J., Schenk, T.: A multisensor data set of an urban and coastal scene. In: Int’l Archives of Photogrammetry and Remote Sensing, vol. 32, pp. 26–31 (1998)Google Scholar
  32. 32.
    Bodt, E., Verleysen, M.C.: Statistical tools to assess the reliability of self-organizing maps. Neural Networks 15, 967–978 (2002)CrossRefPubMedGoogle Scholar
  33. 33.
    Merényi, E., Tasdemir, K., Farrand, W.: Intelligent information extraction to aid science decision making in autonomous space exploration. In: Fink, W. (ed.) Proceedings of DSS 2008 SPIE Defense and Security Symposium, Space Exploration Technologies, Orlando, FL, Mach 17–18, 2008, vol. 6960, pp. 17–18. SPIE (2008) 69600M InvitedGoogle Scholar
  34. 34.
    Tasdemir, K., Merényi, E.: Exploiting data topology in visualization and clustering of Self-Organizing Maps. IEEE Trans. on Neural Networks (2008) (in press)Google Scholar
  35. 35.
    Ultsch, A.: Self-organizing neural networks for visualization and classification. In: Opitz, O., Lausen, B. (eds.) Information and Classification — Concepts, Methods and Applications, pp. 307–313. Springer, Berlin (1993)CrossRefGoogle Scholar
  36. 36.
    Kraaijveld, M., Mao, J., Jain, A.: A nonlinear projection method based on Kohonen’s topology preserving maps. IEEE Trans. on Neural Networks 6(3), 548–559 (1995)CrossRefPubMedGoogle Scholar
  37. 37.
    Merkl, D., Rauber, A.: Alternative ways for cluster visualization in Self-Organizing Maps. In: Proc. 1st Workshop on Self-Organizing Maps (WSOM 1997), Espoo, Finland, June 4-6, 1997, pp. 106–111 (1997)Google Scholar
  38. 38.
    Ultsch, A.: Maps for the visualization of high-dimensional data spaces. In: Proc. 4th Workshop on Self-Organizing Maps (WSOM 2003), Paris, France, vol. 3, pp. 225–230 (2003)Google Scholar
  39. 39.
    Cottrell, M., de Bodt, E.: A Kohonen map representation to avoid misleading interpretations. In: Proc. 4th European Symposium on Artificial Neural Networks (ESANN 1996), pp. 103–110. D-Facto, Bruges (1996)Google Scholar
  40. 40.
    Himberg, J.: A SOM based cluster visualization and its application for false colouring. In: Proc. IEEE-INNS-ENNS International Joint Conf. on Neural Networks, Como, Italy, vol. 3, pp. 587–592 (2000)Google Scholar
  41. 41.
    Kaski, S., Venna, J., Kohonen, T.: Coloring that reveals cluster structures in multivariate data. Australian Journal of Intelligent Information Processing Systems 6, 82–88 (2000)Google Scholar
  42. 42.
    Villmann, T., Merényi, E.: Extensions and modifications of the Kohonen-SOM and applications in remote sensing image analysis. In: Seiffert, U., Jain, L.C. (eds.) Self-Organizing Maps: Recent Advances and Applications, pp. 121–145. Springer, Heidelberg (2001)Google Scholar
  43. 43.
    Vesanto, J.: SOM-Based Data Visualization Methods. Intelligent Data Analysis 3(2), 111–126 (1999)CrossRefGoogle Scholar
  44. 44.
    Kaski, S., Kohonen, T., Venna, J.: Tips for SOM Processing and Colourcoding of Maps. In: Deboeck, G., Kohonen, T. (eds.) Visual Explorations in Finance Using Self-Organizing Maps, London (1998)Google Scholar
  45. 45.
    Pölzlbauer, G., Rauber, A., Dittenbach, M.: Advanced visualization techniques for self-organizing maps with graph-based methods. In: Jun, W., Xiaofeng, L., Zhang, Y. (eds.) Proc. Second Intl. Symp. on Neural Networks (ISSN 2005), Chongqing, China, pp. 75–80. Springer, Heidelberg (2005)Google Scholar
  46. 46.
    Aupetit, M., Catz, T.: High-dimensional labeled data analysis with topology representing graphs. Neurocomputing 63, 139–169 (2005)CrossRefGoogle Scholar
  47. 47.
    Aupetit, M.: Visualizing the trustworthiness of a projection. In: Proc. 14th European Symposium on Artificial Neural Networks, ESANN 2006, Bruges, Belgium, April 26-28, 2006, pp. 271–276 (2006)Google Scholar
  48. 48.
    Howell, E.S., Merényi, E., Lebofsky, L.A.: Classification of asteroid spectra using a neural network. Jour. Geophys. Res. 99(E5), 10, 847–10, 865 (1994)CrossRefGoogle Scholar
  49. 49.
    Merényi, E., Howell, E.S., et al.: Prediction of water in asteroids from spectral data shortward of 3 microns. ICARUS 129, 421–439 (1997)CrossRefGoogle Scholar
  50. 50.
    Tasdemir, K., Merényi, E.: Considering topology in the clustering of self-organizing maps. In: Proc. 5th Workshop On Self-Organizing Maps (WSOM 2005), Paris, France, September 5–8, 2005, pp. 439–446 (2005)Google Scholar
  51. 51.
    Tasdemir, K., Merényi, E.: Data topology visualization for the Self-Organizing Map. In: Proc. 14th European Symposium on Artificial Neural Networks (ESANN 2006), Brussels, Belgium, April 26–28, 2006, pp. 125–130. D facto publications (2006)Google Scholar
  52. 52.
    Vesanto, J., Alhoniemi, E.: Clustering of the self-organizing map. IEEE Transactions on Neural Networks 11(3), 586–600 (2000)CrossRefPubMedGoogle Scholar
  53. 53.
    Merényi, E., Csató, B., Taşdemir, K.: Knowledge discovery in urban environments from fused multi-dimensional imagery. In: Gamba, P., Crawford, M. (eds.) Proc. IEEE GRSS/ISPRS Joint Workshop on Remote Sensing and Data Fusion over Urban Areas (URBAN 2007), Paris, France, IEEE Catalog number 07EX1577, April 11-13, 2007, pp. 1–13 (2007)Google Scholar
  54. 54.
    Csathó, B., Schenk, T., Lee, D.C., Filin, S.: Inclusion of multispectral data into object recognition. Int’l Archives of Photogrammetry and Remote Sensing 32, 53–61 (1999)Google Scholar
  55. 55.
    Schott, J., Brown, S., Raqueño, R., Gross, H., Robinson, G.: An advanced synthetic image generation model and its application to multi/hyperspectral algorithm development. Canadian Journal of Remote Sensing 25(2) (June 1999)Google Scholar
  56. 56.
    Ientilucci, E., Brown, S.: Advances in wide-area hyperspectral image simulation. In: Proceedings of SPIE, May 5–8, 2003, vol. 5075, pp. 110–121 (2003)Google Scholar
  57. 57.
    Green, R.O.: Summaries of the 6th Annual JPL Airborne Geoscience Workshop, 1. In: AVIRIS Workshop, Pasadena, CA, March 4–6 (1996)Google Scholar
  58. 58.
    Green, R.O., Boardman, J.: Exploration of the relationship between information content and signal-to-noise ratio and spatial resolution. In: Proc. 9th AVIRIS Earth Science and Applications Workshop, Pasadena, CA, February 23–25 (2000)Google Scholar
  59. 59.
    Tou, J., Gonzalez, R.C.: Pattern Recognition Principles. Addison-Wesley Publishing Company, Reading (1974)Google Scholar
  60. 60.
    Tasdemir, K., Merényi, E.: A new cluster validity index for prototype based clustering algorithms based on inter- and intra-cluster density. In: Proc. Int’l Joint Conf. on Neural Networks (IJCNN 2007), Orlando, FL, August 12–17, 2007, pp. 2205–2211 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Erzsébet Merényi
    • 1
  • Kadim Tasdemir
    • 1
  • Lili Zhang
    • 1
  1. 1.Department of Electrical and Computer EngineeringRice UniversityHoustonU.S.A.

Personalised recommendations