Skip to main content
SpringerLink
Log in

Learning Highly Structured Manifolds: Harnessing the Power of SOMs

  • Chapter
Similarity-Based Clustering

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 5400))

Abstract

In this paper we elaborate on the challenges of learning manifolds that have many relevant clusters, and where the clusters can have widely varying statistics. We call such data manifolds highly structured. We describe approaches to structure identification through self-organized learning, in the context of such data. We present some of our recently developed methods to show that self-organizing neural maps contain a great deal of information that can be unleashed and put to use to achieve detailed and accurate learning of highly structured manifolds, and we also offer some comparisons with existing clustering methods on real data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Lee, J., Verleysen, M.: Nonlinear Dimensionality Reduction. Information Science and Statistics. Springer, New York (2007)

    Book  Google Scholar 

  2. Gorban, A., Kégl, B., Wunsch, D., Zinovyev, A. (eds.): Principal Manifolds for data Visualization and Dimension Reduction. Lecture Notes in Computational Science and Engineering. Springer, New York (2008)

    Google Scholar 

  3. Cox, T.F., Cox, M.: Multidimensional Scaling. Chapman and Hall/CRC, Boca Raton (2001)

    Google Scholar 

  4. Tenenbaum, J.B., de Silva, V., Langford, J.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)

    Article  CAS  PubMed  Google Scholar 

  5. Roweis, S., Soul, L.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)

    Article  CAS  PubMed  Google Scholar 

  6. Donoho, D.L., Grimes, C.: Hessian eigenmaps: new locally linear embedding techniques for high-dimensional data. Proc. National Academy of Sciences. 100, 5591–5596 (2003)

    Article  CAS  Google Scholar 

  7. Pless, R.: Using Isomap to explore video sequences. In: Proc. International Conference on Computer Vision, pp. 1433–1440 (2003)

    Google Scholar 

  8. Yang, M.: Face Recognition Using Extended Isomap. In: Proc. International Conference on Image Processing ICIP 2002, vol. 2, pp. 117–120 (2002)

    Google Scholar 

  9. Polito, M., Perona, P.: Grouping and dimensionality reduction by locally linear embedding. In: Proc. Neural Information Processing Systems, NIPS (2001)

    Google Scholar 

  10. Vlachos, M., Domeniconi, C., Gunopulos, D., Kollios, G., Koudas, N.: Non-linear dimensionality reduction techniques for classification and visualization. In: Proceedings of 8th SIGKDD, pp. 645–651 (2002)

    Google Scholar 

  11. Zhang, J., Li, S.Z., Wang, J.: Manifold learning and applications in recognition. In: Tan, Y.P., Kim Hui Yap, L.W. (eds.) Intelligent Multimedia Processing with Soft Computing. Springer, Heidelberg (2004)

    Google Scholar 

  12. Kohonen, T.: Self-Organizing Maps, 2nd edn. Springer, Heidelberg (1997)

    Book  Google Scholar 

  13. Martinetz, T., Berkovich, S., Schulten, K.: Neural Gas network for vector quantization and its application to time-series prediction. IEEE Trans. on Neural Networks 4(4), 558–569 (1993)

    Article  CAS  PubMed  Google Scholar 

  14. Cottrell, M., Hammer, B., Hasenfuss, A., Villmann, T.: Batch and median neural gas. Neural Networks 19, 762–771 (2006)

    Article  PubMed  Google Scholar 

  15. Bishop, C.M., Svensen, M., Williams, C.K.I.: GTM: The Generative Topographic Mapping. Neural Computation 10(1), 215–234 (1998)

    Article  Google Scholar 

  16. Aupetit, M.: Learning topology with the Generative Gaussian Graph and the EM Algorithm. In: Weiss, Y., Schölkopf, B., Platt, J. (eds.) Advances in Neural Information Processing Systems 18, pp. 83–90. MIT Press, Cambridge (2006)

    Google Scholar 

  17. Bauer, H.U., Der, R., Herrmann, M.: Controlling the magnification factor of self–organizing feature maps. Neural Computation 8(4), 757–771 (1996)

    Article  Google Scholar 

  18. Merényi, E., Jain, A., Villmann, T.: Explicit magnification control of self-organizing maps for “forbidden” data. IEEE Trans. on Neural Networks 18(3), 786–797 (2007)

    Article  PubMed  Google Scholar 

  19. Villmann, T., Claussen, J.: Magnification control in self-organizing maps and neural gas. Neural Computation 18, 446–469 (2006)

    Article  PubMed  Google Scholar 

  20. Hammer, B., Hasenfuss, A., Villmann, T.: Magnification control for batch neural gas. Neurocomputing 70, 1125–1234 (2007)

    Article  Google Scholar 

  21. DeSieno, D.: Adding a conscience to competitive learning. In: Proc. IEEE Int’l Conference on Neural Networks (ICNN), New York, July 1988, vol. I, pp. I–117–124 (1988)

    Google Scholar 

  22. Cottrell, M., Fort, J., Pages, G.: Theoretical aspects of the SOM algorithm. Neurocomputing 21, 119–138 (1998)

    Article  Google Scholar 

  23. Ritter, H., Schulten, K.: On the stationary state of Kohonen’s self-organizing sensory mapping. Biol. Cyb. 54, 99–106 (1986)

    Article  Google Scholar 

  24. Erwin, E., Obermayer, K., Schulten, K.: Self-organizing maps: ordering, convergence properties and energy functions. Biol. Cyb. 67, 47–55 (1992)

    Article  CAS  Google Scholar 

  25. Hammer, B., Villmann, T.: Mathematical aspects of neural networks. In: Proc. Of European Symposium on Artificial Neural Networks (ESANN 2003), Brussels, Belgium. D facto publications (2003)

    Google Scholar 

  26. Martinetz, T., Schulten, K.: Topology representing networks. Neural Networks 7(3), 507–522 (1994)

    Article  Google Scholar 

  27. Villmann, T., Der, R., Herrmann, M., Martinetz, T.: Topology Preservation in Self–Organizing Feature Maps: Exact Definition and Measurement. IEEE Transactions on Neural Networks 8(2), 256–266 (1997)

    Article  CAS  PubMed  Google Scholar 

  28. Bauer, H.U., Pawelzik, K.: Quantifying the neighborhood preservation of Self-Organizing Feature Maps. IEEE Trans. on Neural Networks 3, 570–579 (1992)

    Article  CAS  PubMed  Google Scholar 

  29. Kiviluoto, K.: Topology preservation in self-organizing maps. In: Proceedings IEEE International Conference on Neural Networks, Bruges, June 3–6, 1996, pp. 294–299 (1996)

    Google Scholar 

  30. Zhang, L., Merényi, E.: Weighted Differential Topographic Function: A Refinement of the Topographic Function. In: Proc. 14th European Symposium on Artificial Neural Networks (ESANN 2006), Brussels, Belgium, pp. 13–18. D facto publications (2006)

    Google Scholar 

  31. Csathó, B., Krabill, W., Lucas, J., Schenk, T.: A multisensor data set of an urban and coastal scene. In: Int’l Archives of Photogrammetry and Remote Sensing, vol. 32, pp. 26–31 (1998)

    Google Scholar 

  32. Bodt, E., Verleysen, M.C.: Statistical tools to assess the reliability of self-organizing maps. Neural Networks 15, 967–978 (2002)

    Article  PubMed  Google Scholar 

  33. Merényi, E., Tasdemir, K., Farrand, W.: Intelligent information extraction to aid science decision making in autonomous space exploration. In: Fink, W. (ed.) Proceedings of DSS 2008 SPIE Defense and Security Symposium, Space Exploration Technologies, Orlando, FL, Mach 17–18, 2008, vol. 6960, pp. 17–18. SPIE (2008) 69600M Invited

    Google Scholar 

  34. Tasdemir, K., Merényi, E.: Exploiting data topology in visualization and clustering of Self-Organizing Maps. IEEE Trans. on Neural Networks (2008) (in press)

    Google Scholar 

  35. Ultsch, A.: Self-organizing neural networks for visualization and classification. In: Opitz, O., Lausen, B. (eds.) Information and Classification — Concepts, Methods and Applications, pp. 307–313. Springer, Berlin (1993)

    Chapter  Google Scholar 

  36. Kraaijveld, M., Mao, J., Jain, A.: A nonlinear projection method based on Kohonen’s topology preserving maps. IEEE Trans. on Neural Networks 6(3), 548–559 (1995)

    Article  CAS  PubMed  Google Scholar 

  37. Merkl, D., Rauber, A.: Alternative ways for cluster visualization in Self-Organizing Maps. In: Proc. 1st Workshop on Self-Organizing Maps (WSOM 1997), Espoo, Finland, June 4-6, 1997, pp. 106–111 (1997)

    Google Scholar 

  38. Ultsch, A.: Maps for the visualization of high-dimensional data spaces. In: Proc. 4th Workshop on Self-Organizing Maps (WSOM 2003), Paris, France, vol. 3, pp. 225–230 (2003)

    Google Scholar 

  39. Cottrell, M., de Bodt, E.: A Kohonen map representation to avoid misleading interpretations. In: Proc. 4th European Symposium on Artificial Neural Networks (ESANN 1996), pp. 103–110. D-Facto, Bruges (1996)

    Google Scholar 

  40. Himberg, J.: A SOM based cluster visualization and its application for false colouring. In: Proc. IEEE-INNS-ENNS International Joint Conf. on Neural Networks, Como, Italy, vol. 3, pp. 587–592 (2000)

    Google Scholar 

  41. Kaski, S., Venna, J., Kohonen, T.: Coloring that reveals cluster structures in multivariate data. Australian Journal of Intelligent Information Processing Systems 6, 82–88 (2000)

    Google Scholar 

  42. Villmann, T., Merényi, E.: Extensions and modifications of the Kohonen-SOM and applications in remote sensing image analysis. In: Seiffert, U., Jain, L.C. (eds.) Self-Organizing Maps: Recent Advances and Applications, pp. 121–145. Springer, Heidelberg (2001)

    Google Scholar 

  43. Vesanto, J.: SOM-Based Data Visualization Methods. Intelligent Data Analysis 3(2), 111–126 (1999)

    Article  Google Scholar 

  44. Kaski, S., Kohonen, T., Venna, J.: Tips for SOM Processing and Colourcoding of Maps. In: Deboeck, G., Kohonen, T. (eds.) Visual Explorations in Finance Using Self-Organizing Maps, London (1998)

    Google Scholar 

  45. Pölzlbauer, G., Rauber, A., Dittenbach, M.: Advanced visualization techniques for self-organizing maps with graph-based methods. In: Jun, W., Xiaofeng, L., Zhang, Y. (eds.) Proc. Second Intl. Symp. on Neural Networks (ISSN 2005), Chongqing, China, pp. 75–80. Springer, Heidelberg (2005)

    Google Scholar 

  46. Aupetit, M., Catz, T.: High-dimensional labeled data analysis with topology representing graphs. Neurocomputing 63, 139–169 (2005)

    Article  Google Scholar 

  47. Aupetit, M.: Visualizing the trustworthiness of a projection. In: Proc. 14th European Symposium on Artificial Neural Networks, ESANN 2006, Bruges, Belgium, April 26-28, 2006, pp. 271–276 (2006)

    Google Scholar 

  48. Howell, E.S., Merényi, E., Lebofsky, L.A.: Classification of asteroid spectra using a neural network. Jour. Geophys. Res. 99(E5), 10, 847–10, 865 (1994)

    Article  Google Scholar 

  49. Merényi, E., Howell, E.S., et al.: Prediction of water in asteroids from spectral data shortward of 3 microns. ICARUS 129, 421–439 (1997)

    Article  Google Scholar 

  50. Tasdemir, K., Merényi, E.: Considering topology in the clustering of self-organizing maps. In: Proc. 5th Workshop On Self-Organizing Maps (WSOM 2005), Paris, France, September 5–8, 2005, pp. 439–446 (2005)

    Google Scholar 

  51. Tasdemir, K., Merényi, E.: Data topology visualization for the Self-Organizing Map. In: Proc. 14th European Symposium on Artificial Neural Networks (ESANN 2006), Brussels, Belgium, April 26–28, 2006, pp. 125–130. D facto publications (2006)

    Google Scholar 

  52. Vesanto, J., Alhoniemi, E.: Clustering of the self-organizing map. IEEE Transactions on Neural Networks 11(3), 586–600 (2000)

    Article  CAS  PubMed  Google Scholar 

  53. Merényi, E., Csató, B., Taşdemir, K.: Knowledge discovery in urban environments from fused multi-dimensional imagery. In: Gamba, P., Crawford, M. (eds.) Proc. IEEE GRSS/ISPRS Joint Workshop on Remote Sensing and Data Fusion over Urban Areas (URBAN 2007), Paris, France, IEEE Catalog number 07EX1577, April 11-13, 2007, pp. 1–13 (2007)

    Google Scholar 

  54. Csathó, B., Schenk, T., Lee, D.C., Filin, S.: Inclusion of multispectral data into object recognition. Int’l Archives of Photogrammetry and Remote Sensing 32, 53–61 (1999)

    Google Scholar 

  55. Schott, J., Brown, S., Raqueño, R., Gross, H., Robinson, G.: An advanced synthetic image generation model and its application to multi/hyperspectral algorithm development. Canadian Journal of Remote Sensing 25(2) (June 1999)

    Google Scholar 

  56. Ientilucci, E., Brown, S.: Advances in wide-area hyperspectral image simulation. In: Proceedings of SPIE, May 5–8, 2003, vol. 5075, pp. 110–121 (2003)

    Google Scholar 

  57. Green, R.O.: Summaries of the 6th Annual JPL Airborne Geoscience Workshop, 1. In: AVIRIS Workshop, Pasadena, CA, March 4–6 (1996)

    Google Scholar 

  58. Green, R.O., Boardman, J.: Exploration of the relationship between information content and signal-to-noise ratio and spatial resolution. In: Proc. 9th AVIRIS Earth Science and Applications Workshop, Pasadena, CA, February 23–25 (2000)

    Google Scholar 

  59. Tou, J., Gonzalez, R.C.: Pattern Recognition Principles. Addison-Wesley Publishing Company, Reading (1974)

    Google Scholar 

  60. Tasdemir, K., Merényi, E.: A new cluster validity index for prototype based clustering algorithms based on inter- and intra-cluster density. In: Proc. Int’l Joint Conf. on Neural Networks (IJCNN 2007), Orlando, FL, August 12–17, 2007, pp. 2205–2211 (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Rice University, Houston, Texas, U.S.A.

    Erzsébet Merényi, Kadim Tasdemir & Lili Zhang

Authors
  1. Erzsébet Merényi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kadim Tasdemir
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lili Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Mathematics and Computing Science, Intelligent Systems Group, University Groningen, P.O. Box 407, 9700 AK, Groningen, Netherlands

    Michael Biehl

  2. Department of Computer Science, Clausthal University of Technology, 38679, Clausthal-Zellerfeld, Germany

    Barbara Hammer

  3. Machine Learning Group, DICE, Place du Levant, Université catholique de Louvain,, 3-B-1348, Louvain-la-Neuve, Belgium

    Michel Verleysen

  4. Dep. of Mathematics/Physics/Computer Sciences, University of Applied Sciences Mittweida, Technikumplatz 17, 09648, Mittweida, Germany

    Thomas Villmann

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Merényi, E., Tasdemir, K., Zhang, L. (2009). Learning Highly Structured Manifolds: Harnessing the Power of SOMs. In: Biehl, M., Hammer, B., Verleysen, M., Villmann, T. (eds) Similarity-Based Clustering. Lecture Notes in Computer Science(), vol 5400. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-01805-3_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-01805-3_8

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-01804-6

  • Online ISBN: 978-3-642-01805-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation