Advertisement

Deep Architectures for Joint Clustering and Visualization with Self-organizing Maps

  • Florent ForestEmail author
  • Mustapha Lebbah
  • Hanane Azzag
  • Jérôme Lacaille
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11607)

Abstract

Recent research has demonstrated how deep neural networks are able to learn representations to improve data clustering. By considering representation learning and clustering as a joint task, models learn clustering-friendly spaces and achieve superior performance, compared with standard two-stage approaches where dimensionality reduction and clustering are performed separately. We extend this idea to topology-preserving clustering models, known as self-organizing maps (SOM). First, we present the Deep Embedded Self-Organizing Map (DESOM), a model composed of a fully-connected autoencoder and a custom SOM layer, where the SOM code vectors are learnt jointly with the autoencoder weights. Then, we show that this generic architecture can be extended to image and sequence data by using convolutional and recurrent architectures, and present variants of these models. First results demonstrate advantages of the DESOM architecture in terms of clustering performance, visualization and training time.

Keywords

Clustering Self-organizing map Representation learning Deep learning Autoencoder 

References

  1. 1.
    Aljalbout, E., Golkov, V., Siddiqui, Y., Cremers, D.: Clustering with Deep Learning: Taxonomy and New Methods (2018). arXiv:1801.07648
  2. 2.
    Bai, S., Kolter, J.Z., Koltun, V.: An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling (2018).  https://doi.org/10.1016/S0925-5273(03)00047-1, arXiv:1803.01271CrossRefGoogle Scholar
  3. 3.
    Fard, M.M., Thonet, T., Gaussier, E.: Deep k-Means: Jointly Clustering with k-Means and Learning Representations (2018). arXiv:1806.10069
  4. 4.
    Fortuin, V., Hüser, M., Locatello, F., Strathmann, H., Rätsch, G.: Deep Self-Organization: Interpretable Discrete Representation Learning on Time Series (2018). arXiv:1806.02199
  5. 5.
    Guo, X., Gao, L., Liu, X., Yin, J.: Improved deep embedded clustering with local structure preservation. In: IJCAI, pp. 1753–1759 (2017)Google Scholar
  6. 6.
    Harchaoui, W., Mattei, P., Alamansa, A., Bouveyron, C.: Wasserstein Adversarial Mixture Clustering (2018). https://hal.archives-ouvertes.fr/hal-01827775/
  7. 7.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  8. 8.
    Jiang, Z., Zheng, Y., Tan, H., Tang, B., Zhou, H.: Variational deep embedding: an unsupervised and generative approach to clustering. In: IJCAI, pp. 1965–1972 (2017).  https://doi.org/10.24963/ijcai.2017/273
  9. 9.
    Kingma, D.P., Ba, J.L.: Adam: a method for stochastic optimization. In: ICLR (2015). arXiv:1412.6980
  10. 10.
    Kohonen, T.: Self-organized formation of topologically correct feature maps. Biol. Cybern. 43(1), 59–69 (1982).  https://doi.org/10.1007/BF00337288MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Kohonen, T.: The self-organizing map. Proc. IEEE 78, 1464–1480 (1990).  https://doi.org/10.1109/5.58325CrossRefGoogle Scholar
  12. 12.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. In: Proceedings of the IEEE (1998).  https://doi.org/10.1109/5.726791CrossRefGoogle Scholar
  13. 13.
    Lewis, D.D., Yang, Y., Rose, T.G., Li, F.: RCV1: a new benchmark collection for text categorization research. J. Mach. Learn. Res. 5, 361–397 (2004). http://dl.acm.org/citation.cfm?id=1005332.1005345
  14. 14.
    Madiraju, N.S., Sadat, S.M., Fisher, D., Karimabadi, H.: Deep Temporal Clustering: Fully Unsupervised Learning of Time-Domain Features, pp. 1–11 (2018). arXiv:1802.01059
  15. 15.
    van den Oord, A., Vinyals, O., Kavukcuoglu, K.: Neural discrete representation learning. In: NIPS (2017). arXiv:1711.00937
  16. 16.
    Srivastava, N., Mansimov, E., Salakhutdinov, R.: Unsupervised Learning of Video Representations using LSTMs (2015). arXiv:1502.04681
  17. 17.
    Xie, J., Girshick, R., Farhadi, A.: Unsupervised deep embedding for clustering analysis. In: ICML, vol. 48 (2015).  https://doi.org/10.1007/JHEP01(2016)157, arXiv:1511.06335
  18. 18.
    Yang, B., Fu, X., Sidiropoulos, N.D., Hong, M.: Towards K-means-friendly spaces: simultaneous deep learning and clustering. In: ICML (2016). arXiv:1610.04794
  19. 19.
    Yang, J., Parikh, D., Batra, D.: Joint Unsupervised Learning of Deep Representations and Image Clusters (2016).  https://doi.org/10.1109/CVPR.2016.556, arXiv:1604.03628

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Université Paris 13, Laboratoire d’Informatique de Paris-Nord (LIPN)VilletaneuseFrance
  2. 2.Safran Aircraft EnginesMoissy-CramayelFrance

Personalised recommendations