Advertisement

Deep Triplet-Driven Semi-supervised Embedding Clustering

  • Dino IencoEmail author
  • Ruggero G. PensaEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11828)

Abstract

In most real world scenarios, experts dispose of limited background knowledge that they can exploit for guiding the analysis process. In this context, semi-supervised clustering can be employed to leverage such knowledge and enable the discovery of clusters that meet the analysts’ expectations. To this end, we propose a semi-supervised deep embedding clustering algorithm that exploits triplet constraints as background knowledge within the whole learning process. The latter consists in a two-stage approach where, initially, a low-dimensional data embedding is computed and, successively, cluster assignment is refined via the introduction of an auxiliary target distribution. Our algorithm is evaluated on real-world benchmarks in comparison with state-of-the-art unsupervised and semi-supervised clustering methods. Experimental results highlight the quality of the proposed framework as well as the added value of the new learnt data representation.

References

  1. 1.
    Banerjee, A., Merugu, S., Dhillon, I.S., Ghosh, J.: Clustering with bregman divergences. J. Mach. Learn. Res. 6, 1705–1749 (2005)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Basu, S., Banerjee, A., Mooney, R.J.: Semi-supervised clustering by seeding. In: ICML, pp. 27–34 (2002)Google Scholar
  3. 3.
    Basu, S., Bilenko, M., Mooney, R.J.: A probabilistic framework for semi-supervised clustering. In: KDD, pp. 59–68 (2004)Google Scholar
  4. 4.
    Bilenko, M., Basu, S., Mooney, R.J.: Integrating constraints and metric learning in semi-supervised clustering. In: ICML, pp. 81–88 (2004)Google Scholar
  5. 5.
    Cucuringu, M., Koutis, I., Chawla, S., Miller, G.L., Peng, R.: Simple and scalable constrained clustering: a generalized spectral method. In: AISTATS, pp. 445–454 (2016)Google Scholar
  6. 6.
    Davidson, I., Ravi, S.S.: Intractability and clustering with constraints. In: ICML, pp. 201–208 (2007)Google Scholar
  7. 7.
    Davis, J.V., Kulis, B., Jain, P., Sra, S., Dhillon, I.S.: Information-theoretic metric learning. In: ICML, pp. 209–216 (2007)Google Scholar
  8. 8.
    Haiyan, W., Haomin, Y., Xueming, L., Haijun, R.: Semi-supervised autoencoder: a joint approach of representation and classification. In: CICN, pp. 1424–1430 (2015)Google Scholar
  9. 9.
    Harwood, B., Kumar B.G., Carneiro, G., Reid, I.D., Drummond, T.: Smart mining for deep metric learning. In: ICCV, pp. 2840–2848 (2017)Google Scholar
  10. 10.
    Hubert, L., Arabie, P.: Comparing partitions. J. Classif. 2(1), 193–218 (1985)CrossRefGoogle Scholar
  11. 11.
    Ienco, D., Pensa, R.G.: Semi-supervised clustering with multiresolution autoencoders. In: IJCNN, pp. 1–8 (2018)Google Scholar
  12. 12.
    Kalintha, W., Ono, S., Numao, M., Fukui, K.: Kernelized evolutionary distance metric learning for semi-supervised clustering. In: AAAI, pp. 4945–4946 (2017)Google Scholar
  13. 13.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014), http://arxiv.org/abs/1412.6980
  14. 14.
    Klein, D., Kamvar, S.D., Manning, C.D.: From instance-level constraints to space-level constraints: making the most of prior knowledge in data clustering. In: ICML, pp. 307–314 (2002)Google Scholar
  15. 15.
    Kumar, N., Kummamuru, K.: Semisupervised clustering with metric learning using relative comparisons. IEEE Trans. Knowl. Data Eng. 20(4), 496–503 (2008)CrossRefGoogle Scholar
  16. 16.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436, May 2015.  https://doi.org/10.1038/nature14539CrossRefGoogle Scholar
  17. 17.
    van der Maaten, L., Hinton, G.E.: Visualizing high-dimensional data using t-sne. JMLR 9, 2579–2605 (2008)zbMATHGoogle Scholar
  18. 18.
    Min, E., Guo, X., Liu, Q., Zhang, G., Cui, J., Long, J.: A survey of clustering with deep learning: from the perspective of network architecture. IEEE Access 6, 39501–39514 (2018)CrossRefGoogle Scholar
  19. 19.
    Nogueira, B.M., Tomas, Y.K.B., Marcacini, R.M.: Integrating distance metric learning and cluster-level constraints in semi-supervised clustering. In: IJCNN, pp. 4118–4125 (2017)Google Scholar
  20. 20.
    Rasmus, A., Berglund, M., Honkala, M., Valpola, H., Raiko, T.: Semi-supervised learning with ladder networks. In: NIPS, pp. 3546–3554 (2015)Google Scholar
  21. 21.
    Ren, Y., Hu, K., Dai, X., Pan, L., Hoi, S.C.H., Xu, Z.: Semi-supervised deep embedded clustering. Neurocomputing 325, 121–130 (2019)CrossRefGoogle Scholar
  22. 22.
    Strehl, A., Ghosh, J.: Cluster ensembles – a knowledge reuse framework for combining multiple partitions. J. Mach. Learn. Res. 3, 583–617 (2002)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Wagstaff, K., Cardie, C., Rogers, S., Schrödl, S.: Constrained k-means clustering with background knowledge. In: ICML, pp. 577–584 (2001)Google Scholar
  24. 24.
    Xiao, H., Rasul, K., Vollgraf, R.: Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. CoRR abs/1708.07747 (2017)Google Scholar
  25. 25.
    Xie, J., Girshick, R.B., Farhadi, A.: Unsupervised deep embedding for clustering analysis. In: ICML, pp. 478–487 (2016)Google Scholar
  26. 26.
    Yu, B., Liu, T., Gong, M., Ding, C., Tao, D.: Correcting the triplet selection bias for triplet loss. In: ECCV, pp. 71–86 (2018)CrossRefGoogle Scholar
  27. 27.
    Zhao, Y., Jin, Z., Qi, G., Lu, H., Hua, X.: An adversarial approach to hard triplet generation. In: ECCV, pp. 508–524 (2018)Google Scholar
  28. 28.
    Zhu, X., Loy, C.C., Gong, S.: Constrained clustering with imperfect oracles. IEEE Trans. Neural Networks Learn. Syst. 27(6), 1345–1357 (2016)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.IRSTEA, UMR TETIS, LIRMM, Univiversity of MontpellierMontpellierFrance
  2. 2.Department of Computer ScienceUniversity of TurinTurinItaly

Personalised recommendations