Skip to main content
SpringerLink
Log in

An Optimal Transport Framework for Collaborative Multi-view Clustering

  • Chapter
  • First Online:
Recent Advancements in Multi-View Data Analytics

Part of the book series: Studies in Big Data ((SBD,volume 106))

  • 404 Accesses

Abstract

Research on Multi-View Clustering (MVC) has become more and more attractive thanks to the richness of its application in several fields. Unlike traditional clustering, each view maintains different information to complete each other. On the other hand, the collaborative framework aims mainly to exchange knowledge between collaborators to improve their local quality. The purpose of this chapter is to present a new framework for Collaborative Multi-View Clustering (Co-MVC) based on Optimal Transport (OT) theory. The main idea of the proposed approaches is to perform a collaborative learning step to create a coordination map between multiple views that will ensure an optimal fusion for building a consensus solution. The intuition behind performing the collaboration step is to exploit the pre-existing structure learned in each view to improve the classical consensus clustering. In this chapter we propose two approaches: Collaborative Consensus Projection Approach (CoCP) that aims to perform the consensus in the global space of the data, and a Collaborative Consensus with New Representation (CoCNR) that seeks to encode a new data representation based on local ones. Both approaches are based on the entropy regularized Wasserstein distance and the Wasserstein barycenters between the data distribution in each view. Extensive experiments are conducted on multiple datasets to analyze and evaluate the proposed approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Agueh, M., Carlier, G.: Barycenters in the Wasserstein space. SIAM J. Math. Anal. 43(2), 904–924 (2011)

    Article  MathSciNet  Google Scholar 

  2. Ayad, H.G., Kamel, M.S.: On voting-based consensus of cluster ensembles. Pattern Recognit. 43(5), 1943–1953 (2010)

    Google Scholar 

  3. Ben-Bouazza, F.-E., Bennani, Y., Cabanes, G., Touzani, A.: Unsupervised collaborative learning based on optimal transport theory. J. Intell. Syst. 30(1), 698–719 (2021)

    Article  Google Scholar 

  4. Ben-Bouazza, F.-E., Bennani, Y., Touzani, A., Cabanes, G.: Subspace guided collaborative clustering based on optimal transport. In: International Conference on Soft Computing and Pattern Recognition, pp. 113–124. Springer (2020)

    Google Scholar 

  5. Ben Bouazza, F.-E., Cabanes, G., Bennani, Y., Touzani, A.: Collaborative clustering through optimal transport. In: International Conference on Artificial Neural Networks. Springer (in press)

    Google Scholar 

  6. Bickel, S., Scheffer, T.: Multi-view clustering. In: ICDM, vol. 4, pp. 19–26 (2004)

    Google Scholar 

  7. Blum, A., Mitchell, T.: Combining labeled and unlabeled data with co-training. In: COLT, pp. 92–100. ACM (1998)

    Google Scholar 

  8. Ben Bouazza, F.E., Bennani, Y., El Hamri, M., Cabanes, G., Matei, B., Touzani, A.: Multi-view clustering through optimal transport. Aust. J. Intell. Inf. Process. Syst. 15(3), 1–9 (2019)

    Google Scholar 

  9. Brefeld, U., Scheffer, T.: Co-em support vector learning. In: ICML, pp. 16–24. ACM (2004)

    Google Scholar 

  10. Brualdi, R.A.: Combinatorial Matrix Classes, vol. 13. Cambridge University Press (2006)

    Google Scholar 

  11. Cornuejols, A., Wemmert, C., Gancarski, P., Bennani, Y.: Collaborative clustering: why, when, what and how. Inf. Fusion 39, 81–95 (2018)

    Article  Google Scholar 

  12. Courty, N., Flamary, R., Habrard, A., Rakotomamonjy, A.: Joint distribution optimal transportation for domain adaptation. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 30, pp. 3730–3739. Curran Associates, Inc. (2017)

    Google Scholar 

  13. Courty, N., Flamary, R., Tuia, D., Rakotomamonjy, A.: Optimal transport for domain adaptation. IEEE Trans. Pattern Anal. Mach. Intell. 39(9), 1853–1865 (2016)

    Article  Google Scholar 

  14. Cuturi, M.: Sinkhorn distances: lightspeed computation of optimal transport. In: Advances in Neural Information Processing Systems, pp. 2292–2300 (2013)

    Google Scholar 

  15. Cuturi, M., Doucet, A.: Fast computation of Wasserstein barycenters. In: International Conference on Machine Learning, pp. 685–693. PMLR (2014)

    Google Scholar 

  16. Davies, D.L., Bouldin, D.W.: A cluster separation measure. IEEE Trans. Pattern Anal. Mach. Intell. 1(2), 224–227 (1979)

    Google Scholar 

  17. Dua, D., Graff, C.: UCI machine learning repository (2017)

    Google Scholar 

  18. El Hamri, M., Bennani, Y., Falih, I.: Inductive semi-supervised learning through optimal transport (in press). In: International Conference on Neural Information Processing. Springer (2021)

    Google Scholar 

  19. El Hamri, M., Bennani, Y., Falih, I.: Label propagation through optimal transport. In: 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2021)

    Google Scholar 

  20. Friedman, M.: The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J. Am. Stat. Assoc. 32(200), 675–701 (1937)

    Article  Google Scholar 

  21. Genevay, A., Chizat, L., Bach, F., Cuturi, M., Peyré, G.: Sample complexity of Sinkhorn divergences. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 1574–1583. PMLR (2019)

    Google Scholar 

  22. Kantorovich, L.V.: On the translocation of masses. In: Dokl. Akad. Nauk. USSR (NS), vol. 37, pp. 199–201 (1942)

    Google Scholar 

  23. Knight, P.A.: The Sinkhorn–Knopp algorithm: convergence and applications. SIAM J. Matrix Anal. Appl. 30(1), 261–275 (2008)

    Google Scholar 

  24. Laclau, C., Redko, I., Matei, B., Bennani, Y., Brault, V.: Co-clustering through optimal transport (2017). arXiv preprint arXiv:1705.06189

  25. Martin Arjovsky, S.C., Bottou, L.: Wasserstein generative adversarial networks. In: Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia (2017)

    Google Scholar 

  26. Mirkin, B.G.: Additive clustering and qualitative factor analysis methods for similarity matrices. J. Classif. 4(1), 7–31 (1987)

    Article  MathSciNet  Google Scholar 

  27. Monge, G.: Mémoire sur la théorie des déblais et des remblais. De l’Imprimerie Royale (1781)

    Google Scholar 

  28. Nemenyi, P.B.: Distribution-Free Multiple Comparisons. Princeton University (1963)

    Google Scholar 

  29. Pele, O., Werman, M.: Fast and robust earth mover’s distances. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 460–467. IEEE (2009)

    Google Scholar 

  30. Peyré, G., Cuturi, M., et al.: Computational optimal transport: with applications to data science. Foundations and Trends® in Machine Learning, 11(5-6), pp. 355–607 (2019)

    Google Scholar 

  31. Rastin, P., Cabanes, G., Grozavu, N., Bennani, Y.: Collaborative clustering: how to select the optimal collaborators? In: 2015 IEEE Symposium Series on Computational Intelligence, pp. 787–794. IEEE (2015)

    Google Scholar 

  32. Rousseeuw, P.J.: Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J. Comput. Appl. Math. 20, 53–65 (1987)

    Google Scholar 

  33. Santambrogio, F.: Optimal Transport for Applied Mathematicians, vol. 55, (58–63), p. 94. Birkäuser, New York (2015)

    Google Scholar 

  34. Solomon, J., Rustamov, R., Guibas, L., Butscher, A.: Wasserstein propagation for semi-supervised learning. In: International Conference on Machine Learning, pp. 306–314 (2014)

    Google Scholar 

  35. Steinley, D.: Properties of the Hubert-Arable adjusted rand index. Psychol. Methods 9(3), 386 (2004)

    Article  Google Scholar 

  36. Tao, Z., Liu, H., Li, S., Ding, Z., Fu, Y.: From ensemble clustering to multi-view clustering. In: IJCAI (2017)

    Google Scholar 

  37. Vega-Pons, S., Ruiz-Shulcloper, J.: A survey of clustering ensemble algorithms. Int. J. Pattern Recognit. Artif. Intell. 25(03), 337–372 (2011)

    Article  MathSciNet  Google Scholar 

  38. Villani, C.: Optimal Transport: Old and New, vol. 338. Springer Science & Business Media (2008)

    Google Scholar 

  39. Wu, J., Liu, H., Xiong, H., Cao, J., Chen, J.: K-means-based consensus clustering: a unified view. IEEE Trans. Knowl. Data Eng. 27(1), 155–169 (2014)

    Google Scholar 

  40. Wu, J., Xiong, H., Chen, J.: Adapting the right measures for k-means clustering. In: SIGKDD, pp. 877–886. ACM (2009)

    Google Scholar 

  41. Xie, X., Sun, S.: Multi-view clustering ensembles. In: International Conference on Machine Learning and Cybernetics, vol. 1, pp. 51–56 (2013)

    Google Scholar 

  42. Yarowsky, D.: Unsupervised word sense disambiguation rivaling supervised methods. In: 33rd Annual Meeting of the Association for Computational Linguistics (1995)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. High School of Biomedical Engineering, Mohammed VI University of Health Sciences, Casablanca, Morocco

    Fatima-Ezzahraa Ben-Bouazza

  2. LIPN–CNRS UMR 7030, Université Sorbonne Paris Nord, Villetaneuse, France

    Fatima-Ezzahraa Ben-Bouazza, Younès Bennani & Mourad El Hamri

  3. LaMSN, La Maison des Sciences Numériques, Saint-Denis, France

    Fatima-Ezzahraa Ben-Bouazza, Younès Bennani & Mourad El Hamri

Authors
  1. Fatima-Ezzahraa Ben-Bouazza
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Younès Bennani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mourad El Hamri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fatima-Ezzahraa Ben-Bouazza .

Editor information

Editors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Alberta, Edmonton, AB, Canada

    Witold Pedrycz

  2. Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan

    Shyi-Ming Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Ben-Bouazza, FE., Bennani, Y., El Hamri, M. (2022). An Optimal Transport Framework for Collaborative Multi-view Clustering. In: Pedrycz, W., Chen, SM. (eds) Recent Advancements in Multi-View Data Analytics. Studies in Big Data, vol 106. Springer, Cham. https://doi.org/10.1007/978-3-030-95239-6_5

Download citation

Publish with us

Policies and ethics

Search

Navigation