Skip to main content
SpringerLink
Log in

Rethinking Collaborative Clustering: A Practical and Theoretical Study Within the Realm of Multi-view Clustering

  • Chapter
  • First Online:
Recent Advancements in Multi-View Data Analytics

Part of the book series: Studies in Big Data ((SBD,volume 106))

Abstract

With distributed and multi-view data being more and more ubiquitous, the last 20 years have seen a surge in the development of new multi-view methods. In unsupervised learning, these are usually classified under the paradigm of multi-view clustering: A broad family of clustering algorithms that tackle data from multiple sources with various goals and constraints. Methods known as collaborative clustering algorithms are also a part of this family. Whereas other multi-view algorithms produce a unique consensus solution based on the properties of the local views, collaborative clustering algorithms aim to adapt the local algorithms so that they can exchange information and improve their local solutions during the multi-view phase, but still produce their own distinct local solutions. In this chapter, we study the connections that collaborative clustering shares with both multi-view clustering and unsupervised ensemble learning. We do so by addressing both practical and theoretical aspects: First we address the formal definition of what is collaborative clustering as well as its practical applications. By doing so, we demonstrate that pretty much everything called collaborative clustering in the literature is either a specific case of multi-view clustering, or misnamed unsupervised ensemble learning. Then, we address the properties of collaborative clustering methods, and in particular we adapt the notion of clustering stability and propose a bound for collaborative clustering methods. Finally, we discuss how some of the properties of collaborative clustering studied in this chapter can be adapted to broader contexts of multi-view clustering and unsupervised ensemble learning.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Formally, it would be incorrect to state that \(A^1 = \dotsc = A^J\), since the algorithms \(A^j\) are defined relatively to different spaces \(\mathbb {X}^j\) and are therefore of different natures.

  2. 2.

    The fact that the Lipschitz constant K must be lower than 1 is due to the convention that the clustering distances are defined between 0 and 1.

References

  1. Arivazhagan, M.G., Aggarwal, V., Singh, A.K., Choudhary, S.: Federated learning with personalization layers (2019)

    Google Scholar 

  2. Ben-David, S., Von Luxburg, U., Pál, D.: A sober look at clustering stability. In: International Conference on Computational Learning Theory. pp. 5–19. Springer (2006)

    Google Scholar 

  3. Bickel, S., Scheffer, T.: Multi-view clustering. In: Proceedings of the 4th IEEE International Conference on Data Mining (ICDM 2004), 1-4 November 2004, Brighton, UK. pp. 19–26. IEEE Computer Society (2004). https://doi.org/10.1109/ICDM.2004.10095

  4. Bishop, C.M., Svensén, M., Williams, C.K.I.: GTM: the generative topographic mapping. Neural Comput. 10(1), 215–234 (1998)

    Google Scholar 

  5. Bonawitz, K., Eichner, H., Grieskamp, W., Huba, D., Ingerman, A., Ivanov, V., Kiddon, C., Konečný, J., Mazzocchi, S., McMahan, B., Overveldt, T.V., Petrou, D., Ramage, D., Roselander, J.: Towards federated learning at scale: system design. In: Talwalkar, A., Smith, V., Zaharia, M. (eds.) Proceedings of Machine Learning and Systems 2019, MLSys 2019, Stanford, CA, USA, March 31 – April 2. mlsys.org. https://proceedings.mlsys.org/book/271.pdf (2019)

  6. Carlsson, G.E., Mémoli, F.: Characterization, stability and convergence of hierarchical clustering methods. J. Mach. Learn. Res. 11, 1425–1470. http://portal.acm.org/citation.cfm?id=1859898 (2010)

  7. Coletta, L.F.S., Vendramin, L., Hruschka, E.R., Campello, R.J.G.B., Pedrycz, W.: Collaborative fuzzy clustering algorithms: some refinements and design guidelines. IEEE Trans. Fuzzy Syst. 20(3), 444–462 (2012)

    Google Scholar 

  8. Cornuéjols, A., Wemmert, C., Gançarski, P., Bennani, Y.: Collaborative clustering: why, when, what and how. Inf. Fusion 39, 81–95 (2018)

    Google Scholar 

  9. Diao, E., Ding, J., Tarokh, V.: Heterofl: computation and communication efficient federated learning for heterogeneous clients. CoRR abs/2010.01264. https://arxiv.org/abs/2010.01264 (2020)

  10. Dunn, J.C.: A fuzzy relative of the isodata process and its use in detecting compact well-separated clusters. J. Cybern. 3(3), 32–57 (1973)

    Google Scholar 

  11. Falih, I., Grozavu, N., Kanawati, R., Bennani, Y., Matei, B.: Collaborative multi-view attributed networks mining. In: 2018 International Joint Conference on Neural Networks, IJCNN 2018, Rio de Janeiro, Brazil, July 8–13, 2018. pp. 1–8. IEEE (2018). https://doi.org/10.1109/IJCNN.2018.8489183

  12. Filali, A., Jlassi, C., Arous, N.: SOM variants for topological horizontal collaboration. In: 2nd International Conference on Advanced Technologies for Signal and Image Processing, ATSIP 2016, Monastir, Tunisia, March 21–23, 2016. pp. 459–464. IEEE (2016). https://doi.org/10.1109/ATSIP.2016.7523117

  13. Filali, A., Jlassi, C., Arous, N.: A hybrid collaborative clustering using self-organizing map. In: 14th IEEE/ACS International Conference on Computer Systems and Applications, AICCSA 2017, Hammamet, Tunisia, October 30 – Nov. 3, 2017. pp. 709–716. IEEE Computer Society (2017). https://doi.org/10.1109/AICCSA.2017.111

  14. Forestier, G., Wemmert, C., Gançarski, P.: Multisource images analysis using collaborative clustering. EURASIP J. Adv. Signal Process. 2008 (2008). https://doi.org/10.1155/2008/374095

  15. Forestier, G., Wemmert, C., Gançarski, P.: Semi-supervised collaborative clustering with partial background knowledge. In: Workshops Proceedings of the 8th IEEE International Conference on Data Mining (ICDM 2008), December 15–19, 2008, Pisa, Italy. pp. 211–217. IEEE Computer Society (2008). https://doi.org/10.1109/ICDMW.2008.116

  16. Forestier, G., Wemmert, C., Gançarski, P.: Towards conflict resolution in collaborative clustering. In: 5th IEEE International Conference on Intelligent Systems, IS 2010, 7–9 July 2010, University of Westminster, London, UK. pp. 361–366. IEEE (2010). https://doi.org/10.1109/IS.2010.5548343

  17. Forestier, G., Wemmert, C., Gançarski, P., Inglada, J.: Mining multiple satellite sensor data using collaborative clustering. In: Saygin, Y., Yu, J.X., Kargupta, H., Wang, W., Ranka, S., Yu, P.S., Wu, X. (eds.) ICDM Workshops 2009, IEEE International Conference on Data Mining Workshops, Miami, Florida, USA, 6 December 2009. pp. 501–506. IEEE Computer Society (2009). https://doi.org/10.1109/ICDMW.2009.42

  18. Foucade, Y., Bennani, Y.: Unsupervised collaborative learning using privileged information. CoRR abs/2103.13145. https://arxiv.org/abs/2103.13145 (2021)

  19. Gançarski, P., Salaou, A.: FODOMUST: une plateforme pour la fouille de données multistratégie multitemporelle. In: de Runz, C., Crémilleux, B. (eds.) 16ème Journées Francophones Extraction et Gestion des Connaissances, EGC 2016, 18-22 Janvier 2016, Reims, France. Revue des Nouvelles Technologies de l’Information, vol. E-30, pp. 481–486. Éditions RNTI. http://editions-rnti.fr/?inprocid=1002204 (2016)

  20. Gançarski, P., Wemmert, C.: Collaborative multi-step mono-level multi-strategy classification. Multimed. Tools Appl. 35(1), 1–27 (2007)

    Google Scholar 

  21. Ghassany, M., Grozavu, N., Bennani, Y.: Collaborative clustering using prototype-based techniques. Int. J. Comput. Intell. Appl. 11(3) (2012). https://doi.org/10.1142/S1469026812500174

  22. Ghassany, M., Grozavu, N., Bennani, Y.: Collaborative multi-view clustering. In: The 2013 International Joint Conference on Neural Networks, IJCNN 2013, Dallas, TX, USA, August 4–9, 2013. pp. 1–8. IEEE (2013). https://doi.org/10.1109/IJCNN.2013.6707037

  23. Grozavu, N., Bennani, Y.: Topological collaborative clustering. Aust. J. Intell. Inf. Process. Syst. 12(3). http://cs.anu.edu.au/ojs/index.php/ajiips/article/view/1216 (2010)

  24. Hafdhellaoui, S., Boualleg, Y., Farah, M.: Collaborative clustering approach based on dempster-shafer theory for bag-of-visual-words codebook generation. In: Meurs, M., Rudzicz, F. (eds.) Advances in Artificial Intelligence - 32nd Canadian Conference on Artificial Intelligence, Canadian AI 2019, Kingston, ON, Canada, May 28-31, 2019, Proceedings. Lecture Notes in Computer Science, vol. 11489, pp. 263–273. Springer (2019). https://doi.org/10.1007/978-3-030-18305-9_21

  25. Jiang, Y., Chung, F.L., Wang, S., Deng, Z., Wang, J., Qian, P.: Collaborative fuzzy clustering from multiple weighted views. IEEE Trans. Cybern. 45(4), 688–701 (2015). https://doi.org/10.1109/TCYB.2014.2334595

    Article  Google Scholar 

  26. Jiang, Z.L., Guo, N., Jin, Y., Lv, J., Wu, Y., Liu, Z., Fang, J., Yiu, S., Wang, X.: Efficient two-party privacy-preserving collaborative k-means clustering protocol supporting both storage and computation outsourcing. Inf. Sci. 518, 168–180 (2020)

    Google Scholar 

  27. Kleinberg, J.M.: An impossibility theorem for clustering. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in Neural Information Processing Systems 15 [Neural Information Processing Systems, NIPS 2002, December 9–14, 2002, Vancouver, British Columbia, Canada]. pp. 446–453. MIT Press. https://proceedings.neurips.cc/paper/2002/hash/43e4e6a6f341e00671e123714de019a8-Abstract.html (2002)

  28. Kohonen, T.: The self-organizing map. Neurocomputing 21(1–3), 1–6 (1998)

    Google Scholar 

  29. von Luxburg, U.: Clustering stability: an overview. Found. Trends Mach. Learn. 2(3), 235–274 (2009)

    Google Scholar 

  30. Mitra, S., Banka, H., Pedrycz, W.: Rough-fuzzy collaborative clustering. IEEE Trans. Syst. Man Cybern. Part B 36(4), 795–805 (2006)

    Google Scholar 

  31. Murena, P., Sublime, J., Matei, B., Cornuéjols, A.: An information theory based approach to multisource clustering. In: Lang, J. (ed.) Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13–19, 2018, Stockholm, Sweden. pp. 2581–2587. ijcai.org (2018). https://doi.org/10.24963/ijcai.2018/358

  32. Ngo, L.T., Dang, T.H., Pedrycz, W.: Towards interval-valued fuzzy set-based collaborative fuzzy clustering algorithms. Pattern Recognit. 81, 404–416 (2018)

    Google Scholar 

  33. Pedrycz, W.: Collaborative fuzzy clustering. Pattern Recognit. Lett. 23(14), 1675–1686 (2002)

    Google Scholar 

  34. Pedrycz, W.: Knowledge-Based Clustering - From Data to Information Granules. Wiley (2005)

    Google Scholar 

  35. Pedrycz, W., Rai, P.: Collaborative clustering with the use of fuzzy c-means and its quantification. Fuzzy Sets Syst. 159(18), 2399–2427 (2008)

    Google Scholar 

  36. Pokhrel, S.R.: Federated learning meets blockchain at 6g edge: A drone-assisted networking for disaster response. In: Proceedings of the 2nd ACM MobiCom Workshop on Drone Assisted Wireless Communications for 5G and Beyond. p. 49–54. DroneCom ’20, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3414045.3415949

  37. Shen, Y., Pedrycz, W.: Collaborative fuzzy clustering algorithm: some refinements. Int. J. Approx. Reason. 86, 41–61 (2017)

    Google Scholar 

  38. Strehl, A., Ghosh, J., Cardie, C.: Cluster ensembles - a knowledge reuse framework for combining multiple partitions. J. Mach. Learn. Res. 3, 583–617 (2002)

    MathSciNet  MATH  Google Scholar 

  39. Sublemontier, J.: Unsupervised collaborative boosting of clustering: an unifying framework for multi-view clustering, multiple consensus clusterings and alternative clustering. In: The 2013 International Joint Conference on Neural Networks, IJCNN 2013, Dallas, TX, USA, August 4–9, 2013. pp. 1–8. IEEE (2013). https://doi.org/10.1109/IJCNN.2013.6706911

  40. Sublime, J., Grozavu, N., Cabanes, G., Bennani, Y., Cornuéjols, A.: From horizontal to vertical collaborative clustering using generative topographic maps. Int. J. Hybrid Intell. Syst. 12(4), 245–256 (2015)

    Google Scholar 

  41. Sublime, J., Lefebvre, S.: Collaborative clustering through constrained networks using bandit optimization. In: 2018 International Joint Conference on Neural Networks, IJCNN 2018, Rio de Janeiro, Brazil, July 8–13, 2018. pp. 1–8. IEEE (2018). https://doi.org/10.1109/IJCNN.2018.8489479

  42. Sublime, J., Matei, B., Cabanes, G., Grozavu, N., Bennani, Y., Cornuéjols, A.: Entropy based probabilistic collaborative clustering. Pattern Recognit. 72, 144–157 (2017)

    Google Scholar 

  43. Sublime, J., Matei, B., Murena, P.: Analysis of the influence of diversity in collaborative and multi-view clustering. In: 2017 International Joint Conference on Neural Networks, IJCNN 2017, Anchorage, AK, USA, May 14–19, 2017. pp. 4126–4133. IEEE (2017). https://doi.org/10.1109/IJCNN.2017.7966377

  44. Sublime, J., Troya-Galvis, A., Puissant, A.: Multi-scale analysis of very high resolution satellite images using unsupervised techniques. Remote Sens. 9(5), 495 (2017)

    Google Scholar 

  45. Vanhaesebrouck, P., Bellet, A., Tommasi, M.: Decentralized collaborative learning of personalized models over networks. In: Singh, A., Zhu, X.J. (eds.) Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 20–22 April 2017, Fort Lauderdale, FL, USA. Proceedings of Machine Learning Research, vol. 54, pp. 509–517. PMLR. http://proceedings.mlr.press/v54/vanhaesebrouck17a.html (2017)

  46. Wemmert, C., Gançarski, P., Korczak, J.J.: A collaborative approach to combine multiple learning methods. Int. J. Artif. Intell. Tools 9(1), 59–78 (2000)

    Google Scholar 

  47. Yu, F., Tang, J., Cai, R.: Partially horizontal collaborative fuzzy c-means. Int. J. Fuzzy Syst. 9, 198–204 (2007)

    MathSciNet  Google Scholar 

  48. Zimek, A., Vreeken, J.: The blind men and the elephant: on meeting the problem of multiple truths in data from clustering and pattern mining perspectives. Mach. Learn. 98(1–2), 121–155 (2015)

    Google Scholar 

  49. Zouinina, S., Grozavu, N., Bennani, Y., Lyhyaoui, A., Rogovschi, N.: Efficient k-anonymization through constrained collaborative clustering. In: IEEE Symposium Series on Computational Intelligence, SSCI 2018, Bangalore, India, November 18–21, 2018. pp. 405–411. IEEE (2018). https://doi.org/10.1109/SSCI.2018.8628635

Download references

Author information

Authors and Affiliations

  1. Helsinki Institute for Information Technology HIIT, Department of Computer Science, Aalto University, Helsinki, Finland

    Pierre-Alexandre Murena

  2. ISEP, School of Engineering, 10 rue de Vanves, 92130, Issy-Les-Moulineaux, France

    Jérémie Sublime

  3. LIPN–CNRS UMR 7030, LaMSN–Sorbonne Paris North University, 93210, St Denis, France

    Jérémie Sublime & Basarab Matei

Authors
  1. Pierre-Alexandre Murena
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jérémie Sublime
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Basarab Matei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pierre-Alexandre Murena .

Editor information

Editors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Alberta, Edmonton, AB, Canada

    Witold Pedrycz

  2. Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan

    Shyi-Ming Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Murena, PA., Sublime, J., Matei, B. (2022). Rethinking Collaborative Clustering: A Practical and Theoretical Study Within the Realm of Multi-view Clustering. In: Pedrycz, W., Chen, SM. (eds) Recent Advancements in Multi-View Data Analytics. Studies in Big Data, vol 106. Springer, Cham. https://doi.org/10.1007/978-3-030-95239-6_4

Download citation

Publish with us

Policies and ethics

Search

Navigation