Socially-Enriched Multimedia Data Co-clustering

  • Lei MengEmail author
  • Ah-Hwee Tan
  • Donald C. Wunsch II
Part of the Advanced Information and Knowledge Processing book series (AI&KP)


Heterogeneous data co-clustering is a commonly used technique for tapping the rich meta-information of multimedia web documents, including category, annotation, and description, for associative discovery. However, most co-clustering methods proposed for heterogeneous data do not consider the representation problem of short and noisy text and their performance is limited by the empirical weighting of the multimodal features. This chapter explains how to use the Generalized Heterogeneous Fusion Adaptive Resonance Theory (GHF-ART) for clustering large-scale web multimedia documents. Specifically, GHF-ART is designed to handle multimedia data with an arbitrarily rich level of meta-information. For handling short and noisy text, GHF-ART employs the representation and learning methods of PF-ART as described in Sect.  3.5, which identify key tags for cluster prototype modeling by learning the probabilistic distribution of tag occurrences of clusters. More importantly, GHF-ART incorporates an adaptive method for effective fusion of the multimodal features, which weights the features of multiple data sources by incrementally measuring the importance of feature modalities through the intra-cluster scatters. Extensive experiments on two web image datasets and one text document set have shown that GHF-ART achieves significantly better clustering performance and is much faster than many existing state-of-the-art algorithms. The content of this chapter is summarized and extended from [12] (©2014 IEEE. Reprinted, with permission, from [12]), and the Python codes of GHF-ART are available at


  1. 1.
    Bekkerman R, Jeon J (2007) Multi-modal clustering for multimedia collections. In: CVPR, pp 1–8Google Scholar
  2. 2.
    Carpenter GA, Grossberg S, Reynolds J (1991) ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network. Neural Netw. 4(5):565–588CrossRefGoogle Scholar
  3. 3.
    Chen Y, Wang L, Dong M (2010) Non-negative matrix factorization for semisupervised heterogeneous data coclustering. TKDE 22(10):1459–1474Google Scholar
  4. 4.
    Chua T, Tang J, Hong R, Li H, Luo Z, Zheng Y (2009) NUS-WIDE: a real-world web image database from national university of singapore. In: CIVR, pp 1–9Google Scholar
  5. 5.
    Gao B, Liu TY, Zheng X, Cheng QS, Ma WY (2005) Consistent bipartite graph co-partitioning for star-structured high-order heterogeneous data co-clustering. In: Proceedings of international conference on knowledge discovery and data mining, pp 41–50Google Scholar
  6. 6.
    He J, Tan AH, Tan CL, Sung SY (2003) On quantitative evaluation of clustering systems. Clustering and information retrieval. Kluwer Academic Publishers, Netherland, pp 105–133Google Scholar
  7. 7.
    Hu X, Sun N, Zhang C, Chua TS (2009) Exploiting internal and external semantics for the clustering of short texts using world knowledge. In: Proceedings of ACM conference on information and knowledge management, pp 919–928Google Scholar
  8. 8.
    Lang K (2005) Newsweeder: Learning to filter netnews. In: Proceedings international conference machine learning, pp 331–339CrossRefGoogle Scholar
  9. 9.
    Li X, Snoek CGM, Worring M (2008) Learning tag relevance by neighbor voting for social image retrieval. Proceedings of ACM multimedia, pp 180–187Google Scholar
  10. 10.
    Liu D, Hua X, Yang L, Wang M, Zhang, H (2009) Tag ranking. In: Proceedings of international conference on World Wide Web, pp 351–360Google Scholar
  11. 11.
    Long B, Wu X, Zhang Z, Yu PS (2006) Spectral clustering for multi-type relational data. In: ICML, pp 585–592Google Scholar
  12. 12.
    Meng L, Tan AH, Xu D (2014) Semi-supervised heterogeneous fusion for multimedia data co-clustering. IEEE Trans Knowl Data Eng 26(9):2293–2306CrossRefGoogle Scholar
  13. 13.
    Rege M, Dong M, Hua J (2008) Graph theoretical framework for simultaneously integrating visual and textual features for efficient web image clustering. In: Proceedings of international conference on World Wide Web, pp 317–326Google Scholar
  14. 14.
    Tan AH (1995) Adaptive resonance associative map. Neural Netw. 8(3):437–446CrossRefGoogle Scholar
  15. 15.
    Xu R, II DCW (2011) BARTMAP: A viable structure for biclustering. Neural Netw. 709–716CrossRefGoogle Scholar
  16. 16.
    Zhao Y, Karypis G (2001) Criterion functions for document clustering: experiments and analysis. Technical report, Department of computer science, University of MinnesotaGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.NTU-UBC Research Center of Excellence in Active Living for the Elderly (LILY)Nanyang Technological UniversitySingaporeSingapore
  2. 2.School of Computer Science and EngineeringNanyang Technological UniversitySingaporeSingapore
  3. 3.Applied Computational Intelligence LaboratoryMissouri University of Science and TechnologyRollaUSA

Personalised recommendations