Multimedia Tools and Applications

, Volume 55, Issue 1, pp 151–178 | Cite as

Towards hierarchical context: unfolding visual community potential for interactive video retrieval

  • Lin Pang
  • Juan Cao
  • Lei Bao
  • Yongdong Zhang
  • Shouxun Lin


Community structure as an interesting property of network has attracted wide attention from many research fields. In this paper, we exploit the visual community structure in visual-temporal correlation network and utilize it to improve interactive video retrieval. Firstly, we propose a hierarchical community-based feedback algorithm. By re-ranking the video shots through diffusion processes respectively on the inter-community and intra-community level, the feedback algorithm can make full use of the limited user feedback. Furthermore, since it avoids entire graph computation, the feedback algorithm can make quick responses to user feedback, which is particularly important for the large video collections. Secondly, we propose a community-based visualization interface called VideoMap. By organizing the video shots following the community structure, the VideoMap presents a comprehensive and informative view of the whole dataset to facilitate users’ access. Moreover, the VideoMap can help users to quickly locate the potential relevant regions and make active annotation according to the distribution of labeled samples on the VideoMap. Experiments on TRECVID 2009 search dataset demonstrate the efficiency of the feedback algorithm and the effectiveness of the visualization interface.


Interactive video retrieval Visual community Hierarchical community-based feedback Active annotation 



This work was supported by the National Basic Research Program of China (973 Program, 2007CB311100), National Nature Science Foundation of China (60902090), Beijing New Star Project on Science & Technology (2007B071), Co-building Program of Beijing Municipal Education Commission.


  1. 1.
    Ahn Y-Y, Bagrow JP, Lehmann S (2009) Communities and hierarchical organization of links in complex networks. arXiv:0903.3178Google Scholar
  2. 2.
    Argyriou A (2004) Efficient approximation methods for harmonic semisupervised learning. Master’s thesis, University College London.Google Scholar
  3. 3.
    Blei D, Ng A, Jordan M (2003) Latent Dirichlet allocation. J Mach Learn Res 3:993–1022MATHCrossRefGoogle Scholar
  4. 4.
    Blondel VD, Guillaume J-L, Lambiotte R, Lefebvre E (2008) Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment 2008(10):10 008+CrossRefGoogle Scholar
  5. 5.
    Cao J, Zhang YD, Feng BL, Bao L, Pang L, Li JT (2009) MCG-ICT-CAS TRECVID2009 search task report. In: Proceedings of TRECVID WorkshopGoogle Scholar
  6. 6.
    Cao J, Zhang YD, Guo JB, Bao L, Li JT (2009) VideoMap: an interactive video retrieval system of MCG-ICT-CAS. In: ACM International Conference on Image and Video RetrievalGoogle Scholar
  7. 7.
    Cao J, Jing HF, Ngo CW, Zhang YD (2009) Distribution-based concept selection for concept-based video retrieval. In: ACM International Conference on Multimedia (MM), pp 19–24Google Scholar
  8. 8.
    Chen Y, Zhou X, and Huang T (2001) One-class SVM for learning in image retrieval. In :Proceedings of the International Conference on Image Processing, Vol. 1,page 34–37.Google Scholar
  9. 9.
    Cheng X, Dale C, and Liu J (2008) Statistics and Social Networking of YouTube Videos. In: Proc. of IEEE IWQoS, page 229-238Google Scholar
  10. 10.
    Clauset A, Newman MEJ, Moore C (2004) Finding community structure in very large networks. Physical Review E 70(6):066111CrossRefGoogle Scholar
  11. 11.
    Clauset A, Moore C, Newman MEJ (2008) Hierarchical structure and the prediction of missing links in networks. Nature 453:98–101CrossRefGoogle Scholar
  12. 12.
    Eckmann J-P, Moses E (2002) Curvature of co-links uncovers hidden thematic layers in the World Wide Web. In: Proc National Acad Sciences, 99, 5825Google Scholar
  13. 13.
    Flake G, Lawrence S, Giles C (2000) Efficient identification of web communities. In: Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 150-160Google Scholar
  14. 14.
    Flake GW, Lawrence S, Giles CL, Coetzee FM (2002) Self-organization and identification of web communities. IEEE Computer.Google Scholar
  15. 15.
    Fortunato S (2009) Community Detection in Graphs. arXiv: 0906.0612Google Scholar
  16. 16.
    Ghosh R, Lerman K (2008) Community detection using a measure of global influence. In: Proceedings of SNAKDD 2008: KDD Workshop on Social Network Mining and AnalysisGoogle Scholar
  17. 17.
    Girvan M and Newman MEJ (2002) Community Structure in Social and Biological Networks, Proceedings of the National Academy of Sciences, Vol. 99, No. 12Google Scholar
  18. 18.
    Gonzalez MC, Hidalgo CA et al (2008) Understanding individual human mobility patterns. Nature 453:779–782CrossRefGoogle Scholar
  19. 19.
    Guo G, Jain A, MaWY, and Zhang HJ. (2001) Learning similarity measure for natural image retrieval with relevance feedback. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, Vol 13, No. 4, page 811-820Google Scholar
  20. 20.
    Hauptmann AG., Lin WH, Yan R, Yang J, Chen MY (2006) Extreme video retrieval: joint maximization of human and computer performance. In: Proceedings of the 14th annual ACM international conference on Multimedia, pp 385–389Google Scholar
  21. 21.
    He JR, Li MJ, Zhang HJ, Tong HH, and Zhang CS (2004) Manifold-ranking based image retrieval.In: Proceedings of the 12th annual ACM international conference on Multimedia, pp 9–16Google Scholar
  22. 22.
    Heesch D, Howarth P, Magalhaes J, May A et al. (2004)Video retrieval using search and browsing. In: Proc. TRECVIDGoogle Scholar
  23. 23.
    Hsu WH, Kennedy LS and Chang SF(2007) Video Search Reranking through Random Walk over Document-Level Context Graph. In: Proceeding of 14th ACM International Conference on Multimedia, page 971–980Google Scholar
  24. 24.
    Jiang YG, Ngo CW, Yang J (2007) Towards optimal bag-of-features for object categorization and semantic video retrieval. In: CIVR, pp 494–501Google Scholar
  25. 25.
    Jing YS, Baluja S (2008) VisualRank: applying pagerank to large-scale image search. Transactions on Pattern Analysis and Machine IntelligenceGoogle Scholar
  26. 26.
    Kamvar SD, Haveliwala TH, Manning CD, Golub GH (2003) Extrapolation methods for accelerating pagerank computations. In International WWW ConferenceGoogle Scholar
  27. 27.
    Kernighan BW, Lin S (1970) An effcient heuristic procedure for partitioning graphs. Bell System Technical JournalGoogle Scholar
  28. 28.
    Leskovec J, Lang KJ, Dasgupta A, Mahoney MW (2008) Community structure in large networks: natural cluster sizes and the absence of large well-defined clusters. arxiv;0810.1355Google Scholar
  29. 29.
    Manevitz L, Yousef M (2004) One-class SVMs for document classification. J Mach Learn Res 2:139–154CrossRefGoogle Scholar
  30. 30.
    Newman M (2006) Modularity and community structure in networks. Proceedings of the National Academy of Sciences of the United States of America 103(23):8577–8582CrossRefGoogle Scholar
  31. 31.
    Newman MEJ (2006) Finding community structure in networks using the eigenvectors of matrices. Physical Review E 74:036104MathSciNetCrossRefGoogle Scholar
  32. 32.
    Newman MEJ, and Girvan M (2004) Finding and evaluating community structure in networks. Physical Review E, 69(2)Google Scholar
  33. 33.
    Onnela J-P, Saram¨aki J, Hyv¨onen J et al. (2007) Structure and tie strengths in mobile communication networks.Proceedings of the National Academy of Sciences, Vol 104, 7332–7336Google Scholar
  34. 34.
    Palla G, Deŕenyi I, Farkas I, Vicsek T (2005) Uncovering the overlapping community structure of complex networks in nature and society. Nature 435:814–818CrossRefGoogle Scholar
  35. 35.
    Palla G, Barabási A-L, Vicsek T (2007) Quantifying social group evolution. Nature 446:664–667CrossRefGoogle Scholar
  36. 36.
    Pan J-Y, et al. (2004) Gcap: graph-based automatic image captioning. In International Workshop on Multimedia Data and Document EngineeringGoogle Scholar
  37. 37.
    Papadopoulos S, Kompatsiaris Y, Vakali A (2010) A graph-based clustering scheme for identifying related tags in folksonomies. In DaWaK 2010Google Scholar
  38. 38.
    Papadopoulos S, Zigkolis C, et al (2010) Image clustering through community detection on hybrid image similarity graphs. In ICIP 2010Google Scholar
  39. 39.
    Pons P, Latapy M (2006) Computing communities in large networks using random walks. Journal of Graph Algorithms and Applications 10(2):191–218MathSciNetMATHGoogle Scholar
  40. 40.
    Porter MA, Onnela JP, Mucha PJ (2009) Communities in Networks. Notices of the American Mathematical Society, Vol. 56, No. 9Google Scholar
  41. 41.
    Rautiainen M, Ojala T, Seppänen T (2003) Cluster-temporal video browsing with semantic filtering. In :Proc. Advanced Concepts for Intelligent Vision Systems, pp 116–123Google Scholar
  42. 42.
    Rautianen M, Ojala T, and Seppänen T (2004) Cluster-temporal browsing of large news video databases. In: IEEE ICME, pp 751–754Google Scholar
  43. 43.
    Snoek CGM, Worring M, Gemert JV, Geusebroek JM et al. (2005) Mediamill: exploring news video archives based on learned semantics. In: ACM Multimedia, pp 225–226Google Scholar
  44. 44.
    Tong S. and Chang E. (2001) Support vector machine active learning for image retrieval. In: Proc. 9th ACM Int. Conf. on Multimedia, pp 107–118Google Scholar
  45. 45.
    TREC Video Retrieval (TRECVID).
  46. 46.
    Wang M, Mei T, Yuan X, Song Y, Dai LR(2007)Video annotation by graph-based learning with neighborhood similarity. In: Proc. of ACM Multimedia, page 325–328Google Scholar
  47. 47.
    White S and Smyth P. (2005) A spectral clustering approach to finding communities in graphs. In: Proceedings of the 5th SIAM International Conference on Data Mining, pp 76–84.Google Scholar
  48. 48.
    Worring M, Snoek CGM, Rooij O, Nguyen GP, and Smeulders A(2007)The MediaMill semantic video search engine. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal ProcessingGoogle Scholar
  49. 49.
    Zhou H and Lipowsky R.(2004) Network Brownian Motion: A New Method to Measure Vertex-Vertex Proximity and to Identify Communities and Subcommunities. LNCS, 3038, 1062–1069.Google Scholar
  50. 50.
    Zhu X (2005) Semi-supervised learning literature survey. Technical report, University of Wisconsin-MadisonGoogle Scholar
  51. 51.
    Zhu X, and Ghahramani Z (2002). Learning from labeled and unlabeled data with label propagation (Technical Report CMU-CALD-02-107). Carnegie Mellon University.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • Lin Pang
    • 2
    • 1
  • Juan Cao
    • 1
  • Lei Bao
    • 1
  • Yongdong Zhang
    • 1
  • Shouxun Lin
    • 1
  1. 1.Laboratory of Advanced Computing Research, Institute of Computing TechnologyChinese Academy of SciencesBeijingChina
  2. 2.Graduate University of the Chinese Academy of SciencesBeijingChina

Personalised recommendations