Temporal-Spatial Refinements for Video Concept Fusion

  • Jie Geng
  • Zhenjiang Miao
  • Hai Chi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7726)


The context-based concept fusion (CBCF) is increasingly used in video semantic indexing, which uses various relations among different concepts to refine the original detection results. In this paper, we present a CBCF method called Temporal-Spatial Node Balance algorithm (TSNB). This method is based on a physical model, in which the concepts are regard as nodes and the relations are regard as forces. Then all the spatial and temporal relations and the moving cost of the nodes will be balanced. This method is intuitive and observable to explain a concept how to influence others or be influenced by others. And it uses both the spatial and temporal information to describe the semantic structure of the video. We use TSNB algorithm on the datasets of TRECVid 2005-2010. The results show that this method outperforms all the existed works as we know. Besides, it is faster.


Spatial Relation Temporal Relation Semantic Concept Video Shot Concept Detection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lew, M.S., et al.: Content-based multimedia information retrieval: State of the art and challenges. ACM Trans. on Mult. Comp. Comm. Appl. 2 (2006)Google Scholar
  2. 2.
    Smeaton, A.F., et al.: Evaluation campaigns and trecvid. In: MIR 2006 (2006)Google Scholar
  3. 3.
    Ramesh Naphade, M., et al.: Factor graph framework for semantic video indexing. IEEE Trans. on CSVT 12 (2002)Google Scholar
  4. 4.
    Weng, M.F., et al.: Multi-cue fusion for semantic video indexing. In: MM 2008 (2008)Google Scholar
  5. 5.
    Zha, Z.J., et al.: Refining video annotation by exploiting pairwise concurrent relation. In: MM 2007 (2007)Google Scholar
  6. 6.
    Zheng, Y., et al.: Semantic video indexing by fusing explicit and implicit context spaces. In: MM 2010 (2010)Google Scholar
  7. 7.
    Zha, Z.J., et al.: Building a comprehensive ontology to refine video concept detection. In: MIR 2007 (2007)Google Scholar
  8. 8.
    Fan, J., et al.: Incorporating concept ontology for hierarchical video classification, annotation, and visualization. IEEE Trans. on MM 9 (2007)Google Scholar
  9. 9.
    Gu, Z., et al.: Multi-layer multi-instance learning for video concept detection. IEEE Trans. on MM 10 (2008)Google Scholar
  10. 10.
    Mylonas, P., et al.: Using visual context and region semantics for high-level concept detection. IEEE Trans. on MM 11 (2009)Google Scholar
  11. 11.
    Wei, X.Y., et. al.: Exploring inter-concept relationship with context space for semantic video indexing. In: CIVR 2009 (2009)Google Scholar
  12. 12.
    Smith, J., et al.: Multimedia semantic indexing using model vectors. In: ICME 2003 (2003)Google Scholar
  13. 13.
    Jiang, W., et al.: Context-based concept fusion with boosted conditional random fields. In: ICASSP 2007 (2007)Google Scholar
  14. 14.
    Jiang, Y.G., et al.: Domain adaptive semantic diffusion for large scale context-based video annotation. In: ICCV 2009 (2009)Google Scholar
  15. 15.
    Golub, G., et al.: A hessenberg-schur method for the problem ax + xb= c. IEEE Trans. on AC 24 (1979)Google Scholar
  16. 16.
  17. 17.
    Naphade, M., et al.: Large-scale concept ontology for multimedia. IEEE Multimedia (2006)Google Scholar
  18. 18.
    Yanagawa, A., et al.: Columbia university’s baseline detectors for 374 lscom semantic visual concepts. Columb. Univ. ADVENT Techn. Report (2007)Google Scholar
  19. 19.
    Jiang, Y.G., et al.: Vireo-374: Keypoint-based lscom semantic concept detectors,
  20. 20.
    Jiang, Y.G., et al.: CU-VIREO374: Fusing Columbia374 and VIREO374 for Large Scale Semantic Concept Detection. Technical report, Columb. Univ. ADVENT #223-2008-1 (2008)Google Scholar
  21. 21.
    Yilmaz, E., et al.: Estimating average precision with incomplete and imperfect judgments. In: CIKM 2006 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Jie Geng
    • 1
  • Zhenjiang Miao
    • 1
  • Hai Chi
    • 2
  1. 1.Institute of Information ScienceBeijing Jiaotong UniversityBeijingChina
  2. 2.Jilin Electric Power Maintenance CompanyChina

Personalised recommendations