Skip to main content
SpringerLink
Log in

Self-similarity-based partial near-duplicate video retrieval and alignment

  • Regular Paper
  • Published:
International Journal of Multimedia Information Retrieval Aims and scope Submit manuscript

Abstract

There have been recent studies on partial near-duplicate videos, which involve segments of videos that are near duplicates of each other. State-of-the-art searching schemes usually segment the input video into clips and implement clip-level near-duplicate retrieval. However, the segmentation results are always poorly aligned, which lead to a difficult “unbalance” problem. In this paper, we introduce a self-similarity-based feature representation called the Self-Similarity Belt (SSBelt), which derives from the Self-Similarity Matrix (SSM). In addition, a distinctive pattern in SSBelt called the Interest Corner is detected and described by a bag-of-words representation. The visual words are then combined into visual shingles and indexed by an inverted file index for fast retrieval. Another important task is to accurately align the unbalanced clips, for which we propose the Intensity Mark (IMark) and design a coarse-to-fine near-duplicate video localization scheme. Experimental results show the effectiveness of our approach for both web-based near-duplicate video and unbalanced video datasets. The near-duplicate alignment capacity of IMark is also shown to be effective.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Wu X, Ngo CW, Hauptmann AG, Tan HK (2009) Real-time near-duplicate elimination for web video search with content and context. IEEE Trans Multimed 11(2):196–207

    Article  Google Scholar 

  2. Cherubini M, de Oliveira R, Oliver N (2009) Understanding near-duplicate videos: a user-centric approach. In: ACM international conference on multimedia, pp 35–44

  3. Tan HK, Ngo CW, Chua TS (2010) Efficient mining of multiple partial near-duplicate alignments by temporal network. IEEE Trans Circuits Syst Video Technol 20(11):1486–1498

    Article  Google Scholar 

  4. Zhang DQ, Chang SF (2004) Detecting image near-duplicate by stochastic attributed relational graph matching with learning. In: ACM international conference on multimedia, pp 877–884

  5. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  6. Ke Y, Sukthankar R, Huston L (2004) Efficient near-duplicate detection and sub-image retrieval. In: ACM international conference on multimedia, pp 869–876

  7. Zhao WL, Ngo CW, Tan HK, Wu X (2007) Near-duplicate keyframe identification with interest point matching and pattern learning. IEEE Trans Multimed 9(5):1037–1048

    Article  Google Scholar 

  8. Zhu J, Hoi SC, Lyu MR, Yan S (2011) Near-duplicate keyframe retrieval by semi-supervised learning and nonrigid image matching. ACM Trans Multimed Comput Commun Appl 7(1):4

    Google Scholar 

  9. Liu H, Lu H, Wen Z, Xue X (2012) Gradient ordinal signature and fixed-point embedding for efficient near-duplicate video detection. IEEE Trans Circuits Syst Video Technol 22(4):555–566

    Article  Google Scholar 

  10. Song J, Yang Y, Huang Z, Shen HT, Hong R (2011) Multiple feature hashing for real-time large scale near-duplicate video retrieval. In: ACM international conference on multimedia, pp 423–432

  11. Chum O, Philbin J, Isard M, Zisserman A (2007) Scalable near identical image and shot detection. In: ACM international conference on image and video retrieval, pp 549–556

  12. Xu D, Cham TJ, Yan S, Duan L, Chang SF (2010) Near duplicate identification with spatially aligned pyramid matching. IEEE Trans Circuits Syst Video Technol 20(8):1068–1079

    Article  Google Scholar 

  13. Zheng YT, Neo SY, Chua TS, Tian Q (2007) The use of temporal, semantic and visual partitioning model for efficient near-duplicate keyframe detection in large scale news corpus. In: ACM international conference on image and video retrieval, pp 409–416

  14. Wei S, Zhao Y, Zhu C, Xu C, Zhu Z (2011) Frame fusion for video copy detection. IEEE Trans Circuits Syst Video Technol 21(1):15–28

    Article  Google Scholar 

  15. H-s Min, Choi JY, De Neve W, Ro YM (2012) Near-duplicate video clip detection using model-free semantic concept detection and adaptive semantic distance measurement. IEEE Trans Circuits Syst Video Technol 22(8):1174–1187

    Article  Google Scholar 

  16. Zhou X, Chen L (2010) Monitoring near duplicates over video streams. In: ACM international conference on multimedia, pp 521–530

  17. Wu Z, Huang Q, Jiang S (2009) Robust copy detection by mining temporal self-similarities. In: IEEE international conference on multimedia and expo, pp 554–557

  18. Cui P, Wu Z, Jiang S, Huang Q (2010) Fast copy detection based on Slice Entropy Scattergraph. In: IEEE international conference on multimedia and expo, pp 1236–1241

  19. Zhang X, Hua G, Zhang L, Shum H (2010) Interest seam image. In: IEEE conference on computer vision and pattern recognition, pp 3296–3303

  20. Wu Z, Jiang S, Huang Q (2009) Near-duplicate video matching with transformation recognition. In: ACM international conference on multimedia, pp 549–552

  21. MUSCLE-VCD-2007: bench mark for video copy detection. https://www.rocq.inria.fr/imedia/civr-bench/

  22. Chang HW, Chen HT (2010) A square-root sampling approach to fast histogram-based search. In: IEEE conference on computer vision and pattern recognition, pp 3043–3049

  23. CC\_WEB\_VIDEO: Near-duplicate web video dataset. http://vireo.cs.cityu.edu.hk/webvideo/

  24. Hampapur A, Hyun K, Bolle R (2002) Comparison of sequence matching techniques for video copy detection. In: SPIE storage and retrieval for media databases, pp 194–201

  25. Shang L, Yang L, Wang F, Chan KP, Hua XS (2010) Real-time large scale near-duplicate web video retrieval. In: ACM international conference on multimedia, pp 531–540

  26. Hua X, Chen X, Zhang H (2005) Robust video signature based on ordinal measure. In: IEEE international conference on image processing, pp 685–688

  27. Foote J, Uchihashi S (2001) The beat spectrum: a new approach to rhythm analysis. In: IEEE international conference on multimedia and expo, pp 224–227

  28. Junejo IN, Dexter E, Laptev I, Prez P (2010) View-independent action recognition from temporal self-similarities. IEEE Trans Pattern Anal Mach Intell 33(1):172–185

    Google Scholar 

  29. Bober M, Bober SK (2002) Description of mpeg-7 visual core experiments. ISO/IEC JTC1/SC29/WG11 N, 2002, 5166

  30. Heikkil M, Pietikainen M, Schmid C (2009) Description of interest regions with local binary patterns. Pattern Recognit 42(3):425–436

    Article  Google Scholar 

  31. Sivic J, Zisserman A (2003) Video google: a text retrieval approach to object matching in videos. In: IEEE international conference on computer vision, pp 1470–1477

  32. Nister D, Stewenius H (2006) Scalable recognition with a vocabulary tree. In: IEEE conference on computer vision and pattern recognition, pp 2161–2168

  33. Broder AZ (1997) On the resemblance and containment of documents. In: Compression and complexity of sequences, pp 21–29

  34. Viola P, Jones M (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154

    Article  Google Scholar 

  35. Zhou X, Chen L, Zhou X (2012) Structure tensor series-based large scale near-duplicate video retrieval. IEEE Trans Multimed 14(4):1220–1233

    Article  Google Scholar 

  36. Wu X, Hauptmann AG, Ngo CW (2007) Practical elimination of near-duplicates from web video search. In: ACM international conference on multimedia, pp 218–227

  37. Yeh M-C, Cheng K-T (2009) A compact, effective descriptor for video copy detection. In: ACM international conference on multimedia, pp 633–636

  38. Poullot S, Crucianu M, Buisson O (2008) Scalable mining of large video databases using copy detection. In: ACM international conference on multimedia, pp 61–70

  39. Zheng L, Qiu G, Huang J, Fu H (2011) Salient covariance for near-duplicate image and video detection. In: IEEE international conference on image processing, pp 2537–2540

  40. TRECVID2008: TREC Video Retrieval Evaluation. http://www-nlpir.nist.gov/projects/trecvid

Download references

Author information

Authors and Affiliations

  1. Department of Information and Communication Engineering, The University of Tokyo, Tokyo, Japan

    Zhipeng Wu & Kiyoharu Aizawa

Authors
  1. Zhipeng Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kiyoharu Aizawa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhipeng Wu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wu, Z., Aizawa, K. Self-similarity-based partial near-duplicate video retrieval and alignment. Int J Multimed Info Retr 3, 1–14 (2014). https://doi.org/10.1007/s13735-013-0049-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13735-013-0049-1

Keywords

Search

Navigation