Multimedia Systems

, Volume 18, Issue 4, pp 337–358 | Cite as

Concept-based near-duplicate video clip detection for novelty re-ranking of web video search results

  • Chidansh A. Bhatt
  • Pradeep K. Atrey
  • Mohan S. Kankanhalli
Regular Paper


State-of-the-art near-duplicate video clip (NDVC) detection for novelty re-ranking uses non-semantic low-level features (color/texture) to detect and eliminate “content-based NDVC” and increases content level novelty in the top results. However, humans may perceive a video as near duplicate from a semantic perspective as well. In this paper, we propose concept-based near-duplicate video clip (CBNDVC) detection technique for novelty re-ranking. We identify “semantic NDVC”, making use of the semantic features (events/concepts) and re-rank the top results to increase the content as well as semantic novelty. Videos are represented as a multivariate time series of confidence values of relevant concepts and thereafter discovery of CBNDVC clusters is achieved by conceptual clustering. Obtained results show higher precision and recall from the user’s perspective.


Near-duplicate video clip detection Semantic novelty and redundancy detection Concept-based near duplicates Conceptual clustering of videos Video semantic time-series representation Multimedia data mining 


  1. 1.
    Adjeroh, D.A., Lee, M.C., King, I.: A distance measure for video sequences. Comput. Vis. Image Underst. 75, 25–45 (1999)Google Scholar
  2. 2.
    Alexis Joly, O.B., Frlicot, C.: Content-based copy retrieval using distortion-based probabilistic similarity search. IEEE Trans. Multimed. 9, 293–306 (2007)Google Scholar
  3. 3.
    Aly, R., Hiemstra, D.: Concept detectors: how good is good enough? In: ACM International Conference on Multimedia, pp. 233–242 (2009)Google Scholar
  4. 4.
    Basharat, A., Zhai, Y., Shah, M.: Content based video matching using spatiotemporal volumes. Comput. Vis. Image Underst. 110, 360–377 (2008)Google Scholar
  5. 5.
    Cherubini, M., de Oliveira, R., Oliver, N.: Understanding near-duplicate videos: a user-centric approach. In: ACM International Conference on Multimedia, pp. 35–44 (2009)Google Scholar
  6. 6.
    Fanizzi, N., d’Amato, C., Esposito, F.: Conceptual clustering its application to concept drift and novelty detection. In: European Semantic Web Conference, pp. 318–332. Springer, Berlin (2008)Google Scholar
  7. 7.
    Fisher, D.H.: Knowledge acquisition via incremental conceptual clustering. Mach. Learn. 2, 139–172 (1987)Google Scholar
  8. 8.
    Han, J.: Data Mining: Concepts and Techniques. Morgan Kaufmann, San Francisco (2005)Google Scholar
  9. 9.
    Hauptmann, A., Yan, R., Lin, W.H.: How many high-level concepts will fill the semantic gap in news video retrieval? In: ACM International Conference on Image and Video Retrieval, pp. 627–634 (2007)Google Scholar
  10. 10.
    Huang, Z., Hu, B., Cheng, H., Shen, H.T., Hongyan, L., Zhou, X.: Mining near-duplicate graph for cluster-based reranking of web video search results. In: ACM Transactions on Information Systems (2010)Google Scholar
  11. 11.
    Keogh, E., Ratanamahatana, C.A.: Exact indexing of dynamic time warping. Knowl. Inf. Syst. 7, 358–386 (2005)Google Scholar
  12. 12.
    Law-To, J., Buisson, O., Gouet-Brunet, V., Boujemaa, N.: Robust voting algorithm based on labels of behavior for video copy detection. In: ACM International Conference on Multimedia, pp. 835–844 (2006)Google Scholar
  13. 13.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)Google Scholar
  14. 14.
    Min, H.S., Choi, J., De Neve, W., Ro, Y.M.: Near-duplicate video detection using temporal patterns of semantic concepts. In: IEEE International Symposium on Multimedia, pp. 65–71 (2009)Google Scholar
  15. 15.
    Rand, W.M.: Objective criteria for the evaluation of clustering methods. J. Am. Stat. Assoc. 66, 846–850 (1971)Google Scholar
  16. 16.
    Sen Ching, S.C., Zakhor, A.: Fast similarity search and clustering of video sequences on the world-wide-web. IEEE Trans. Multimed. 7, 524–537 (2005)Google Scholar
  17. 17.
    Shen, H.T., Zhou, X., Huang, Z., Shao, J., Zhou, X.: Uqlips: a real-time near-duplicate video clip detection system. In: International Conference on Very Large Databases, pp. 1374–1377 (2007)Google Scholar
  18. 18.
    Snoek, C.G.M., Worring, M.: Concept-based video retrieval foundations and trends in information retrieval, vol. 2, pp. 215–322. Now Publishers Inc., Hanover (2009)Google Scholar
  19. 19.
    Snoek, C.G.M., Worring, M., van Gemert, J.C., Geusebroek, J.M., Smeulders, A.W.M.: The challenge problem for automated detection of 101 semantic concepts in multimedia. In: ACM International Conference on Multimedia, pp. 421–430 (2006)Google Scholar
  20. 20.
    Stricker, M., Orengo, M.: Similarity of Color Images. vol. 2185, pp. 381–392 (1995)Google Scholar
  21. 21.
    Tian, X., Yang, L., Wang, J., Yang, Y., Wu, X., Hua, X-S.: Bayesian video search reranking. In: ACM International Conference on Multimedia, pp. 131–140. ACM, New York (2008)Google Scholar
  22. 22.
    Wei, W.: Time Series Analysis Univariate Multivariate Methods. Pearson Addison Wesley, Boston (2006)Google Scholar
  23. 23.
    Wu, X., Hauptmann, A.G., Ngo, C.W.: Practical elimination of near-duplicates from web video search. In: ACM International Conference on Multimedia, pp. 218–227 (2007)Google Scholar
  24. 24.
    Wu, X., Ngo, C.W., Hauptmann, A.G.: Carnegie Mellon University (Informedia group) and City University of Hong Kong (VIREO group)
  25. 25.
    Wu, X., Ngo, C.W., Hauptmann, A.G., Tan, H.K.: Real-time near-duplicate elimination for web video search with content and context. Trans. Multimed. 11, 196–207 (2009)Google Scholar
  26. 26.
    Wu, X., Zhao, W.L., Ngo, C.W., Hauptmann, A.G.: Near-duplicate web video detection. In: Hua, X.S., Worring, M., Chua, T.S. (eds.) Internet Multimedia Search and Mining, Bentham Science Publishers (2010)Google Scholar
  27. 27.
    Yanagawa, A., Chang, S.F., Kennedy, L., Hsu, W.: Columbia university’s baseline detectors for 374 lscom semantic visual concepts. Columbia University ADVENT Technical Report, pp. 846–850 (2007)Google Scholar
  28. 28.
    Yuan, J., Duan, L.Y., Tian, Q., Xu, C.: Fast and robust short video clip search using an index structure. In: ACM SIGMM International Workshop on Multimedia Information Retrieval, pp. 61–68 (2004)Google Scholar
  29. 29.
    Tian, X., Yang, L., Wang, J., Yang, Y., Wu, X., Hua, X.S.: Bayesian video search reranking. In: ACM International Conference on Multimedia, pp. 131–140 (2008)Google Scholar
  30. 30.
    Tan, H.-K., Ngo C.-W.,Wu, X.: Modeling video hyperlinks with hypergraph for Web video reranking. In: ACM International Conference on Multimedia, pp. 659–662 (2008)Google Scholar
  31. 31.
    Liu, Y., Mei, T., Hua, X.-S., Tang, J., Wu, X., Li, S.: Learning to video search rerank via pseudo preference feedback. In: International Conference on Multimedia and Expo, pp. 297–300 (2008)Google Scholar
  32. 32.
    Hsu, W., Kennedy, L., Chang, S.F.: Video search reranking through random walk over document-level context graph. In: ACM International Conference on Multimedia, pp. 971–980 (2007)Google Scholar
  33. 33.
    McKusick, K.,Thompson, K.: COBWEB/3: a portable implementation. Technical Report No. FIA-90-6-18-2, pp. 139–172 (1990)Google Scholar
  34. 34.
    Reich, Y., Fenves, S.J.: The formation and use of abstract concepts in design. In: Concept formation knowledge and experience in unsupervised learning, pp. 323–353. Morgan Kaufmann Publishers Inc., San Francisco (1991)Google Scholar
  35. 35.
    Cheeseman, P., Stutz, J.: Bayesian classification (AutoClass): theory and results. In: Advances in knowledge discovery and data mining, pp. 53–180. American Association for Artificial Intelligence, Menlo Park (1996)Google Scholar
  36. 36.
    Talavera, L., Bejar, J.J.: Generality-based conceptual clustering with probabilistic concepts. IEEE Trans. Pattern Anal. Mach. Intell. 23, 196–206 (2001)Google Scholar
  37. 37.
    Chu, W.W., Chiang, K., Hsu, C., Yau, H.: An error-based conceptual clustering method for providing approximate query answers. vol. 39, pp. 216–230. Commun. ACM (1996)Google Scholar
  38. 38.
    Wei, X., Ngo, C., Jiang, Y.: Selection of concept detectors for video search by ontology-enriched semantic spaces. IEEE Trans. Multimed. 10, 1085–1096 (2008)Google Scholar
  39. 39.
    Snoek, C.G.M., Huurnink, B., Hollink, L., Rijke, M., Schreiber, G., Worring, M.: Adding semantics to detectors for video retrieval. IEEE Trans. Multimed. 9, 975–986 (2007)Google Scholar
  40. 40.
    Bhatt, C., Kankanhalli, M.: Multimedia data mining: state of the art and challenges. Multimed. Tools Appl. 51, 35–76 (2011)Google Scholar
  41. 41.
    Bhatt, C., Kankanhalli, M.: Probabilistic temporal multimedia data mining. In: ACM Transactions on Intelligent Systems and Technology, vol. 2, pp. 17:1–17:19. ACM, New York (2011)Google Scholar
  42. 42.
    Bather, J.: An Introduction to Dynamic Programming and Sequential Decisions. Wiley-Interscience Series in Systems Optimisation, Wiley, Hoboken (2000)Google Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  • Chidansh A. Bhatt
    • 1
  • Pradeep K. Atrey
    • 2
  • Mohan S. Kankanhalli
    • 1
  1. 1.School of ComputingNational University of SingaporeSingaporeSingapore
  2. 2.Applied Computer ScienceUniversity of WinnipegWinnipegCanada

Personalised recommendations