Advertisement

Journal of Real-Time Image Processing

, Volume 16, Issue 6, pp 1987–1997 | Cite as

Dual-dissimilarity measure-based statistical video cut detection

  • Gyujin Bae
  • Sung In Cho
  • Suk-Ju Kang
  • Young Hwan KimEmail author
Original Research Paper

Abstract

Video cut detection is an essential process of temporal continuity-based video applications such as video segmentation, video retargeting, and frame rate up-conversion. The performance of these applications highly depends on the performance of cut detection. This paper proposes an effective and low-complexity approach for detecting video cuts. The proposed method uses two simple dissimilarity measures for video cut detection: inter-frame luminance variation and temporal variation of inter-frame variations over several frames. The first is used to detect abrupt changes, and the second is used to reduce the influence of disturbances, e.g., object or camera motion. The proposed method is comprised of the following three steps. First, it computes the two dissimilarity measures. Then, it combines them using Bayesian estimation and linear regression. Finally, it decides on the possibility of cuts using the combined dissimilarity measure. Experimental results show that the average F1 score of the proposed method was up to 0.252 (37.0%) higher than those of the benchmark methods. Moreover, the algorithmic simplicity of the proposed method reduced the average computation time per pixel by up to 99.8%, when compared with state-of-the-art methods. Thus, the proposed method is superior to existing methods in terms of computational complexity and detection accuracy.

Keywords

Video cut detection Shot boundary detection Motion invariance Bayesian estimation Linear regression 

Notes

Acknowledgements

This research was supported by LG Display Co., Ltd., IDEC, and the MSIP Ministry of Science, ICT and Future Planning), Korea, under the “ICT Consilience Creative Program” (IITP-R0346-16-1007) supervised by the ITTP (Institute for Information and communications Technology Promotion).

References

  1. 1.
    Guan, G., Wang, Z., Lu, S., Deng, J.D.: Keypoint based keyframe selection. IEEE Trans. Circuits Syst. Video Technol. 23(4), 729–734 (2013)CrossRefGoogle Scholar
  2. 2.
    Hampapur, A., Weymouth, T., Jain, R.: Digital video segmentation.In: Proceedings ACM Multimedia, pp. 357–364, (1994)Google Scholar
  3. 3.
    Tarabalka, Y., Charpiat, G., Brucker, L., Menze, B.H.: Spatio-temporal video segmentation with shape growth or shrinkage constraint. IEEE Trans. Image Process. 23(9), 3829–3840 (2014)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Li, C., Lin, L., Zuo, W., Yan, S., Tang, J.:Sold: sub-optimal low-rank decomposition for efficient video segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5519–5527, (2015)Google Scholar
  5. 5.
    Tambo, A.L., Bhanu, B.: Segmentation of pollen tube growth videos using dynamic bi-modal fusion and seam carving. IEEE Trans. Image Process. 25(5), 1993–2004 (2016)CrossRefGoogle Scholar
  6. 6.
    Avidan, S., Shamir, A.: Seam carving for content-aware image resizing. ACM Trans. Graphics 26(3), 10 (2007)CrossRefGoogle Scholar
  7. 7.
    Krahenbuhl, P., Lang, M., Hornung, A., Markus, G.: A system for retargeting of streaming video. ACM Trans. Graphics 28(5), 126 (2009)CrossRefGoogle Scholar
  8. 8.
    Cheng, W.-H., Wang, C.-W., Wu, J.-L.: Video adaptation for small display based on content recomposition. IEEE Trans. Circuits Syst. Video Technol. 17(1), 43–58 (2007)CrossRefGoogle Scholar
  9. 9.
    Patti, A.J., Sezan, M.I., Tekalp, A.M.: High resolution standards conversion of low resolution video. In: Proceedings IEEE International Conference Acoustics, Speech, Signal Processing, vol. 4, pp. 2197–2200, (1995)Google Scholar
  10. 10.
    Kang, S.-J., Cho, K.R., Kim, Y.H.: Motion compensated frame rate up-conversion using extended bilateral motion estimation. IEEE Trans. Consumer Electron. 53(4), 1759–1767 (2007)CrossRefGoogle Scholar
  11. 11.
    Kang, S.-J., Yoo, S., Kim, Y.H.: Dual motion estimation for frame rate up-conversion. IEEE Trans. Circuits Syst. Video Technol. 20(12), 1909–1914 (2010)CrossRefGoogle Scholar
  12. 12.
    Kang, S.-J.: Adaptive luminance coding-based scene-change detection for frame rate up-conversion. IEEE Trans. Consumer Electron. 59(2), 370–375 (2013)CrossRefGoogle Scholar
  13. 13.
    Zhang, H., Kankanhalli, A., Smoliar, S.: Automatic partitioning of full-motion video. Multimedia Syst. 1(1), 10–28 (1993)CrossRefGoogle Scholar
  14. 14.
    Yeo, B.L., Liu, B.: Rapid scene analysis on compressed video. IEEE Trans. Circuits Syst. Video Technol. 5(6), 90–105 (1995)Google Scholar
  15. 15.
    Hanjalic, A.: Shot-boundary detection: unraveled and resolved? IEEE Trans. Circuits Syst. Video Technol. 12(2), 90–105 (2002)CrossRefGoogle Scholar
  16. 16.
    Ma, Y.F., Sheng, J., Chen, Y., Zhang, H.J.: MSR-ASIA at TREC-10 video track: shot boundary detection task. In: Proceedings 10th Text Retrieval Conference (TREC), pp. 371–377, (2001)Google Scholar
  17. 17.
    Dimou, A., Nemethova, O., Rupp, M.: Scene change detection for H. 264 using dynamic threshold techniques. In: Proceedings 5th EURASIP Conference Speech Image Process. Multimedia Common. Service, pp. 1–6, (2005)Google Scholar
  18. 18.
    Kim, J.R., Suh, S., Sull, S.: Fast scene change detection for personal video recorder. IEEE Trans. Consumer Electron. 49(3), 683–688 (2003)CrossRefGoogle Scholar
  19. 19.
    Tan, Y.P., Nagamani, J., Lu, H.: Modified Kolmogorov-Smirnov metric for shot boundary detection. Electron. Lett. 39(18), 1313–1315 (2003)CrossRefGoogle Scholar
  20. 20.
    Chasanis, V., Lias, A., Galatsanos, N.: Simultaneous detection of abrupt cuts and dissolves in videos using support vector machines. Pattern Recogn. Lett. 30(1), 55–65 (2009)CrossRefGoogle Scholar
  21. 21.
    Bendraou, V., Essannouni, F., Aboutajdine, D., Salam, A.: Video shot boundary detection method using histogram differences and local image descriptor. In: Complex Systems (WCCS), 2014 Second World Conference on, (2014)Google Scholar
  22. 22.
    Shu, H., Chau, L.-P.: A new scene change feature for video transcoding. In: Proceedings IEEE International Symposium on Circuits and Systems, pp. 4582–4585, (2005)Google Scholar
  23. 23.
    Lin, W., Sun, M.T., Li, H., Hu, H.M.: A new shot change detection method using information from motion estimation. In: Pacific-Rim Conference on Multimedia, pp. 264–275, (2010)CrossRefGoogle Scholar
  24. 24.
    Lee, J., Kim, S.-J., Lee, C.S.: Effective scene change detection by using statistical analysis of optical flows. Appl. Math Info. Sci. 6(1), 177–183 (2012)Google Scholar
  25. 25.
    Yuan, J., Wang, H., Xiao, L., Zheng, W., Li, J., Lin, F., Zhang, B.: A formal study of shot boundary detection. IEEE Trans. Circuits Syst. Video Technol. 17(2), 168–186 (2007)CrossRefGoogle Scholar
  26. 26.
    Kucuktunc, O., Gudukbay, U., Ulusoy, O.: Fuzzy color histogram-based video segmentation. Comput. Vis. Image Underst. 114(1), 125–134 (2010)CrossRefGoogle Scholar
  27. 27.
    Kang, S.-J., Cho, S.I., Yoo, S., Kim, Y.H.: Scene change detection using multiple histograms for motion-compensated frame rate up- conversion. J. Display Technol. 8(3), 121–126 (2012)CrossRefGoogle Scholar
  28. 28.
    (2001). TREC video retrieval test collection (Online). http://www.open-video.org/collection_detail.php?cid=7
  29. 29.
    Sun, J., Wan, Y.: A novel metric for efficient video shot boundary detection. In: Proceedings 2014 International Conference on Visual Communication and Image Processing, pp. 45–48, (2014)Google Scholar
  30. 30.
    Yang, Y., Liu, X.: A re-examination of text categorization methods. In: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 42–49, (1999)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  1. 1.Department of Electrical EngineeringPohang University of Science and Technology (POSTECH)PohangKorea
  2. 2.Department of Electronic Engineering, College of Information and Communication EngineeringDaegu UniversityGyeongbukKorea
  3. 3.Department of Electronic EngineeringSogang UniversitySeoulKorea

Personalised recommendations