Video shot boundary detection using multiscale geometric analysis of nsct and least squares support vector machine
- 102 Downloads
- 1 Citations
Abstract
The fundamental step in video content analysis is the temporal segmentation of video stream into shots, which is known as Shot Boundary Detection (SBD). The sudden transition from one shot to another is known as Abrupt Transition (AT), whereas if the transition occurs over several frames, it is called Gradual Transition (GT). A unified framework for the simultaneous detection of both AT and GT have been proposed in this article. The proposed method uses the multiscale geometric analysis of Non-Subsampled Contourlet Transform (NSCT) for feature extraction from the video frames. The dimension of the feature vectors generated using NSCT is reduced through principal component analysis to simultaneously achieve computational efficiency and performance improvement. Finally, cost efficient Least Squares Support Vector Machine (LS-SVM) classifier is used to classify the frames of a given video sequence based on the feature vectors into No-Transition (NT), AT and GT classes. A novel efficient method of training set generation is also proposed which not only reduces the training time but also improves the performance. The performance of the proposed technique is compared with several state-of-the-art SBD methods on TRECVID 2007 and TRECVID 2001 test data. The empirical results show the effectiveness of the proposed algorithm.
Keywords
Shot boundary detection Abrupt transition Gradual transition Principal component analysis Non-subsampled contourlet transform Least squares support vector machineNotes
Acknowledgements
The first author acknowledges Tata Consultancy Services (TCS) for providing fellowship to carry out the research work. Malay K. Kundu acknowledges the Indian National Academy of Engineering (INAE) for their support through INAE Distinguished Professor fellowship. The authors would like to thank the National Institute of Standards & Technology (NIST) for providing TRECVID data set.
References
- 1.Arman F, Hsu A, Chiu MY (1994) Image Processing on encoded video sequences. Multimedia Syst 1(5):211–219CrossRefGoogle Scholar
- 2.Bescos J, Cisneros G, Martinez JM, Menendez JM, Cabrera J (2005) A unified model for techniques on video-shot transition detection. IEEE Trans Multimedia 7(2):293–307CrossRefGoogle Scholar
- 3.Brabanter KD, Karsmakers P, Ojeda F, Alzate C, Brabanter JD, Pelckmans K, Moor BD, Vandewalle J, Suykens JAK (2011) LS-SVMlab Toolbox Users Guide version 1.8, ESAT-SISTA Technical Report 10-146 pp 1–115Google Scholar
- 4.Chasanis V, Likas A, Galatsanos N (2009) Simultaneous detection of abrupt cuts and dissolves in videos using support vector machines. Pattern Recogn Lett 30 (2009):55–65CrossRefGoogle Scholar
- 5.Choudhury A, Medioni G (2012) A framework for robust online video contrast enhancement using modularity optimization. IEEE Trans Circuits Syst Video Technol 22(9):1266–1279CrossRefGoogle Scholar
- 6.Chowdhury M, Kundu MK (2014) Comparative assessment of efficiency for content based image retrieval systems using different wavelet features and pre-classifier. Multimedia Tools and Applications 72(3):1–36Google Scholar
- 7.Chua T-S, Feng H, Chandrashekhara A (2003) An unified framework for shot boundary detection via active learning Proceedings Int. Conf. Acoust. Speech Signal Proces, pp 845–848Google Scholar
- 8.Cooper M, Liu T, Rieffel E (2007) Video segmentation via temporal pattern classification. IEEE Trans Multimedia 9(3):610–618CrossRefGoogle Scholar
- 9.da Cunha AL, Zhou J, Do MN (2006) The nonsubsampled contourlet transform: theory, Design, and Applications. IEEE Trans Image Process 15:3089–3101CrossRefGoogle Scholar
- 10.Do MN, Vetterli M (2005) The Contourlet Transform: an efficient directional multiresolution image representation. IEEE Trans Image Process 14(12):2091–2106CrossRefGoogle Scholar
- 11.Duda RO, Hart PE, David G (2012) Pattern classification, John Wiley & SonsGoogle Scholar
- 12.Garcia-Perez AM (1992) The perceived image: Efficient modelling of visual inhomogeneity. Spat Vis 6(2):89–99CrossRefGoogle Scholar
- 13.Gianluigi C, Raimondo S (2006) An innovative algorithm for key frame extraction in video summarization. J Real-Time Image Proc 1(1):69–88CrossRefGoogle Scholar
- 14.Hanjalic A (2002) Shot-boundary detection: unraveled and resolved?. IEEE Trans Circuits Syst Video Technol 12(2):90–105CrossRefGoogle Scholar
- 15.Hsu CW, Lin CJ (2002) A comparison of methods for multi-class support vector machines. IEEE Trans Neural Netw 13(2):415–425CrossRefGoogle Scholar
- 16.Jang H (2006) Gradual shot boundary detection using localized edge blocks, vol 28Google Scholar
- 17.Kawai Y, Sumiyoshi H, Yagi N (2007) Shot boundary detection at TRECVID 2007 Proceedings TREC Video Retr. Eval OnlineGoogle Scholar
- 18.Kundu MK, Mondal J (2012) A novel technique for automatic abrupt shot transition detection Proceedings Int. Conf. Communications, Devices and Intelligent Systems, pp 628–631Google Scholar
- 19.Lakshmi Priya GG, Domnic S (2014) Walsh-Hadamard Transform kernel-based feature vector for shot boundary detection. IEEE Trans Image Process 12:23MathSciNetMATHGoogle Scholar
- 20.Li S, Yang B, Hu J (2011) Performance comparison of different multi-resolution transforms for image fusion. Information Fusion 12(2):74–84CrossRefGoogle Scholar
- 21.Li W-K, Lai S-H (2002) A motion-aided video shot segmentation algorithm Pacific rim Conference Multimedia, pp 336–343Google Scholar
- 22.Liu Z, Zavesky E, Gibbon D, Shahraray B, Haffner P (2007) AT&T research at TRECVID 2007 Proceedings TRECVID WorkshopGoogle Scholar
- 23.Lopez F, Valiente JM, Baldrich R, Vanrell M (2005) Fast surface grading using color statistics in the CIE lab space Proceedings Pattern Recognition and Image Analysis, pp 666–673CrossRefGoogle Scholar
- 24.Ma YF, Sheng J, Chen Y, Zhang HJ (2001) Msr-asia at trec-10 video track: Shot boundary detection Proceedings TRECGoogle Scholar
- 25.Miene A, Dammeyer A, Hermes T, Herzog O (2001) Advanced and adaptive shot boundary detection Proceedings ECDL WS Generalized Documents, pp 39–43Google Scholar
- 26.Mithling M, Ewerth R, Stadelmann T, Zofel C, Shi B, Freislchen B (2007) University of Marburg at TRECVID 2007: Shot boundary detection and high level feature extraction Proceedings REC Video Retr. Eval OnlineGoogle Scholar
- 27.Mohanta PP, Saha SK, Chanda B (2012) A model-based shot boundary detection technique using frame transition parameters. IEEE Trans Multimedia 14 (1):223–233CrossRefGoogle Scholar
- 28.Omidyeganeh M, Ghaemmaghami S, Shirmohammadi S (2011) Video keyframe analysis using a segment-based statistical metric in a visually sensitive parametric space. IEEE Trans Image Process 20(10):2730–2737MathSciNetCrossRefMATHGoogle Scholar
- 29.Ren J, Jiang J, Chen J (2007) Determination of Shot boundary in MPEG videos for TRECVID 2007 Proceedings TREC Video Retr. Eval OnlineGoogle Scholar
- 30.Sasithradevi A, Roomi SMdM, Raja R (2016) Non-subsampled Contourlet Transform based Shot Boundary Detection. IJCTA 9(7):3231–3228Google Scholar
- 31.Smeaton AF, Over P, Doherty AR (2010) Video shot boundary detection: Seven years of trecvid activity. Comput Vis Image Underst 114(4):411–418CrossRefGoogle Scholar
- 32.Suykens JAK, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300CrossRefMATHGoogle Scholar
- 33.TRECVID Dataset. Available: http://trecvid.nist.gov/
- 34.Youseff SM (2012) IC,TEDCT-CBIR: Integrating curvelet transform with enhanced dominant colors extraction and texture analysis for efficient content-based image retrieval. Comput Electr Eng 38(5):1358–1376CrossRefGoogle Scholar
- 35.Yuan et al (2007) THU And ICRC at TRECVID 2007 Proceedings TREC video retr. Eval. OnlineGoogle Scholar
- 36.Yuan J, Wang H, Xiao L, Zheng W, Li J, Lin F, Zhang B (2007) A formal study of shot boundary detection. IEEE Trans Circuits Syst Video Technol 17 (2):168–186CrossRefGoogle Scholar
- 37.Zhang HJ, Kankanhalli A, Smolier SW (1993) Automatic partitioning of full-motion video. Multimedia Systems 1(1):10–28CrossRefGoogle Scholar