Abstract
In this paper we present a new approach for automatic summarization of rushes, or unstructured video. Our approach is composed of three major steps. First, based on shot and sub-shot segmentations, we filter sub-shots with low information content not likely to be useful in a summary. Second, a method using maximal matching in a bipartite graph is adapted to measure similarity between the remaining shots and to minimize inter-shot redundancy by removing repetitive retake shots common in rushes video. Finally, the presence of faces and motion intensity are characterised in each sub-shot. A measure of how representative the sub-shot is in the context of the overall video is then proposed. Video summaries composed of keyframe slideshows are then generated. In order to evaluate the effectiveness of this approach we re-run the evaluation carried out by TRECVid, using the same dataset and evaluation metrics used in the TRECVid video summarization task in 2007 but with our own assessors. Results show that our approach leads to a significant improvement on our own work in terms of the fraction of the TRECVid summary ground truth included and is competitive with the best of other approaches in TRECVid 2007.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Because of the relatively excellent performance of baseline runs in TRECVid Summarisation 2007, we use these as a basis for comparison against our own work since they were almost the best.
References
Byrne D, Kehoe P, Lee H, Ó’Conaire C, Smeaton AF, O’Connor NE, Jones GJ (2007) A user-centered approach to rushes summarisation via highlight-detected keyframes. In: TVS ’07: proceedings of the international workshop on TRECVID video summarization. ACM, New York, pp 35–39
Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 8(6):679–698
Cooray S, O’Connor N (2005) Hybrid technique for face detection in color images. In: IEEE conference on advanced video and signal based surveillance, AVSS, Italy, pp 253–258
Dai Y, Hu G, Chen W (1995) Graph theory and algebra structure. Tsinghua University Press, Beijing, pp 89–91 (in Chinese)
Ferman A, Tekalp A (2003) Two-stage hierarchical video summary extraction to match low-level user browsing preferences. IEEE Trans Multimedia 5(2):244–256
Liu C (2003) A Bayesian discriminating features method for face detection. IEEE Trans Pattern Anal Mach Intell 25:741–754
Ma Y, Lu L, Zhang H, Li M (2002) A user attention model for video summarization. In: Proceedings of the tenth ACM international conference on multimedia. ACM, New York, pp 533–542
Ngo C, Zhao W, Jiang Y (2006) Fast tracking of near-duplicate keyframes in broadcast domain with transitivity propagation. In: Proceedings of the 14th annual ACM international conference on multimedia. ACM, New York, pp 845–854
O’Connor N, Cooke E, le Borgne H, Blighe M, Adamek T (2005) The AceToolbox: low-Level audiovisual feature extraction for retrieval and classification. In: 2nd IEE European workshop on the integration of knowledge, semantic and digital media technologies
Over P, Smeaton AF, Awad G (2008) The TRECVid 2008 BBC rushes summarization evaluation. In TVS ’08: Proceedings of the 2nd ACM TRECVid video summarization workshop. ACM, New York, pp 1–20
Over P, Smeaton AF, Kelly P (2007) The TRECVid 2007 BBC rushes summarization evaluation pilot. In TVS ’07: Proceedings of the international workshop on TRECVID video summarization. ACM, New York, pp 1–15
Smeaton AF, Over P, Doherty AR (2009) Video shot boundary detection: Seven years of TRECVid activity. Comput Vis Image Underst. doi:10.1016/j.cviu.2009.03.011
Smeaton AF, Over P, Kraaij W (2006) Evaluation campaigns and TRECVid. In: MIR ’06: proceedings of the 8th ACM international workshop on multimedia information retrieval. ACM, New York, pp 321–330
Taskiran C, Pizlo Z, Amir A, Ponceleon D, Delp E (2006) Automated video program summarization using speech transcripts. IEEE Trans Multimedia 8(4):775–791
Acknowledgements
This work was funded by the National High Technology Development 863 Program of China (2006AA01Z316), the National Natural Science Foundation of China (60572137 and 60875048) and by Science Foundation Ireland as part of the CLARITY CSET (07/CE/I1147). The authors thank the reviewers for their helpful and insightful feedback.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bai, L., Hu, Y., Lao, S. et al. Automatic summarization of rushes video using bipartite graphs. Multimed Tools Appl 49, 63–80 (2010). https://doi.org/10.1007/s11042-009-0398-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-009-0398-1