Skip to main content
SpringerLink
Log in

Unsupervised video summarization via clustering validity index

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Although lots of the prior works have been proposed to solve the representative selection problem of video summarization, the main difficulty is still left for determining the optimal representatives’ number of the raw videos that are not annotated. In this paper, we propose an unsupervised video summarization method by motion-based frame selection and a novel clustering validity indexes to determine the optimal representatives of the original video. The proposed framework segments shots and selects candidate frames by evaluating their forward and backward motion and can automatically select representatives to highlight all the significant visual properties. Shots are segmented uniformly and the frame with the largest motion is extracted in each segmentation to form the video candidate frame subset. Then Affinity Propagation combined with the validity index is used to automatically select the optimal representatives from the candidate frame subset. Our experimental result on several benchmark datasets demonstrates the robustness and effectiveness of our proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Bezdek JC (1998) Some new indexes of cluster validity. IEEE Trans Syst Man Cybern 28(3):301–315

    Article  Google Scholar 

  2. Bharill N (2014) Enhanced cluster validity index for the evaluation of optimal number of clusters for Fuzzy C-Means algorithm. IEEE International Conference on Fuzzy Systems, p 1526–1533

  3. Calinski T (1974) A dendrite method for cluster analysis. Communications in Statistics 3(1):1–27

    MathSciNet  MATH  Google Scholar 

  4. Davies DL (1979) A cluster separation measure. IEEE Trans Pattern Anal Mach Intell 1(2):224–227

    Article  Google Scholar 

  5. Dimitriadou E (2002) An examination of indexes for determining the number of cluster in binary data sets. Psychometrika 67(1):137–160

    Article  MathSciNet  MATH  Google Scholar 

  6. Dudoit S (2002) A prediction-based resampling method for estimating the number of clusters in a dataset. Genome Biol 3(7):1–21

    Article  Google Scholar 

  7. Frey BJ (2007) Clustering by passing messages between data points. Science 315:972–976

    Article  MathSciNet  MATH  Google Scholar 

  8. Gygli M (2014) Creating summaries from user videos. European Conference on Computer Viision (ECCV), p 505–520

  9. Hong R (2016) Multi-View Object Retrieval via Multi-Scale Topic Models. IEEE Trans Image Process 25(12):5814–5827

    Article  MathSciNet  MATH  Google Scholar 

  10. James C (2016) The generalized C index for internal fuzzy cluster validity. IEEE Trans Fuzzy Syst 24(6):1500–1512

    Article  Google Scholar 

  11. Kapp AV, Tibshirani R (2007) Are clusters found in one dataset present in another dataset? Biostatistics 8(1):9–31

    Article  MATH  Google Scholar 

  12. Lin P-L (2016) An efficient validity index method for datasets with complex-shaped clusters. International Conference on Machine Learning and Cybernetics (ICMLC), p 2160–1348

  13. Liu X (2015) Event-based media enrichment using an adaptive probabilistic hypergraph model. IEEE Transactions on Cybernetics 45(11):2461–2471

    Article  Google Scholar 

  14. Luo C (2016) Image classification by selective regularized subspace learning. IEEE Trans Multimedia 18(1):40–50

    Article  Google Scholar 

  15. Ma M (2017) Nonlinear kernel sparse dictionary selection for video summarization. Proceeding of the IEEE International Conference on Multimedia and Expo (ICME), p 637–642

  16. Mahasseni B (2017) Unsupervised video summarization with adversarial LSTM networks. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), p 2982–2991

  17. Meng J (2018) Video summarization via Multiview representative selection. IEEE Trans Image Process 27(5):2134–2145

    Article  MathSciNet  MATH  Google Scholar 

  18. Panda R (2017) Weakly supervised summarization of web videos. IEEE International Conference on Computer Vision (ICCV), p 3677–3686

  19. Rousseeuw P (1987) Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math 20(1):53–65

    Article  MATH  Google Scholar 

  20. Rui XU (2012) A comparison study of validity indices on swarm-intelligence-based clustering. IEEE Trans Syst Man Cybern 42(4):1243–1256

    Article  Google Scholar 

  21. Said AB (2017) Cluster validity index based on Jeffrey divergence. Pattern Anal Applic 20(1):21–31

    Article  MathSciNet  Google Scholar 

  22. Song Y (2015) TVSum: Summatizing Web Video Using Titles, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), p 5179–5187

  23. Tasdemir K (2011) A validity index for prototype-based clustering of data sets with complex cluster structures. IEEE Trans Syst Man Cybern 41(4):1039–1053

    Article  Google Scholar 

  24. Varini P (2017) Personalized egocentric video summarization of cultural tour on user preferences input. IEEE Trans Multimedia 19(12):2832–2844

    Article  Google Scholar 

  25. Wang M (2012) Event driven web video summarization by tag localization and key-shot identification. IEEE Trans Multimedia 14(4):975–985

    Article  Google Scholar 

  26. Wang M (2013) View-based discriminative probabilistic modeling for 3D object retrieval and recognition. IEEE Trans Image Processing 22(4):1395–1407

  27. Wang M (2016) Scalable semi-supervised learning by efficient anchor graph regularization. IEEE Trans Knowl Data Eng 28(7):1864–1877

    Article  Google Scholar 

  28. Wang M (2017) Learning on big graph: label inference and regularization with anchor hierarchy. IEEE Trans Knowl Data Eng 29(5):1101–1114

    Article  Google Scholar 

  29. Youtube statistics. https://www.youtube.com/yt/about/press/. Accessed 2018 May 30

  30. Zhang K (2016) Summary transfer: exemplar-based subset selection for video summarization. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), p 1059–1067

  31. Zhang Y (2017) Motion-state-adaptive video summarization via spatiotemporal analysis. IEEE Trans Circuits Syst Video Techn 27(6):1340–1352

  32. Zhao Y (2016) Visual summarization of image collections by fast RANSAC. Neurocomputing 172:48–52

    Article  Google Scholar 

  33. Zheng Y (2017) Driver types and their behaviors within a high level of pedestrian activity environment. The International Journal of Transportation Research 9(1):1–11

    Google Scholar 

  34. Zhou S (2011) Comparative study on method for determining optimal number of clusters based on affinity propagation clustering. Computer Science 38(2):225–228

    Google Scholar 

Download references

Acknowledgements

This work was partially supported by National Key R&D Program of China, No. 2017YFC0820604; Anhui Provincial Natural Science Foundation, No. 18808085QF188 and the National Nature Science Foundation of China under Grant No. 61502138, Grant No. 61702156, Grant No. 61472116, and Grant No. 61471154.

Author information

Authors and Affiliations

  1. Hefei University of Technology, Hefei, 230601, Anhui, China

    Ye Zhao, Yanrong Guo, Rui Sun, Zhengqiong Liu & Dan Guo

Authors
  1. Ye Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanrong Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rui Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhengqiong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dan Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ye Zhao.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, Y., Guo, Y., Sun, R. et al. Unsupervised video summarization via clustering validity index. Multimed Tools Appl 79, 33417–33430 (2020). https://doi.org/10.1007/s11042-019-7582-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-7582-8

Keywords

Search

Navigation