Multi-layer Spectral Clustering for Video Segmentation

  • Xiaofei Di
  • Hong Chang
  • Xilin Chen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7725)


Video segmentation with spatial priority suffers from incoherence problem, since the presegments of consecutive frames may be very different. To address this problem, this paper proposes an effective and scalable approach for video segmentation, aiming to cluster video pixels that are coherent in both appearance and motion. We build up a multi-layer graph based on multiple segmentations of the video frames, where each presegment corresponds to a vertex in the graph and each layer corresponds to the segmentation result using mean shift algorithm under specific granularity. Three types of edges are connected in the graph and the corresponding affinities are defined which convey local grouping cues of intra-frame, inter-frame and inter-layer neighborhoods. Then the task of video segmentation is formulated into graph partition, which can be solved efficiently by power iteration clustering algorithm. Both qualitative and quantitative experimental results demonstrate the efficacy of our proposed method.


Image Segmentation Segmentation Result Motion Segmentation Video Segmentation Shift Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Tron, R., Vidal, R.: A benchmark for the comparison of 3-d motion segmentation algorithms. In: CVPR (2007)Google Scholar
  2. 2.
    Rao, S.R., Tron, R., Vidal, R.: Motion segmentation via robust subspace seperation in the presence of outlying, incomplete, or corrupted trajectories. PAMI (2010)Google Scholar
  3. 3.
    DeMenthon, D., Megret, R.: Spatio-temporal segmentation of video by hierarchical mean shift analysis. In: CVPR (2000)Google Scholar
  4. 4.
    Khan, S., Shah, M.: Object based segmentation of video using color, motion and spatial information. PAMI (2005)Google Scholar
  5. 5.
    Torsello, A., Pavan, M., Pelillo, M.: Object based segmentation of video using color, motion and spatial information. In: EMMCVPR (2005)Google Scholar
  6. 6.
    Grundmann, M., Kwatra, V., Han, M., Essa, I.: Efficient hierarchical graph-based video segmentation. In: CVPR (2010)Google Scholar
  7. 7.
    Nagahashi, T., Fujiyoshi, H., Kanade, T.: Video Segmentation Using Iterated Graph Cuts Based on Spatio-temporal Volumes. In: Zha, H., Taniguchi, R.-i., Maybank, S. (eds.) ACCV 2009, Part II. LNCS, vol. 5995, pp. 655–666. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  8. 8.
    Brendel, W., Todorovic, S.: Video object segmentation by tracking regions. In: ICCV (2009)Google Scholar
  9. 9.
    Ke, Q., Kanade, T.: A subspace approach to layer extraction. In: CVPR (2001)Google Scholar
  10. 10.
    Hedau, V., Arora, H., Ahuja, N.: Matching images under unstable segmentations. In: CVPR (2008)Google Scholar
  11. 11.
    Kim, T.H., Lee, K.M., Lee, S.U.: Learning full pairwise affinities for spectral segmentation. In: CVPR (2010)Google Scholar
  12. 12.
    Weiss, Y.: Segmentation using eigenvectors: a unifying view. In: ICCV (1999)Google Scholar
  13. 13.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. PAMI (2000)Google Scholar
  14. 14.
    Cour, T., Bébézit, F., Shi, J.: Segmentation using eigenvectors: a unifying view. In: CVPR (2005)Google Scholar
  15. 15.
    Lin, F., Cohen, W.W.: Power iteration clustering. In: ICML (2010)Google Scholar
  16. 16.
    Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. PAMI (2002)Google Scholar
  17. 17.
    Kohli, P., Ladicky, L., Torr, P.: Robust higher order potentials for enforcing label consistency. In: CVPR (2008)Google Scholar
  18. 18.
    Hu, G., Gao, Q.: A non-parametric statistics based method for generic curve partition and classification. In: ICIP (2010)Google Scholar
  19. 19.
    Vazquez-Reina, A., Avidan, S., Pfister, H., Miller, E.: Multiple Hypothesis Video Segmentation from Superpixel Flows. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 268–281. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  20. 20.
    Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: From contours to regions:an empirical evaluation. In: CVPR (2009)Google Scholar
  21. 21.
    Gorelick, L., Blank, M., Shechtman, E., Irani, M., Basri, R.: Action as space-time shapes. IEEE TPAMI (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Xiaofei Di
    • 1
    • 2
    • 3
  • Hong Chang
    • 1
    • 2
  • Xilin Chen
    • 1
    • 2
  1. 1.Key Lab of Intelligent Information ProcessingChinese Academy of Sciences (CAS)BeijingChina
  2. 2.Institute of Computing TechnologyCASBeijingChina
  3. 3.Graduate SchoolChinese Academy of SciencesBeijingChina

Personalised recommendations