Learning and Propagation of Dominant Colors for Fast Video Segmentation

  • Cédric Verleysen
  • Christophe De Vleeschouwer
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8192)


Color segmentation is an essential problem in image processing. While most of the recent works focus on the segmentation of individual images, we propose to use the temporal color redundancy to segment arbitrary videos. In an initial phase, a k-medoids clustering is applied on histogram peaks observed on few frames to learn the dominant colors composing the recorded scene. In a second phase, these dominant colors are used as reference colors to speed up a color-based segmentation process and, are updated on-the-fly when the scene changes. Our evaluation first shows that the proprieties of k-medoids clustering make it well suited to learn the dominant colors. Then, the efficiency and the effectiveness of the proposed method are demonstrated and compared to standard segmentation benchmarks. This assessment reveals that our approach is more than 250 times faster than the conventional mean-shift segmentation, while preserving the segmentation accuracy.


Segmentation clustering color learning k-medoids box-cox transform 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alpert, S., Galun, M., Basri, R., Brandt, A.: Image segmentation by probabilistic bottom-up aggregation and cue integration. In: Proc. of IEEE CVPR 2007, pp. 1–8 (June 2007)Google Scholar
  2. 2.
    Bai, X., Wang, J., Simons, D., Sapiro, G.: Video snapcut: robust video object cutout using localized classifiers. ACM SIGGRAPH 28(3), 70 (2009)CrossRefGoogle Scholar
  3. 3.
    Box, G., Cox, D.: An analysis of transformations. Journal of the Royal Statistical Society. Series B (Methodological), 211–252 (1964)Google Scholar
  4. 4.
    Boykov, Y., Funka-Lea, G.: Graph cuts and efficient N-D image segmentation. IJCV 70(2), 109–131 (2006)CrossRefGoogle Scholar
  5. 5.
    Brendel, W., Todorovic, S.: Video object segmentation by tracking regions. In: Proc. of IEEE ICCV 2009, pp. 833–840 (2009)Google Scholar
  6. 6.
    Carreira, M.: Gaussian mean-shift is an EM algorithm. IEEE PAMI 29(5), 767–776 (2007)CrossRefGoogle Scholar
  7. 7.
    Cheng, Y.: Mean shift, mode seeking, and clustering. IEEE PAMI 17(8), 790–799 (1995)CrossRefGoogle Scholar
  8. 8.
    Comaniciu, D., Meer, P.: Mean shift: A robust approach toward feature space analysis. IEEE PAMI 24(5), 603–619 (2002)CrossRefGoogle Scholar
  9. 9.
    Dorea, C., de Queiroz, R.: Depth map reconstruction using color-based region merging. In: Proc. of IEEE ICIP 2011, pp. 1977–1980 (2011)Google Scholar
  10. 10.
    Ellis, L., Zografos, V.: Online learning for fast segmentation of moving objects. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012, Part II. LNCS, vol. 7725, pp. 52–65. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  11. 11.
    Fitzmaurice, G.M., Lipsitz, S.R., Parzen, M.: Approximate median regression via the box-cox transformation. The American Statistician 61(3), 233–238 (2007)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Fukuchi, K., Miyazato, K., Kimura, A., Takagi, S., Yamato, J.: Saliency-based video segmentation with graph cuts and sequentially updated priors. In: Proc. of IEEE ICME 2009, pp. 638–641 (2009)Google Scholar
  13. 13.
    Fulkerson, B., Soatto, S.: Really quick shift: Image segmentation on a GPU. In: Kutulakos, K.N. (ed.) ECCV 2010 Workshops, Part II. LNCS, vol. 6554, pp. 350–358. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  14. 14.
    Galun, M., Sharon, E., Basri, R., Brandt, A.: Texture segmentation by multiscale aggregation of filter responses and shape elements. In: Proc. of IEEE CVPR 2003, pp. 716–723 (2003)Google Scholar
  15. 15.
    Grundmann, M., Kwatra, V., Han, M., Essa, I.: Efficient hierarchical graph based video segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2010)Google Scholar
  16. 16.
    Huang, Y., Liu, Q., Metaxas, D.: Video object segmentation by hypergraph cut. In: Proc. of IEEE CVPR 2009, pp. 1738–1745 (2009)Google Scholar
  17. 17.
    Li, Y., Sun, J., Shum, H.: Video object cut and paste. ACM SIGGRAPH 24(3), 595–600 (2005)CrossRefGoogle Scholar
  18. 18.
    Mignotte, M.: A de-texturing and spatially constrained k-means approach for image segmentation. Pattern Recognition Letters 32(2), 359–367 (2011)CrossRefGoogle Scholar
  19. 19.
    Moscheni, F., Bhattacharjee, S., Kunt, M.: Spatio-temporal segmentation based on region merging. IEEE PAMI 20(9), 897–915 (1998)CrossRefGoogle Scholar
  20. 20.
    Price, B., Morse, B., Cohen, S.: Livecut: Learning-based interactive video segmentation by evaluation of multiple propagated cues. In: Proc. of IEEE ICCV 2009, pp. 779–786 (2009)Google Scholar
  21. 21.
    Seber, G.: Multivariate observations. Wiley (1984)Google Scholar
  22. 22.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE PAMI 22(8), 888–905 (2000)CrossRefGoogle Scholar
  23. 23.
    Velmurugan, T., Santhanam, T.: Computational complexity between k-means and k-medoids clustering algorithms for normal and uniform distributions of data points. Journal of Computer Science 6(3), 363–368 (2010)CrossRefGoogle Scholar
  24. 24.
    Verleysen, C., De Vleeschouwer, C.: Recognition of sport players’ numbers using fast color segmentation. In: Proc. of the SPIE-IS&T Electronic Imaging (SPIE 2012), vol. 8305 (2012)Google Scholar
  25. 25.
    Wang, T., Guillemaut, J., Collomosse, J.: Multi-label propagation for coherent video segmentation and artistic stylization. In: Proc. of IEEE ICIP 2010, pp. 3005–3008 (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Cédric Verleysen
    • 1
  • Christophe De Vleeschouwer
    • 1
  1. 1.ICTEAM instituteUniversité catholique de LouvainLouvain-La-NeuveBelgique

Personalised recommendations