Graph-Based Hierarchical Video Cosegmentation

  • Franciele Rodrigues
  • Pedro Leal
  • Yukiko Kenmochi
  • Jean Cousty
  • Laurent Najman
  • Silvio Guimarães
  • Zenilton PatrocínioJr.Email author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10484)


The goal of video cosegmentation is to jointly extract the common foreground regions and/or objects from a set of videos. In this paper, we present an approach for video cosegmentation that uses graph-based hierarchical clustering as its basic component. Actually, in this work, video cosegmentation problem is transformed into a graph-based clustering problem in which a cluster represents a set of similar supervoxels belonging to the analyzed videos. Our graph-based Hierarchical Video Cosegmentation method (or HVC) is divided in two main parts: (i) supervoxel generation and (ii) supervoxel correlation. The former explores only intra-video similarities, while the latter seeks to determine relationships between supervoxels belonging to the same video or to distinct videos. Experimental results provide comparison between HVC and other methods from the literature on two well known datasets, showing that HVC is a competitive one. HVC outperforms on average all the compared methods for one dataset; and it was the second best for the other one. Actually, HVC is able to produce good quality results without being too computational expensive, taking less than 50% of the time spent by any other approach.


Graph-based segmentation Video cosegmentation Hierarchical clustering 


  1. 1.
    Alexe, B., Deselaers, T., Ferrari, V.: Measuring the objectness of image windows. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2189–2202 (2012)CrossRefGoogle Scholar
  2. 2.
    Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33, 898–916 (2011)CrossRefGoogle Scholar
  3. 3.
    Belo, L., Caetano, C., Patrocínio Jr., Z.K.G., Guimarães, S.J.F.: Summarizing video sequence using a graph-based hierarchical approach. Neurocomputing 173(3), 1001–1016 (2016)CrossRefGoogle Scholar
  4. 4.
    Chiu, W.C., Fritz, M.: Multi-class video co-segmentation with a generative multi-video model. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 321–328 (2013)Google Scholar
  5. 5.
    Endres, I., Hoiem, D.: Category independent object proposals. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6315, pp. 575–588. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-15555-0_42 CrossRefGoogle Scholar
  6. 6.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. Int. J. Comput. Vis. 59(2), 167–181 (2004)CrossRefGoogle Scholar
  7. 7.
    Fu, H., Xu, D., Zhang, B., Lin, S., Ward, R.K.: Object-based multiple foreground video co-segmentation via multi-state selection graph. IEEE Trans. Image Process. 24(11), 3415–3424 (2015)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Fu, H., Xu, D., Zhang, B., Lin, S.: Object-based multiple foreground video co-segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3166–3173 (2014)Google Scholar
  9. 9.
    Guigues, L., Cocquerez, J.P., Le Men, H.: Scale-sets image analysis. Int. J. Comput. Vis. 68(3), 289–317 (2006)CrossRefGoogle Scholar
  10. 10.
    Guimarães, S.J.F., Cousty, J., Kenmochi, Y., Najman, L.: A hierarchical image segmentation algorithm based on an observation scale. In: Gimel’farb, G., et al. (eds.) SSPR/SPR 2012. LNCS, vol. 7626, pp. 116–125. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-34166-3_13 CrossRefGoogle Scholar
  11. 11.
    Rubio, J.C., Serrat, J., López, A.: Video co-segmentation. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012. LNCS, vol. 7725, pp. 13–24. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-37444-9_2 CrossRefGoogle Scholar
  12. 12.
    Souza, K.J.F., Araújo, A.A., Guimarães, S.J.F., Patrocínio Jr., Z.K.G., Cord, M.: Streaming graph-based hierarchical video segmentation by a simple label propagation. In: 26th Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 119–125 (2015)Google Scholar
  13. 13.
    Souza, K.J.F., Araújo, A.A., Patrocínio Jr., Z.K.G., Cousty, J., Najman, L., Kenmochi, Y., Guimarães, S.J.F.: Hierarchical video segmentation using an observation scale. In: 25th Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 320–327 (2013)Google Scholar
  14. 14.
    Souza, K.J.F., Araújo Jr., A.A., Patrocínio, Z.K.G., Guimarães, S.J.F.: Graph-based hierarchical video segmentation based on a simple dissimilarity measure. Pattern Recogn. Lett. 47, 85–92 (2014)CrossRefGoogle Scholar
  15. 15.
    Zhang, D., Javed, O., Shah, M.: Video object co-segmentation by regulated maximum weight cliques. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8695, pp. 551–566. Springer, Cham (2014). doi: 10.1007/978-3-319-10584-0_36 Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Franciele Rodrigues
    • 1
  • Pedro Leal
    • 1
  • Yukiko Kenmochi
    • 2
  • Jean Cousty
    • 2
  • Laurent Najman
    • 2
  • Silvio Guimarães
    • 1
    • 2
  • Zenilton PatrocínioJr.
    • 1
    Email author
  1. 1.PUC Minas - ICEI - DCC - VIPLABBelo HorizonteBrazil
  2. 2.Université Paris-Est, LIGM, ESIEE Paris - CNRSChamps-sur-MarneFrance

Personalised recommendations