Superpixels and Supervoxels in an Energy Optimization Framework

  • Olga Veksler
  • Yuri Boykov
  • Paria Mehrani
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6315)


Many methods for object recognition, segmentation, etc., rely on a tessellation of an image into “superpixels”. A superpixel is an image patch which is better aligned with intensity edges than a rectangular patch. Superpixels can be extracted with any segmentation algorithm, however, most of them produce highly irregular superpixels, with widely varying sizes and shapes. A more regular space tessellation may be desired. We formulate the superpixel partitioning problem in an energy minimization framework, and optimize with graph cuts. Our energy function explicitly encourages regular superpixels. We explore variations of the basic energy, which allow a trade-off between a less regular tessellation but more accurate boundaries or better efficiency. Our advantage over previous work is computational efficiency, principled optimization, and applicability to 3D “supervoxel” segmentation. We achieve high boundary recall on images and spatial coherence on video. We also show that compact superpixels improve accuracy on a simple application of salient object segmentation.


Patch Size Salient Object Variable Patch Superpixel Segmentation Ground Truth Segment 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

978-3-642-15555-0_16_MOESM1_ESM.avi (1.7 mb)
Electronic Supplementary Material (1,729 KB)
978-3-642-15555-0_16_MOESM2_ESM.avi (1.1 mb)
Electronic Supplementary Material (1,111 KB)
978-3-642-15555-0_16_MOESM3_ESM.avi (997 kb)
Electronic Supplementary Material (997 KB)
978-3-642-15555-0_16_MOESM4_ESM.avi (605 kb)
Electronic Supplementary Material (605 KB)
978-3-642-15555-0_16_MOESM5_ESM.avi (1.1 mb)
Electronic Supplementary Material (1,099 KB)
978-3-642-15555-0_16_MOESM6_ESM.avi (683 kb)
Electronic Supplementary Material (683 KB)
978-3-642-15555-0_16_MOESM7_ESM.avi (284 kb)
Electronic Supplementary Material (285 KB)
978-3-642-15555-0_16_MOESM8_ESM.avi (331 kb)
Electronic Supplementary Material (331 KB)


  1. 1.
    Ren, X., Malik, J.: Learning a classification model for segmentation. In: ICCV, vol. 1, pp. 10–17 (2003)Google Scholar
  2. 2.
    Mori, G., Ren, X., Efros, A.A., Malik, J.: Recovering human body configurations: combining segmentation and recognition. In: CVPR, vol. 2, pp. 326–333 (2004)Google Scholar
  3. 3.
    Hoiem, D., Efros, A., Hebert, M.: Geometric context from a single image. In: ICCV, pp. 654 – 661 (2005)Google Scholar
  4. 4.
    Mori, G.: Guiding model search using segmentation. In: ICCV, pp. 1417–1423 (2005)Google Scholar
  5. 5.
    He, X., Zemel, R.S., Ray, D.: Learning and incorporating top-down cues in image segmentation. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 338–351. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  6. 6.
    Malisiewicz, T., Efros, A.A.: Improving spatial support for objects via multiple segmentations. In: BMVC (2007)Google Scholar
  7. 7.
    Pantofaru, C., Schmid, C., Hebert, M.: Object recognition by integrating multiple image segmentations. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 481–494. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  8. 8.
    Fulkerson, B., Vedaldi, A., Soatto, S.: Class segmentation and object localization with superpixel neighborhoods. In: ICCV (2009)Google Scholar
  9. 9.
    van den Hengel, A., Dick, A., Thormählen, T., Ward, B., Torr, P.H.S.: Videotrace: rapid interactive scene modelling from video. ACM SIGGRAPH 26, 86 (2007)CrossRefGoogle Scholar
  10. 10.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: CVPR, vol. 1, pp. 511–518 (2001)Google Scholar
  11. 11.
    Comaniciu, D., Meer, P., Member, S.: Mean shift: A robust approach toward feature space analysis. TPAMI 24, 603–619 (2002)Google Scholar
  12. 12.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. IJCV 59, 167–181 (2004)CrossRefGoogle Scholar
  13. 13.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. TPAMI 22, 888–905 (1997)Google Scholar
  14. 14.
    Levinshtein, A., Stere, A., Kutulakos, K., Fleet, D., Dickinson, S., Siddiqi, K.: Fast superpixels using geometric flows. TPAMI 31, 2290–2297 (2009)Google Scholar
  15. 15.
    Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV, vol. 2, pp. 416–423 (2001)Google Scholar
  16. 16.
    Kwatra, V., Schödl, A., Essa, I., Turk, G., Bobick, A.: Graphcut textures: Image and video synthesis using graph cuts. ACM SIGGRAPH 22, 277–286 (2003)CrossRefGoogle Scholar
  17. 17.
    Osher, S., Sethian, J.A.: Fronts propagating with curvature-dependent speed: Algorithms based on hamilton-Jacobi formulations. Journal of Computational Physics 79, 12–49 (1988)zbMATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Boykov, Y., Veksler, O., Zabih, R.: Efficient approximate energy minimization via graph cuts. TPAMI 21, 1222–1239 (2001)Google Scholar
  19. 19.
    Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M., Rother, C.: A comparative study of energy minimization methods for markov random fields with smoothness-based priors. TPAMI 30, 1068–1080 (2008)Google Scholar
  20. 20.
    Moore, A., Prince, S., Warrell, J., Mohammed, U., Jones, G.: Superpixel lattices. In: CVPR (2008)Google Scholar
  21. 21.
    Moore, A., Prince, S.J., Warrel, J.: Lattice cut - constructing superpixels using layer constraints. In: CVPR (2010)Google Scholar
  22. 22.
    Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. TPAMI 24, 137–148 (2004)Google Scholar
  23. 23.
    Boykov, Y., Kolmogorov, V.: Computing geodesics and minimal surfaces via graph cuts. In: ICCV, pp. 26–33 (2003)Google Scholar
  24. 24.
    Boykov, Y., Funka Lea, G.: Graph cuts and efficient n-d image segmentation. IJCV 70, 109–131 (2006)CrossRefGoogle Scholar
  25. 25.
    Truong, B.T., Venkatesh, S.: Video abstraction: A systematic review and classification. ACM SIGGRAPH 3, 3 (2007)Google Scholar
  26. 26.
    Wang, J., Xu, Y., Shum, H., Cohen, M.F.: Video tooning. ACM SIGGRAPH, 574–583 (2004)Google Scholar
  27. 27.
    Martin, D., Fowlkes, C., Malik, J.: Learning to find brightness and texture boundaries in natural images. NIPS (2002)Google Scholar
  28. 28.
    Liu, T., Sun, J., Zheng, N.N., Tang, X., Shum, H.Y.: Learning to detect a salient object (2007)Google Scholar
  29. 29.
    Friedman, J., Hastie, T., Tibshirani, R.: Additive logistic regression: A statistical view of boosting. The Annals of Statistics 38, 337–374 (2000)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Olga Veksler
    • 1
  • Yuri Boykov
    • 1
  • Paria Mehrani
    • 1
  1. 1.Computer Science DepartmentUniversity of Western OntarioLondonCanada

Personalised recommendations