Segmentation-Based Motion with Occlusions Using Graph-Cut Optimization

  • Michael Bleyer
  • Christoph Rhemann
  • Margrit Gelautz
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4174)


We propose to tackle the optical flow problem by a combination of two recent advances in the computation of dense correspondences, namely the incorporation of image segmentation and robust global optimization via graph-cuts. In the first step, each segment (extracted by colour segmentation) is assigned to an affine motion model from a set of sparse correspondences. Using a layered model, we then identify those motion models that represent the dominant image motion. This layer extraction task is accomplished by optimizing a simple energy function that operates in the domain of segments via graph-cuts. We then estimate the spatial extent that is covered by each layer and identify occlusions. Since treatment of occlusions is hardly possible when using entire segments as matching primitives, we propose to use the pixel level in addition. We therefore define an energy function that measures the quality of an assignment of segments and pixels to layers. This energy function is then extended to work on multiple input frames and minimized via graph-cuts. In the experimental results, we show that our method produces good-quality results, especially in regions of low texture and close to motion boundaries, which are challenging tasks in optical flow computation.


Energy Function Motion Model Segment Level Pixel Level Motion Segmentation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Horn, B., Schunck, B.: Determining optical flow. Artificial Intelligence 17, 185–203 (1981)CrossRefGoogle Scholar
  2. 2.
    Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo. In: IJCAI, pp. 121–130 (1981)Google Scholar
  3. 3.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  4. 4.
    Zitnick, C., Kanade, T.: A cooperative algorithm for stereo matching and occlusion detection. TPAMI 22(7), 675–684 (2000)CrossRefGoogle Scholar
  5. 5.
    Black, M., Jepson, A.: Estimating optical flow in segmented images using variable-order parametric models with local deformations. TPAMI 18(10), 972–986 (1996)CrossRefGoogle Scholar
  6. 6.
    Tao, H., Sawhney, H., Kumar, R.: A global matching framework for stereo computation. In: ICCV, pp. 532–539 (2001)Google Scholar
  7. 7.
    Hong, L., Chen, G.: Segment-based stereo matching using graph cuts. In: CVPR, vol. 1, pp. 74–81 (2004)Google Scholar
  8. 8.
    Ke, Q., Kanade, T.: A subspace approach to layer extraction. In: CVPR, pp. 255–262 (2001)Google Scholar
  9. 9.
    Altunbasak, Y., Eren, P., Tekalp, A.: Region-based parametric motion segmentation using color information. GMIP 60(1), 13–23 (1998)Google Scholar
  10. 10.
    Ayer, S., Sawhney, H.: Layered representation of motion video using robust maximum-likelihood estimation of mixture models and mdl encoding. In: ICCV, pp. 777–784 (1995)Google Scholar
  11. 11.
    Willis, J., Agarwal, S., Belongie, S.: What went where. In: CVPR, pp. 37–44 (2003)Google Scholar
  12. 12.
    Xiao, J., Shah, M.: Motion layer extraction in the presence of occlusion using graph cuts. TPAMI 27(10), 1644–1659 (2005)CrossRefGoogle Scholar
  13. 13.
    Christoudias, C., Georgescu, B., Meer, P.: Synergism in low-level vision. In: ICPR, vol. 4, pp. 150–155 (2002)Google Scholar
  14. 14.
    Shi, J., Tomasi, C.: Good features to track. In: CVPR, pp. 593–600 (1994)Google Scholar
  15. 15.
    Wang, J., Adelson, E.: Representing moving images with layers. Transactions on Image Processing 3(5), 625–638 (1994)CrossRefGoogle Scholar
  16. 16.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. TPAMI 23(11), 1222–1239 (2001)CrossRefGoogle Scholar
  17. 17.
    Bleyer, M.: Segmentation-based Stereo and Motion with Occlusions. PhD thesis, Vienna University of Technology (2006)Google Scholar
  18. 18.
    Xiao, J., Shah, M.: Accurate motion layer segmentation and matting. In: CVPR, vol. 2, pp. 698–703 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Michael Bleyer
    • 1
  • Christoph Rhemann
    • 1
  • Margrit Gelautz
    • 1
  1. 1.Institute for Software Technology and Interactive SystemsVienna University of TechnologyViennaAustria

Personalised recommendations