Dense and Deformable Motion Segmentation for Wide Baseline Images

  • Juho Kannala
  • Esa Rahtu
  • Sami S. Brandt
  • Janne Heikkilä
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5575)


In this paper we describe a dense motion segmentation method for wide baseline image pairs. Unlike many previous methods our approach is able to deal with deforming motions and large illumination changes by using a bottom-up segmentation strategy. The method starts from a sparse set of seed matches between the two images and then proceeds to quasi-dense matching which expands the initial seed regions by using local propagation. Then, the quasi-dense matches are grouped into coherently moving segments by using local bending energy as the grouping criterion. The resulting segments are used to initialize the motion layers for the final dense segmentation stage, where the geometric and photometric transformations of the layers are iteratively refined together with the segmentation, which is based on graph cuts. Our approach provides a wider range of applicability than the previous approaches which typically require a rigid planar motion model or motion with small disparity. In addition, we model the photometric transformations in a spatially varying manner. Our experiments demonstrate the performance of the method with real images involving deforming motion and large changes in viewpoint, scale and illumination.


Color Channel Geometric Transformation Motion Segmentation Seed Match Motion Layer 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Wills, J., Agarwal, S., Belongie, S.: A feature-based approach for dense segmentation and estimation of large disparity motion. IJCV 68, 125–143 (2006)CrossRefGoogle Scholar
  2. 2.
    Wang, J.Y.A., Adelson, E.H.: Representing moving images with layers. IEEE Transactions on Image Processing 3(5), 625–638 (1994)CrossRefGoogle Scholar
  3. 3.
    Ferrari, V., Tuytelaars, T., Van Gool, L.: Simultaneous object recognition and segmentation from single or multiple model views. IJCV 67, 159–188 (2006)CrossRefGoogle Scholar
  4. 4.
    Weiss, Y.: Smoothness in layers: Motion segmentation using nonparametric mixture estimation. In: CVPR (1997)Google Scholar
  5. 5.
    Torr, P.H.S., Szeliski, R., Anandan, P.: An integrated bayesian approach to layer extraction from image sequences. TPAMI 23(3), 297–303 (2001)CrossRefGoogle Scholar
  6. 6.
    Xiao, J., Shah, M.: Motion layer extraction in the presence of occlusion using graph cuts. TPAMI 27, 1644–1659 (2005)CrossRefGoogle Scholar
  7. 7.
    Simon, I., Seitz, S.M.: A probabilistic model for object recognition, segmentation, and non-rigid correspondence. In: CVPR (2007)Google Scholar
  8. 8.
    Kannala, J., Rahtu, E., Brandt, S.S., Heikkilä, J.: Object recognition and segmentation by non-rigid quasi-dense matching. In: CVPR (2008)Google Scholar
  9. 9.
    Kumar, M.P., Torr, P.H.S., Zisserman, A.: Learning layered motion segmentations of video. IJCV 76, 301–319 (2008)CrossRefGoogle Scholar
  10. 10.
    Jackson, J.D., Yezzi, A.J., Soatto, S.: Dynamic shape and appearance modeling via moving and deforming layers. IJCV 79, 71–84 (2008)CrossRefGoogle Scholar
  11. 11.
    Lowe, D.: Distinctive image features from scale invariant keypoints. IJCV 60, 91–110 (2004)CrossRefGoogle Scholar
  12. 12.
    Donato, G., Belongie, S.: Approximate thin plate spline mappings. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 21–31. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  13. 13.
    Vedaldi, A., Soatto, S.: Local features, all grown up. In: CVPR (2006)Google Scholar
  14. 14.
    Čech, J., Matas, J., Perd’och, M.: Efficient sequential correspondence selection by cosegmentation. In: CVPR (2008)Google Scholar
  15. 15.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. TPAMI 23(11), 1222–1239 (2001)CrossRefGoogle Scholar
  16. 16.
    Hansen, P.C.: Rank-Deficient and Discrete Ill-Posed Problems. SIAM, Philadelphia (1998)CrossRefGoogle Scholar
  17. 17.
    Horn, B.K.P., Schunk, B.G.: Determining optical flow. Artificial Intelligence (1981)Google Scholar
  18. 18.
    Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. TPAMI 26(9), 1124–1137 (2004)CrossRefzbMATHGoogle Scholar
  19. 19.
    Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? TPAMI 26(2), 147–159 (2004)CrossRefzbMATHGoogle Scholar
  20. 20.
    Bagon, S.: Matlab wrapper for graph cut (2006),

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Juho Kannala
    • 1
  • Esa Rahtu
    • 1
  • Sami S. Brandt
    • 1
  • Janne Heikkilä
    • 1
  1. 1.Machine Vision GroupUniversity of OuluFinland

Personalised recommendations