Semi-automatic Motion Segmentation with Motion Layer Mosaics

  • Matthieu Fradet
  • Patrick Pérez
  • Philippe Robert
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5304)


A new method for motion segmentation based on reference motion layer mosaics is presented. We assume that the scene is composed of a set of layers whose motion is well described by parametric models. This usual assumption is compatible with the notion of motion layer mosaic, which allows a compact representation of the sequence with a small number of mosaics only. We segment the sequence using a reduced number of distant image-to-mosaic comparisons instead of a larger number of close image-to-image comparisons. Apart from computational advantage, another interest lies in the fact that motions estimated between distant images are more likely to be different from one region to another than when estimated between consecutive images. This helps the segmentation process. The segmentation is obtained by graph cut minimization of a cost function which includes an original image-to-mosaic data term. At the end of the segmentation process, it may happen that the obtained boundaries are not precisely the expected ones. Often the user has no other possibility than modifying manually every segmentation one after another or than starting over all again the process with different parameters. We propose an original easy way for the user to manually correct the possible errors on the mosaics themselves. These corrections are then propagated to all the images of the corresponding video interval thanks to a second segmentation pass. Experimental results demonstrate the potential of our approach.


Segmentation Process Foreground Object Minimum Description Length Consecutive Image Distant Image 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Wang, J., Adelson, E.: Representing moving images with layers. IEEE Trans. on Image Processing 3(5), 625–638 (1994)CrossRefGoogle Scholar
  2. 2.
    Ayer, S., Sawhney, H.S.: Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding. In: ICCV, pp. 777–784 (1995)Google Scholar
  3. 3.
    Black, M.J., Anandan, P.: The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding 63(1), 75–104 (1996)CrossRefGoogle Scholar
  4. 4.
    Mémin, E., Pérez, P.: Hierarchical estimation and segmentation of dense motion fields. IJCV 46(2), 129–155 (2002)CrossRefzbMATHGoogle Scholar
  5. 5.
    Bouthemy, P., François, E.: Motion segmentation and qualitative dynamic scene analysis from an image sequence. IJCV 10(2), 157–182 (1993)CrossRefGoogle Scholar
  6. 6.
    Cremers, D., Soatto, S.: Motion competition: A variational approach to piecewise parametric motion segmentation. IJCV 62(3), 249–265 (2005)CrossRefGoogle Scholar
  7. 7.
    Odobez, J.M., Bouthemy, P.: Direct incremental model-based image motion segmentation for video analysis. Signal Processing 66(2), 143–155 (1998)CrossRefzbMATHGoogle Scholar
  8. 8.
    Dupont, R., Paragios, N., Keriven, R., Fuchs, P.: Extraction of layers of similar motion through combinatorial techniques. In: Rangarajan, A., Vemuri, B.C., Yuille, A.L. (eds.) EMMCVPR 2005. LNCS, vol. 3757, pp. 220–234. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  9. 9.
    Xiao, J., Shah, M.: Motion layer extraction in the presence of occlusion using graph cuts. PAMI 27(10), 1644–1659 (2005)CrossRefGoogle Scholar
  10. 10.
    Dupont, R., Juan, O., Keriven, R.: Robust segmentation of hidden layers in video sequences. Technical report, CERTIS - ENPC (January 2006)Google Scholar
  11. 11.
    Min, C., Yu, Q., Medioni, G.: Multi-layer mosaics in the presence of motion and depth effects. In: ICPR, pp. 992–995 (2006)Google Scholar
  12. 12.
    Irani, M., Anandan, P., Bergen, J., Kumar, R., Hsu, S.: Efficient representations of video sequences and their applications. Signal Processing: Image Communication 8(4), 327–351 (1996)Google Scholar
  13. 13.
    Shum, H.Y., Szeliski, R.: Construction and refinement of panoramic mosaics with global and local alignment. In: ICCV (1998)Google Scholar
  14. 14.
    Boykov, Y., Jolly, M.P.: Interactive graph cuts for optimal boundary & region segmentation of objects in n-d images. In: ICCV (2001)Google Scholar
  15. 15.
    Rother, C., Kolmogorov, V., Blake, A.: “Grabcut”: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23(3), 309–314 (2004)CrossRefGoogle Scholar
  16. 16.
    Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. PAMI 26(9), 1124–1137 (2004)CrossRefzbMATHGoogle Scholar
  17. 17.
    Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? PAMI 26(2), 147–159 (2004)CrossRefzbMATHGoogle Scholar
  18. 18.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. PAMI 23(11), 1222–1239 (2001)CrossRefGoogle Scholar
  19. 19.
    Brown, M., Lowe, D.: Recognising panoramas. In: ICCV, pp. 1218–1225 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Matthieu Fradet
    • 1
    • 2
  • Patrick Pérez
    • 2
  • Philippe Robert
    • 1
  1. 1.Thomson Corporate ResearchRennesFrance
  2. 2.INRIA, Rennes-Bretagne AtlantiqueFrance

Personalised recommendations