Variational Motion Segmentation with Level Sets

  • Thomas Brox
  • Andrés Bruhn
  • Joachim Weickert
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3951)


We suggest a variational method for the joint estimation of optic flow and the segmentation of the image into regions of similar motion. It makes use of the level set framework following the idea of motion competition, which is extended to non-parametric motion. Moreover, we automatically determine an appropriate initialization and the number of regions by means of recursive two-phase splits with higher order region models. The method is further extended to the spatiotemporal setting and the use of additional cues like the gray value or color for the segmentation. It need not fear a quantitative comparison to pure optic flow estimation techniques: For the popular Yosemite sequence with clouds we obtain the currently most accurate result. We further uncover a mistake in the ground truth. Coarsely correcting this, we get an average angular error below 1 degree.


Computer Vision Ground Truth Motion Estimation Active Contour Coarse Scale 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alvarez, L., Weickert, J., Sánchez, J.: Reliable estimation of dense optical flow fields with large displacements. International Journal of Computer Vision 39(1), 41–56 (2000)CrossRefzbMATHGoogle Scholar
  2. 2.
    Amiaz, T., Kiryati, N.: Dense discontinuous optical flow via contour-based segmentation. In: Proc. International Conference on Image Processing, Genoa, Italy, September 2005, vol. 3, pp. 1264–1267 (2005)Google Scholar
  3. 3.
    Amiaz, T., Kiryati, N.: Piecewise-smooth dense optical flow via level sets. Technical Report VIA-2005-6-2, Vision and Image Analysis Laboratory, School of Electrical Engineering, Tel Aviv University, Israel (June 2005)Google Scholar
  4. 4.
    Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. International Journal of Computer Vision 12(1), 43–77 (1994)CrossRefGoogle Scholar
  5. 5.
    Black, M.J., Anandan, P.: The robust estimation of multiple motions: parametric and piecewise smooth flow fields. Computer Vision and Image Understanding 63(1), 75–104 (1996)CrossRefGoogle Scholar
  6. 6.
    Bouthemy, P., François, E.: Motion segmentation and qualitative dynamic scene analysis from an image sequence. International Journal of Computer Vision 10(2), 157–182 (1993)CrossRefGoogle Scholar
  7. 7.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  8. 8.
    Brox, T., Weickert, J.: Level set based image segmentation with multiple regions. In: Rasmussen, C.E., Bülthoff, H.H., Schölkopf, B., Giese, M.A. (eds.) DAGM 2004. LNCS, vol. 3175, pp. 415–423. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  9. 9.
    Bruhn, A., Weickert, J.: Towards ultimate motion estimation: Combining highest accuracy with real-time performance. In: Proc. 10th International Conference on Computer Vision, October 2005, pp. 749–755. IEEE Computer Society Press, Beijing (2005)Google Scholar
  10. 10.
    Bruhn, A., Weickert, J., Schnörr, C.: Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods. Int. Journal of Computer Vision 61(3), 211–231 (2005)CrossRefGoogle Scholar
  11. 11.
    Caselles, V., Kimmel, R., Sapiro, G.: Geodesic active contours. International Journal of Computer Vision 22, 61–79 (1997)CrossRefzbMATHGoogle Scholar
  12. 12.
    Chan, T., Vese, L.: Active contours without edges. IEEE Transactions on Image Processing 10(2), 266–277 (2001)CrossRefzbMATHGoogle Scholar
  13. 13.
    Cremers, D., Soatto, S.: Motion competition: A variational framework for piecewise parametric motion segmentation. Internatonal Journal of Computer Vision 62(3), 249–265 (2005)CrossRefGoogle Scholar
  14. 14.
    Dervieux, A., Thomasset, F.: A finite element method for the simulation of Rayleigh–Taylor instability. In: Rautman, R. (ed.) Approximation Methods for Navier–Stokes Problems. Lecture Notes in Mathematics, vol. 771, pp. 145–158. Springer, Berlin (1979)CrossRefGoogle Scholar
  15. 15.
    Horn, B., Schunck, B.: Determining optical flow. Artificial Intelligence 17, 185–203 (1981)CrossRefGoogle Scholar
  16. 16.
    Kass, M., Witkin, A., Terzopoulos, D.: Snakes: Active contour models. International Journal of Computer Vision 1, 321–331 (1988)CrossRefzbMATHGoogle Scholar
  17. 17.
    Kichenassamy, S., Kumar, A., Olver, P., Tannenbaum, A., Yezzi, A.: Conformal curvature flows: from phase transitions to active vision. Archive for Rational Mechanics and Analysis 134, 275–301 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Mémin, E., Pérez, P.: A multigrid approach for hierarchical motion estimation. In: Proc. 6th International Conference on Computer Vision, Bombay, India, pp. 933–938 (1998)Google Scholar
  19. 19.
    Mémin, E., Pérez, P.: Hierarchical estimation and segmentation of dense motion fields. International Journal of Computer Vision 46(2), 129–155 (2002)CrossRefzbMATHGoogle Scholar
  20. 20.
    Mumford, D., Shah, J.: Boundary detection by minimizing functionals, I. In: Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, June 1985, pp. 22–26. IEEE Computer Society Press, Los Alamitos (1985)Google Scholar
  21. 21.
    Nagel, H.-H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence 8, 565–593 (1986)CrossRefGoogle Scholar
  22. 22.
    Osher, S., Sethian, J.A.: Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton–Jacobi formulations. Journal of Computational Physics 79, 12–49 (1988)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Papenberg, N., Bruhn, A., Brox, T., Didas, S., Weickert, J.: Highly accurate optic flow computation with theoretically justified warping. Technical Report 124, Dept. of Mathematics, Saarland University, Saarbrücken, Germany (January 2005); International Journal of Computer Vision (to appear)Google Scholar
  24. 24.
    Paragios, N., Deriche, R.: Geodesic active regions: A new paradigm to deal with frame partition problems in computer vision. Journal of Visual Communication and Image Representation 13(1/2), 249–268 (2002)CrossRefGoogle Scholar
  25. 25.
    Paragios, N., Deriche, R.: Geodesic active regions and level set methods for motion estimation and tracking. Computer Vision and Image Understanding 97(3), 259–282 (2005)CrossRefGoogle Scholar
  26. 26.
    Potter, J.L.: Velocity as a cue to segmentation. IEEE Transactions on Systems, Man and Cybernetics 5, 390–394 (1975)CrossRefGoogle Scholar
  27. 27.
    Schnörr, C.: Determining optical flow for irregular domains by minimizing quadratic functionals of a certain class. International Journal of Computer Vision 6(1), 25–38 (1991)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Thompson, W.B.: Combining motion and contrast for segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 2(6), 543–549 (1980)CrossRefGoogle Scholar
  29. 29.
    Zhu, S.-C., Yuille, A.: Region competition: unifying snakes, region growing, and Bayes/MDL for multiband image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 18(9), 884–900 (1996)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Thomas Brox
    • 1
  • Andrés Bruhn
    • 2
  • Joachim Weickert
    • 2
  1. 1.CVPR Group, Department of Computer ScienceUniversity of BonnBonnGermany
  2. 2.Mathematical Image Analysis Group, Faculty of Mathematics and Computer ScienceSaarland UniversitySaarbrückenGermany

Personalised recommendations