Abstract
We present a novel variational approach for segmenting the image plane into a set of regions of parametric motion on the basis of two consecutive frames from an image sequence. Our model is based on a conditional probability for the spatio-temporal image gradient, given a particular velocity model, and on a geometric prior on the estimated motion field favoring motion boundaries of minimal length.
Exploiting the Bayesian framework, we derive a cost functional which depends on parametric motion models for each of a set of regions and on the boundary separating these regions. The resulting functional can be interpreted as an extension of the Mumford-Shah functional from intensity segmentation to motion segmentation. In contrast to most alternative approaches, the problems of segmentation and motion estimation are jointly solved by continuous minimization of a single functional. Minimizing this functional with respect to its dynamic variables results in an eigenvalue problem for the motion parameters and in a gradient descent evolution for the motion discontinuity set.
We propose two different representations of this motion boundary: an explicit spline-based implementation which can be applied to the motion-based tracking of a single moving object, and an implicit multiphase level set implementation which allows for the segmentation of an arbitrary number of multiply connected moving objects.
Numerical results both for simulated ground truth experiments and for real-world sequences demonstrate the capacity of our approach to segment objects based exclusively on their relative motion.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Ayer, S. and Sawhney, H.S. 1995. Layered representation of motion video using robust maximum likelihood estimation of mixture models and MDL encoding. In Proc. of the Int. Conf. on Comp. Vis., Boston, USA, pp. 777–784.
BigBigünn, J., Granlund, G.H., and Wiklund, J. 1991. Multidimensional orientation estimation with applications to texture analysis and optical flow. IEEE PAMI, 13(8):775–790.
Black, M.J. 1994. Recursive non-linear estimation of discontinuous flow fields. In Proc. of the Europ. Conf. on Comp. Vis., J.O. Eklundh, (Ed.), vol. 800 of LNCS, Springer-Verlag, pp. 138–145.
Black, M.J. and Anandan, P. 1996. The robust estimation of multiple motions: Parametric and piecewise–smooth flow fields. Comp. Vis. Graph. Image Proc.: IU, 63(1):75–104.
Bouthemy, P. and Francois, E. 1993. Motion segmentation and qualitative dynamic scene analysis from an image sequence. Int. J. of Computer Vision, 10(2):157–182.
Brox, T., Bruhn, A., Papenberg, N., and Weickert, J. 2004. High accuracy optical flow estimation based on a theory for warping. In European Conf. on Computer Vision, T. Pajdla and V. Hlavac (Eds.), vol. 3024 of LNCS, Springer, Prague, pp. 25–36.
Caselles, V. and Coll, B. 1996. Snakes in movement. SIAM J. Numer. Anal., 33:2445–2456.
Caselles, V., Kimmel, R., and Sapiro, G. 1995. Geodesic active contours. In Proc. IEEE Intl. Conf. on Comp. Vis., Boston, USA, pp. 694–699.
Chan, T. and Vese, L. 2001. Active contours without edges. IEEE Trans. Image Processing, 10(2):266–277.
Cremers, D. 2003a. A multiphase level set framework for variational motion segmentation. In Int. Conf. on Scale Space Theories in Computer Vision, L. Griffith (Ed.), vol. 2695 of LNCS, Isle of Skye, Springer, pp. 599–614.
Cremers, D. 2003b. A variational framework for image segmentation combining motion estimation and shape regularization. In IEEE Conf. on Comp. Vis. and Patt. Recog., C. Dyer and P. Perona (Eds.), vol. 1, pp. 53–58.
Cremers, D. and SchnSchnörrrr, C. 2003. Statistical shape knowledge in variational motion segmentation. Image and Vision Computing, 21(1):77–86.
Cremers, D. and Soatto, S. 2003. Variational space-time motion segmentation. In IEEE Int. Conf. on Computer Vision, B. Triggs and A. Zisserman (Eds.), Nice, vol. 2, pp. 886–892.
Cremers, D., Tischhäuser, F., Weickert, J., and Schnörr, C. 2002. Diffusion Snakes: Introducing statistical shape knowledge into the Mumford–Shah functional. Int. J. of Computer Vision, 50(3):295–313.
Cremers, D. and Yuille, A.L. 2003. A generative model based approach to motion segmentation. In Pattern Recognition, B. Michaelis and G. Krell (Eds.), vol. 2781 of LNCS, Springer, Magdeburg, pp. 313–320.
Delingette, H. and Montagnat, J. 2000. New algorithms for controlling active contours shape and topology. In Proc. of the Europ. Conf. on Comp. Vis., D. Vernon (Ed.), vol. 1843 of LNCS, Springer, pp. 381–395.
Farnebäck, G. 1999. Spatial Domain Methods for Orientation and Velocity Estimation. PhD thesis, Dept. of Electrical Engineering, Linköpings universitet.
Farnebäck, G. 2001. Very high accuracy velocity estimation using orientation tensors, parametric motion, and segmentation of the motion field. In ICCV, vol. 1, pp. 171–177.
Gomes, J. and Faugeras, O.D. 2000. Level sets and distance functions. In Proc. of the Europ. Conf. on Comp. Vis., D. Vernon (Ed.), vol. 1842 of LNCS, Dublin, Ireland, Springer, pp. 588–602.
Heitz, F. and Bouthemy, P. 1993. Multimodal estimation of discontinuous optical flow using markov random fields. IEEE PAMI, 15(12):1217–1232.
Horn, B.K.P. and Schunck, B.G. 1981. Determining optical flow. A.I., 17:185–203.
Jehan-Besson, S., Barlaud, M., and Aubert, G. 2003. DREAM2S: Deformable regions driven by an eulerian accurate minimization method for image and video segmentation. Int. J. of Computer Vision, 53(1):45–70.
Jepson, A. and Black, M.J. 1993. Mixture models for optic flow computation. In Proc. IEEE Conf. on Comp. Vision Patt. Recog., New York, pp. 760–761.
Kass, M., Witkin, A., and Terzopoulos, D. 1988. Snakes: Active contour models. Int. J. of Computer Vision, 1(4):321–331.
Kichenassamy, S., Kumar, A., Olver, P.J., Tannenbaum, A., and Yezzi, A.J. 1995. Gradient flows and geometric active contour models. In Proc. IEEE Intl. Conf. on Comp. Vis., Boston, USA, pp. 810–815.
Konrad, J. and Dubois, E. 1992. Bayesian estimation of motion vector fields. IEEE PAMI, 14(9):910–927.
Kornprobst, P., Deriche, R., and Aubert, G. 1999. Image sequence analysis via partial differential equations. J. Math. Im. Vis., 11(1):5–26.
Lie, J., Lysaker, M., and Tai, X.-C. 2003. A variant of the level set method and applications to image segmentation. Technical Report 03–50, Computational Applied Mathematics, UCLA, Los Angeles.
Lucas, B.D. and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In Proc.7th International Joint Conference on Artificial Intelligence, Vancouver, pp. 674–679.
Malladi, R., Sethian, J.A., and Vemuri, B.C. 1995. Shape modeling with front propagation: A level set approach. IEEE PAMI, 17(2):158–175.
McInerney, T. and Terzopoulos, D. 1995. Topologically adaptable snakes. In Proc. 5th Int. Conf. on Computer Vision, IEEE Comp. Soc. Press: Los Alamitos, California, pp. 840–845.
Memin, E. and Perez, P. 1998. Dense estimation and object-based segmentation of the optical flow with robust techniques. IEEE Trans. on Im. Proc., 7(5):703–719.
Memin, E. and Perez, P. 2002. Hierarchical estimation and segmentation of dense motion fields. Int. J. of Computer Vision, 46(2):129–155.
Morel, J.-M. and Solimini, S. 1995. Variational Methods in Image Segmentation. Birkhäuser: Boston.
Mumford, D. and Shah, J. 1989. Optimal approximations by piecewise smooth functions and associated variational problems. Comm. Pure Appl. Math., 42:577–685.
Nagel, H.H. and Enkelmann, W. 1986. An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE PAMI, 8(5):565–593.
Nestares, O., Fleet, D.J., and Heeger, D.J. 2000. Likelihood functions and confidence bounds for total-least-squares problems. In Proc. Conf. Computer Vis. and Pattern Recog., vol. 1, Hilton Head Island, SC, pp. 760–767.
Odobez, J.-M. and Bouthemy, P. 1995. Robust multiresolution estimation of parametric motion models. J. of Visual Commun. and Image Repr., 6(4):348–365.
Odobez, J.-M. and Bouthemy, P. 1998. Direct incremental model-based image motion segmentation for video analysis. Signal Proc., 66:143–155.
Osher, S.J. and Sethian, J.A. 1988. Fronts propagation with curvature dependent speed: Algorithms based on Hamilton–Jacobi formulations. J. of Comp. Phys., 79:12–49.
Paragios, N. and Deriche, R. 2000. Geodesic active contours and level sets for the detection and tracking of moving objects. IEEE PAMI, 22(3):266–280.
Samson, C., Blanc-FBlanc-Féraudraud, L., Aubert, G., and Zerubia, J. 2000. A level set model for image classification. Int. J. of Computer Vision, 40(3):187–197.
Schnörr, C. 1992. Computation of discontinuous optical flow by domain decomposition and shape optimization. Int. J. of Computer Vision, 8(2):153–165.
Shi, J. and Malik, J. 1998. Motion segmentation and tracking using normalized cuts. In Intl. Conf. on Comp. Vision, Bombay, India.
Sussman, M. and Fatemi, E. 1999. An efficient, interface-preserving level set redistancing algorithm and its application to interfacial incompressible fluid flow. SIAM J. Sci. Comput., 20(4):1165–1191.
Sussman, M., Smereka P., and Osher, S.J. 1994. A level set approach for computing solutions to incompressible twophase flow. J. of Comp. Phys., 94:146–159.
Unal, G., Krim, H., and Yezzi, A. (to appear). Fast incorporation of optic flow into active polygons. IEEE Trans. on Image Processing.
Wang, J.Y.A. and Adelson, E.H. 1994. Representating moving images with layers. IEEE Trans. on Image Processing, 3(5):625–638.
Weickert, J. and Schnörr, C. 2001. A theoretical framework for convex regularizers in PDE–based computation of image motion. Int. J. of Computer Vision, 45(3):245–264.
Weiss, Y. 1997. Smoothness in layers: Motion segmentation using nonparametric mixture estimation. In Proc. IEEE Conf. on Comp. Vision Patt. Recog., Puerto Rico, pp. 520–527.
Weiss, Y. and Fleet, D.J. 2001. Velocity likelihoods in biological and machine vision. In Probabilistic Models of the Brain: Perception and Neural Function, M.S. Lewicki, R.P.N. Rao, and B.A. Olshausen (Eds.), MIT Press, pp. 81–100.
Yezzi, A. and Soatto, S. 2003. Stereoscopic segmentation. Int. J. of Computer Vision, 53(1):31–43.
Zhao, H.-K., Chan, T., Merriman, B., and Osher, S. 1996. A variational level set approach to multiphase motion. J. of Comp. Phys., 127:179–195.
Zheng, H. and Blostein, S.D. 1995. Motion-based object segmentation and estimation. IEEE Trans. on Image Processing, 4(9):1225–1235.
Zhu, S.C. and Yuille, A. 1996. Region competition: Unifying snakes, region growing, and Bayes/MDL for multiband image segmentation. IEEE PAMI, 18(9):884–900.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Cremers, D., Soatto, S. Motion Competition: A Variational Approach to Piecewise Parametric Motion Segmentation. Int J Comput Vision 62, 249–265 (2005). https://doi.org/10.1007/s11263-005-4882-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-005-4882-4