Abstract
Dynamic analysis of video sequences often relies on the segmentation of the sequence into regions of consistent motions. Approaching this problem requires a definition of which motions are regarded as consistent. Common approaches to motion segmentation usually group together points or image regions that have the same motion between successive frames (where the same motion can be 2D, 3D, or non-rigid). In this paper we define a new type of motion consistency, which is based on temporal consistency of behaviors across multiple frames in the video sequence. Our definition of consistent “temporal behavior” is expressed in terms of multi-frame linear subspace constraints. This definition applies to 2D, 3D, and some non-rigid motions without requiring prior model selection. We further show that our definition of motion consistency extends to data with directional uncertainty, thus leading to a dense segmentation of the entire image. Such segmentation is obtained by applying the new motion consistency constraints directly to covariance-weighted image brightness measurements. This is done without requiring prior correspondence estimation nor feature tracking.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Anandan, P. and Irani, M. 2002. Factorization with uncertainty. International Journal of Computer Vision, 49(2–3):101–116.
Ayer, S. and Sawhney, H. 1995. Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding. In International Conference on Computer Vision, pp. 777–784.
Boult, T.E. and Brown, L.G. 1991. Factorization-based segmentation of motions. In Proc. of the IEEE Workshop on Motion Understanding, pp. 179–186.
Brand, M. 2001. Morphable 3D models from video. IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, vol. II, pp. 456–463.
Bregler, C., Hertzmann, A., and Biermann, H. 2000. Recovering non-rigid 3D shape from image streams. In IEEE Conference on Computer Vision and Pattern Recognition, South-Carolina, vol. II, pp. 690–696.
Costeira, J. and Kanade, T. 1995. A multi-body factorization method for motion analysis. In International Conference on Computer Vision, Cambridge, MA, pp. 1071–1076.
Gear, C.W. 1998. Multibody grouping from motion images. International Journal of Computer Vision, 2(29):133–150.
Ichimura, N. 2000. A robust and efficient motion segmentation based on orthogonal projection matrix of shape space. In IEEE Conference on Computer Vision and Pattern Recognition, South-Carolina, Vol. II, pp. 446–452.
Irani, M. 2002. Multi-frame correspondence estimation using subspace constraints. International Journal of Computer Vision, 48(3):173–194.
Irani, M., Rousso, B., and Peleg, S. 1992. Detecting and tracking multiple moving objects using temporal integration. In European Conference on Computer Vision, Santa Margarita Ligure, pp. 282–287.
Kanatani, K. 2001. Motion segmentation by subspace separation and model selection. In International Conference on Computer Vision, Vancouver, Canada, vol. 1, pp. 301–306.
Lucas, B.D. and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In Image Understanding Workshop, pp. 121–130.
Machline, M. Zelnik-Manor, L., and Irani, M. 2002. Multi-body segmentation: Revisiting motion consistency. In Workshop on Vision and Modeling of Dynamic Scene.
Morris, D.D. and Kanade, T. 1998. A unified factorization algorithm for points, line segments and planes with uncertainty models. In International Conference on Computer Vision, Bombay, India, pp. 696–702.
Torr, P.H.S. 1998. Geometric motion segmentation and model selection. In Philosophical Transactions of the Royal Society A, J. Lasenby, A. Zisserman, R. Cipolla, and H. Longuet-Higgins, (eds.), Roy. Soc., pp. 1321–1340.
Torr, P.H.S. and Zisserman, A. 1998. Concerning Bayesian motion segmentation, model averaging, matching and the trifocal tensor. In European Conference on Computer Vision, pp. 511–527.
Torresani, L., Yang, D.B., Alexander E.J., and Bregler, C. 2001. Tracking and modeling non-rigid objects with rank constraints. In IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, vol. I, pp. 493–500.
Wang, J. and Adelson, E. 1993. Layered representation for motion analysis. In IEEE Conference on Computer Vision and Pattern Recognition, New York, pp. 361–366.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zelnik-Manor, L., Machline, M. & Irani, M. Multi-body Factorization with Uncertainty: Revisiting Motion Consistency. Int J Comput Vision 68, 27–41 (2006). https://doi.org/10.1007/s11263-005-4840-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-005-4840-1