Abstract
We propose a model of the shape, motion and appearance of a sequence of images that captures occlusions, scene deformations, arbitrary viewpoint variations and changes in irradiance. This model is based on a collection of overlapping layers that can move and deform, each supporting an intensity function that can change over time. We discuss the generality and limitations of this model in relations to existing ones such as traditional optical flow or motion segmentation, layers, deformable templates and deformotion. We then illustrate how this model can be used for inference of shape, motion, deformation and appearance of the scene from a collection of images. The layering structure allows for automatic inpainting of partially occluded regions. We illustrate the model on synthetic and real sequences where existing schemes fail; we implement our gradient-based infinite-dimensional optimization using level set methods.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Alvarez, L., Weickert, J., Sanchez, J.: A scale-space approach to nonlocal optical flow calculations. In: Scale Space 1999, pp. 235–246 (1999)
Baker, S., Matthews, I., Schneider, J.: Image coding with active appearance models. Technical report, Carnegie Mellon University, The Robotics Institute (2003)
Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Akeley, K. (ed.) Siggraph 2000, Computer Graphics Proceedings, pp. 417–424. ACM Press / ACM SIGGRAPH / Addison Wesley Longman (2000)
Blake, A., Isard, M.: Active contours. Springer, Heidelberg (1998)
Caselles, V., Kimmel, R., Sapiro, G.: Geodesic active contours. Int. J. of Computer Vision 22(1), 61–79 (1997)
Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. In: Proc. of the Eur. Conf. on Comp. Vis., pp. 484–496 (1998)
Cremers, D.: Multiphase levelset framework for variational motion segmentation. In: Intl. Conf. on Scale Space Theories in Computer Vision, June 2003, pp. 599–614 (2003)
Deriche, R., Kornprobst, P., Aubert, G.: Optical flow estimation while preserving its discontinuities: a variational approach. In: Proc. of ACCV, vol. 2, pp. 290–295 (1995)
Grenander, U.: General Pattern Theory. Oxford University Press, Oxford (1993)
Haussecker, H.W., Fleet, D.J.: Computing optical flow with physical models of brightness variation. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 661–673 (2001)
Horn, B.K.P., Schunk, B.G.: Determining optical flow. Artificial Intell. 17, 185–203 (1981)
Hsu, S., Anandan, P., Peleg, S.: Accurate computation of optical flow by using layered motion representations. In: Proc. IEEE Conf. on Comp. Vision and Pattern Recogn., pp. 1621–1626 (1992)
Kass, M., Witkin, A., Terzopoulos, D.: Snakes: active contour models. Int. J. of Computer Vision 1(4), 321–331 (1987)
Miller, M.I., Younes, L.: Group action, diffeomorphism and matching: a general framework. In: Proc. of SCTV (1999)
Osher, S., Sethian, J.: Fronts propagating with curvature-dependent speed: algorithms based on hamilton-jacobi equations. J. of Comp. Physics 79, 12–49 (1988)
Paragios, N., Deriche, R.: Geodesic active contours and level sets for the detection and tracking of moving objects. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(3), 266–280 (2000)
Paragios, N., Rousson, M., Ramesh, V.: Non-rigid registration using distance functions. Comput. Vis. Image Underst. 89(2-3), 142–165 (2003)
Schnörr, C.: Computation of discontinuous optical flow by domain decomposition and shape optimization. Int. J. of Computer Vision 8(2), 153–165 (1992)
Soatto, S., Yezzi, A.: Deformotion: deforming motion, shape average and the joint segmentation and registration of images. In: Proc. of the Eur. Conf. on Computer Vision (ECCV), vol. 3, pp. 32–47 (2002)
Wang, J., Adelson, E.: Representing moving images with layers. IEEE Trans. on Image Processing 3(5), 625–638 (1994)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jackson, J.D., Yezzi, A., Soatto, S. (2005). Dynamic Shape and Appearance Modeling Via Moving and Deforming Layers. In: Rangarajan, A., Vemuri, B., Yuille, A.L. (eds) Energy Minimization Methods in Computer Vision and Pattern Recognition. EMMCVPR 2005. Lecture Notes in Computer Science, vol 3757. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11585978_28
Download citation
DOI: https://doi.org/10.1007/11585978_28
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-30287-2
Online ISBN: 978-3-540-32098-2
eBook Packages: Computer ScienceComputer Science (R0)