International Journal of Computer Vision

, Volume 46, Issue 2, pp 129–155 | Cite as

Hierarchical Estimation and Segmentation of Dense Motion Fields

  • Etienne Mémin
  • Patrick Pérez


In this paper we present a comprehensive energy-based framework for the estimation and the segmentation of the apparent motion in image sequences. The robust cost functions and the associated hierarchical minimization techniques that we propose mix efficiently non-parametric (dense) representations, local interacting parametric representations, and global non-interacting parametric representations related to a partition into regions. Experimental comparisons, both on synthetic and real images, demonstrate the merit of the approach on different types of photometric and kinematic contents ranging from moving rigid objects to moving fluids.

apparent motion robust discontinuity-preserving estimation motion-based segmentation hierarchical non-linear minimization dense and parametric representations 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Adiv, G. 1985. Determining three-dimensional motion and structure from optical flow generated by several moving objects. IEEE Trans. Pattern Anal. Machine Intell., 7:384–401.Google Scholar
  2. Ayer, S. and Sawhney, H.S. 1995. Layered representation of motion video using robust maximum-likelihood estimation of mixture models and Mdl encoding. In Proc. Int. Conf. Computer Vision, pp. 777–784.Google Scholar
  3. Bab-Hadashiar, A. and Sutter, D. 1998. Robust optic flow computation. Int. J. Computer Vision, 29(1):59–77.Google Scholar
  4. Barron, J., Fleet, D., and Beauchemin, S. 1994. Performance of optical flow techniques. Int. J. Computer Vision, 12(1):43–77.Google Scholar
  5. Bergen, J., Anandan, P., Hanna, K., and Hingorani, R. 1992. Hierarchical model-based motion estimation. In Proc. Europ. Conf. Computer Vision, G. Sandini (Ed.), vol. 558 of LNCS, pp. 237–252, Springer-Verlag: Berlin.Google Scholar
  6. Besag, J. 1986. On the statistical analysis of dirty pictures. J. Royal Statist. Soc., 48B(3):259–302.Google Scholar
  7. Black, M. 1994. Recursive non-linear estimation of discontinuous flow fields. In Proc. Europ. Conf. Computer Vision, Stockholm, Sweden, pp. 138–145.Google Scholar
  8. Black, M. and Anandan, P. 1996. The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding, 63(1):75–104.Google Scholar
  9. Black, M. and Jepson, P. 1996. Estimating optical flow in segmented images using variable-order parametric models with local deformations. IEEE Trans. Pattern Anal. Machine Intell., 18(10):972–986.Google Scholar
  10. Black, M. and Rangarajan, A. 1996. On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. Int. J. Computer Vision, 19(1):75–104.Google Scholar
  11. Bouthemy, P. and Francois, E. 1993. Motion segmentation and qualitative dynamic scene analysis from an image sequence. Int. J. Computer Vision, 10(2):157–182.Google Scholar
  12. Chang, M.M., Tekalp, A.M., and Sezan, M.I. 1997. Simultaneous motion estimation and segmentation. IEEE Trans. Image Processing, 6(9):1326–1333.Google Scholar
  13. Charbonnier, P., Blanc-Féraud, L., Aubert, G., and Barlaud, M. 1997. Deterministic edge-preserving regularization in computed imaging. IEEE Trans. Image Processing, 6(2):298–311.Google Scholar
  14. Cohen, I. and Herlin, I. 1999. Non uniform multiresolution method for optical flow and phase portrait models: Environmental applications. Int. J. Computer Vision, 33(1):29–49.Google Scholar
  15. Corpetti, T., Mémin, E., and Pérez, P. 2000. Dense fluid flow estimation. IRISA, Technical Report No. 1352.Google Scholar
  16. Delanay, A. and Bresler, Y. 1998. Globally convergent edge-preserving regularized reconstruction: An application to limited-angle tomography. IEEE Trans. Image Processing, 7(2):204–221.Google Scholar
  17. Deriche, R., Kornprobst, P., and Aubert, G. 1995. Optical flow estimation while preserving its discontinuities: A variational approach. In Proc. Asian Conf. Computer Vision, Singapore, vol. 1, pp. 290–295.Google Scholar
  18. Enkelmann, W. 1988. Investigation of multigrid algorithms for the estimation of optical flow fields in image sequences. Comp. Vision Graph. and Image Proces., 43:150–177.Google Scholar
  19. Geman, D. and Reynolds, G. 1992. Constrained restoration and the recovery of discontinuities. IEEE Trans. Pattern Anal. Machine Intell., 14(3):367–383.Google Scholar
  20. Hackbusch, W. 1985. Multi-Grid Methods and Applications. Springer-Verlag: Berlin.Google Scholar
  21. Hellier, P., Barillot, C., Mémin, E., and Pérez, P. (2001). Hierarchical estimation of a dense deformation field for 3D robust registration. IEEE Transaction on Medical Imaging, 20(5):388–402.Google Scholar
  22. Horn, B. and Schunck, B. 1981. Determining optical flow. Artificial Intelligence, 17:185–203.Google Scholar
  23. Huang, Y., Palaniappan, K., Zhuand, X., and Cavanaugh, J. 1995. Optic flow field segmentation and motion estimation using a robust genetic partitioning algorithm. IEEE Trans. Pattern Anal. Machine Intell., 17(12):1177–1190.Google Scholar
  24. Ju, X., Black, M.J., and Jepson, A.D. 1996. Skin and bones: Multi-layer, locally affine, optical flow and regularization with transparency. In Proc. Conf. Comp. Vision Pattern Rec., pp. 307–314.Google Scholar
  25. Kornprobst, P., Deriche, R., and Aubert, G. 1999. Image sequence analysis via partial differential equations. Journal of Mathematical Imaging and Vision, 11(1):5–26.Google Scholar
  26. Lai, S. and Vemuri, B. 1998. Reliable and efficient computation of optical flow. Int. J. Computer Vision, 29(2):87–105.Google Scholar
  27. Leclerc, Y. 1989. Constructing simple stable descriptions for image partitioning. Int. J. Computer Vision, 3:73–102.Google Scholar
  28. Mémin, E. and Pérez, P. 1998a. Dense estimation and object-based segmentation of the optical flow with robust techniques. IEEE Trans. Image Processing, 7(5):703–719.Google Scholar
  29. Mémin, E. and Pérez, P. 1998b. Amultigrid approach for hierarchical motion estimation. In Proc. Int. Conf. Computer Vision, Bombay, India, pp. 933–938.Google Scholar
  30. Mémin, E. and Pérez, P. 1999. Fluid motion recovery by coupling dense and parametric motion fields. In Int. Conf. on Computer, ICCV'99, pp. 620–625.Google Scholar
  31. Mitiche, A. and Bouthemy, P. 1996. Computation of image motion: A synopsis of current problems and methods. Int. Journ. of Comp. Vis., 19(1):29–55.Google Scholar
  32. Mumford, D. and Shah, J. 1989. Optimal approximation by piecewise smooth functions and associated variational problems. Comm. Pure and Appl. Math., 42:577–685.Google Scholar
  33. Murray, D.W. and Buxton, H. 1987. Scene segmentation from visual motion using global optimization. IEEE Trans. Pattern Anal. Machine Intell., 9(2):220–228.Google Scholar
  34. Musse, O., Heitz, F., and Armspach, J.P. 1999. 3D deformable image matching using multiscale minimization of global energy functions. In Proc. Conf. Comp. Vision Pattern Rec., Fort Collins, Colorado, vol. 2, pp. 478–485Google Scholar
  35. Nesi, P. 1993. Variational approach to optical flow estimation managing discontinuities. Image and Vision Computing, 11(7):419–439.Google Scholar
  36. Oisel, L., Mémin, E., and Morin, L. 2000. Geometric driven optical flow estimation and segmentation for 3d reconstruction. In Proc. Europ. Conf. Computer Vision, Dublin, Irland, and D. Vernon, (Ed.), vol. II, pp. 847–863. Springer Notes in Computer Science vol. 1843.Google Scholar
  37. Samson, C., Blanc-Ferraud, L., Aubert, G., and Zerubia, J. 1999. Simultaneous image classification and restoration using a variational approach. In Proc. Conf. Comp. Vision Pattern Rec., Fort Collins, Colorado, vol. 2, pp. 618–623.Google Scholar
  38. Schnorr, C., Sprengel, R., and Neumann, B. 1996. A variational approach to the design of early vision algorithms. Computing Suppl., vol. 11, pp. 149–165.Google Scholar
  39. Schunck, B.G. 1986. The image flow constraint equation. Comput. Vision, Graphics, Image Proc., 35:20–46.Google Scholar
  40. Stiller, C. 1997. Object-based estimation of dense motion fields. IEEE Trans. Image Processing, 6(2):234–250.Google Scholar
  41. Szeliski, R. and Coughlan, J. 1994 Hierarchical spline-based image registration. In Proc. Conf. Comp. Vision Pattern Rec., Seattle, Washington, pp. 194–201.Google Scholar
  42. Szeliski, R. and Shum, H.-Y. 1996. Motion estimation with quadtree splines. IEEE Trans. Pattern Anal. Machine Intell., 18(12):1199–1210.Google Scholar
  43. Thisted, R.A. 1988. Elements of Statistical Computing. Chapman and Hall: London.Google Scholar
  44. Viéville, T. and Faugeras, O. 1992. Robust and fast computation of unbiased intensity derivatives in images. In Proc. Europ. Conf. Computer Vision, S. Margherita Ligure, pp. 203–212.Google Scholar
  45. Yezzi, A., Tsai, A., and Willsky, A. 1999. A statistical approach to image segmentation for bimodal and trimodal imagery. In Proc. Int. Conf. Computer Vision, Corfu, Greece.Google Scholar
  46. Zhu, S. and Yuille, A. 1996. Region competition: Unifying snakes, region growing, and Bayes/MDL for multiband image segmentation. IEEE Trans. Pattern Anal. Machine Intell., 18:884–900.Google Scholar

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  • Etienne Mémin
    • 1
  • Patrick Pérez
    • 2
  1. 1.IRISA/Univ. Rennes IRennes CedexFrance
  2. 2.Microsoft ResearchCambridgeUK

Personalised recommendations