A Variational Technique for Time Consistent Tracking of Curves and Motion



In this paper, a new framework for the tracking of closed curves and their associated motion fields is described. The proposed method enables a continuous tracking along an image sequence of both a deformable curve and its velocity field. Such an approach is formalized through the minimization of a global spatio-temporal continuous cost functional, w.r.t a set of variables representing the curve and its related motion field. The resulting minimization process relies on optimal control approach and consists in a forward integration of an evolution law followed by a backward integration of an adjoint evolution model. This latter pde includes a term related to the discrepancy between the current estimation of the state variable and discrete noisy measurements of the system. The closed curves are represented through implicit surface modeling, whereas the motion is described either by a vector field or through vorticity and divergence maps depending on the kind of targeted applications. The efficiency of the approach is demonstrated on two types of image sequences showing deformable objects and fluid motions.


Variational method Data assimilation Curve tracking Dynamical model 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Amiaz, T., Kiryati, N.: Piecewise-smooth dense optical flow via level sets. Int. J. Comput. Vis. 68(2), 111–124 (2006) CrossRefGoogle Scholar
  2. 2.
    Anderson, B., Moore, J.: Optimal Filtering. Prentice Hall, Englewood Cliffs (1979) MATHGoogle Scholar
  3. 3.
    Barron, J., Fleet, D., Beauchemin, S.: Performance of optical flow techniques. Int. J. Comput. Vis. 12(1), 43–77 (1994) CrossRefGoogle Scholar
  4. 4.
    Bartesaghi, A., Sapiro, G.: Tracking of moving objects under severe and total occlusions. In Proc. Int. Conf. Image Processing, ICIP’05, vol. 1, pp. 301–304 (2005) Google Scholar
  5. 5.
    Beg, M., Miller, M., Trouvé, A., Younes, L.: Computing large deformation metric mappings via geodesic flows of diffeomorphisms. Int. J. Comput. Vis. 61(2), 139–157 (2005) CrossRefGoogle Scholar
  6. 6.
    Bennet, A.: Inverse Methods in Physical Oceanography. Cambridge University Press, Cambridge (1992) Google Scholar
  7. 7.
    Blake, A., Isard, M.: Active Contours. Springer, London (1998) Google Scholar
  8. 8.
    Bouttier, F., Courtier, P.: Data Assimilation Concepts and Methods. ECMWF Meteorological Training Course Lecture Series (1999) Google Scholar
  9. 9.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Proc. Eur. Conf. Comput. Vis., ECCV’04, Prague, Czech Republic, pp. 25–36. Springer, Berlin (2004) Google Scholar
  10. 10.
    Brox, T., Bruhn, A., Weickert, J.: Variational motion segmentation with level sets. In Proc. Eur. Conf. Comput. Vis., vol. 1, pp. 471–483 (2006) Google Scholar
  11. 11.
    Bruhn, A., Weickert, J., Schnörr, C.: Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods. Int. J. Comput. Vis. 61(3), 211–231 (2005) CrossRefGoogle Scholar
  12. 12.
    Carlier, J., Wieneke, B.: Report on production and diffusion of fluid mechanics images and data. Technical report, Fluid Project deliverable 1.2 (2005) Google Scholar
  13. 13.
    Chan, T., Vese, L.: Active contours without edges. IEEE Trans. Image Process. 10(2), 266–277 (2001) MATHCrossRefGoogle Scholar
  14. 14.
    Corpetti, T., Heitz, D., Arroyo, G., Mémin, E., Santa-Cruz, A.: Fluid experimental flow estimation based on an optical-flow scheme. Int. J. Exp. Fluid 40(1), 80–97 (2006) CrossRefGoogle Scholar
  15. 15.
    Corpetti, T., Mémin, E., Pérez, P.: Dense estimation of fluid flows. IEEE Trans. Pattern Anal. Mach. Intell. 24(3), 365–380 (2002) CrossRefGoogle Scholar
  16. 16.
    Courtier, P., Talagrand, O.: Variational assimilation of meteorological observations with the direct and adjoint shallow-water equations. Tellus 42, 531–549 (1990) CrossRefGoogle Scholar
  17. 17.
    Cremers, D.: Dynamical statistical shape priors for level set based tracking. IEEE Trans. Pattern Anal. Mach. Intell. 28(8), 1262–1273 (2006) CrossRefGoogle Scholar
  18. 18.
    Cremers, D., Soatto, S.: Variational space-time motion segmentation. In: Proc. IEEE Int. Conf. Comput. Vis. ICCV’03, vol. 2, pp. 886–892 (2003) Google Scholar
  19. 19.
    Cremers, D., Soatto, S.: Motion competition: a variational framework for piecewise parametric motion segmentation. Int. J. Comput. Vis. 62(3), 249–265 (2005) CrossRefGoogle Scholar
  20. 20.
    Cremers, D., Tischhäuser, F., Weickert, J., Schnoerr, C.: Diffusion snakes: introducing statistical shape knowledge into the Mumford-Shah functional. Int. J. Comput. Vis. 50(3), 295–313 (2002) MATHCrossRefGoogle Scholar
  21. 21.
    Cuzol, A., Hellier, P., Memin, E.: A low dimensional fluid motion estimator. Int. J. Comput. Vis. 75(3), 329–349 (2007) CrossRefGoogle Scholar
  22. 22.
    Dambreville, S., Rathi, Y., Tannenbaum, A.: Tracking deformable objects with unscented Kalman filtering and geometric active contours. In: American Control Conf., ACC’06, June 2006 Google Scholar
  23. 23.
    Faugeras, B.: Assimilation variationnelle de données dans un modèle couplé océan-biogéochimie. PhD thesis, Université Joseph-Fourier–Grenoble I, Octobre 2002 Google Scholar
  24. 24.
    Goldenberg, R., Kimmel, R., Rivlin, E., Rudzsky, M.: Fast geodesic active contours. IEEE Trans. Image Process. 10(10), 1467–1475 (2001) CrossRefMathSciNetGoogle Scholar
  25. 25.
    Héas, P., Mémin, E., Papadakis, N., Szantai, A.: Layered estimation of atmospheric mesoscale dynamics from satellite imagery. IEEE Trans. Geosci. Remote Sens. 45(12), 4087–4104 (2007) CrossRefGoogle Scholar
  26. 26.
    Héas, P., Papadakis, N., Mémin, E.: Time-consistent estimators of 2d/3d motion of atmospheric layers from pressure images. Technical Report 6292, INRIA, September 2007 Google Scholar
  27. 27.
    Jackson, J., Yezzi, A., Soatto, S.: Tracking deformable moving objects under severe occlusions. In: IEEE Conference on Decision and Control, December 2004 Google Scholar
  28. 28.
    Kimmel, R., Bruckstein, A.M.: Tracking level sets by level sets: a method for solving the shape from shading problem. Comput. Vis. Image Underst. 62(1), 47–58 (1995) CrossRefGoogle Scholar
  29. 29.
    Kohlberger, T., Mémin, E., Schnörr, C.: Variational dense motion estimation using the Helmholtz decomposition. In: Proc. Int. Conf. on Scale-Space and PDE methods in Comput. Vis., Scale-Space’03, Isle of Skye, UK, pp. 432–448 (2003) Google Scholar
  30. 30.
    Kurganov, A., Levy, D.: A third-order semidiscrete central scheme for conservation laws and convection-diffusion equations. SIAM J. Sci. Comput. 22(4), 1461–1488 (2000) MATHCrossRefMathSciNetGoogle Scholar
  31. 31.
    Kurganov, A., Tadmor, E.: New high-resolution central schemes for nonlinear conservation laws and convection-diffusion equations. J. Comput. Phys. 160(1), 241–282 (2000) MATHCrossRefMathSciNetGoogle Scholar
  32. 32.
    Le Dimet, F.-X., Talagrand, O.: Variational algorithms for analysis and assimilation of meteorological observations: theoretical aspects. Tellus 38(A), 97–110 (1986) Google Scholar
  33. 33.
    Leventon, M., Grimson, E., Faugeras, O.: Statistical shape influence in geodesic active contours. In: Proc. IEEE Comput. Vis. and Pat. Rec. CVPR’00 (2000) Google Scholar
  34. 34.
    Li, Z., Navon, I.M.: Optimality of 4d-var and its relationship with the Kalman filter and Kalman smoother. Q. J. R. Meteorol. Soc. 127(572), 661–683 (2001) CrossRefGoogle Scholar
  35. 35.
    Lions, J.: Contrôle Optimal de Systèmes Gouvernés par des Équations aux Dérivées Partielles. Dunod, Paris (1968) MATHGoogle Scholar
  36. 36.
    Lions, J.: Optimal Control of Systems Governed by PDEs. Springer, New York (1971) Google Scholar
  37. 37.
    Mansouri, A.: Region tracking via level set PDEs without motion computation. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 947–961 (2003) CrossRefGoogle Scholar
  38. 38.
    Martin, P., Réfrégier, P., Goudail, F., Guérault, F.: Influence of the noise model on level set active contours segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 26(6), 799–803 (2004) CrossRefGoogle Scholar
  39. 39.
    Mémin, E., Pérez, P.: Dense estimation and object-based segmentation of the optical flow with robust techniques. IEEE Trans. Image Process. 7(5), 703–719 (1998) CrossRefGoogle Scholar
  40. 40.
    Niethammer, M., Tannenbaum, A.: Dynamic geodesic snakes for visual tracking. In: Proc. IEEE Comput. Vis. and Pat. Rec. CVPR’04, vol. 1, pp. 660–667 (2004) Google Scholar
  41. 41.
    Niethammer, M., Vela, P., Tannenbaum, A.: Geometric observers for dynamically evolving curves. In: Proc. Conf. Decision and Control Eur. Control Conf. CDC-ECC’05, pp. 6071–6077 (2005) Google Scholar
  42. 42.
    Nir, T., Bruckstein, A., Kimmel, R.: Over parameterized optical-flow. Technical Report CIS-2006-05, Technion—Israel Institute of Technology (2006) Google Scholar
  43. 43.
    Odobez, J.-M., Bouthemy, P.: Robust multiresolution estimation of parametric motion models. J. Vis. Commun. Image Represent. 6(4), 348–365 (1995) CrossRefGoogle Scholar
  44. 44.
    Oksendal, B.: Stochastic Differential Equations. Springer, Berlin (1998) Google Scholar
  45. 45.
    Osher, S., Sethian, J.: Fronts propagating with curvature dependent speed: algorithms based on Hamilton-Jacobi formulation. J. Comput. Phys. 79, 12–49 (1988) MATHCrossRefMathSciNetGoogle Scholar
  46. 46.
    Papadakis, N., Mémin, E.: A variational method for joint tracking of curve and motion. Research Report 6283, INRIA, September 2007 Google Scholar
  47. 47.
    Papadakis, N., Mémin, E.: Variational optimal control technique for the tracking of deformable objects. In: Proc. IEEE Int. Conf. Comput. Vis., ICCV’07, Rio de Janeiro, Brazil, October 2007 Google Scholar
  48. 48.
    Papenberg, N., Bruhn, A., Brox, T., Didas, S., Weickert, J.: Highly accurate optic flow computation with theoretically justified warping. Int. J. Comput. Vis. 67(2), 141–158 (2005) CrossRefGoogle Scholar
  49. 49.
    Paragios, N.: A level set approach for shape-driven segmentation and tracking of the left ventricle. IEEE Trans. Med. Imaging 22(6), 773–776 (2003) CrossRefGoogle Scholar
  50. 50.
    Paragios, N., Deriche, R.: Geodesic active regions: a new framework to deal with frame partition problems in computer vision. J. Vis. Commun. Image Represent. 13, 249–268 (2002) CrossRefGoogle Scholar
  51. 51.
    Peterfreund, N.: Robust tracking of position and velocity with Kalman snakes. IEEE Trans. Pattern Anal. Mach. Intell. 21(6), 564–569 (1999) CrossRefGoogle Scholar
  52. 52.
    Rathi, Y., Vaswani, N., Tannenbaum, A., Yezzi, A.: Tracking deforming objects using particle filtering for geometric active contours. IEEE Trans. Pattern Anal. Mach. Intell. 29(8), 1470–1475 (2007) CrossRefGoogle Scholar
  53. 53.
    Ruhnau, P., Stahl, A., Schnörr, C.: Variational estimation of experimental fluid flows with physics-based spatio-temporal regularization. Meas. Sci. Technol. 18(3), 755–763 (2007) CrossRefGoogle Scholar
  54. 54.
    Sethian, J.: Level Set Methods. Cambridge University Press, Cambridge (1996) MATHGoogle Scholar
  55. 55.
    Shu, C.-W.: Essentially non-oscillatory and weighted essentially non-oscillatory schemes for hyperbolic conservation laws. In: Advanced Numerical Approximation of Nonlinear Hyperbolic Equations. Lecture Notes in Mathematics, vol. 1697, pp. 325–432. Springer, Berlin (1998) CrossRefGoogle Scholar
  56. 56.
    Soatto, S., Yezzi, A., Duci, A.: Region matching and tracking under deformations and occlusions. In: Geometric Level Sets Methods in Imaging Vision and Graphics. Springer, Berlin (2003) Google Scholar
  57. 57.
    Talagrand, O., Courtier, P.: Variational assimilation of meteorological observations with the adjoint vorticity equation, I: theory. Q. J. R. Meteorol. Soc. 113, 1311–1328 (1987) CrossRefGoogle Scholar
  58. 58.
    Trémolet, Y.: Incremental 4d-var convergence study. Tellus A 59(5), 706–718 (2007) CrossRefGoogle Scholar
  59. 59.
    Weickert, J., Schnörr, C.: Variational optic-flow computation with a spatio-temporal smoothness constraint. J. Math. Imaging Vis. 14(3), 245–255 (2001) MATHCrossRefGoogle Scholar
  60. 60.
    Yezzi, A., Zöllei, L., Kapur, T.: A variational framework for joint segmentation and registration. In: MMBIA, pp. 44–51 (2001) Google Scholar
  61. 61.
    Yuan, J., Schnörr, C., Mémin, E.: Discrete orthogonal decomposition and variational fluid flow estimation. J. Math. Imaging Vis. 28(1), 67–80 (2007) CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.Barcelona MediaUniversitat Pompeu FabraBarcelonaSpain
  2. 2.CEFIMASCentro de Física y Matemática de América del SurBuenos AiresArgentina
  3. 3.VISTAINRIA Rennes Bretagne-AtlantiqueRennes cedexFrance

Personalised recommendations