A New Framework for Motion Estimation in Image Sequences Using Optimal Flow Control

  • Annette Stahl
  • Ole Morten Aamo
Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 89)


Application of tools from optimal flow control to the field of computer vision and image sequence processing, has recently led to a new and promising research direction. We present an approach to image motion estimation that uses an optimal flow control formulation subject to a physical constraint. Motion fields are forced to satisfy appropriate equations of motion. Although the framework presented is flexible with respect to selection of equations of motion, we employ the Burgers equation from fluid mechanics as physical prior knowledge in this study. To solve the resulting time-dependent optimisation problem we introduce an iterative method to uncouple the derived state and adjoint equations. We perform numerical experiments on synthetic and real image sequences and compare our results with other well-known methods to demonstrate performance of the optimal control formulation in determining image motion from video and image sequences. The results indicate improved performance.


Optimal control Motion estimation Physical prior knowledge Optimisation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alvarez, L., Esclarìn, J., Lefebure, M., Sànchez, J.: A PDE model for computing the optical flow. In: Proceedings of CEDYA XVI, pp. 1349–1356 (1999)Google Scholar
  2. 2.
    Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. Int. J. of Computer Vision 12(1), 43–77 (1994)CrossRefGoogle Scholar
  3. 3.
    Borzi, A., Ito, K., Kunisch, K.: Optimal control formulation for determining optical flow. SIAM J. Sci. Comput. 24(3), 818–847 (2002)MathSciNetMATHCrossRefGoogle Scholar
  4. 4.
    Burgers, J.M.: A mathematical model illustrating the theory of turbulence. Adv. Appl. Mech. 1, 171–199 (1948)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Colella, P., Puckett, E.G.: Modern Numerical Methods for Fluid Flow. Lecture Notes, Dep. of Mech. Eng., Uni. of California, Berkeley, CA (1998), http://www.rzg.mpg.de/~bds/numerics/cfd-lectures.html Google Scholar
  6. 6.
    Deriche, R., Kornprobst, P., Aubert, G.: Optical-flow estimation while preserving its discontinuities: A variational approach. In: ACCV, pp. 71–80 (1995)Google Scholar
  7. 7.
    Gunzburger, M.: Perspectives in Flow Control and Optimization. Society for Industrial and Applied Mathematics (2002)Google Scholar
  8. 8.
    Hirsch, C.: Numerical Computation of Internal and External Flows, vol. I+II. John Wiley & Sons, Chichester (2000)Google Scholar
  9. 9.
    Horn, B., Schunck, B.: Determining optical flow. Artificial Intelligence 17, 185–203 (1981)CrossRefGoogle Scholar
  10. 10.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision (darpa). In: Proc. of the 1981 DARPA Image Understanding Workshop, pp. 121–130 (1981)Google Scholar
  11. 11.
    McCane, B., Novins, K., Crannitch, D., Galvin, B.: On benchmarking optical flow. Comput. Vis. Image Underst. 84(1), 126–143 (2001)MATHCrossRefGoogle Scholar
  12. 12.
    Nagel, H.H.: Constraints for the estimation of displacement vector fields from image sequences. In: Int. Joint Conf. on Artificial Intelligence, pp. 945–951 (1983)Google Scholar
  13. 13.
    Nagel, H.H.: Extending the oriented smoothness constraint into the temporal domain and the estimation of derivatives of optical flow. In: Faugeras, O. (ed.) ECCV 1990. LNCS, vol. 427, pp. 139–148. Springer, Heidelberg (1990)CrossRefGoogle Scholar
  14. 14.
    Ruhnau, P., Schnörr, C.: Optical Stokes flow: An imaging-based control approach. Experiments in Fluids 42, 61–78 (2007)CrossRefGoogle Scholar
  15. 15.
    Schnörr, C.: Determining optical flow for irregular domains by minimizing quadratic functionals of a certain class. Int. J. of Computer Vision 6(1), 25–38 (1991)CrossRefGoogle Scholar
  16. 16.
    Schnörr, C.: Segmentation of visual motion by minimizing convex non-quadratic functionals. In: 12th Int. Conf. on Pattern Recognition, vol. A, pp. 661–663. IEEE Computer Society Press, Los Alamitos (1994)CrossRefGoogle Scholar
  17. 17.
    Stahl, A., Ruhnau, P., Schnörr, C.: A Distributed Parameter Approach to Dynamic Image Motion. In: Int. Workshop on The Representation and Use of Prior Knowledge in Vision, ECCV Workshop (2006)Google Scholar
  18. 18.
    Wedel, A., Pock, T., Zach, C., Bischof, H., Cremers, D.: An improved algorithm for TV-11 optical flow. In: Cremers, D., Rosenhahn, B., Yuille, A.L., Schmidt, F.R. (eds.) Statistical and Geometrical Approaches to Visual Motion Analysis. LNCS, vol. 5604, pp. 23–45. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  19. 19.
    Weickert, J., Schnörr, C.: A theoretical framework for convex regularizers in PDE-based computation of image motion. Int. J. of Computer Vision 45(3), 245–264 (2001)MATHCrossRefGoogle Scholar
  20. 20.
    Weickert, J., Schnörr, C.: Variational optic flow computation with a spatio-temporal smoothness constraint. J. Math. Imaging and Vision 14(3), 245–255 (2001)MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Annette Stahl
    • 1
  • Ole Morten Aamo
    • 1
  1. 1.Department of Engineering CyberneticsNorwegian University of Science and Technology (NTNU)TrondheimNorway

Personalised recommendations