Machine Vision and Applications

, Volume 24, Issue 2, pp 275–287 | Cite as

Spatial and temporal constraints in variational correspondence methods

  • Jarno RalliEmail author
  • Javier Díaz
  • Eduardo Ros
Original Paper


This paper proposes a new method for optical-flow and stereo estimation based on the inclusion of both spatial and temporal constraints in a variational framework. These constraints bound the solution based on a priori information, or in other words, based on what is known of a possible solution or how it is expected to change temporally. This knowledge can be something that (a) is known since the geometrical properties of the scene are known or (b) is deduced by a higher-level algorithm capable of inferring this information. In the latter case, the constraint terms enable the exchange of information between high- and low-level vision systems: the high-level system makes a hypothesis of a possible scene setup which is then tested by the low-level one, recurrently. Since high-level vision systems incorporate knowledge of the world that surrounds us, this kind of hypothesis testing loop between the high- and low-level vision systems should converge to a more coherent solution.


A priori information Spatial constraint Temporal constraint Variational calculation Data fusion Spatio-temporal coherency Hypothesis-forming validation-loop 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hirschmuller, H., Scharstein, D.: Evaluation of cost functions for stereo matching. In: CVPR07, pp. 1–8 (2007)Google Scholar
  2. 2.
    Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. In: Computer vision, 2007. ICCV 2007. IEEE 11th International Conference, pp. 1–8 (2007)Google Scholar
  3. 3.
    Ralli, J., Díaz, J., Ros, E.: Disparity disambiguation by fusion of signal- and symbolic-level information. Mach. Vis. Appl., pp. 1–13 (2010). doi: 10.1007/s00138-010-0266-z
  4. 4.
    Krüger, N.: Three dilemmas of signal- and symbol-based representations in computer vision. In: Proceedings of the Workshop Brain, Vision and Artificial Intelligence, vol. 3704, pp. 167–176 (2005)Google Scholar
  5. 5.
    Kalkan, S., Yan, S., Krüger, V., Wörgötter, F., Krüger, N.: A signal-symbol loop mechanism for enhanced edge extraction. In: International Conference on Computer Vision Theory and Applications VISAPP’08, pp. 214–221 (2008)Google Scholar
  6. 6.
    Black, M., Anandan, P.: Robust dynamic motion estimation over time. In: Proceedings of Computer Vision and Pattern Recognition, pp. 296–302 (1991)Google Scholar
  7. 7.
    Black, M.: Recursive non-linear estimation of discontinuous flow fields. In: In Third European Conference on Computer Vision, pp. 138–145. Springer, Berlin (1994)Google Scholar
  8. 8.
    Werlberger, M., Trobin, W., Pock, T., Wedel, A., Cremers, D., Bischof, H.: Anisotropic huber-l1 optical flow. In: Proceedings of the British Machine Vision Conference (BMVC) (2009)Google Scholar
  9. 9.
    Weickert J., Schnörr C.: Variational optic flow computation with a spatio-temporal smoothness constraint. JMIV 14(3), 245–255 (2001)zbMATHCrossRefGoogle Scholar
  10. 10.
    Salgado, A., Sánchez, J.: Temporal constraints in large optical flow estimation. In: EUROCAST, pp. 709–716 (2007)Google Scholar
  11. 11.
    Bruhn A., Weickert J., Feddern C., Kohlberger T., Schnorr C.: Variational optical flow computation in real time. IEEE Trans. Image Process. 14(5), 608–615 (2005)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Alvarez L., Weickert J., Sánchez J.: Reliable estimation of dense optical flow fields with large displacements. Int. J. Comput. Vis. 39(1), 41–56 (2000)zbMATHCrossRefGoogle Scholar
  13. 13.
    Weickert J., Schnörr C.: A theoretical framework for convex regularizers in pde-based computation of image motion. Int. J. Comput. Vis. 45(3), 245–264 (2001)zbMATHCrossRefGoogle Scholar
  14. 14.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: ECCV04, pp. 25–36 (2004)Google Scholar
  15. 15.
    Bruhn, A.: Variational optic flow computation: accurate modelling and efficient numerics. PhD thesis, Saarland University, Saarbrücken, Germany (2006)Google Scholar
  16. 16.
    Ralli, J., Díaz, J., Ros, E.: Complementary image representation spaces in variational disparity calculation. EURASIP J. Adv. Signal Process. (2011, in press)Google Scholar
  17. 17.
    Nagel H.H., Enkelmann W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. PAMI 8(5), 565–593 (1986)CrossRefGoogle Scholar
  18. 18.
    Brox, T.: From pixels to regions: partial differential equations in image analysis. PhD thesis, Saarland University, Saarbrücken, Germany (2005)Google Scholar
  19. 19.
    Zimmer, H., Bruhn, A., Weickert, J., Valgaerts, L., Salgado, A., Rosenhahn, B., Seidel, H.P.: Complementary optic flow. In: EMMCVPR, Lecture Notes in Computer Science, vol. 5681, pp. 207–220 (2009)Google Scholar
  20. 20.
    Acar, R., Vogel, C.R.: Analysis of bounded variation penalty methods for ill-posed problems. In: Inverse Problems, pp. 1217–1229 (1994)Google Scholar
  21. 21.
    Alvarez, L., Escalarín, J., Lefébure, M., Sánchez, J.: A pde model for computing the optical flow. In: CEDYA XVI, Universidad de Las Palmas de Gran Canaria, pp. 1349–1356 (1999)Google Scholar
  22. 22.
    Perona P., Malik J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12, 629–639 (1990)CrossRefGoogle Scholar
  23. 23.
    Ralli, J., Díaz, J., Ros, E., Ilonen, J., Kyrki, V.: External constraints in variational disparity calculation: hypothesis-forming-validation-loops and segmentation. Mach. Vis. Appl. (2011, in press)Google Scholar
  24. 24.
    Faugeras, O.D.: What can be seen in three dimensions with an uncalibrated stereo rig. In: ECCV ’92: Proceedings of the Second European Conference on Computer Vision, pp. 563–578 (1992)Google Scholar
  25. 25.
    Hartley, R.I.: Estimation of relative camera positions for uncalibrated cameras. In: ECCV ’92: Proceedings of the Second European Conference on Computer Vision, pp. 579–587 (1992)Google Scholar
  26. 26.
    Wedel, A., Pock, T., Braun, J., Franke, U., Cremers, D.: Duality tv-l1 flow with fundamental matrix prior. In: IVCNZ08, pp. 1–6 (2008)Google Scholar
  27. 27.
    Valgaerts, L., Bruhn, A., Weickert, J.: A variational model for the joint recovery of the fundamental matrix and the optical flow. In: Proceedings of the 30th DAGM symposium on Pattern Recognition, pp. 314–324 (2008)Google Scholar
  28. 28.
    Blake A., Zisserman A.: Visual Reconstruction. The MIT Press, Cambridge (1987)Google Scholar
  29. 29.
    Bruhn A., Weickert J., Kohlberger T., Schnörr C.: A multigrid platform for real-time motion computation with discontinuity-preserving variational methods. Int. J. Comput. Vis. 70(3), 257–277 (2003)CrossRefGoogle Scholar
  30. 30.
    Trottenberg U., Oosterlee C., Schüller A.: Multigrid. Academic Press, San Diego (2001)zbMATHGoogle Scholar
  31. 31.
    Barron J.L., Fleet D.J., Beauchemin S.S.: Performance of optical flow techniques. Int. J. Comput. Vis. 12, 43–77 (1994)CrossRefGoogle Scholar
  32. 32.
    Fischler M.A., Bolles R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  1. 1.Departamento de Arquitectura y Tecnología de Computadores, Escuela Técnica Superior de Ingeniería Informatica y de TelecomunicacíonUniversidad de GranadaGranadaSpain

Personalised recommendations