Skip to main content

Computing optical flow across multiple scales: An adaptive coarse-to-fine strategy

Abstract

Single-scale approaches to the determination of the optical flow field from the time-varying brightness pattern assume that spatio-temporal discretization is adequate for representing the patterns and motions in a scene. However, the choice of an appropriate spatial resolution is subject to conflicting, scene-dependent, constraints. In intensity-base methods for recovering optical flow, derivative estimation is more accurate for long wavelengths and slow velocities (with respect to the spatial and temporal discretization steps). On the contrary, short wavelengths and fast motions are required in order to reduce the errors caused by noise in the image acquisition and quantization process.

Estimating motion across different spatial scales should ameliorate this problem. However, homogeneous multiscale approaches, such as the standard multigrid algorithm, do not improve this situation, because an optimal velocity estimate at a given spatial scale is likely to be corrupted at a finer scale. We propose an adaptive multiscale method, where the discretization scale is chosen locally according to an estimate of the relative error in the velocity estimation, based on image properties.

Results for synthetic and video-acquired images show that our coarse-to-fine method, fully parallel at each scale, provides substantially better estimates of optical flow than do conventional algorithms, while adding little computational cost.

This is a preview of subscription content, access via your institution.

References

  1. Adelson E.H., and Bergen J.R. 1985. Spatio-temporal energy models for the perception of motion. J. Opt. Soc. Amer. A 2: 284–299.

    Google Scholar 

  2. Battiti, R. 1989. Surface reconstruction and discontinuity detection: a fast hierarchical approach on a two-dimensional mesh. Proc. 4th Conf. Hypercube Concurrent Computers and Applications, Monterey, CA.

  3. Battiti, R. 1990. Multiscale methods, parallel computation and neural networks for real-time computer vision. Ph.D. Dissertation, California Institute of Technology.

  4. Brandt A. 1977. Multi-level adaptive solutions to boundary-value problems. Mathematics of Computations 31: 333–390.

    Google Scholar 

  5. Bülthoff H.H., Little J.J., and Poggio T. 1989. Parallel computation of motion: computation, psychophysics and physiology. Nature 337: 549–553.

    Google Scholar 

  6. Burt, P.J. 1984. The pyramid as a structure for efficient computation. In Multiresolution Image Processing and Analysis, Rosenfeld, A. (ed.), Springer-Verlag, pp. 6–35.

  7. Enkelmann W. 1988. Investigations of multigrid algorithms for the estimation of optical flow fields in image sequences. Comput. Vision, Graph. Image Process. 43: 150–177.

    Google Scholar 

  8. Fennema C.L. and Thompson W.B. 1979. Velocity determination in scenes containing several moving objects. Comput. Graph. Image Process. 9: 301–315.

    Google Scholar 

  9. Geman S., and Geman D. 1984. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Patt. Anal. Mach. Intell. PAMI-6: 721–741.

    Google Scholar 

  10. Girosi, F., Verri, A., and Torre, V. 1989. Constraints for the computation of the optical flow. Proc. IEEE Workshop on Visual Motion, Irvine, CA, March, pp. 116–124.

  11. Glazer, F. 1984. Multilevel relaxation in low-level computer vision. In Multiresolution Image Processing and Analysis, A. Rosenfeld (ed.). Springer-Verlag, pp. 312–330.

  12. Harris J., Koch C., Staats E., and Luo J. 1990. Analog hardware for detecting discontinuities in early vision. Intern. J. Comput. Vision 4: 211–223.

    Google Scholar 

  13. Hassenstein B., and Reichardt W. (1956). Systemtheoretische Analyse der Zeit, Reihenfolgen, und Vorzeichenauswertung bei der Bewegungsperzepion des Rüsselkäfers Chlorophanus. Z. Naturforsch. 11b: 513–524.

    Google Scholar 

  14. Hildreth E.C. 1984. Computations underlying the measurement of visual motion. Artificial Intelligence 23: 309–354.

    Google Scholar 

  15. Hildreth E., and Koch C. 1987. The analysis of visual motion: from computational theory to neuronal mechanisms. Annu. Rev. Neurosci. 10: 477–533.

    Google Scholar 

  16. Horn B.K.P., and Schunck G. 1981. Determining optical flow. Artificial Intelligence 17: 185–203.

    Google Scholar 

  17. Hutchinson J., Koch C., Luo J., and Mead C. 1988. Computing motion using analog and binary resistive networks. IEEE Computer 21: 52–61.

    Google Scholar 

  18. Kamgar-Parsi B., and Kamgar-Parsi B. 1989. Evaluation of quantization error in computer vision. IEEE Trans Patt. Anal. Mach. Intell. PAMI-11: 929–940.

    Google Scholar 

  19. Kearney J.K., Thompson W.B., and Boley D.L. 1984. Optical flow estimation: an error analysis of gradient-based methods with local optimization. IEEE Trans. Patt. Anal. Mach. Intell. PAMI-9: 229–244.

    Google Scholar 

  20. Koch C., Wang H.T., and Mathur B. 1989. Computing motion in the primate visual system. J. Exper. Biol. 146: 115–139.

    Google Scholar 

  21. Little, J., and Verri, A. 1989. Analysis of differential and matching methods for optical flow. Proc. IEEE Workshop Visual Motion, Irvine CA, March, pp. 173–180.

  22. Marr D., and Ullman S. 1981. Directional selectivity and its use in early visual processing. Proc. Roy. Soc. London B 211: 151–180.

    Google Scholar 

  23. Marroquin J. 1984. Surface reconstruction preserving discontinuities. M.I.T. Artif. Intell. Lab. Memo 792, MIT: Cambridge, MA.

    Google Scholar 

  24. Maunsell J.H.R., and Van Essen D.C. 1983. Functional properties of neurons in middle temporal visual area of the macaque monkey. I. Selectivity for stimulus direction, speed and orientation. J. Neurophysiol. 49: 1127–1147.

    Google Scholar 

  25. Nagel, H.H. 1978. Analysis techniques for image sequences. Proc. 4th Intern. Joint Conf. Patt. Recog., Kyoto, Japan, November.

  26. Nagel H.H., and Enkelmann W. 1986. An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Trans. Patt. Anal. Mach. Intell. PAMI-8 (5): 565–593.

    Google Scholar 

  27. Poggio T., and Reichardt W., 1973. Considerations on models of movement detection. Kybernetik 13: 223–227.

    Google Scholar 

  28. Poggio T., Gamble E.B., and Little J.J. 1988. Parallel integration of vision modules. Science 242: 436–440.

    Google Scholar 

  29. Poggio T., Torre V., and Koch C. 1985. Computational vision and regularization theory. Nature 317: 314–419.

    Google Scholar 

  30. Reichardt W., Schlögel R.W., and Egelhaaf M. 1988. Movement detectors of the correlation type provide sufficient information for local computation of 2-D velocity field. Naturwissenschaften 75: 313–315.

    Google Scholar 

  31. Terzopoulos D. 1986. Image analysis using multigrid relaxation methods. IEEE Trans. Patt. Anal. Mach. Intell. PAMI-8: 129–139.

    Google Scholar 

  32. Ullman S. 1981. Analysis of visual motion by biological and computer systems. IEEE Computer, 14: 57–69.

    Google Scholar 

  33. Cras S., Girosi F., Verri A., and Torre V. 1988. A computational approach to motion perception. Biological Cybernetics 60: 79–87.

    Google Scholar 

  34. vanSanten J.P.H., and Sperling G. 1984. A temporal covariance model of motion perception. J. Opt. Soc. Amer. A 1: 451–473.

    Google Scholar 

  35. Verri A., and Poggio T. 1989. Motion field and optical flow: qualitative properties. IEEE Trans. Patt. Anal. Mach. Intell. PAMI-11: 490–498.

    Google Scholar 

  36. Wang H.T., Mathur B., and Koch C. 1989. Computing optical flow in the primate visual system. Neural Computation, 1: 92–103.

    Google Scholar 

  37. Wang H.T., Mathur B., and Koch C. 1991. A multiscale adaptive network model of motion computation in primates. In: Advances in Neural Information Processing Systems, Touretzky D.S. and Lippman R., eds., Morgan Kaufmann, San Mateo, in press.

    Google Scholar 

  38. Watson A.B., and Ahumada A.J. 1985. Model of human visual-motion sensing. J. Opt. Soc. Amer. A 2: 322–341.

    Google Scholar 

  39. Yuille A.L., and Grzywacz N.M. 1988. A computational theory for the perception of coherent visual motion. Nature 333: 71–73.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Battiti, R., Amaldi, E. & Koch, C. Computing optical flow across multiple scales: An adaptive coarse-to-fine strategy. Int J Comput Vision 6, 133–145 (1991). https://doi.org/10.1007/BF00128153

Download citation

Keywords

  • Optical Flow
  • Velocity Estimate
  • Discretization Step
  • Quantization Process
  • Multiscale Method