Abstract
Single-scale approaches to the determination of the optical flow field from the time-varying brightness pattern assume that spatio-temporal discretization is adequate for representing the patterns and motions in a scene. However, the choice of an appropriate spatial resolution is subject to conflicting, scene-dependent, constraints. In intensity-base methods for recovering optical flow, derivative estimation is more accurate for long wavelengths and slow velocities (with respect to the spatial and temporal discretization steps). On the contrary, short wavelengths and fast motions are required in order to reduce the errors caused by noise in the image acquisition and quantization process.
Estimating motion across different spatial scales should ameliorate this problem. However, homogeneous multiscale approaches, such as the standard multigrid algorithm, do not improve this situation, because an optimal velocity estimate at a given spatial scale is likely to be corrupted at a finer scale. We propose an adaptive multiscale method, where the discretization scale is chosen locally according to an estimate of the relative error in the velocity estimation, based on image properties.
Results for synthetic and video-acquired images show that our coarse-to-fine method, fully parallel at each scale, provides substantially better estimates of optical flow than do conventional algorithms, while adding little computational cost.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adelson E.H., and Bergen J.R. 1985. Spatio-temporal energy models for the perception of motion. J. Opt. Soc. Amer. A 2: 284–299.
Battiti, R. 1989. Surface reconstruction and discontinuity detection: a fast hierarchical approach on a two-dimensional mesh. Proc. 4th Conf. Hypercube Concurrent Computers and Applications, Monterey, CA.
Battiti, R. 1990. Multiscale methods, parallel computation and neural networks for real-time computer vision. Ph.D. Dissertation, California Institute of Technology.
Brandt A. 1977. Multi-level adaptive solutions to boundary-value problems. Mathematics of Computations 31: 333–390.
Bülthoff H.H., Little J.J., and Poggio T. 1989. Parallel computation of motion: computation, psychophysics and physiology. Nature 337: 549–553.
Burt, P.J. 1984. The pyramid as a structure for efficient computation. In Multiresolution Image Processing and Analysis, Rosenfeld, A. (ed.), Springer-Verlag, pp. 6–35.
Enkelmann W. 1988. Investigations of multigrid algorithms for the estimation of optical flow fields in image sequences. Comput. Vision, Graph. Image Process. 43: 150–177.
Fennema C.L. and Thompson W.B. 1979. Velocity determination in scenes containing several moving objects. Comput. Graph. Image Process. 9: 301–315.
Geman S., and Geman D. 1984. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Patt. Anal. Mach. Intell. PAMI-6: 721–741.
Girosi, F., Verri, A., and Torre, V. 1989. Constraints for the computation of the optical flow. Proc. IEEE Workshop on Visual Motion, Irvine, CA, March, pp. 116–124.
Glazer, F. 1984. Multilevel relaxation in low-level computer vision. In Multiresolution Image Processing and Analysis, A. Rosenfeld (ed.). Springer-Verlag, pp. 312–330.
Harris J., Koch C., Staats E., and Luo J. 1990. Analog hardware for detecting discontinuities in early vision. Intern. J. Comput. Vision 4: 211–223.
Hassenstein B., and Reichardt W. (1956). Systemtheoretische Analyse der Zeit, Reihenfolgen, und Vorzeichenauswertung bei der Bewegungsperzepion des Rüsselkäfers Chlorophanus. Z. Naturforsch. 11b: 513–524.
Hildreth E.C. 1984. Computations underlying the measurement of visual motion. Artificial Intelligence 23: 309–354.
Hildreth E., and Koch C. 1987. The analysis of visual motion: from computational theory to neuronal mechanisms. Annu. Rev. Neurosci. 10: 477–533.
Horn B.K.P., and Schunck G. 1981. Determining optical flow. Artificial Intelligence 17: 185–203.
Hutchinson J., Koch C., Luo J., and Mead C. 1988. Computing motion using analog and binary resistive networks. IEEE Computer 21: 52–61.
Kamgar-Parsi B., and Kamgar-Parsi B. 1989. Evaluation of quantization error in computer vision. IEEE Trans Patt. Anal. Mach. Intell. PAMI-11: 929–940.
Kearney J.K., Thompson W.B., and Boley D.L. 1984. Optical flow estimation: an error analysis of gradient-based methods with local optimization. IEEE Trans. Patt. Anal. Mach. Intell. PAMI-9: 229–244.
Koch C., Wang H.T., and Mathur B. 1989. Computing motion in the primate visual system. J. Exper. Biol. 146: 115–139.
Little, J., and Verri, A. 1989. Analysis of differential and matching methods for optical flow. Proc. IEEE Workshop Visual Motion, Irvine CA, March, pp. 173–180.
Marr D., and Ullman S. 1981. Directional selectivity and its use in early visual processing. Proc. Roy. Soc. London B 211: 151–180.
Marroquin J. 1984. Surface reconstruction preserving discontinuities. M.I.T. Artif. Intell. Lab. Memo 792, MIT: Cambridge, MA.
Maunsell J.H.R., and Van Essen D.C. 1983. Functional properties of neurons in middle temporal visual area of the macaque monkey. I. Selectivity for stimulus direction, speed and orientation. J. Neurophysiol. 49: 1127–1147.
Nagel, H.H. 1978. Analysis techniques for image sequences. Proc. 4th Intern. Joint Conf. Patt. Recog., Kyoto, Japan, November.
Nagel H.H., and Enkelmann W. 1986. An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Trans. Patt. Anal. Mach. Intell. PAMI-8 (5): 565–593.
Poggio T., and Reichardt W., 1973. Considerations on models of movement detection. Kybernetik 13: 223–227.
Poggio T., Gamble E.B., and Little J.J. 1988. Parallel integration of vision modules. Science 242: 436–440.
Poggio T., Torre V., and Koch C. 1985. Computational vision and regularization theory. Nature 317: 314–419.
Reichardt W., Schlögel R.W., and Egelhaaf M. 1988. Movement detectors of the correlation type provide sufficient information for local computation of 2-D velocity field. Naturwissenschaften 75: 313–315.
Terzopoulos D. 1986. Image analysis using multigrid relaxation methods. IEEE Trans. Patt. Anal. Mach. Intell. PAMI-8: 129–139.
Ullman S. 1981. Analysis of visual motion by biological and computer systems. IEEE Computer, 14: 57–69.
Cras S., Girosi F., Verri A., and Torre V. 1988. A computational approach to motion perception. Biological Cybernetics 60: 79–87.
vanSanten J.P.H., and Sperling G. 1984. A temporal covariance model of motion perception. J. Opt. Soc. Amer. A 1: 451–473.
Verri A., and Poggio T. 1989. Motion field and optical flow: qualitative properties. IEEE Trans. Patt. Anal. Mach. Intell. PAMI-11: 490–498.
Wang H.T., Mathur B., and Koch C. 1989. Computing optical flow in the primate visual system. Neural Computation, 1: 92–103.
Wang H.T., Mathur B., and Koch C. 1991. A multiscale adaptive network model of motion computation in primates. In: Advances in Neural Information Processing Systems, Touretzky D.S. and Lippman R., eds., Morgan Kaufmann, San Mateo, in press.
Watson A.B., and Ahumada A.J. 1985. Model of human visual-motion sensing. J. Opt. Soc. Amer. A 2: 322–341.
Yuille A.L., and Grzywacz N.M. 1988. A computational theory for the perception of coherent visual motion. Nature 333: 71–73.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Battiti, R., Amaldi, E. & Koch, C. Computing optical flow across multiple scales: An adaptive coarse-to-fine strategy. Int J Comput Vision 6, 133–145 (1991). https://doi.org/10.1007/BF00128153
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF00128153