International Journal of Computer Vision

, Volume 1, Issue 4, pp 279–302 | Cite as

Optical flow using spatiotemporal filters

  • David J. Heeger


A model is presented, consonant with current views regarding the neurophysiology and psychophysics of motion perception, that combines the outputs of a set of spatiotemporal motion-energy filters to estimate image velocity. A parallel implementation computes a distributed representation of image velocity. A measure of image-flow uncertainty is formulated; preliminary results indicate that this uncertainty measure may be used to recognize ambiguity due to the aperture problem. The model appears to deal with the aperture problem as well as the human visual system since it extracts the correct velocity for some patterns that have large differences in contrast at different spatial orientations.


Image Processing Artificial Intelligence Computer Vision Visual System Computer Image 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    S.T., Barnard and W.B., Thomson, “Disparity analysis of images,” IEEE Trans. Pami-2(4), pp. 333–340, 1980.Google Scholar
  2. 2.
    B.K.P., Horn and B.G., Schunk, “Determining optical flow,” Artificial Intelligence, vol. 17; pp. 185–203, 1981.Google Scholar
  3. 3.
    J.K., Kearney and W.B., Thompson, “An error analysis of gradient-based methods for optical flow estimation,” IEEE Trans. Pami-9(2), pp. 229–244, 1987.Google Scholar
  4. 4.
    H., Gafni and Y., Zeevi, “A model for separation of spatial and temporal information in the visual system,” Biological Cybernetics, vol. 28; pp. 73–82, 1977.Google Scholar
  5. 5.
    H., Gafni and Y., Zeevi, “A model for processing of movement in the visual system,” Biological Cybernetics, vol. 32; pp. 165–173, 1979.Google Scholar
  6. 6.
    M., Fahle and T., Poggio, “Visual hyperacuity: Spatiotemporal interpolation in human vision,” Proc. R. Soc. (London), vol. 213; pp. 451–477, 1981.Google Scholar
  7. 7.
    A.B. Watson and A.J. Ahumada, “A look at motion in the frequency domain,” Tech. Rep. 84352, NASA-Ames Research Center, 1983.Google Scholar
  8. 8.
    A.B., Watson and A.J., Ahumada, “Model of human visual-motion sensing,” J. Opt. Soc. Amer. vol. A 2(2); pp. 322–342, 1985.Google Scholar
  9. 9.
    E.H., Adelson and J.R., Bergen, “Spatiotemporal energy models for the perception of motion,” J. Opt. Soc. Amer., vol. A2(2), pp. 284–299, 1985.Google Scholar
  10. 10.
    J.P.H.van, Santen and G., Sperling, “Elaborated reichardt detectors,” J. Opt. Soc. Amer., vol. A2(2); pp. 300–321, 1985.Google Scholar
  11. 11.
    D.J. Fleet, The early processing of spatio-temporal visual information, Master's thesis, Dept. of Computer Science, Univ. of Toronto, 1984. (available as Tech. Report RBCVTR-84-7.)Google Scholar
  12. 12.
    D.J. Fleet and A.D. Jepson, “A cascaded filter approach to the construction of velocity selective mechanisms,” Tech. Report RBCV-TR-84-6, Dept. of Computer Science, Univ. Toronto, 1984.Google Scholar
  13. 13.
    E.C., Hildreth, “Computations underlying the measurement of visual motion,” Artificial Intelligence, vol. 23(3); pp. 309–355, 1984.Google Scholar
  14. 14.
    D., Gabor, “Theory of communication,” J. IEE (London), vol. 93; pp. 429–457, 1946.Google Scholar
  15. 15.
    J.G., Daugman, “Two-dimensional analysis of cortical receptive field profiles,” Vision Research, vol. 20; pp. 846–856, 1980.Google Scholar
  16. 16.
    J.G., Daugman, “Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters,” J. of the Opt. Soc. of Amer., vol. A2(7); pp. 1160–1169, 1985.Google Scholar
  17. 17.
    David J., Heeger, “A model for the extraction of image flow,” J. Opt. Soc. Amer., vol A4(8); pp. 1455–1471, 1987.Google Scholar
  18. 18.
    David J. Heeger, “Models for motion perception,” Ph.D. thesis, CIS Department, Univ. of Pennsylvania, 1987. (Available as technical report MS-CIS-87-91.)Google Scholar
  19. 19.
    S.G., Mallat, “Scale change versus scale space representation,” in Proc. First Int Conf. on Computer Vision, pp. 592–596, IEEE, London, 1987.Google Scholar
  20. 20.
    P., Burt, “Fast algorithms for estimating local image properties,” Computer Vision, Graphics, and Image Processing, vol. 21; pp. 368–382, 1983.Google Scholar
  21. 21.
    D.C., Burr and J., Ross, “Contrast sensitivity at high velocities,” Vision Research, vol. 22; pp. 479–484, 1982.Google Scholar
  22. 22.
    P.E., Gill, W., Murray, and M.H., Wright, Practical Optimization. Academic Press: New York, 1981.Google Scholar
  23. 23.
    R.A., Hummel and S.W., Zucker, “On the foundations of relaxing labelling processes,” IEEE Pami-5(3); pp. 267–287, 1983.Google Scholar
  24. 24.
    D., Terzopoulos, “Regularization of inverse visual problems involving discontinuities,” IEEE Pami-8(4), pp. 413–424, 1986.Google Scholar
  25. 25.
    T., Poggio, V., Torre, and C., Koch, “Computational vision and regularization theory,” Nature, vol. 317(6035); pp. 314–319, 1985.Google Scholar
  26. 26.
    D., Marr and E., Hildreth, “Theory of edge detection,” Proc. Roy. Soc. (London), vol. B207; pp. 187–217, 1980.Google Scholar
  27. 27.
    D.E., Rummelhart and J.L., McClelland, eds., Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Mit Press: Cambridge, Mass., 1986.Google Scholar
  28. 28.
    B.B., Mandlebrot, The Fractal Geometry of Nature. W.H. Freeman: New York, 1983.Google Scholar
  29. 29.
    M.H., DeGroot, Probability and Statistics. Addison-Wesley: Menlo Park, Calif., 1975.Google Scholar
  30. 30.
    J., Melsa and D., Cohn, Decision and Estimation Theory. McGraw-Hill: New York, 1978.Google Scholar
  31. 31.
    E.H., Adelson and J.A., Movshon, “Phenomenal coherence of moving visual patterns,” Nature vol. 300(5892); pp. 523–525, 1982.Google Scholar
  32. 32.
    E.H. Adelson, Media-Technology Laboratory, MIT, personal communication.Google Scholar
  33. 33.
    M.P., doCarmo, Differential Geometry of Curves and Surfaces. Prentice-Hall: Englewood Cliffs, N.J., 1976.Google Scholar
  34. 34.
    E.H. Adelson and E. Simonelli, “Orthogonal pyramid transfers for image coding,” in PROC. SPIE, VISUAL COMMUN. and IMAGE PROC. II, pp. 50–58, Cambridge, MA, 1987.Google Scholar

Copyright information

© Kluwer Academic Publishers 1987

Authors and Affiliations

  • David J. Heeger
    • 1
  1. 1.Vision Sciences Group, Media LaboratoryMassachusetts Institute of TechnologyCambridge

Personalised recommendations