Advertisement

International Journal of Computer Vision

, Volume 2, Issue 3, pp 283–310 | Cite as

A computational framework and an algorithm for the measurement of visual motion

  • P. Anandan
Article

Abstract

The robust measurement of visual motion from digitized image sequences has been an important but difficult problem in computer vision. This paper describes a hierarchical computational framework for the determination of dense displacement fields from a pair of images, and an algorithm consistent with that framework. Our framework is based on a scale-based separation of the image intensity information and the process of measuring motion. The large-scale intensity information is first used to obtain rough estimates of image motion, which are then refined by using intensity information at smaller scales. The estimates are in the form of displacement (or velocity) vectors for pixels and are accompanied by a direction-dependent confidence measure. A smoothness constraint is employed to propagate measurements with high confidence to neighboring areas where the confidences are low. At all levels, the computations are pixel-parallel, uniform across the image, and based on information from a small neighborhood of a pixel. Results of applying our algorithm to pairs of real images are included. In addition to our own matching algorithm, we also show that two different hierarchical gradient-based algorithms are consistent with our framework.

Keywords

Computer Vision Image Sequence Displacement Field Image Intensity Small Neighborhood 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    E.H.Adelson and J.R.Bergen, “Spatiotemporal energy models for the perception of motion”, J. Opt. Soc. Am. A, 2(2):284–299, 1985.Google Scholar
  2. 2.
    G.Adiv, “Determining 3-D motion and structure from optical flows generated by several moving objects”, IEEE Trans. PAMI 7(4):384–401, 1985.Google Scholar
  3. 3.
    G. Adiv, “Inherent ambiguities in recovering 3-D motion and structure from a noisy flow field”, Proc. Conf. Comput. Vision and Pattern Recognition, pp. 70–77, 1985.Google Scholar
  4. 4.
    J.K.Aggarwal, L.S.Davis, and W.N.Martin, “Correspondence processes in dynamic scene analysis”, Proc. IEEE 69(5):562–572, 1981.Google Scholar
  5. 5.
    P.Anandan, “Computing dense displacement fields with confidence measures in scenes containing occlusion”, SPIE Int. Conf. Robots and Comput. Vision, 521:184–194, 1984; also as COINS Tech. Rept. 84–32, Univ. of Massachusetts, December 1984.Google Scholar
  6. 6.
    P. Anandan, “Measuring visual motion from image sequences”, PhD dissertation, COINS TR 87-21, Univ. of Massachusetts, Amherst, MA, 1987.Google Scholar
  7. 7.
    P. Anandan and R. Weiss, “Introducing a smoothness constraint in a matching approach for the computation of displacement fields”, Proc. DARPA IU Workshop pp. 186–196, 1985.Google Scholar
  8. 8.
    S.T.Barnard and W.B.Thompson, “Disparity analysis of images”, IEEE Trans. PAMI 2(4):333–340, 1980.Google Scholar
  9. 9.
    P. Beaudet, “Rotationally invariant image operators”, Proc. Int. Conf. Pattern Recognition, pp. 579–583, 1978.Google Scholar
  10. 10.
    M.Boldt and R.Weiss, “Token-based extraction of straight lines”, COINS TR 87–104, Univ. of Massachusetts, Amherst, MA, 1987.Google Scholar
  11. 11.
    P.J.Burt, “Fast filter transforms for image processing”, Comput. Graphics Image Processing 16:20–51, 1981.Google Scholar
  12. 12.
    P.J.Burt, T.H.Hong, and A.Rosenfeld, “Image segmentation and region property computation by cooperative hierarchical computation”, IEEE Trans. Syst., Man, Cybern. 11:802–809, 1981.Google Scholar
  13. 13.
    P.J. Burt, C. Yen, and X. Xu, “Local correlation measures for motion analysis: A comparative study”, Proc. IEEE Conf. Pattern Recognition Image Processing. pp. 269–274, 1982.Google Scholar
  14. 14.
    P.J. Burt, C. Yen, and X. Xu, “Multiresolution flow-through motion analysis”, Proc. IEEE Conf. Comput. Vision Pattern Recognition, pp. 246–252, 1983.Google Scholar
  15. 15.
    M.P.doCarmo, Differential Geometry of Curves and Surfaces. Prentice-Hall: Englewood Cliffs, NJ, 1976.Google Scholar
  16. 16.
    J.L.Crowley and R.M.Stern, “Fast computations of the difference of low-pass transform”, IEEE Trans. PAMI 6:212–222, 1984.Google Scholar
  17. 17.
    W. Enkelmann, “Investigations of multigrid algorithms for the estimation of optical flow fields in image sequences”, Proc. Workshop Motion: Representation and Control, Kiawah Island, SC, 1986, pp. 81–87.Google Scholar
  18. 18.
    J.Fang and T.S.Huang, “Some experiments on estiamting the 3-D motion parameters of a rigid body from two consecutive image frames”, IEEE Trans. PAMI 6(5):545–554, 1984.Google Scholar
  19. 19.
    S.Geman and D.Geman, “Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images”, IEEE Trans. PAMI 6(6):721–741, 1984.Google Scholar
  20. 20.
    D. Gennery, “Modeling the environment of an exploring vehicle by means of stereo vision”, PhD dissertation, Standard Artificial Intelligence Laboratory, AIM-339, 1980.Google Scholar
  21. 21.
    F. Glazer, “Hierarchical motion detection”. PhD dissertation, COINS TR 87-02, Univ. of Massachusetts, Amherst, MA, 1987.Google Scholar
  22. 22.
    F. Glazer, G. Reynolds, and P. Anandan, “Scene matching by hierarchical correlation”, Proc. IEEE Conf. Comput. Vision and Pattern. Recognition, Annapolis, MD, pp. 432–441, 1983.Google Scholar
  23. 23.
    W.E.L.Grimson, “Computational experiments with a feature based stereo algorithm”, IEEE Trans. PAMI 7(1):17–34, 1985.Google Scholar
  24. 24.
    A.R.Hanson and E.M.Riseman, “Processing cones: A computational structure for image analysis”. In Structured Computer Vision, S.Tanimato and A.Klinger (eds.), Academic Press: New York, 1980.Google Scholar
  25. 25.
    D. Heeger, “Optical flow from spatiotemporal filters”, Proc. 1st Int. Conf. Comput. Vision, London, UK, pp. 181–190, 1987.Google Scholar
  26. 26.
    E.C.Hildreth, The Measurement of Visual Motion. MIT Press: Cambridge, MA, 1984.Google Scholar
  27. 27.
    B.K.P.Horn and B.G.Schunck, “Determining Optical Flow”, Artificial Intelligence 17:185–203, 1981.Google Scholar
  28. 28.
    A.Klinger and R.D.Dyer, “Experiments on picture representations using regular decomposition”, Comput. Graphics Image Processing 5(1):68–105, 1976.Google Scholar
  29. 29.
    B.D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision”, Proc. 7th Int. Joint Conf. Artif. Intell., Vancouver, Canada, pp. 674–679, 1981.Google Scholar
  30. 30.
    D. Marr and T. Poggio, “A computational theory of human stereo vision”, Proc. Roy. Soc. London B-204, pp. 301–308, 1979.Google Scholar
  31. 31.
    J. Marroquin, S. Mitter, and T. Poggio, “Probabilistic solutions for ill-posed problems in computational vision”, Proc. DARPA IU Workshop, Miami Beach, FL, pp. 293–309, 1986.Google Scholar
  32. 32.
    J.E.W.Mayhew and J.P.Frisby, “Psychophysical and computational studies towards a theory of human stereopsis”, Artificial Intelligence 17:349–385, 1981.Google Scholar
  33. 33.
    H.H.Nagel, “Displacement vectors derived from second order intensity variations in image sequences”, Comput. Vision Pattern Recognition Image Processing 21:85–117, 1983.Google Scholar
  34. 34.
    H.H. Nagel, “Image sequences—ten (octal) years—from phenomenology towards a theoretical foundation”, Proc. 8th Int. Conf. Pattern Recognition, Paris, France, 1986.Google Scholar
  35. 35.
    H.H.Nagel and W.Enkelmann, “An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences”, IEEE Trans. PAMI 8:565–593, 1986.Google Scholar
  36. 36.
    A.Rosenfeld and A.C.Kak, Digital Picture Processing. Academic Press: New York, 1976.Google Scholar
  37. 37.
    S.L.Tanimato and T.Pavlidis, “A hierarchical data structure for picture processing”, Comput. Graphics Image Processing 4(2):104–119, 1975.Google Scholar
  38. 38.
    D. Terzopoulos, “Multiresolution computation of visible-surface representations”, PhD dissertation, Massachusetts Institute of Technology, 1984.Google Scholar
  39. 39.
    S. Ullman, “Analysis of visual motion by biological and computer systems”, IEEE Computer, pp. 57–69, 1981.Google Scholar
  40. 40.
    J.P.H.vanSanten and G.Sperling, “Elaborated Reichhardt detectors”, J. Opt. Soc. Amer. 2(7):300–321, 1985.Google Scholar
  41. 41.
    A.B. Watson and A.J. Ahmuda, “Model of human visual-motion sensing”, J. Opt. Soc. Amer. 2(7), 1985.Google Scholar
  42. 42.
    A. Waxman, “An image-flow paradigm”, Proc. Workshop Comput. Vision, Annapolis, MD, pp. 49–57, 1984.Google Scholar
  43. 43.
    A. Waxman and K. Wohn, “Contour evaluation, neighborhood deformation, and global image flow: planar surfaces in motion”, Univ. of Maryland Tech. Rept. CS-TR-1394, 1984.Google Scholar
  44. 44.
    R. Weiss and M. Boldt, “Geometric grouping applied to straight lines”, Proc. IEEE Conf. Comput. Vision Pattern Recognition, Miami Beach, FL, pp. 489–493, 1986.Google Scholar
  45. 45.
    L.R. Williams and P. Anandan, “A coarse-to-fine control strategy for stereo and motion on a mesh-connected computer”, Proc. IEEE Conf. Comput. Vision Pattern Recognition, Miami Beach, FL, pp. 219–226, 1986.Google Scholar
  46. 46.
    L.R. Williams and A. Hanson, “Translating optical flow into token matching”, Proc. DARPA IU Workshop, Boston, MA, pp. 970–980, 1988.Google Scholar
  47. 47.
    R.Y.Wong and E.L.Hall, “Sequential hierarchical scene matching”, IEEE Trans. Comput. 27(4):359–366, 1978.Google Scholar

Copyright information

© Kluwer Academic Publishers 1989

Authors and Affiliations

  • P. Anandan
    • 1
  1. 1.Computer Science DepartmentYale UniversityNew HavenUSA

Personalised recommendations