International Journal of Computer Vision

, Volume 1, Issue 3, pp 239–258 | Cite as

Closed-form solutions to image flow equations for 3D structure and motion

  • Allen M. Waxman
  • Behrooz Kamgar-Parsi
  • Muralidhara Subbarao
Article

Abstract

A major source of three-dimensional (3D) information about objects in the world is available to the observer in the form of time-varying imagery. Relative motion between textured objects and the observer generates a time-varying optic array at the image, from which image motion of contours, edge fragments, and feature points can be extracted. These dynamic features serve to sample the underlying “image flow” field. New, closed-form solutions are given for the structure and motion of planar and curved surface patches from monocular image flow and its derivatives through second order. Both planar and curved surface solutions require at most, the solution of a cubic equation. The analytic solution for curved surface patches combines the transformation of Longuet-Higgins and Prazdny [25] with the planar surface solution of Subbarao and Waxman [43]. New insights regarding uniqueness of solutions also emerge. Thus, the “structure-motion coincidence” of Waxman and Ullman [54] is interpreted as the “duality of tangent plane solutions.” The multiplicity of transformation angles (up to three) is related to the sign of the Gaussian curvature of the surface patch. Ovoid patches (i.e., bowls) are shown to possess a unique transform angle, though they are subject to the local structure-motion coincidence. Thus, ovoid patches almost always yield a unique 3D interpretation. In general, ambiguous solutions can be resolved by requiring continuity of the solution over time.

Keywords

Feature Point Curve Surface Planar Surface Gaussian Curvature Tangent Plane 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    E.H. Adelson and J.R. Bergen, “Spatiotemporal energy models for the perception of motion,” J. Opt. Soc. Amer. vol. A2, pp. 284–299, 1985.Google Scholar
  2. 2.
    G. Adiv, “Determining 3-D motion and structure from optical flow generated by several moving objects,” PROC. DARPA IMAGE UNDERSTANDING WORKSHOP, New Orleans: SAIC October 1984, pp. 113–129.Google Scholar
  3. 3.
    C.H. Anderson, P.J. Burt, and G.S. van der Wal, “Change detection and tracking using pyramid transform techniques,” in PROC. SPIE CONF. ON INTELLIGENT ROBOTS AND COMPUTER VISION, Boston, September, 1985.Google Scholar
  4. 4.
    R. Aris, Vectors, Tensors, and the Basic Equations of Fluid Mechanics. Prentice-Hall: Englewood Cliffs, 1962.Google Scholar
  5. 5.
    A. Bandyopadhyay, “Constraints on the computation of rigid motion parameters from retinal displacements,” University of Rochester, Dept. of Computer Science Tech. Report 168, October, 1985.Google Scholar
  6. 6.
    M.L. Braunstein, Depth Perception Through Motion. Academic Press: New York, 1976.Google Scholar
  7. 7.
    M.L. Braunstein, “Perception of rotation in depth: The psychophysical evidence,” in Proc. Workshop on Motion: Representation and Perception, Toronto: ACM, April, 1983, pp. 119–124.Google Scholar
  8. 8.
    B.F. Buxton, H. Buxton, D.W. Murray, and N.S. Williams, “3-D solutions to the aperture problem,” PROC. 6TH EUROPEAN CONF. ARTIF. INTEL., Pisa, 1984, pp. 631–640.Google Scholar
  9. 9.
    J. Doner, J.S. Lappin, and G. Perfetto, “Detection of three-dimensional structure in moving optical patterns,” J. Exp. Psychology: Human Perception and Performance, vol. 10, 1984, pp. 1–11.Google Scholar
  10. 10.
    D.J. Fleet and A.D. Jepson, “Velocity extraction without form perception,” in Proc. Workshop on Computer Vision: Representation and Control, Bellaire: IEEE, October, 1985, pp. 179–185.Google Scholar
  11. 11.
    J.J. Gibson. The Senses Considered as Perceptual Systems. Houghton Mifflin: Boston, 1966.Google Scholar
  12. 12.
    J.J. Gibson. The Ecological Approach to Visual Perception. Houghton Mifflin: Boston, 1979.Google Scholar
  13. 13.
    J.C. Hay, “Optical motions and space perception: An extension of Gibson's analysis,” Psychological Review vol. 73, pp. 550–565, 1966.Google Scholar
  14. 14.
    E.C. Hildreth, “The measurement of visual motion,” Ph.D. thesis, Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science; also “Computing the velocity field along contours,” in PROC. WORKSHOP ON MOTION: REPRESENTATION AND PERCEPTION, Toronto: ACM, April, 1983, pp. 26–32.Google Scholar
  15. 15.
    D.D. Hoffman and B.M. Bennett, “Inferring the relative three-dimensional positions of two moving points,” J. Opt. Soc. Amer. vol. A2, pp. 350–353, 1985.Google Scholar
  16. 16.
    B.K.P. Horn and B.G. Schunck, “Determining optical flow,” Artificial Intelligence vol. 17, pp. 185–203, 1981.Google Scholar
  17. 17.
    T.S. Huang, Image Sequence Processing and Dynamic Scene Analysis. Springer-Verlag: Berlin, 1983.Google Scholar
  18. 18.
    M.R.M. Jenkin, “The stereopsis of time-varying images,” Univ. of Toronto, Dept. of Computer Science Tech. Report RBCV-TR-83-3, September, 1984.Google Scholar
  19. 19.
    G. Johansson, “Visual motion perception.” Sci. Amer. vol. 232, pp. 76–88, 1975.Google Scholar
  20. 20.
    K. Kanatani, “Structure from motion without correspondence: General principle,” in Proc. Darpa Image Understanding Workshop, Miami: SAIC, December, 1985, pp. 107–116.Google Scholar
  21. 21.
    J.J. Koenderink and A.J. van Doorn, “Invariant properties of the motion parallax field due to the movement of rigid bodies relative to an observer,” Optica Acta vol. 22, pp. 773–791, 1975.Google Scholar
  22. 22.
    J.J. Koenderink and A.J. van Doorn, “Local structure of movement parallax of the plane,” J. Opt. Soc. Amer. vol. 66, pp. 717–723, 1976.Google Scholar
  23. 23.
    H.C. Longuet-Higgins, “A computer algorithm for reconstructing a scene from two projections,” Nature vol. 293, pp. 133–135, 1981.Google Scholar
  24. 24.
    H.C. Longuet-Higgins, “The visual ambiguity of a moving plane,” in PROC. ROY. SOC. LONDON B223, pp. 165–175, 1984.Google Scholar
  25. 25.
    H.C. Longuet-Higgins and K. Prazdny, “The interpretation of a moving retinal image,” in PROC. ROY. SOC. LONDON B208, pp. 385–397, 1980.Google Scholar
  26. 26.
    D. Marr, Vision. Freeman: San Francisco: 1982.Google Scholar
  27. 27.
    D. Marr and S. Ullman, “Directional selectivity and its use in early visual processing,” in PROC. ROY. SOC. LONDON B211, pp. 151–180, 1981.Google Scholar
  28. 28.
    S.J. Maybank, “The angular velocity associated with the optical flowfied arising from motion through a rigid environment,” in PROC. ROY. SOC. LONDON A401, pp. 317–326, 1985.Google Scholar
  29. 29.
    A.J. McConnell, Applications of Tensor Analysis. Dover Press: New York, 1957.Google Scholar
  30. 30.
    A. Mitiche, “On combining stereopsis and kineopsis for space perception; First Conf. Art. Intell. Applications. Denver: IEEE, December, 1984, pp. 156–160.Google Scholar
  31. 31.
    D.W. Murray and B.F. Buxton, “Scene segmentation from visual motion using global optimization,” IEEE Trans. Pat. Anal. Mach. Intel. vol. 9, pp. 220–228, 1987.Google Scholar
  32. 32.
    H.-H. Nagel, “On the estimation of dense displacement vector fields from image sequences,” in Proc. Workshop on Motion: Representation and Perception, Toronto: ACM, April, 1983, pp. 59–65.Google Scholar
  33. 33.
    H.-H. Nagel, “Dynamic stereo vision in a robot feedback loop based on the evaluation of multiple interconnected displacement vector fields,” in Proc. 3rd Int. Symp. Robotics Res., Gouvieux, MIT Press: Cambridge; October, 1985.Google Scholar
  34. 34.
    K. Prazdny, “Egomotion and relative depth map from optical flow,” Biological Cybernetics vol. 36, pp. 87–102, 1980.Google Scholar
  35. 35.
    D. Regan and K.I. Beverley, “Binocular and monocular stimuli for motion in depth: Changing-disparity and changing-size feed the same motion-in-depth stage,” Vision Research vol. 19, pp. 1331–1342, 1979.Google Scholar
  36. 36.
    D. Regan and K.I. Beverley, “Visually gudied locomotion: Psychophysical evidence for a neural mechanism sensitive to flow patterns,” Science vol. 205, pp. 311–313, 1979.Google Scholar
  37. 37.
    D. Regan and K.I. Beverley, “Visual responses to vorticity and the neural analysis of optic flow,” J. Opt. Soc. Amer. A2, pp. 280–283, 1985.Google Scholar
  38. 38.
    W. Richards, “Structure from stereo and motion,” M.I.T. Artif. Intell. Laboratory Memo 731, 1983; also J. OPT. SOC. AMER. A2, pp. 343–349, 1985.Google Scholar
  39. 39.
    B.G. Schunck, “Motion segmentation and estimation by constraint line clustering,” in Proc. Workshop on Computer Vision: Representation and Control, Annapolis: IEEE, April, 1984, pp. 58–62.Google Scholar
  40. 40.
    B.G. Schunck, “Image flow continuity equations for motion and density,” in Proc. of Workshop on Motion: Representation and Analysis, Charleston: ACM, May, 1986, pp. 89–94.Google Scholar
  41. 41.
    M. Subbarao, “Interpretation of visual motion: A computational study,” Ph.D. thesis, Univ. of Maryland, Dept. of Computer Science, 1986.Google Scholar
  42. 42.
    M. Subbarao, “Interpretation of image motion fields: Rigid curved surfaces in motion,” Univ. of Maryland, Cent. Autom. Res. Tech. Report; also submitted to INT. J. COMPUTER VISION, 1986.Google Scholar
  43. 43.
    M. Subbarao and A.M. Waxman, “On the uniqueness of image flow solutions for planar surfaces in motion,” in Proc. Workshop on Computer Vision: Representation and Control, Bellaire: IEEE, october, 1985, pp. 129–140; also in Computer Vision, Graphics and Image Processing vol. 36, pp. 208–228, 1986.Google Scholar
  44. 44.
    W.B. Thompson, K.M. Mutch and V.A. Berzins, “Dynamic occlusion analysis in optical flow fields,” IEEE Trans. Pami vol. 4, pp. 374–383, 1985.Google Scholar
  45. 45.
    J.T. Tood, “Perception of structure from motion: Is projective correspondence of moving elements a necessary condition?” J. Exp. Psychology: Human Perception and Performance vol. 11, pp. 689–710, 1985.Google Scholar
  46. 46.
    R.Y. Tsai and T.S. Huang, “Uniqueness and estimation of 3-D motion parameters of rigid objects with curved surfaces,” Univ. of Illinois, Coord. Sci. Lab. Report R-921, 1981.Google Scholar
  47. 47.
    R.Y. Tsai and T.S. Huang, “Estimating 3-D motion parameters of a rigid planar patch,” Univ. of Illinois, Coord. Sci. Lab. Report R-922, 1981.Google Scholar
  48. 48.
    S. Ullman, The Interpretation of Visual Motion. MIT Press: Cambridge; 1979.Google Scholar
  49. 49.
    S. Ullman, “Maximizing rigidity: the incremental recovery of structure from rigid and rubbery motion,” M.I.T. Artif. Intell. Lab. Memo 721, 1982.Google Scholar
  50. 50.
    H. Wallach and D.N. O'Connell, “The kinetic depth effect,” J. Exp. Psychology vol. 45, pp. 205–217, 1953.Google Scholar
  51. 51.
    A.M. Waxman, “An image flow paradigm,” in Proc. Workshop on Computer Vision: Representation and Control, Annapolis: IEEE, April, 1984, pp. 49–57.Google Scholar
  52. 52.
    A.M. Waxman and J.H. Duncan, ”Binocular image flows: Steps toward stereo-motion fusion,” Univ. of Maryland, Cent. Autom. Res. Tech. Report 119, October, 1985; also IEEE TRANS PAMI vol. 8, pp. 715–729, 1986.Google Scholar
  53. 53.
    A.M. Waxman and S. Sinha, “Dynamic Stereo: Passive ranging to moving objects from relative image flows,” in Proc. Darpa Image Understanding Workshop, New Orleans: SAIC, October, 1984, pp. 130–136.Google Scholar
  54. 54.
    A.M. Waxman, A.M. and S. Ullman, “Surface structure and 3-D motion from image flow: A kinematic analysis,” Univ. of Maryland, Cent. Autom. Res. Tech. Report 24, October, 1983; also INT. J. ROBOTICS RES. vol. 4(3), pp. 72–94, 1985.Google Scholar
  55. 55.
    A.M. Waxman and K. Wohn, “Contour evolution, neighborhood deformation and global image flow: Planar surfaces in motion,” Univ. of Maryland, Cent. Autom. Res. Tech. Report 58, April, 1984; also INT. J. ROBOTICS RES. vol. 4(3), pp. 95–108, 1985.Google Scholar
  56. 56.
    A.M. Waxman and K. Wohn, “Image flow theory: A framework for 3-D inference from time-varuing imagery,” in ADVANCES IN COMPUTER VISION (ed. C. Brown), Erlbaum Publ. (in press).Google Scholar
  57. 57.
    R.M. Winger, Projective Geometry. Dover: New York, 1962.Google Scholar
  58. 58.
    K. Wohn, “A contour-based approach to image flow,” Ph.D. thesis, Univ. of Maryland, Depart. of Computer Science, September, 1984.Google Scholar
  59. 59.
    K. Wohn and A.M. Waxman, “Contour evolution, neighborhood deformation and local image flow: Curved surfaces in motion,” Univ. of Maryland, Cent. Autom. Res. Tech. Report 134.Google Scholar
  60. 60.
    K. Wohn and A.M. Waxman, “The analytic structure of image flows: Deformation and segmentation.” Univ. of Maryland, Cent. Autom. Res. Tech. Report, January, 1987; submitted to COMPUTER VISION, GRAPHICS AND IMAGE PROCESSING, January, 1987.Google Scholar

Copyright information

© Kluwer Academic Publishers 1987

Authors and Affiliations

  • Allen M. Waxman
    • 1
  • Behrooz Kamgar-Parsi
    • 1
  • Muralidhara Subbarao
    • 1
  1. 1.Computer Vision Laboratory, Center for Automation ResearchUniversity of MarylandCollege Park

Personalised recommendations