3-D Motion and Structure from 2-D Motion Causally Integrated over Time: Implementation

  • Alessandro Chiuso
  • Paolo Favaro
  • Hailin Jin
  • Stefano Soatto
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1843)


The causal estimation of three-dimensional motion from a sequence of two-dimensional images can be posed as a nonlinear filtering problem. We describe the implementation of an algorithm whose uniform observability, minimal realization and stability have been proven analytically in [5]. We discuss a scheme for handling occlusions, drift in the scale factor and tuning of the filter. We also present an extension to partially calibrated camera models and prove its observability. We report the performance of our implementation on a few long sequences of real images. More importantly, however, we have made our real-time implementation - which runs on a personal computer - available to the public for first-hand testing.


Pattern Anal Rigid Motion Motion Error Minimal Realization Correspondence Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    G. Adiv. Determining three-dimensional motion and structure from optical flow generated by several moving objects. IEEE Trans. Pattern Anal. Mach. Intell., 7(4):348–401, 1985.Google Scholar
  2. 2.
    A. Azarbayejani and A. Pentland. Recursive estimation of motion, structure and focal length. IEEE Trans. Pattern Anal. Mach. Intell., 17(6):562–575, 1995.CrossRefGoogle Scholar
  3. 3.
    M.S. Bartlett. An Introduction to Stochastic Processes. Cambridge University Press, 1956.Google Scholar
  4. 4.
    T. Broida and R. Chellappa. Estimation of object motion parameters from noisy images. IEEE Trans. Pattern Anal. Mach. Intell., Jan. 1986.Google Scholar
  5. 5.
    A. Chiuso and S. Soatto. 3-d motion and structure causally integrated over time: Theory. In Tutorial lecture notes of IEEE Intl. Conf. on Robotics and Automation, April 2000.Google Scholar
  6. 6.
    W. Dayawansa, B. Ghosh, C. Martin, and X. Wang. A necessary and sufficient condition for the perspective observability problem. Systems and Control Letters, 25(3):159–166, 1994.CrossRefMathSciNetGoogle Scholar
  7. 7.
    E. D. Dickmanns and V. Graefe. Applications of dynamic monocular machine vision. Machine Vision and Applications, 1:241–261, 1988.CrossRefGoogle Scholar
  8. 8.
    O. Faugeras. Three dimensional vision, a geometric viewpoint. MIT Press, 1993.Google Scholar
  9. 9.
    C. Fermüller and Y. Aloimonos. Tracking facilitates 3-d motion estimation. Biological Cybernetics (67), 259–268, 1992.Google Scholar
  10. 10.
    D.B. Gennery. Tracking known 3-dimensional object. In Proc. AAAI 2nd Natl. Conf. Artif. Intell., pages 13–17, Pittsburg, PA, 1982.Google Scholar
  11. 11.
    J. Heel. Dynamic motion vision. Robotics and Autonomous Systems, 6(1), 1990.Google Scholar
  12. 12.
    X. Hu and N. Ahuja. Motion and structure estimation using long sequence motion models. Image and Vision Computing, 11(9):549–569, 1993.CrossRefGoogle Scholar
  13. 13.
    A. Jepson and D. Heeger. Subspace methods for recovering rigid motion ii: theory. RBCV TR-90-35, University of Toronto-CS dept., November 1990. Revised July 1991.Google Scholar
  14. 14.
    H. Jin, P. Favaro, and S. Soatto. Real-time 3-d motion and structure from point features: a front-end system for vision-based control and interaction. In Computer Vision and Pattern Recognition; code available from, June 2000.
  15. 15.
    J. J. Koenderink and A. J. Van Doorn. Affine structure from motion. J. Optic. Soc. Am., 8(2):377–385, 1991.CrossRefGoogle Scholar
  16. 16.
    R. Kumar, P. Anandan, and K. Hanna. Shape recovery from multiple views: a parallax based approach. Proc. of the Image Understanding Workshop, 1994.Google Scholar
  17. 17.
    B.D. Lucas and T. Kanade. An iterative image registration technique with an application to stereo vision. Proc. 7th Int. Joinnt Conf. on Art. Intell., 1981.Google Scholar
  18. 18.
    L. Matthies, R. Szelisky, and T. Kanade. Kalman filter-based algorithms for estimating depth from image sequences. Int. J. of computer vision, pages 2989–2994, 1989.Google Scholar
  19. 19.
    P. McLauchlan, I. Reid, and D. Murray. Recursive affine structure and motion from image sequences. Proc. of the 3rd Eur. Conf. Comp. Vision, Stockholm, May 1994.Google Scholar
  20. 20.
    P. F. McLauchlan. Gauge invariance in projective 3d reconstruction. In IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes, Fort Collins, CO, June 1999, 1999.Google Scholar
  21. 21.
    J. Oliensis. Provably correct algorithms for multi-frame structure from motion. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1996.Google Scholar
  22. 22.
    J. Oliensis and J. Inigo-Thomas. Recursive multi-frame structure from motion incorporating motion error. Proc. DARPA Image Understanding Workshop, 1992.Google Scholar
  23. 23.
    J. Philip. Estimation of three dimensional motion of rigid objects from noisy observations. IEEE Trans. Pattern Anal. Mach. Intell., 13(1):61–66, 1991.CrossRefGoogle Scholar
  24. 24.
    C. Poelman and T. Kanade. A paraperspective factorization method for shape and motion recovery. Proc. of the 3 ECCV, LNCS Vol 810, Springer Verlag, 1994.Google Scholar
  25. 25.
    H. S. Sawhney. Simplifying motion and structure analysis using planar parallax and image warping. Proc. of the Int. Conf. on Pattern Recognition, Seattle, June 1994.Google Scholar
  26. 26.
    L. Shapiro, A. Zisserman, and M. Brady. Motion from point matches using affine epipolar geometry. Proc. of the ECCV94, Vol. 800 of LNCS, Springer Verlag, 1994.Google Scholar
  27. 27.
    S. Soatto. Observability/identifiability of rigid motion under perspective projection. In Proc. of the 33rd IEEE Conf. on Decision and Control, pages 3235–3240, Dec. 1994.Google Scholar
  28. 28.
    S. Soatto. 3-d structure from visual motion: modeling, representation and observability. Automatica, 33:1287–1312, 1997.zbMATHCrossRefMathSciNetGoogle Scholar
  29. 29.
    S. Soatto and P. Perona. Reducing “structure from motion”: a general framework for dynamic vision. part 1: modeling. IEEE Trans. Pattern Anal. Mach. Intell., 20(9):993–942, September 1998.Google Scholar
  30. 30.
    M. Spetsakis and J. Aloimonos. A multi-frame approach to visual motion perception. Int. J. Computer Vision 6(3), 1991.Google Scholar
  31. 31.
    R. Szeliski. Recovering 3d shape and motion from image streams using nonlinear least squares. J. visual communication and image representation, 1994.Google Scholar
  32. 32.
    M. A. Taalebinezhaad. Direct recovery of motion and shape in the general case by fixation. IEEE Trans. Pattern Anal. Mach. Intell., 14(8):847–853, 1992.CrossRefGoogle Scholar
  33. 33.
    C. Tomasi and T. Kanade. Shape and motion from image streams under orthography: a factorization method. Int. J. of Computer Vision, 9(2):137–154, 1992.CrossRefGoogle Scholar
  34. 34.
    J. Weng, N. Ahuja, and T. Huang. Optimal motion and structure estimation. IEEE Trans. Pattern Anal. Mach. Intell., 15:864–884, 1993.CrossRefGoogle Scholar
  35. 35.
    Z. Zhang and O. D. Faugeras. Three dimensional motion computation and object segmentation in a long sequence of stereo frames. Int. J. of Computer Vision, 7(3):211–241, 1992.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Alessandro Chiuso
    • 1
    • 2
  • Paolo Favaro
    • 1
  • Hailin Jin
    • 1
  • Stefano Soatto
    • 1
  1. 1.Washington UniversitySaint Louis
  2. 2.Università di PadovaPadovaItaly

Personalised recommendations