Motion estimation on the essential manifold

  • Stefano Soatto
  • Ruggero Frezza
  • Pietro Perona
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 801)


We introduce a novel perspective for viewing the “ego-motion reconstruction” problem as the estimation of the state of a dynamical system having an implicit measurement constraint and unknown inputs. Such a system happens to be “linear”, but it is defined on a space (the “Essential Manifold”) which is not a linear (vector) space.

We propose two recursive schemes for performing the estimation task: the first consists in “flattening the space” and solving a nonlinear estimation problem on the flat (euclidean) space. The second consists in viewing the system as embedded in a larger euclidean space, and solving at each step a linear estimation problem on a linear space, followed by a “projection” onto the Essential Manifold.

Both schemes output motion estimates together with the joint second order statistics of the estimation error, which can be used by any “structure from motion” module which incorporates motion error [18, 22] in order to estimate 3D scene structure.

Experiments are presented with real and synthetic image sequences.


Motion Estimation Visual Motion Rigid Motion Unknown Input Estimation Task 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Azarbayejani, A., Horowitz, B., and Pentland, A. Recursive estimation of structure and motion using relative orientation constraints. Proc. CVPR (New York, 1993).Google Scholar
  2. 2.
    Boothby, W. Introduction to Differentiable Manifolds and Riemannian Geometry. Academic Press, 1986.Google Scholar
  3. 3.
    Broida, T., and Chellappa, R. Estimating the kinematics and structure of a rigid object from a sequence of monocular frames. IEEE Trans. Pattern Anal. Mach. Intell. (1991).Google Scholar
  4. 4.
    Darmon. A recursive method to apply the hough transform to a set of moving objects. Proc. IEEE, CH 1746 7/82 (1982).Google Scholar
  5. 5.
    Di-Bernardo, E., Toniutti, L., Frezza, R., and Picci, G. Stima del moto dell'osservatore e della struttura della scena mediante visione monoculare. Tesi di Laurea-Università di Padova (1993).Google Scholar
  6. 6.
    Faugeras, O. Three dimensional vision, a geometric viewpoint. MIT Press, 1993.Google Scholar
  7. 7.
    Gennery, D. Tracking known 3-dimensional object. In Proc. AAAI 2nd Natl. Conf. Artif. Intell. (Pittsburg, PA, 1982), pp. 13–17.Google Scholar
  8. 8.
    Heel, J. Direct estimation of structure and motion from multiple frames. AI Memo 1190, MIT AI Lab (March 1990).Google Scholar
  9. 9.
    Isidori, A. Nonlinear Control Systems. Springer Verlag, 1989.Google Scholar
  10. 10.
    Jazwinski, A. Stochastic Processes and Filtering Theory. Academic Press, 1970.Google Scholar
  11. 11.
    Kalman, R. A new approach to linear filtering and prediction problems. Trans. of the ASME-Journal of basic engineering. 35–45 (1960).Google Scholar
  12. 12.
    Krener, A. J., and Respondek, W. Nonlinear observers with linearizable error dynamics. SIAM J. Control Optim. vol. 23 (2) (1985).Google Scholar
  13. 13.
    Longuet-Higgins, H. C. A computer algorithm for reconstructing a scene from two projections. Nature 293 (1981), 133–135.Google Scholar
  14. 14.
    Matthies, L., Szelisky, R., and Kanade, T. Kalman filter-based algorithms for estimating depth from image sequences. Int. J. of computer vision (1989).Google Scholar
  15. 15.
    Maybank, S. Theory of reconstruction from image motion, vol. 28 of Information Sciences. Springer-Verlag, 1992.Google Scholar
  16. 16.
    Mundy, J., and Zisserman, A-, Eds, Geometric invariance in computer vision. MIT Press, Cambridge, Mass., 1992.Google Scholar
  17. 17.
    Murray, R., Li, Z., and Sastry, S. A Mathematical Introduction to Robotic Manipulation. Preprint, 1993.Google Scholar
  18. 18.
    Oliensis, J., and Inigo-Thomas, J. Recursive multi-frame structure from motion incorporating motion error. Proc. DARPA Image Understanding Workshop (1992).Google Scholar
  19. 19.
    Soatto, S. Observability of rigid motion under perspective projection with application to visual motion estimation. Technical Report CIT-CDS 94-001, California Institute of Technology (1994).Google Scholar
  20. 20.
    Soatto, S., Frezza, R., and Perona, P. Recursive motion estimation on the essential manifold. Technical Report CIT-CDS 93-021 and CIT-CNS 32/93, California Institute of Technology (1993).Google Scholar
  21. 21.
    Soatto, S., and Perona, P. Three dimensional transparent structure segmentation and multiple 3d motion estimation from monocular perspective image sequences. Technical Report CIT-CDS 93-022, California Institute of Technology (1993).Google Scholar
  22. 22.
    Soatto, S., Perona, P., Frezza, R., and Picci, G. Recursive motion and structure estimation with complete error characterization. In Proc. IEEE Comput. Soc. Conf. Comput. Vision and Pattern Recogn. (New York, June 1993), pp. 428–433.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1994

Authors and Affiliations

  • Stefano Soatto
    • 1
  • Ruggero Frezza
    • 2
  • Pietro Perona
    • 1
    • 2
  1. 1.California Institute of Technology 116-81Pasadena
  2. 2.Dipartimento di Elettronica ed InformaticaUniversità di PadovaPadovaItaly

Personalised recommendations