Two-View Multibody Structure from Motion

  • René Vidal
  • Yi Ma
  • Stefano Soatto
  • Shankar Sastry
Article

Abstract

We present an algebraic geometric approach to 3-D motion estimation and segmentation of multiple rigid-body motions from noise-free point correspondences in two perspective views. Our approach exploits the algebraic and geometric properties of the so-called multibody epipolar constraint and its associated multibody fundamental matrix, which are natural generalizations of the epipolar constraint and of the fundamental matrix to multiple motions. We derive a rank constraint on a polynomial embedding of the correspondences, from which one can estimate the number of independent motions as well as linearly solve for the multibody fundamental matrix. We then show how to compute the epipolar lines from the first-order derivatives of the multibody epipolar constraint and the epipoles by solving a plane clustering problem using Generalized PCA (GPCA). Given the epipoles and epipolar lines, the estimation of individual fundamental matrices becomes a linear problem. The clustering of the feature points is then automatically obtained from either the epipoles and epipolar lines or from the individual fundamental matrices. Although our approach is mostly designed for noise-free correspondences, we also test its performance on synthetic and real data with moderate levels of noise.

Keywords

multibody structure from motion 3-D motion segmentation multibody epipolar constraint multibody fundamental matrix Generalized PCA (GPCA) 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Avidan, S. and Shashua, A. 2000.Trajectory triangulation: 3D reconstruction of moving points from a monocular image sequence, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(4):348–357.CrossRefGoogle Scholar
  2. Costeira, J. and Kanade, T. 1995. Multibody factorization methods for motion analysis, In International Conference on Computer Vision, pp. 1071–1076.Google Scholar
  3. Fan, X. and Vidal, R. 2005. The space of multibody fundamental matrices: Rank geometry and projection. In ICCV Workshop on Dynamical Vision.Google Scholar
  4. Feng, X. and Perona, P. 1998. Scene segmentation from 3D motion, In IEEE Conference on Computer Vision and Pattern Recognition, pp. 225–231.Google Scholar
  5. Han, M. and Kanade, T. 2000. Reconstruction of a scene with multiple linearly moving objects. In IEEE Conference on Computer Vision and Pattern Recognition, volume 2, pp. 542–549.Google Scholar
  6. Harris, J. 1992. Algebraic Geometry: A First Course, Springer-Verlag.Google Scholar
  7. Hartley R. and Zisserman A. 2000. Multiple View Geometry in Computer Vision, Cambridge.Google Scholar
  8. Hartley, R. and Vidal, R. 2004. The multibody trifocal tensor: Motion segmentation from 3 perspective views, In IEEE Conference on Computer Vision and Pattern Recognition, volume I, pp. 769–775.Google Scholar
  9. Kanatani, K. 2001. Motion segmentation by subspace separation and model selection, In International Conference on Computer Vision, vol. 2, pp. 586–591.Google Scholar
  10. Kanatani, K. 2002. Evaluation and selection of models for motion segmentation, In Asian Conference on Computer Vision, pp. 7–12.Google Scholar
  11. Lang, S. 1993. Algebra, Addison-Wesley Publishing Company, 3rd editionGoogle Scholar
  12. Longuet-Higgins, H., C. 1981. A computer algorithm for reconstructing a scene from two projections, Nature, 293:133–135.Google Scholar
  13. Ma, Y., Košecká, J. and Sastry, S. 2001. Optimization criteria and geometric algorithms for motion and structure estimation, International Journal of Computer Vision, 44(3):219–249.CrossRefGoogle Scholar
  14. Ma, Y. Soatto, S., Kosecka, J., and Sastry, S. 2003. An Invitation to 3D Vision: From Images to Geometric Models, Springer Verlag.Google Scholar
  15. Shashua, A. and Levin, A. 2001. Multi-frame infinitesimal motion model for the reconstruction of (dynamic) scenes with multiple linearly moving objects. In International Conference on Computer Vision, vol. 2 :592–599.Google Scholar
  16. Sturm, P. 2002. Structure and motion for dynamic scenes—the case of points moving in planes. In European Conference on Computer Vision, pp. 867–882.Google Scholar
  17. Torr, P., Szeliski, R., and Anandan, P. 2001. An integrated Bayesian approach to layer extraction from image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(3):297–303.CrossRefGoogle Scholar
  18. Torr, P. H. S. 1998. Geometric motion segmentation and model selection, Philosophical Transactions Royal Society of London, 356(1740):1321–1340.CrossRefMATHMathSciNetGoogle Scholar
  19. Vidal, R. and Hartley, R. 2004. Motion segmentation with missing data by PowerFactorization and Generalized PCA. In IEEE Conference on Computer Vision and Pattern Recognition, vol. II, pp. 310–316.Google Scholar
  20. Vidal, R. and Ma, Y. 2004. A unified algebraic approach to 2-D and 3-D motion segmentation, In European Conference on Computer Vision, pp. 1–15.Google Scholar
  21. Vidal, R., Ma, Y., and Piazzi. J. 2004. A new GPCA algorithm for clustering subspaces by fitting, differentiating and dividing polynomials. In IEEE Conference on Computer Vision and Pattern Recognition, (1):510–517.Google Scholar
  22. Vidal, R., Ma, Y., and Sastry, S. 2005. Generalized principal component analysis (GPCA). IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(12):1–15.CrossRefGoogle Scholar
  23. Vidal, R., Ma, Y., and Sastry, S. 2003. Generalized Principal Component Analysis (GPCA). In IEEE Conference on Computer Vision and Pattern Recognition, (1):621–628.Google Scholar
  24. Vidal, R. and Sastry, S. 2003. Optimal segmentation of dynamic scenes from two perspective views. In IEEE Conference on Computer Vision and Pattern Recognition, (2):281–286.Google Scholar
  25. Vidal, R., Soatto, S., Ma, Y., and Sastry, S. 2002. Segmentation of dynamic scenes from the multibody fundamental matrix. In ECCV Workshop on Visual Modeling of Dynamic Scenes.Google Scholar
  26. Vidal, R., Soatto, S., and Sastry, S. 2002. Two-view segmentation of dynamic scenes from the multibody fundamental matrix. Technical Report, UCB/ERL M02/02, UC Berkeley.Google Scholar
  27. Wolf, L. and Shashua, A. 2001. Affine 3-D reconstruction from two projective images of independently translating planes. In International Conference on Computer Vision, pp. 238–244.Google Scholar
  28. Wolf, L. and Shashua, A. 2001. Two-body segmentation from two perspective views. In Conference on Computer Vision and Pattern Recognition, pp. 263–270.Google Scholar

Copyright information

© Springer Science + Business Media, LLC. 2006

Authors and Affiliations

  • René Vidal
    • 1
  • Yi Ma
    • 2
  • Stefano Soatto
    • 3
  • Shankar Sastry
    • 4
  1. 1.Center for Imaging Science, Department of Biomedical EngineeringJohns Hopkins UniversityBaltimore
  2. 2.Department of ECEUniversity of Illinois at Urbana-ChampaignUrbana
  3. 3.Computer Science DepartmentUniversity of California at Los AngelesLos Angeles
  4. 4.Department of EECSUniversity of California at BerkeleyBerkeley

Personalised recommendations