International Journal of Computer Vision

, Volume 79, Issue 1, pp 85–105 | Cite as

Multiframe Motion Segmentation with Missing Data Using PowerFactorization and GPCA

Article

Abstract

We consider the problem of segmenting multiple rigid-body motions from point correspondences in multiple affine views. We cast this problem as a subspace clustering problem in which point trajectories associated with each motion live in a linear subspace of dimension two, three or four. Our algorithm involves projecting all point trajectories onto a 5-dimensional subspace using the SVD, the PowerFactorization method, or RANSAC, and fitting multiple linear subspaces representing different rigid-body motions to the points in ℝ5 using GPCA. Unlike previous work, our approach does not restrict the motion subspaces to be four-dimensional and independent. Instead, it deals gracefully with all the spectrum of possible affine motions: from two-dimensional and partially dependent to four-dimensional and fully independent. Our algorithm can handle the case of missing data, meaning that point tracks do not have to be visible in all images, by using the PowerFactorization method to project the data. In addition, our method can handle outlying trajectories by using RANSAC to perform the projection. We compare our approach to other methods on a database of 167 motion sequences with full motions, independent motions, degenerate motions, partially dependent motions, missing data, outliers, etc. On motion sequences with complete data our method achieves a misclassification error of less that 5% for two motions and 29% for three motions.

Keywords

Multibody factorization Multibody grouping 3-D motion segmentation Structure from motion Subspace clustering PowerFactorization and Generalized Principal Component Analysis (GPCA) Missing data 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Boult, T. E., & Brown, L. G. (1991). Factorization-based segmentation of motions. In IEEE workshop on motion understanding (pp. 179–186). Google Scholar
  2. Buchanan, A., & Fitzgibbon, A. (2000). Damped Newton algorithms for matrix factorization with missing data. In IEEE conference on computer vision and pattern recognition (pp. 316–322). Google Scholar
  3. Costeira, J., & Kanade, T. (1998). A multibody factorization method for independently moving objects. International Journal of Computer Vision, 29(3), 159–179. CrossRefGoogle Scholar
  4. De la Torre, F., & Black, M. J. (2001). Robust principal component analysis for computer vision. In IEEE international conference on computer vision (pp. 362–369). Google Scholar
  5. Fan, Z., Zhou, J., & Wu, Y. (2006). Multibody grouping by inference of multiple subspaces from high-dimensional data using oriented-frames. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(1), 91–105. CrossRefGoogle Scholar
  6. Fischler, M. A., & Bolles, R. C. (1981). RANSAC random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 26, 381–395. CrossRefMathSciNetGoogle Scholar
  7. Gear, C. W. (1998). Multibody grouping from motion images. International Journal of Computer Vision, 29(2), 133–150. CrossRefGoogle Scholar
  8. Gruber, A., & Weiss, Y. (2004). Multibody factorization with uncertainty and missing data using the EM algorithm. In IEEE conference on computer vision and pattern recognition (Vol. I, pp. 707–714). Google Scholar
  9. Hartley, R. (2003). PowerFactorization: an approach to affine reconstruction with missing and uncertain data. In Australia–Japan advanced workshop on computer vision. Google Scholar
  10. Ichimura, N. (1999). Motion segmentation based on factorization method and discriminant criterion. In IEEE international conference on computer vision (pp. 600–605). Google Scholar
  11. Kanatani, K. (2001). Motion segmentation by subspace separation and model selection. In IEEE international conference on computer vision (Vol. 2, pp. 586–591). Google Scholar
  12. Kanatani, K., & Matsunaga, C. (2002). Estimating the number of independent motions for multibody motion segmentation. In European conference on computer vision (pp. 25–31). Google Scholar
  13. Ng, A., Weiss, Y., & Jordan, M. (2001). On spectral clustering: analysis and an algorithm. In Neural information processing systems. Google Scholar
  14. Oliensis, J., & Genc, Y. (2001). Fast and accurate algorithms for projective multi–image structure from motion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 546–559. CrossRefGoogle Scholar
  15. Poelman, C. J., & Kanade, T. (1997). A paraperspective factorization method for shape and motion recovery. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(3), 206–218. CrossRefGoogle Scholar
  16. Shum, H. Y., Ikeuchi, K., & Reddy, R. (1995). Principal component analysis with missing data and its application to polyhedral object modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(9), 854–867. CrossRefGoogle Scholar
  17. Sugaya, Y., & Kanatani, K. (2002). Outlier removal for feature tracking by subspace separation. In Proceedings of the 8th symposium on sensing via imaging information (pp. 603–608). Google Scholar
  18. Sugaya, Y., & Kanatani, K. (2004). Geometric structure of degeneracy for multi-body motion segmentation. In Workshop on statistical methods in video processing. Google Scholar
  19. Tomasi, C., & Kanade, T. (1992). Shape and motion from image streams under orthography: a factorization method. International Journal of Computer Vision, 9, 137–154. CrossRefGoogle Scholar
  20. Tron, R., & Vidal, R. (2007). A benchmark for the comparson of 3-D motion segmentation algorithms. In IEEE conference on computer vision and pattern recognition (pp. 1–8). Google Scholar
  21. Vidal, R., & Hartley, R. (2004). Motion segmentation with missing data by PowerFactorization and Generalized PCA. In IEEE conference on computer vision and pattern recognition (Vol. II, pp. 310–316). Google Scholar
  22. Vidal, R., Ma, Y., & Sastry, S. (2005). Generalized Principal Component Analysis (GPCA). IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(12), 1–15. CrossRefGoogle Scholar
  23. Vidal, R., Ma, Y., Soatto, S., & Sastry, S. (2006). Two-view multibody structure from motion. International Journal of Computer Vision, 68(1), 7–25. CrossRefGoogle Scholar
  24. Vidal, R., & Oliensis, J. (2002). Structure from planar motions with small baselines. In European conference on computer vision (pp. 383–398). Google Scholar
  25. Weiss, Y. (1999). Segmentation using eigenvectors: a unifying view. In IEEE international conference on computer vision (pp. 975–982). Google Scholar
  26. Wu, Y., Zhang, Z., Huang, T. S., & Lin, J. Y. (2001). Multibody grouping via orthogonal subspace decomposition. In IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 252–257). Google Scholar
  27. Yan, J., & Pollefeys, M. (2005). A factorization approach to articulated motion recovery. In IEEE conference on computer vision and pattern recognition (Vol. II, pp. 815–821). Google Scholar
  28. Yan, J., & Pollefeys, M. (2006). A general framework for motion segmentation: independent, articulated, rigid, non-rigid, degenerate and non-degenerate. In European conference on computer vision (pp. 94–106). Google Scholar
  29. Zelnik-Manor, L., & Irani, M. (2003). Degeneracies, dependencies and their implications in multi-body and multi-sequence factorization. In IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 287–293). Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.Center for Imaging Science, Department of Biomedical EngineeringJohns Hopkins UniversityBaltimoreUSA
  2. 2.Vision Science Technology and Applications Program, National ICT Australia, Department of Information EngineeringAustralian National UniversityCanberraAustralia

Personalised recommendations