Abstract
We consider the problem of segmenting multiple rigid-body motions from point correspondences in multiple affine views. We cast this problem as a subspace clustering problem in which point trajectories associated with each motion live in a linear subspace of dimension two, three or four. Our algorithm involves projecting all point trajectories onto a 5-dimensional subspace using the SVD, the PowerFactorization method, or RANSAC, and fitting multiple linear subspaces representing different rigid-body motions to the points in ℝ5 using GPCA. Unlike previous work, our approach does not restrict the motion subspaces to be four-dimensional and independent. Instead, it deals gracefully with all the spectrum of possible affine motions: from two-dimensional and partially dependent to four-dimensional and fully independent. Our algorithm can handle the case of missing data, meaning that point tracks do not have to be visible in all images, by using the PowerFactorization method to project the data. In addition, our method can handle outlying trajectories by using RANSAC to perform the projection. We compare our approach to other methods on a database of 167 motion sequences with full motions, independent motions, degenerate motions, partially dependent motions, missing data, outliers, etc. On motion sequences with complete data our method achieves a misclassification error of less that 5% for two motions and 29% for three motions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Boult, T. E., & Brown, L. G. (1991). Factorization-based segmentation of motions. In IEEE workshop on motion understanding (pp. 179–186).
Buchanan, A., & Fitzgibbon, A. (2000). Damped Newton algorithms for matrix factorization with missing data. In IEEE conference on computer vision and pattern recognition (pp. 316–322).
Costeira, J., & Kanade, T. (1998). A multibody factorization method for independently moving objects. International Journal of Computer Vision, 29(3), 159–179.
De la Torre, F., & Black, M. J. (2001). Robust principal component analysis for computer vision. In IEEE international conference on computer vision (pp. 362–369).
Fan, Z., Zhou, J., & Wu, Y. (2006). Multibody grouping by inference of multiple subspaces from high-dimensional data using oriented-frames. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(1), 91–105.
Fischler, M. A., & Bolles, R. C. (1981). RANSAC random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 26, 381–395.
Gear, C. W. (1998). Multibody grouping from motion images. International Journal of Computer Vision, 29(2), 133–150.
Gruber, A., & Weiss, Y. (2004). Multibody factorization with uncertainty and missing data using the EM algorithm. In IEEE conference on computer vision and pattern recognition (Vol. I, pp. 707–714).
Hartley, R. (2003). PowerFactorization: an approach to affine reconstruction with missing and uncertain data. In Australia–Japan advanced workshop on computer vision.
Ichimura, N. (1999). Motion segmentation based on factorization method and discriminant criterion. In IEEE international conference on computer vision (pp. 600–605).
Kanatani, K. (2001). Motion segmentation by subspace separation and model selection. In IEEE international conference on computer vision (Vol. 2, pp. 586–591).
Kanatani, K., & Matsunaga, C. (2002). Estimating the number of independent motions for multibody motion segmentation. In European conference on computer vision (pp. 25–31).
Ng, A., Weiss, Y., & Jordan, M. (2001). On spectral clustering: analysis and an algorithm. In Neural information processing systems.
Oliensis, J., & Genc, Y. (2001). Fast and accurate algorithms for projective multi–image structure from motion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 546–559.
OpenCV (2000). http://sourceforge.net/projects/opencvlibrary.
Poelman, C. J., & Kanade, T. (1997). A paraperspective factorization method for shape and motion recovery. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(3), 206–218.
Shum, H. Y., Ikeuchi, K., & Reddy, R. (1995). Principal component analysis with missing data and its application to polyhedral object modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(9), 854–867.
Sugaya, Y., & Kanatani, K. (2002). Outlier removal for feature tracking by subspace separation. In Proceedings of the 8th symposium on sensing via imaging information (pp. 603–608).
Sugaya, Y., & Kanatani, K. (2004). Geometric structure of degeneracy for multi-body motion segmentation. In Workshop on statistical methods in video processing.
Tomasi, C., & Kanade, T. (1992). Shape and motion from image streams under orthography: a factorization method. International Journal of Computer Vision, 9, 137–154.
Tron, R., & Vidal, R. (2007). A benchmark for the comparson of 3-D motion segmentation algorithms. In IEEE conference on computer vision and pattern recognition (pp. 1–8).
Vidal, R., & Hartley, R. (2004). Motion segmentation with missing data by PowerFactorization and Generalized PCA. In IEEE conference on computer vision and pattern recognition (Vol. II, pp. 310–316).
Vidal, R., Ma, Y., & Sastry, S. (2005). Generalized Principal Component Analysis (GPCA). IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(12), 1–15.
Vidal, R., Ma, Y., Soatto, S., & Sastry, S. (2006). Two-view multibody structure from motion. International Journal of Computer Vision, 68(1), 7–25.
Vidal, R., & Oliensis, J. (2002). Structure from planar motions with small baselines. In European conference on computer vision (pp. 383–398).
Weiss, Y. (1999). Segmentation using eigenvectors: a unifying view. In IEEE international conference on computer vision (pp. 975–982).
Wu, Y., Zhang, Z., Huang, T. S., & Lin, J. Y. (2001). Multibody grouping via orthogonal subspace decomposition. In IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 252–257).
Yan, J., & Pollefeys, M. (2005). A factorization approach to articulated motion recovery. In IEEE conference on computer vision and pattern recognition (Vol. II, pp. 815–821).
Yan, J., & Pollefeys, M. (2006). A general framework for motion segmentation: independent, articulated, rigid, non-rigid, degenerate and non-degenerate. In European conference on computer vision (pp. 94–106).
Zelnik-Manor, L., & Irani, M. (2003). Degeneracies, dependencies and their implications in multi-body and multi-sequence factorization. In IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 287–293).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Vidal, R., Tron, R. & Hartley, R. Multiframe Motion Segmentation with Missing Data Using PowerFactorization and GPCA. Int J Comput Vis 79, 85–105 (2008). https://doi.org/10.1007/s11263-007-0099-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-007-0099-z