A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-rigid, Degenerate and Non-degenerate

  • Jingyu Yan
  • Marc Pollefeys
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3954)


We cast the problem of motion segmentation of feature trajectories as linear manifold finding problems and propose a general framework for motion segmentation under affine projections which utilizes two properties of trajectory data: geometric constraint and locality. The geometric constraint states that the trajectories of the same motion lie in a low dimensional linear manifold and different motions result in different linear manifolds; locality, by which we mean in a transformed space a data and its neighbors tend to lie in the same linear manifold, provides a cue for efficient estimation of these manifolds. Our algorithm estimates a number of linear manifolds, whose dimensions are unknown beforehand, and segment the trajectories accordingly. It first transforms and normalizes the trajectories; secondly, for each trajectory it estimates a local linear manifold through local sampling; then it derives the affinity matrix based on principal subspace angles between these estimated linear manifolds; at last, spectral clustering is applied to the matrix and gives the segmentation result. Our algorithm is general without restriction on the number of linear manifolds and without prior knowledge of the dimensions of the linear manifolds. We demonstrate in our experiments that it can segment a wide range of motions including independent, articulated, rigid, non-rigid, degenerate, non-degenerate or any combination of them. In some highly challenging cases where other state-of-the-art motion segmentation algorithms may fail, our algorithm gives expected results.


Segmentation Result Spectral Cluster Rigid Motion Trajectory Data Linear Manifold 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Fischler, M.A., Bolles, R.C.: Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. Comm. of the ACM 24, 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Costeira, J.P., Kanade, T.: A Multibody Factorization Method for Independently Moving Objects. IJCV 29(3), 159–179 (1998)CrossRefGoogle Scholar
  3. 3.
    Bregler, C., Hertzmann, A., Biermann, H.: Recovering Non-Rigid 3D Shape from Image Streams. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2000) (June 2000)Google Scholar
  4. 4.
    Brand, M.: Morphable 3D models from video. In: CVPR, vol. II, pp. 456–463 (2001)Google Scholar
  5. 5.
    Xiao, J., Chai, J.-x., Kanade, T.: A closed-form solution to non-rigid shape and motion recovery. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 573–587. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  6. 6.
    Ullman, S.: Maximizing rigidity: The incremental recovery of 3D structure from rigid and rubbery motion. Technical Report A.I. Memo No. 721, MIT (1983)Google Scholar
  7. 7.
    Sinclair, D.: Motion segmentation and local structure. In: Proceedings of the 4th International Conference on Computer Vision (1993)Google Scholar
  8. 8.
    Boult, T., Brown, L.: Factorization-based segmentation of motions. In: Proceedings of the IEEE Workshop on Visual Motion (1991)Google Scholar
  9. 9.
    Gear, C.W.: Feature grouping in moving objects. In: Proceedings of the Workshop on Motion of Non-Rigid and Articulated Objects, Austin, Texas (1994)Google Scholar
  10. 10.
    Ichimura, N.: Motion segmentation based on factorization method and discriminant criterion. In: Proc. IEEE Int. Conf. Computer Vision, pp. 600–605 (1999)Google Scholar
  11. 11.
    Kanatani, K.: Motion segmentation by subspace separation and model selection:model selection and reliability evaluation. Intl. J. of Image and Graphics 2(2), 179–197 (2002)CrossRefGoogle Scholar
  12. 12.
    Zelnik-Manor, L., Irani, M.: Degeneracies, dependencies and their implications in multi-body and multi-sequence factorizations. In: Proc. IEEE Computer Vision and Pattern Recognition (2003)Google Scholar
  13. 13.
    Ferrari-Trecate, G., Muselli, M., Liberati, D., Morari, M.: A clustering technique for the identification of piecewise affine and hybrid systems. Automatica 39, 205–217 (2003)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Weiss, Y.: Segmentation using eigenvectors: A unifying view. In: International Conference on Computer Vision, Corfu, Greece, September 1999, pp. 975–982 (1999)Google Scholar
  15. 15.
    Ng, A., Jordan, M., Weiss, Y.: On spectral clustering: analysis and an algorithm. In: Advances in Neural Information Processing Systems 14, MIT Press, Cambridge (2002)Google Scholar
  16. 16.
    Shi, J., Malik, J.: Normalized Cuts and Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) (2000)Google Scholar
  17. 17.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. IJCV 9(2), 137–154 (1992)CrossRefGoogle Scholar
  18. 18.
    Vidal, R., Hartley, R.: Motion Segmentation with Missing Data using PowerFactorization and GPCA. In: IEEE Conference on Computer Vision and Pattern Recognition (2004)Google Scholar
  19. 19.
    Vidal, R., Ma, Y., Sastry, S.: Generalized Principal Component Analysis (GPCA). In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2003) (June 2003)Google Scholar
  20. 20.
    Vidal, R., Ma, Y., Piazzi, J.: A New GPCA Algorithm for Clustering Subspaces by Fitting, Differentiating and Dividing Polynomials. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2004), June 27-July 02 (2004)Google Scholar
  21. 21.
    Yan, J., Pollefeys, M.: A Factorization-based Approach to Articulated Motion Recovery. In: IEEE Conf. on Computer Vision and Pattern Recognition (2005)Google Scholar
  22. 22.
    Tresadern, P., Reid, I.: Articulated Structure From Motion by Factorization. In: Proc. IEEE Conf on Computer Vision and Pattern Recognition (2005)Google Scholar
  23. 23.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2002)MATHGoogle Scholar
  24. 24.
    Golub, G., van Loan, A.: Matrix Computations. Johns Hopkins U. Press (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Jingyu Yan
    • 1
  • Marc Pollefeys
    • 1
  1. 1.Department of Computer ScienceThe University of North Carolina at Chapel HillChapel HillUSA

Personalised recommendations