Semi-Nonnegative Matrix Factorization for Motion Segmentation with Missing Data

  • Quanyi Mo
  • Bruce A. Draper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7578)


Motion segmentation is an old problem that is receiving renewed interest because of its role in video analysis. In this paper, we present a Semi-Nonnegative Matrix Factorization (SNMF)method that models dense point tracks in terms of their optical flow, and decomposes sets of point tracks into semantically meaningful motion components. We show that this formulation of SNMF with missing values outperforms the state-of-the-art algorithm of Brox and Malik in terms of accuracy on 10-frame video segments from the Berkeley test set, while being over 100 times faster. We then show how SNMF can be applied to longer videos using sliding windows. The result is competitive in terms of accuracy with Brox and Malik’s algorithm, while still being two orders of magnitude faster.


Motion Segmentation Semi-Nonnegative Matrix Factorization(SNMF) Missing Data 


  1. 1.
    Brox, T., Malik, J.: Object Segmentation by Long Term Analysis of Point Trajectories. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 282–295. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  2. 2.
    Vidal, R., Ma, Y.: A Unified Algebraic Approach to 2-D and 3-D Motion Segmentation. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3021, pp. 1–15. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  3. 3.
    Yan, J., Pollefeys, M.: A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-rigid, Degenerate and Non-degenerate. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 94–106. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  4. 4.
    Rao, S., Tron, R., Vidal, R., Ma, Y.: Motion segmentation via robust subspace separation in the presence of outlying, incomplete, or corrupted trajectories. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2008)Google Scholar
  5. 5.
    Fan, Z., Zhou, J., Wu, Y.: Multibody grouping by inference of multiple subspaces from high-dimensional data using oriented-frames. IEEE Transactions on Pattern Analysis and Machine Intelligence 28, 91–105 (2006)CrossRefGoogle Scholar
  6. 6.
    Elhamifar, E., Vidal, R.: Sparse subspace clustering. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2790–2797. IEEE (2009)Google Scholar
  7. 7.
    Cheriyadat, A., Radke, R.: Non-negative matrix factorization of partial track data for motion segmentation. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 865–872. IEEE (2009)Google Scholar
  8. 8.
    Lee, D., Seung, H., et al.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)CrossRefGoogle Scholar
  9. 9.
    Ding, C., Li, T., Jordan, M.: Convex and semi-nonnegative matrix factorizations. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 45–55 (2010)CrossRefGoogle Scholar
  10. 10.
    Hoyer, P.: Non-negative matrix factorization with sparseness constraints. The Journal of Machine Learning Research 5, 1457–1469 (2004)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Hartley, R., Schaffalitzky, F.: Powerfactorization: 3d reconstruction with missing or uncertain data. In: Australia-Japan Advanced Workshop on Computer Vision, vol. 74, pp. 76–85 (2003)Google Scholar
  12. 12.
    Sundaram, N., Brox, T., Keutzer, K.: Dense point trajectories by gpu-accelerated large displacement optical flow. In: Proceedings of the 11th European Conference on Computer Vision, pp. 438–451 (2010)Google Scholar
  13. 13.
    Thurau, C.: PyMF: Python matrix factorization library (2010),
  14. 14.
    Tron, R., Vidal, R.: A benchmark for the comparison of 3-d motion segmentation algorithms. In: IEEE Conference on Computer Vision and Pattern Recognition 2007, pp. 1–8. IEEE (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Quanyi Mo
    • 1
  • Bruce A. Draper
    • 1
  1. 1.Colorado State UniversityUSA

Personalised recommendations