Advertisement

Temporal Factorization vs. Spatial Factorization

  • Lihi Zelnik-Manor
  • Michal Irani
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3022)

Abstract

The traditional subspace-based approaches to segmentation (often referred to as multi-body factorization approaches) provide spatial clustering/segmentation by grouping together points moving with consistent motions. We are exploring a dual approach to factorization, i.e., obtaining temporal clustering/segmentation by grouping together frames capturing consistent shapes. Temporal cuts are thus detected at non-rigid changes in the shape of the scene/object. In addition it provides a clustering of the frames with consistent shape (but not necessarily same motion). For example, in a sequence showing a face which appears serious at some frames, and is smiling in other frames, all the “serious expression” frames will be grouped together and separated from all the “smile” frames which will be classified as a second group, even though the head may meanwhile undergo various random motions.

Keywords

Video Clip Temporal Factorization Spectral Cluster Spatial Factorization Temporal Cluster 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Birchfield, S.: Klt: An implementation of the kanade-lucas-tomasi feature tracker, http://robotics.stanford.edu/~birch/klt/
  2. 2.
    Black, M.J.: Dense optical flow: robust regularization, http://www.cs.brown.edu/people/black/
  3. 3.
    Black, M.J., Anandan, P.: A framework for the robust estimation of optical flow. In: International Conference on Computer Vision, Berlin, Germany, pp. 231–236 (1993)Google Scholar
  4. 4.
    Black, M.J., Anandan, P.: The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding 63(1), 75–104 (1996)CrossRefGoogle Scholar
  5. 5.
    Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3d shape from image streams. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. II, pp. 690–696 (2000)Google Scholar
  6. 6.
    Costeira, J., Kanade, T.: A multi-body factorization method for motion analysis. In: International Conference on Computer Vision, Cambridge, MA, June 1995, pp. 1071–1076 (1995)Google Scholar
  7. 7.
    Gear, C.W.: Multibody grouping from motion images. International Journal of Computer Vision 2(29), 133–150 (1998)CrossRefGoogle Scholar
  8. 8.
    Irani, M.: Multi-frame correspondence estimation using subspace constraints. International Journal of Computer Vision 48(3), 173–194 (2002)zbMATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Kanatani, K.: Motion segmentation by subspace separation and model selection. In: International Conference on Computer Vision, Vancouver, Canada, vol. 1, pp. 301–306 (2001)Google Scholar
  10. 10.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Image Understanding Workshop, pp. 121–130 (1981)Google Scholar
  11. 11.
    Machline, M., Zelnik-Manor, L., Irani, M.: Multi-body segmentation: Revisiting motion consistency. In: Tistarelli, M., Bigun, J., Jain, A.K. (eds.) ECCV 2002. LNCS, vol. 2359, Springer, Heidelberg (2002)Google Scholar
  12. 12.
    Nagasaka, A., Tanaka, Y.: Automatic video indexing and full-video search for object appearances. In: Visual Databas Systems II, IFIP (1992)Google Scholar
  13. 13.
    Ng, A., Jordan, M., Weiss, Y.: On spectral clustering: Analysis and an algorithm. In: Advances in Neural Information Processing Systems 14 (2001)Google Scholar
  14. 14.
    Rao, C., Shah, M.: Motion segmentation by subspace separation and model selection. In: International Conference on Computer Vision, Vancouver, Canada, vol. 1, pp. 301–306 (2001)Google Scholar
  15. 15.
    Rui, Y., Anandan, P.: Segmenting visual actions based on spatio-temporal motion patterns. In: IEEE Conference on Computer Vision and Pattern Recognition (June 2000)Google Scholar
  16. 16.
    Swanberg, S., Shu, D.F., Jain, R.: Knowledge guided parsing in video databases. In: SPIE (1993)Google Scholar
  17. 17.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: A factorization method. International Journal of Computer Vision 9, 137–154 (1992)CrossRefGoogle Scholar
  18. 18.
    Weiss, Y.: Segmentation using eigenvectors: A unifying view. In: International Conference on Computer Vision, Corfu, Greece, September 1999, pp. 975–982 (1999)Google Scholar
  19. 19.
    Zelnik-Manor, L., Irani, M.: Event-based video analysis. In: IEEE Conference on Computer Vision and Pattern Recognition, Kauai, Hawaii (2001)Google Scholar
  20. 20.
    Zhang, H., Kankanhali, A., Smoliar, W.: Automatic partitioning of full-motion video. In: Multimedia Systems (1993)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Lihi Zelnik-Manor
    • 1
  • Michal Irani
    • 2
  1. 1.California Institute of TechnologyPasadenaUSA
  2. 2.Weizmann Institute of ScienceRehovotIsrael

Personalised recommendations