Temporal Factorization vs. Spatial Factorization
Conference paper
Abstract
The traditional subspace-based approaches to segmentation (often referred to as multi-body factorization approaches) provide spatial clustering/segmentation by grouping together points moving with consistent motions. We are exploring a dual approach to factorization, i.e., obtaining temporal clustering/segmentation by grouping together frames capturing consistent shapes. Temporal cuts are thus detected at non-rigid changes in the shape of the scene/object. In addition it provides a clustering of the frames with consistent shape (but not necessarily same motion). For example, in a sequence showing a face which appears serious at some frames, and is smiling in other frames, all the “serious expression” frames will be grouped together and separated from all the “smile” frames which will be classified as a second group, even though the head may meanwhile undergo various random motions.
Keywords
Video Clip Temporal Factorization Spectral Cluster Spatial Factorization Temporal Cluster
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Download
to read the full conference paper text
References
- 1.Birchfield, S.: Klt: An implementation of the kanade-lucas-tomasi feature tracker, http://robotics.stanford.edu/~birch/klt/
- 2.Black, M.J.: Dense optical flow: robust regularization, http://www.cs.brown.edu/people/black/
- 3.Black, M.J., Anandan, P.: A framework for the robust estimation of optical flow. In: International Conference on Computer Vision, Berlin, Germany, pp. 231–236 (1993)Google Scholar
- 4.Black, M.J., Anandan, P.: The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding 63(1), 75–104 (1996)CrossRefGoogle Scholar
- 5.Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3d shape from image streams. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. II, pp. 690–696 (2000)Google Scholar
- 6.Costeira, J., Kanade, T.: A multi-body factorization method for motion analysis. In: International Conference on Computer Vision, Cambridge, MA, June 1995, pp. 1071–1076 (1995)Google Scholar
- 7.Gear, C.W.: Multibody grouping from motion images. International Journal of Computer Vision 2(29), 133–150 (1998)CrossRefGoogle Scholar
- 8.Irani, M.: Multi-frame correspondence estimation using subspace constraints. International Journal of Computer Vision 48(3), 173–194 (2002)zbMATHCrossRefMathSciNetGoogle Scholar
- 9.Kanatani, K.: Motion segmentation by subspace separation and model selection. In: International Conference on Computer Vision, Vancouver, Canada, vol. 1, pp. 301–306 (2001)Google Scholar
- 10.Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Image Understanding Workshop, pp. 121–130 (1981)Google Scholar
- 11.Machline, M., Zelnik-Manor, L., Irani, M.: Multi-body segmentation: Revisiting motion consistency. In: Tistarelli, M., Bigun, J., Jain, A.K. (eds.) ECCV 2002. LNCS, vol. 2359, Springer, Heidelberg (2002)Google Scholar
- 12.Nagasaka, A., Tanaka, Y.: Automatic video indexing and full-video search for object appearances. In: Visual Databas Systems II, IFIP (1992)Google Scholar
- 13.Ng, A., Jordan, M., Weiss, Y.: On spectral clustering: Analysis and an algorithm. In: Advances in Neural Information Processing Systems 14 (2001)Google Scholar
- 14.Rao, C., Shah, M.: Motion segmentation by subspace separation and model selection. In: International Conference on Computer Vision, Vancouver, Canada, vol. 1, pp. 301–306 (2001)Google Scholar
- 15.Rui, Y., Anandan, P.: Segmenting visual actions based on spatio-temporal motion patterns. In: IEEE Conference on Computer Vision and Pattern Recognition (June 2000)Google Scholar
- 16.Swanberg, S., Shu, D.F., Jain, R.: Knowledge guided parsing in video databases. In: SPIE (1993)Google Scholar
- 17.Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: A factorization method. International Journal of Computer Vision 9, 137–154 (1992)CrossRefGoogle Scholar
- 18.Weiss, Y.: Segmentation using eigenvectors: A unifying view. In: International Conference on Computer Vision, Corfu, Greece, September 1999, pp. 975–982 (1999)Google Scholar
- 19.Zelnik-Manor, L., Irani, M.: Event-based video analysis. In: IEEE Conference on Computer Vision and Pattern Recognition, Kauai, Hawaii (2001)Google Scholar
- 20.Zhang, H., Kankanhali, A., Smoliar, W.: Automatic partitioning of full-motion video. In: Multimedia Systems (1993)Google Scholar
Copyright information
© Springer-Verlag Berlin Heidelberg 2004