Abstract
Spatiotemporal segmentation is an essential task for video analysis. The strong interconnection between finding an object’s spatial support and finding its motion characteristics makes the problem particularly challenging. Motivated by closure detection techniques in 2D images, this paper introduces the concept of spatiotemporal closure. Treating the spatiotemporal volume as a single entity, we extract contiguous “tubes” whose overall surface is supported by strong appearance and motion discontinuities. Formulating our closure cost over a graph of spatiotemporal superpixels, we show how it can be globally minimized using the parametric maxflow framework in an efficient manner. The resulting approach automatically recovers coherent spatiotemporal components, corresponding to objects, object parts, and object unions, providing a good set of multiscale spatiotemporal hypotheses for high-level video analysis.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Wang, D.: Unsupervised video segmentation based on watersheds and temporal tracking. CirSysVideo 8, 539–546 (1998)
Moscheni, F., Bhattacharjee, S., Kunt, M.: Spatiotemporal segmentation based on region merging. PAMI 20, 897–915 (1998)
Gelgon, M., Bouthemy, P.: A region-level motion-based graph representation and labeling for tracking a spatial image partition. Pattern Recognition 33, 725–740 (2000)
Deng, Y., Manjunath, B.: Unsupervised segmentation of color-texture regions in images and video. PAMI 23, 800–810 (2001)
DeMenthon, D.: Spatio-temporal segmentation of video by hierarchical mean shift analysis. In: SMVP (2002)
Shi, J., Malik, J.: Motion segmentation and tracking using normalized cuts. In: ICCV, pp. 1154–1160 (1998)
Fowlkes, C., Belongie, S., Malik, J.: Efficient spatiotemporal grouping using the nyström method. In: CVPR, pp. 231–238 (2001)
Huang, Y., Liu, Q., Metaxas, D.: Video object segmentation by hypergraph cut. In: CVPR, pp. 1738–1745 (2009)
Greenspan, H., Goldberger, J., Mayer, A.: Probabilistic space-time video modeling via piecewise gmm. PAMI 26, 384–396 (2004)
Levinshtein, A., Sminchisescu, C., Dickinson, S.: Optimal Contour Closure by Superpixel Grouping. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6312, pp. 480–493. Springer, Heidelberg (2010)
Levinshtein, A., Stere, A., Kutulakos, K.N., Fleet, D.J., Dickinson, S.J., Siddiqi, K.: Turbopixels: Fast superpixels using geometric flows. PAMI 31, 2290–2297 (2009)
Carreira, J., Sminchisescu, C.: Constrained parametric min-cuts for automatic object segmentation. In: CVPR (2010)
Kolmogorov, V., Boykov, Y., Rother, C.: Applications of parametric maxflow in computer vision. In: ICCV (2007)
Li, F., Carreira, J., Sminchisescu, C.: Object Recognition as Ranking Holistic Figure-Ground Hypotheses. In: CVPR (2010)
Pritch, Y., Rav-Acha, A., Peleg, S.: Nonchronological video synopsis and indexing. PAMI 30, 1971–1984 (2008)
Welch, G., Bishop, G.: An introduction to the kalman filter. Technical report (1995)
Black, M., Jepson, A.: Eigentracking: Robust matching and tracking of articulated objects using a view-based representation. IJCV 26, 63–84 (1998)
Isard, M., Blake, A.: Condensation - conditional density propagation for visual tracking. IJCV 29, 5–28 (1998)
Comaniciu, D., Ramesh, V., Meer, P.: Kernel-based object tracking. PAMI 25, 564–577 (2003)
Megret, R., DeMenthon, D.: A survey of spatio-temporal grouping techniques. Technical report, University of Maryland, College Park (2002)
Wang, J., Adelson, E.: Representing moving images with layers. TIP 3, 625–638 (1994)
Weiss, Y., Adelson, E.H.: A unified mixture framework for motion segmentation: Incorporating spatial coherence and estimating the number of models. In: CVPR, p. 321 (1996)
Weiss, Y.: Smoothness in layers: Motion segmentation using nonparametric mixture estimation. In: CVPR, p. 520 (1997)
Jojic, N., Frey, B.J.: Learning flexible sprites in video layers. In: CVPR, vol. 1, p. 199 (2001)
Jepson, A.D., Fleet, D.J., Black, M.J.: A layered motion representation with occlusion and compact spatial support. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 692–706. Springer, Heidelberg (2002)
Bascle, B., Deriche, R.: Region tracking through image sequences. In: ICCV, p. 302 (1995)
Paragios, N., Deriche, R.: Geodesic active contours and level sets for the detection and tracking of moving objects. PAMI 22, 266–280 (2000)
Chung, D., MacLean, W., Dickinson, S.: Integrating region and boundary information for spatially coherent object tracking. IVC 24, 680–692 (2006)
Cremers, D., Soatto, S.: Motion competition: A variational approach to piecewise parametric motion segmentation. IJCV 62, 249–265 (2005)
Patras, I., Lagendijk, R.L., Hendriks, E.A.: Video segmentation by map labeling of watershed segments. PAMI 23, 326–332 (2001)
Martin, D.R., Fowlkes, C.C., Malik, J.: Learning to detect natural image boundaries using local brightness, color, and texture cues. PAMI 26, 530–549 (2004)
Stein, A., Hoiem, D., Hebert, M.: Learning to find object boundaries using motion cues. In: ICCV (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Levinshtein, A., Sminchisescu, C., Dickinson, S. (2011). Spatiotemporal Closure. In: Kimmel, R., Klette, R., Sugimoto, A. (eds) Computer Vision – ACCV 2010. ACCV 2010. Lecture Notes in Computer Science, vol 6492. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-19315-6_29
Download citation
DOI: https://doi.org/10.1007/978-3-642-19315-6_29
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-19314-9
Online ISBN: 978-3-642-19315-6
eBook Packages: Computer ScienceComputer Science (R0)