Abstract
We address the problem of annotating a video sequence with partial supervision. Given the pixel-wise annotations in the first frame, we aim to propagate these labels ideally throughout the whole video. While some labels can be propagated using optical flow, disocclusion and unreliable flow in some areas require additional cues. To this end, we propose to train localized classifiers on the annotated frame. In contrast to a global classifier, localized classifiers allow to distinguish colors that appear in both the foreground and the background but at very different locations. We design a multi-scale hierarchy of localized random forests, which collectively takes a decision. Cues from optical flow and the classifier are combined in a variational framework. The approach can deal with multiple objects in a video. We present qualitative and quantitative results on the Berkeley Motion Segmentation Dataset.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence (2011)
Badrinarayanan, V., Galasso, F., Cipolla, R.: Label propagation in video sequences. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2010)
Breiman, L.: Random forests. Machine Learning 45, 5–32 (2001)
Brendel, W., Todorovic, S.: Video object segmentation by tracking regions. In: IEEE International Conference on Computer Vision, ICCV (2009)
Brox, T., Malik, J.: Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence (2010)
Brox, T., Malik, J.: Object Segmentation by Long Term Analysis of Point Trajectories. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 282–295. Springer, Heidelberg (2010)
Budvytis, I., Badrinarayanan, V., Cipolla, R.: Semi-supervised video segmentation using tree structured graphical models. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2011)
Catanzaro, B., Su, B., Sundaram, N., Lee, Y., Murphy, M., Keutzer, K.: Efficient, high-quality image contour detection. In: International Conference on Computer Vision, ICCV (2009)
Chockalingam, P., Pradeep, N., Birchfield, S.: Adaptive fragments-based tracking of non-rigid objects using level sets. In: IEEE International Conference on Computer Vision, ICCV (2009)
Godec, M., Roth, P., Bischof, H.: Hough-based tracking of non-rigid objects. In: IEEE International Conference on Computer Vision, ICCV (2011)
Grundmann, M., Kwatra, V., Han, M., Essa, I.: Efficient hierarchical graph-based video segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2010)
Li, Y., Sun, J., Shum, H.Y.: Video object cut and paste. ACM Trans. Graph. (2005)
Liu, C., Yuen, J., Torralba, A.: Nonparametric scene parsing- label transfer via dense scene alignment. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2009)
Price, B.L., Morse, B.S., Cohen, S.: Livecut: Learning-based interactive video segmentation by evaluation of multiple propagated cues. In: IEEE International Conference on Computer Vision, ICCV (2009)
Ren, X., Malik, J.: Tracking as repeated figure/ground segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2007)
Saffari, A., Leistner, C., Santner, J., Godec, M., Bischof, H.: On-line random forests. In: ICCV 2009 Workshop on On-line Computer Vision (2009)
Stalder, S., Grabner, H., Van Gool, L.: Beyond semi-supervised tracking: Tracking should be as simple as detection, but not simpler than recognition. In: ICCV 2009 Workshop on On-line Learning for Computer Vision (2009)
Tsai, D., Flagg, M., Rehg, J.M.: Motion coherent tracking with multi-label mrf optimization. In: British Machine Vision Conference, BMVC (2010)
Vazquez-Reina, A., Avidan, S., Pfister, H., Miller, E.: Multiple Hypothesis Video Segmentation from Superpixel Flows. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 268–281. Springer, Heidelberg (2010)
Yuen, J., Russell, B., Liu, C., Torralba, A.: Labelme video- building a video database with human annotations. In: IEEE International Conference on Computer Vision, ICCV (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nagaraja, N.S., Ochs, P., Liu, K., Brox, T. (2012). Hierarchy of Localized Random Forests for Video Annotation. In: Pinz, A., Pock, T., Bischof, H., Leberl, F. (eds) Pattern Recognition. DAGM/OAGM 2012. Lecture Notes in Computer Science, vol 7476. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32717-9_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-32717-9_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-32716-2
Online ISBN: 978-3-642-32717-9
eBook Packages: Computer ScienceComputer Science (R0)