Occlusion Handling in Video Segmentation via Predictive Feedback
We present a method for unsupervised on-line dense video segmentation which utilizes sequential Bayesian estimation techniques to resolve partial and full occlusions. Consistent labeling through occlusions is vital for applications which move from low-level object labels to high-level semantic knowledge - tasks such as activity recognition or robot control. The proposed method forms a predictive loop between segmentation and tracking, with tracking predictions used to seed the segmentation kernel, and segmentation results used to update tracked models. All segmented labels are tracked, without the use of a-priori models, using parallel color-histogram particle filters. Predictions are combined into a probabilistic representation of image labels, a realization of which is used to seed segmentation. A simulated annealing relaxation process allows the realization to converge to a minimal energy segmented image. Found segments are subsequently used to repopulate the particle sets, closing the loop. Results on the Cranfield benchmark sequence demonstrate that the prediction mechanism allows on-line segmentation to maintain temporally consistent labels through partial & full occlusions, significant appearance changes, and rapid erratic movements. Additionally, we show that tracking performance matches state-of-the art tracking methods on several challenging benchmark sequences.
Unable to display preview. Download preview PDF.
- 2.Liu, S., Dong, G., Yan, C.H., Ong, S.H.: Video segmentation: Propagation, validation and aggregation of a preceding graph. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2008)Google Scholar
- 3.Hedau, V., Arora, H., Ahuja, N.: Matching images under unstable segmentations. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2008)Google Scholar
- 4.Brendel, W., Todorovic, S.: Video object segmentation by tracking regions. In: IEEE International Conference on Computer Vision, ICCV (2009)Google Scholar
- 7.Ablavsky, V., Thangali, A., Sclaroff, S.: Layered graphical models for tracking partially-occluded objects. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2008)Google Scholar
- 9.Abramov, A., Pauwels, K., Papon, J., Wörgötter, F., Dellen, B.: Real-time segmentation of stereo videos on a portable system with a mobile gpu. IEEE Transactions on Circuits and Systems for Video Technology (in press)Google Scholar
- 11.Doucet, A., De Freitas, N., Gordon, N. (eds.): Sequential Monte Carlo methods in practice (2001)Google Scholar
- 12.Santner, J., Leistner, C., Saffari, A., Pock, T., Bischof, H.: Prost: Parallel robust online simple tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2010)Google Scholar
- 13.Babenko, B., Yang, M.H., Belongie, S.: Visual tracking with online multiple instance learning. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2009)Google Scholar
- 14.Adam, A., Rivlin, E., Shimshoni, I.: Robust fragments-based tracking using the integral histogram. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2006)Google Scholar
- 15.Saffari, A., Leistner, C., Santner, J., Godec, M., Bischof, H.: On-line random forests. In: IEEE International Conference on Computer Vision Workshops, ICCV Workshops (2009)Google Scholar