Occlusion Handling in Video Segmentation via Predictive Feedback

  • Jeremie Papon
  • Alexey Abramov
  • Florentin Wörgötter
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7585)


We present a method for unsupervised on-line dense video segmentation which utilizes sequential Bayesian estimation techniques to resolve partial and full occlusions. Consistent labeling through occlusions is vital for applications which move from low-level object labels to high-level semantic knowledge - tasks such as activity recognition or robot control. The proposed method forms a predictive loop between segmentation and tracking, with tracking predictions used to seed the segmentation kernel, and segmentation results used to update tracked models. All segmented labels are tracked, without the use of a-priori models, using parallel color-histogram particle filters. Predictions are combined into a probabilistic representation of image labels, a realization of which is used to seed segmentation. A simulated annealing relaxation process allows the realization to converge to a minimal energy segmented image. Found segments are subsequently used to repopulate the particle sets, closing the loop. Results on the Cranfield benchmark sequence demonstrate that the prediction mechanism allows on-line segmentation to maintain temporally consistent labels through partial & full occlusions, significant appearance changes, and rapid erratic movements. Additionally, we show that tracking performance matches state-of-the art tracking methods on several challenging benchmark sequences.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Vazquez-Reina, A., Avidan, S., Pfister, H., Miller, E.: Multiple Hypothesis Video Segmentation from Superpixel Flows. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 268–281. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  2. 2.
    Liu, S., Dong, G., Yan, C.H., Ong, S.H.: Video segmentation: Propagation, validation and aggregation of a preceding graph. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2008)Google Scholar
  3. 3.
    Hedau, V., Arora, H., Ahuja, N.: Matching images under unstable segmentations. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2008)Google Scholar
  4. 4.
    Brendel, W., Todorovic, S.: Video object segmentation by tracking regions. In: IEEE International Conference on Computer Vision, ICCV (2009)Google Scholar
  5. 5.
    Vermaak, J., Godsill, S., Perez, P.: Monte carlo filtering for multi target tracking and data association. IEEE Transactions on Aerospace and Electronic Systems 41, 309–332 (2005)CrossRefGoogle Scholar
  6. 6.
    Papadakis, N., Bugeau, A.: Tracking with occlusions via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 144–157 (2011)CrossRefGoogle Scholar
  7. 7.
    Ablavsky, V., Thangali, A., Sclaroff, S.: Layered graphical models for tracking partially-occluded objects. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2008)Google Scholar
  8. 8.
    Pérez, P., Hue, C., Vermaak, J., Gangnet, M.: Color-Based Probabilistic Tracking. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part I. LNCS, vol. 2350, pp. 661–675. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  9. 9.
    Abramov, A., Pauwels, K., Papon, J., Wörgötter, F., Dellen, B.: Real-time segmentation of stereo videos on a portable system with a mobile gpu. IEEE Transactions on Circuits and Systems for Video Technology (in press)Google Scholar
  10. 10.
    Blatt, M., Wiseman, S., Domany, E.: Superparamagnetic clustering of data. Physical Review Letters 76, 3251–3254 (1996)CrossRefGoogle Scholar
  11. 11.
    Doucet, A., De Freitas, N., Gordon, N. (eds.): Sequential Monte Carlo methods in practice (2001)Google Scholar
  12. 12.
    Santner, J., Leistner, C., Saffari, A., Pock, T., Bischof, H.: Prost: Parallel robust online simple tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2010)Google Scholar
  13. 13.
    Babenko, B., Yang, M.H., Belongie, S.: Visual tracking with online multiple instance learning. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2009)Google Scholar
  14. 14.
    Adam, A., Rivlin, E., Shimshoni, I.: Robust fragments-based tracking using the integral histogram. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2006)Google Scholar
  15. 15.
    Saffari, A., Leistner, C., Santner, J., Godec, M., Bischof, H.: On-line random forests. In: IEEE International Conference on Computer Vision Workshops, ICCV Workshops (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Jeremie Papon
    • 1
  • Alexey Abramov
    • 1
  • Florentin Wörgötter
    • 1
  1. 1.Bernstein Center for Computational Neuroscience (BCCN), III Physikalisches Institut - BiophysikGeorg-August University of GöttingenGermany

Personalised recommendations