Graph Based Energy for Active Object Removal

  • Yimin Yu
  • Duanqing Xu
  • Chun Chen
  • Lei Zhao
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4270)


In this paper, we present a system for completing the blank hole in an image list or a video sequence, which can be used in movie-making industry to produce some special montage effect. To achieve this, we apply a 3D coordinate to depict the video clip. As the time information representing different frames is considered, the available information is enriched than the single image. Our method takes the global character into account for avoiding visual inconsistencies. We view the video completion problem as a labeling problem and apply the belief propagation algorithm to minimize the energy defined on the label graph. By measuring the similarity between source and target patches, we could discover the appropriate patch to fill the hole. Furthermore, we introduce some techniques to speedup the belief propagation algorithm for gaining better performance efficiency. Examples demonstrate the effectiveness of our approach and the results are encouraging.


Video Sequence Unknown Region Image Completion Belief Propagation Algorithm Sample Patch 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bhat, K., Saptharishi, M., Khosla, P.: Motion Detection and Segmentation Using Image Mosaics. In: IEEE International Conference on Multimedia and Expo., July 2000, vol. 3, pp. 1577–1580 (2000)Google Scholar
  2. 2.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(11), 1222–1239 (2001)CrossRefGoogle Scholar
  3. 3.
    Criminisi, A., Pérez, P., Toyama, K.: Object removal by exemplar-based inpainting. In: CVPR (2003)Google Scholar
  4. 4.
    Drori, I., Cohen-Or, D., Yeshurun, H.: Fragment-based image completion. In: ACM TOG. SIGGRAPH (2003)Google Scholar
  5. 5.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient belief propagation for early vision. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 261–268 (2004)Google Scholar
  6. 6.
    Li, Y., Sun, J., Shum, H.-Y.: Video object cut and paste. ACM Transactions on Graphics (TOG) 24(3) (July 2005)Google Scholar
  7. 7.
    Sand, P., Teller, S.: Video Matching. ACM Transactions on Graphics 22(3), 592–599 (2004)CrossRefGoogle Scholar
  8. 8.
    Sun, J., Yuan, L., Jia, J., Shum, H.-Y.: Image completion with structure propagation. In: SIGGRAPH (2005)Google Scholar
  9. 9.
    Wexler, Y., Shechtman, E., Irani, M.: Space-Time Video Completion. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2004), July 2004, vol. 1, pp. 120–127 (2004)Google Scholar
  10. 10.
    Zhang, Y., Xiao, J., Shah, M.: Motion Layer Based Object Removal in Videos. In: IEEE Workshop on Application on Computer Vision, January 5-6, 2005, Breckenridge, Colorado (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Yimin Yu
    • 1
  • Duanqing Xu
    • 1
  • Chun Chen
    • 1
  • Lei Zhao
    • 1
  1. 1.College of Computer ScienceZhejiang UniversityHangzhouP.R. China

Personalised recommendations