Video Completion for Indoor Scenes

  • Vardhman Jain
  • P. J. Narayanan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4338)


In this paper, we present a new approach for object removal and video completion of indoor scenes. In indoor images, the frames are not affine related. The region near the object to be removed can have multiple planes with sharply different motions. Dense motion estimation may fail for such scenes due to missing pixels. We use feature tracking to find dominant motion between two frames. The geometry of the motion of multiple planes is used to segment the motion layers into component planes. The homography corresponding to each hole pixel is used to warp a frame in the future or past for filling it. We show the application of our technique on some typical indoor videos.


Feature Tracking Texture Synthesis Motion Segmentation Image Inpainting Unknown Region 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: SIGGRAPH 2000: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, pp. 417–424. ACM Press/Addison-Wesley Publishing Co, New York (2000)CrossRefGoogle Scholar
  2. 2.
    Efros, A., Leung, T.: Texture synthesis by non-parametric sampling. In: IEEE International Conference on Computer Vision, Corfu, Greece, pp. 1033–1038 (1999)Google Scholar
  3. 3.
    Efros, A.A., Freeman, W.T.: Image quilting for texture synthesis and transfer. Proceedings of SIGGRAPH 2001, 341–346 (2001)Google Scholar
  4. 4.
    Rother, C., Kolmogorov, V., Blake, A.: “grabcut”: Interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23, 309–314 (2004)CrossRefGoogle Scholar
  5. 5.
    Bertalmio, M., Vese, L., Sapiro, G., Osher, S.: Simultaneous structure and texture image inpainting. In: CVPR 2003, pp. 707–712 (2003)Google Scholar
  6. 6.
    Criminisi, A., Perez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on Image Processing 13, 1200–1212 (2004)CrossRefGoogle Scholar
  7. 7.
    Kang, S., Chan, T., Soatto, S.: Landmark based inpainting from multiple views. Technical report, UCLA Math CAM (2002)Google Scholar
  8. 8.
    Wexler, Y., Shechtman, E., Irani, M.: Space-time video completion. In: CVPR 2004, pp. 120–127 (2004)Google Scholar
  9. 9.
    Jia, J., Wu, T.P., Tai, Y.W., Tang, C.K.: Video repairing: Inference of foreground and background under severe occlusion. In: CVPR 2004, vol. 1 (2004)Google Scholar
  10. 10.
    Kokaram, A., Collis, B., Robinson, S.: A bayesian framework for recursive object removal in movie post-production. In: ICIP 2003, pp. 937–940 (2003)Google Scholar
  11. 11.
    Matsushita, Y., Ofek, E., Tang, X., Shum, H.: Full frame video stabilization. In: CVPR 2005 (2005)Google Scholar
  12. 12.
    Zhang, Y., Xiao, J., Shah, M.: Motion layer based object removal in videos. WACV/Motion 01, 516–521 (2005)Google Scholar
  13. 13.
    Vincent, E., Laganiere, R.: Detecting planar homographies in an image pair. In: Symposium on Image and Signal Processing and Analysis (ISPA 2001) (2001)Google Scholar
  14. 14.
    Fraundorfer, F., Schindler, K., Bischof, H.: Piecewise planar scene reconstruction from sparse correspondences. Image and Vision Computing (2006)Google Scholar
  15. 15.
    Wills, J., Agarwal, S., Belongie, S.: What went where. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 37–44 (2003)Google Scholar
  16. 16.
    Jain, V., Narayanan, P.: Layer extraction using graph cuts and feature tracking. In: Proceedings of the third, International Conference on Visual Information Engineering, pp. 292–297 (2006)Google Scholar
  17. 17.
    Li, Y., Sun, J., Shum, H.: Video object cut and paste. ACM Trans. Graph. 24, 595–600 (2005)CrossRefGoogle Scholar
  18. 18.
    Fischler, M., Bolles, R.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communication of the ACM 24, 381–395 (1981)CrossRefMathSciNetGoogle Scholar
  19. 19.
    Hartley, R.: In defence of the 8-point algorithm. In: ICCV 1995: Proceedings of the Fifth International Conference on Computer Vision, Washington, DC, USA, p. 1064. IEEE Computer Society, Los Alamitos (1995)CrossRefGoogle Scholar
  20. 20.
    Johansson, B.: View synthesis and 3d reconstruction of piecewise planar scenes using intersection lines between the planes. In: IEEE International Conference on Computer Vision, vol. 1, pp. 54–59 (1999)Google Scholar
  21. 21.
    Bhat, P., Zheng, K., Snavely, N., Agarwala, A., Agrawala, M., Cohen, M., Curless, B.: Piecewise image registration in the presence of multiple large motions. In: CVPR 2006, pp. 2491–2497 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Vardhman Jain
    • 1
  • P. J. Narayanan
    • 1
  1. 1.Center for Visual Information TechnologyInternational Institute of Information TechnologyHyderabadIndia

Personalised recommendations