Video Inpainting Based on Re-weighted Tensor Decomposition
Video inpainting is the process of improving the information content in a video by removing irrelevant video objects and restoring lost or deteriorated parts utilizing the spatiotemporal features that are available from adjacent frames. This paper proposes an effective video inpainting technique utilizing the multi-dimensional data decomposition technique. In Tensor Robust Principal Component Analysis (TRPCA), a multi-dimensional data corrupted by gross errors is decomposed into a low multi-rank component and a sparse component. The proposed method employs an improved version of TRPCA called Re-weighted low-rank Tensor Decomposition (RWTD) to separate the true information and the irrelevant sparse components in a video. Through this, manual identification of the components which have to be removed is avoided. Subsequent inpainting algorithm fills the region with appropriate and visually plausible data. The capabilities of the proposed method are validated by applying into videos having moving sparse outliers in it. The experimental results reveal that the proposed method performs well compared with other techniques.
KeywordsVideo inpainting Tensor decomposition Sparsity Low-rank tensor recovery
- 1.S. Moran, “Video inpainting,” vol. 1, pp. 12–25, 2009.Google Scholar
- 2.W. Zhang, S. Cheung, and M. Chen, “Hiding privacy information in video surveillance system,” in Image Processing, 2005. ICIP 2005. IEEE International Conference on, vol. 3. IEEE, 2005, pp. II–868.Google Scholar
- 3.Y. Umeda and K. Arakawa, “Removal of film scratches using exemplar-based inpainting with directional median filter,” in Communications and Information Technologies (ISCIT), 2012 International Symposium on. IEEE, 2012, pp. 6–11.Google Scholar
- 5.V. V. Mahalingam, Digital inpainting algorithms and evaluation. University of Kentucky, 2010.Google Scholar
- 6.M. Bertalmio, A. L. Bertozzi, and G. Sapiro, “Navier-stokes, fluid dynamics, and image and video inpainting,” in Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on, vol. 1. IEEE, 2001, pp. I–I.Google Scholar
- 7.W.-Q. Yan and M. S. Kankanhalli, “Erasing video logos based on image inpainting,” in Multimedia and Expo, 2002. ICME’02. Proceedings. 2002 IEEE International Conference on, vol. 2. IEEE, 2002, pp. 521–524.Google Scholar
- 11.P. Kumar and P. Puttaswamy, “Moving text line detection and extraction in tv video frames,” in Advance Computing Conference (IACC), 2015 IEEE International. IEEE, 2015, pp. 24–28.Google Scholar
- 12.C. Lu, J. Feng, Y. Chen, W. Liu, Z. Lin, and S. Yan, “Tensor robust principal component analysis: Exact recovery of corrupted low-rank tensors via convex optimization,” in Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, (CVPR), 2016.Google Scholar
- 14.B. M. and S. N. George, “Reweighted low-rank tensor decomposition and its applications in video denoising,” CoRR, vol. abs/1611.05963, 2016. [Online]. Available.Google Scholar
- 19.M. Yan and W. Yin, “Self equivalence of the alternating direction method of multipliers,” arXiv preprint arXiv:1407.7400, 2014.
- 20.X. Yuan, “Alternating direction methods for sparse covariance selection,” preprint, 2009.Google Scholar
- 21.Z. Lin, R. Liu, and Z. Su, “Linearized alternating direction method with adaptive penalty for low-rank representation,” in Advances in neural information processing systems, 2011, pp. 612–620.Google Scholar
- 23.https://media.xiph.org/video/derf/. [Online]. Available.