MARLow: A Joint Multiplanar Autoregressive and Low-Rank Approach for Image Completion
In this paper, we propose a novel multiplanar autoregressive (AR) model to exploit the correlation in cross-dimensional planes of a similar patch group collected in an image, which has long been neglected by previous AR models. On that basis, we then present a joint multiplanar AR and low-rank based approach (MARLow) for image completion from random sampling, which exploits the nonlocal self-similarity within natural images more effectively. Specifically, the multiplanar AR model constraints the local stationarity in different cross-sections of the patch group, while the low-rank minimization captures the intrinsic coherence of nonlocal patches. The proposed approach can be readily extended to multichannel images (e.g. color images), by simultaneously considering the correlation in different channels. Experimental results demonstrate that the proposed approach significantly outperforms state-of-the-art methods, even if the pixel missing rate is as high as 90 %.
KeywordsImage completion Multiplanar autoregressive model Low-rank minimization
This work was supported by National High-tech Technology R&D Program (863 Program) of China under Grant 2014AA015205, National Natural Science Foundation of China under contract No. 61472011 and Beijing Natural Science Foundation under contract No. 4142021.
- 9.Goh, W., Chong, M., Kalra, S., Krishnan, D.: Bi-directional 3D auto-regressive model approach to motion picture restoration. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 4, pp. 2275–2278, May 1996Google Scholar
- 11.Heide, F., Heidrich, W., Wetzstein, G.: Fast and flexible convolutional sparse coding. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5135–5143, June 2015Google Scholar
- 12.Ji, H., Liu, C., Shen, Z., Xu, Y.: Robust video denoising using low rank matrix completion. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1791–1798, June 2010Google Scholar
- 13.Kokaram, A., Rayner, P.: Detection and interpolation of replacement noise in motion picture sequences using 3D autoregressive modelling. In: IEEE International Symposium on Circuits and Systems, vol. 3, pp. 21–24, May 1994Google Scholar
- 15.Liu, J., Musialski, P., Wonka, P., Ye, J.: Tensor completion for estimating missing values in visual data. In: IEEE International Conference on Computer Vision, pp. 2114–2121, September 2009Google Scholar
- 17.Mairal, J., Bach, F., Ponce, J., Sapiro, G., Zisserman, A.: Non-local sparse models for image restoration. In: IEEE International Conference on Computer Vision, pp. 2272–2279, September 2009Google Scholar
- 19.Roth, S., Black, M.: Fields of experts: a framework for learning image priors. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 860–867, June 2005Google Scholar
- 21.Takeda, H., Farsiu, S., Milanfar, P.: Robust kernel regression for restoration and reconstruction of images from sparse noisy data. In: IEEE International Conference on Image Processing, pp. 1257–1260 (2006)Google Scholar
- 22.Zhang, D., Hu, Y., Ye, J., Li, X., He, X.: Matrix completion by truncated nuclear norm regularization. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2192–2199, June 2012Google Scholar
- 26.Zhang, Z., Ely, G., Aeron, S., Hao, N., Kilmer, M.: Novel methods for multilinear data completion and de-noising based on tensor-svd. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3842–3849, June 2014Google Scholar