Abstract
The topic of image completion has received increasing attention in recent years, motivated by many important applications in computer vision, data mining and image processing. In this study, we consider the problem of recovering missing values of pixels in highly incomplete images with a random or irregular structure. The analyzed gray-scale or colour images are transformed to multi-way arrays which are then recursively approximated by low-rank tensor decomposition models. In our approach, the multi-way array is represented by the tensor train model, and in each iterative step, the low-rank approximation is filtered with the Gaussian low-pass filter. As a result, the proposed algorithms considerably outperform the state-of-the art methods for matrix and tensor completion problems, especially when an incompleteness degree is very high, e.g. with 90% of missing pixels.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
The tensor train contraction between the tensors \(\mathcal {A} \in \mathbb R_+^{I_1 \times \ldots \times I_N}\) and \(\mathcal {B} \in \mathbb R_+^{J_1 \times \ldots \times J_M}\) with \(I_N = J_1\) performs a tensor contraction between the last mode of \(\mathcal {A}\) and the first mode of \(\mathcal {B}\), which results in \(\mathcal {C} = \mathcal {A} \bullet \mathcal {B} \in \mathbb R_+^{I_1 \times \ldots \times I_{N-1} \times J_2 \times \ldots \times J_M}\).
References
Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 417–424. ACM Press/Addison-Wesley Publishing Co. (2000)
Criminisi, A., Perez, P., Toyama, K.: Object removal by exemplar-based inpainting. In: IEEE Computer Vision and Pattern Recognition (CVPR) (2003)
Sun, J., Yuan, L., Jia, J., Shum, H.Y.: Image completion with structure propagation. ACM Trans. Graph. 24(3), 861–868 (2005)
Darabi, S., Shechtman, E., Barnes, C., Goldman, D.B., Sen, P.: Image melding: combining inconsistent images using patch-based synthesis. ACM Trans. Graph. 31(4), 82:1–82:10 (2012)
Iizuka, S., Simo-Serra, E., Ishikawa, H.: Globally and locally consistent image completion. ACM Trans. Graph. 36(4), 107:1–107:14 (2017)
Pathak, D., Krähenbühl, P., Donahue, J., Darrell, T., Efros, A.: Context encoders: feature learning by inpainting. In: IEEE Computer Vision and Pattern Recognition (CVPR) (2016)
Cai, J.F., Candès, E.J., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20(4), 1956–1982 (2010)
Zhang, D., Hu, Y., Ye, J., Li, X., He, X.: Matrix completion by truncated nuclear norm regularization. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2192–2199 (2012)
Zhang, M., Desrosiers, C.: Image completion with global structure and weighted nuclear norm regularization. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 4187–4193 (2017)
Wen, Z., Yin, W., Zhang, Y.: Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm. Math. Program. Comput. 4(4), 333–361 (2012)
Wang, Y., Zhang, Y.: Image inpainting via weighted sparse non-negative matrix factorization. In: 18th IEEE International Conference on Image Processing, pp. 3409–3412 (2011)
Sadowski, T., Zdunek, R.: Image completion with smooth nonnegative matrix factorization. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds.) ICAISC 2018. LNCS (LNAI), vol. 10842, pp. 62–72. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91262-2_6
Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.I.: Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. Wiley, Hoboken (2009)
Acar, E., Dunlavy, D.M., Kolda, T.G., Mørup, M.: Scalable tensor factorizations with missing data. In: 2010 SIAM International Conference on Data Mining, pp. 701–712 (2010)
Zhao, Q., Zhang, L., Cichocki, A.: Bayesian CP factorization of incomplete tensors with automatic rank determination. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1751–1763 (2015)
Yokota, T., Zhao, Q., Cichocki, A.: Smooth PARAFAC decomposition for tensor completion. IEEE Trans. Signal Process. 64(20), 5423–5436 (2016)
Latorre, J.I.: Image compression and entanglement. arXiv:quant-ph/0510031, Quantum Physics (2005)
Oseledets, I.: Tensor-train decomposition. SIAM J. Sci. Comput. 33(5), 2295–2317 (2011)
Bengua, J.A., Phien, H.N., Tuan, H.D., Do, M.N.: Efficient tensor completion for color image and video recovery: low-rank tensor train. IEEE Trans. Image Process. 26(5), 2466–2479 (2017)
Ko, C.Y., Batselier, K., Yu, W., Wong, N.: Fast and accurate tensor completion with tensor trains: a system identification approach. CoRR abs/1804.06128 (2018)
Wang, W., Aggarwal, V., Aeron, S.: Efficient low rank tensor ring completion. In: IEEE International Conference on Computer Vision (ICCV), pp. 5698–5706 (2018)
Cichocki, A., Lee, N., Oseledets, I.V., Phan, A.H., Zhao, Q., Mandic, D.P.: Tensor networks for dimensionality reduction and large-scale optimization: Part 1 low-rank tensor decompositions. Found. Trends Mach. Learn. 9(4–5), 249–429 (2016)
Sadowski, T., Zdunek, R.: Modified HALS algorithm for image completion and recommendation system. In: Świątek, J., Borzemski, L., Wilimowska, Z. (eds.) ISAT 2017. AISC, vol. 656, pp. 17–27. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-67229-8_2
Xu, Y., Hao, R., Yin, W., Su, Z.: Parallel matrix factorization for low-rank tensor completion. Inverse Probl. Imaging 9(2), 601–624 (2015)
Acknowledgment
This work was partially supported by the statutory grant, no. 401/0034/18, and partially by the grant 2015/17/B/ST6/01865 funded by National Science Center (NCN) in Poland.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Zdunek, R., Fonał, K., Sadowski, T. (2019). Image Completion with Filtered Low-Rank Tensor Train Approximations. In: Rojas, I., Joya, G., Catala, A. (eds) Advances in Computational Intelligence. IWANN 2019. Lecture Notes in Computer Science(), vol 11507. Springer, Cham. https://doi.org/10.1007/978-3-030-20518-8_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-20518-8_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-20517-1
Online ISBN: 978-3-030-20518-8
eBook Packages: Computer ScienceComputer Science (R0)