Repairing Sparse Low-Rank Texture

  • Xiao Liang
  • Xiang Ren
  • Zhengdong Zhang
  • Yi Ma
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7576)


In this paper, we show how to harness both low-rank and sparse structures in regular or near regular textures for image completion. Our method leverages the new convex optimization for low-rank and sparse signal recovery and can automatically correctly repair the global structure of a corrupted texture, even without precise information about the regions to be completed. Through extensive simulations, we show our method can complete and repair textures corrupted by errors with both random and contiguous supports better than existing low-rank matrix recovery methods. Through experimental comparisons with existing image completion systems (such as Photoshop) our method demonstrate significant advantage over local patch based texture synthesis techniques in dealing with large corruption, non-uniform texture, and large perspective deformation.


Low-Rank and Sparse Matrix Recovery Texture Completion Image Repairing 


  1. 1.
    Efros, A.A., Leung, T.K.: Texture synthesis by non-parametric sampling. In: IEEE International Conference on Computer Vision (ICCV), pp. 1033–1038 (1999)Google Scholar
  2. 2.
    Efros, A.A., Freeman, W.T.: Image quilting for texture synthesis and transfer. In: Proceedings of SIGGRAPH 2001, pp. 341–346 (2001)Google Scholar
  3. 3.
    Liang, L., Liu, C., Xu, Y., Guo, B., Shum, H.: Real-time texture synthesis by patch-based sampling. ACM Transactions on Graphics (SIGGRAPH) 20, 127–150 (2001)CrossRefGoogle Scholar
  4. 4.
    Komodakis, N., Tziritas, G.: Image completion using global optimization (2006)Google Scholar
  5. 5.
    Sun, J., Yuan, L., Jia, J., Shum, H.Y.: Image completion with structure propagation. ACM Trans. Graph. 24, 861–868 (2005)CrossRefGoogle Scholar
  6. 6.
    Criminisi, A., Pérez, P., Toyama, K.: Object removal by exemplar-based inpainting. In: Proc. IEEE Computer Vision and Pattern Recognition, CVPR (2003)Google Scholar
  7. 7.
    Liu, Y., Lin, W.C., Hays, J.: Near-regular texture analysis and manipulation. ACM Transactions on Graphics (SIGGRAPH) 23, 368–376 (2004)CrossRefGoogle Scholar
  8. 8.
    Bertalmio, M., Sapiro, G.: Image inpainting. ACM Transactions on Graphics (SIGGRAPH), 417–424 (2000)Google Scholar
  9. 9.
    Bertalmio, M., Bertozzi, A.L., Sapiro, G.: Navier-stokes, fluid dynamics, and image and video inpainting. In: Proc. IEEE Computer Vision and Pattern Recognition (CVPR), pp. 355–362 (2001)Google Scholar
  10. 10.
    Oliveira, M.M., Bowen, B., Mckenna, R., Sung Chang, Y.: Fast digital image inpainting. In: Proceedings of the International Conference on Visualization, Imaging and Image Processing, VIIP (2001)Google Scholar
  11. 11.
    Levin, A., Zomet, A., Weiss, Y.: Learning how to inpaint from global image statistics. In: Proceedings of the Internetional Conference on Computer Vision, ICCV (2003)Google Scholar
  12. 12.
    Mairal, J., Elad, M., Sapiro, G.: Sparse representation for color image restoration. The IEEE Trans. on Image Processing (TIP) 17, 53–69 (2007)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Fadili, M.J., Starck, J.-L., Murtagh, F.: Inpainting and zooming using sparse representations. The Computer Journal, 64–79 (2009)Google Scholar
  14. 14.
    Elad, M., Starck, J.-L., Querre, P., Donoho, D.: Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA). The Computer Journal, 340–358 (2005)Google Scholar
  15. 15.
    Nigeau, A., Bertalmio, M., Caselles, V., Sapiro, G.: A comprehensive framework for image inpainting. The IEEE Trans. on Image Processing (TIP) 19, 2634–2645 (2010)CrossRefGoogle Scholar
  16. 16.
    Kwatra, V., Schodl, A., Essa, I., Turk, G., Bobick, A.: Graphcut textures: Image and video synthesis using graph cuts. ACM Transactions on Graphics (SIGGRAPH) 22, 277–286 (2003)CrossRefGoogle Scholar
  17. 17.
    Hays, J., Efros, A.A.: Scene completion using millions of photographs. ACM Transactions on Graphics (SIGGRAPH) 26 (2007)Google Scholar
  18. 18.
    Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.B.: PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics (SIGGRAPH) 28 (2009)Google Scholar
  19. 19.
    Pritch, Y., Kav-Venaki, E., Peleg, S.: Shift-map image editing. In: Proceedings of the Internetional Conference on Computer Vision, ICCV (2009)Google Scholar
  20. 20.
    Lin, Z., Chen, M., Wu, L., Ma, Y.: The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices. UIUC Technical Report UILU-ENG-09-2215 (2010)Google Scholar
  21. 21.
    Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. Foundations of Computational Mathematics 9, 717–772 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  22. 22.
    Candès, E., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? Journal of the ACM 58 (2011)Google Scholar
  23. 23.
    Zhuang, L.S., Gao, H.Y., L.Z.M.Y.Z.X.Y.N.: Non-negative low rank and sparse graph for semi-supervised learning (2012)Google Scholar
  24. 24.
    Lin, Z., Liu, R., Su, Z.: Linearized alternating direction method with adaptive penalty for low-rank representation. In: Advances in Neural Information Processing Systems, NIPS (2011)Google Scholar
  25. 25.
    Liu, J., Musialski, P., Wonka, P., Ye, J.: Tensor completion for estimating missing values in visual data, pp. 2114–2121 (2009)Google Scholar
  26. 26.
    Zhou, Z., Wagner, A., Mobahi, H., Wright, J., Ma, Y.: Face recognition with contiguous occlusion using markov random fields. In: Proceedings of IEEE International Conference on Computer Vision, ICCV (2009)Google Scholar
  27. 27.
    Zhang, Z., Liang, X., Ma, Y.: Unwrapping low-rank textures on generalized cylindrical surfaces. In: International Conference on Computer Vision, ICCV (2011)Google Scholar
  28. 28.
    Zhang, Z., Ganesh, A., Liang, X., Ma, Y.: TILT: Transform-invariant low-rank textures. To Appear in International Journal of Computer Vision (IJCV) (2011)Google Scholar
  29. 29.
    Barnes, C., Shechtman, E., Goldman, D.B., Finkelstein, A.: The Generalized PatchMatch Correspondence Algorithm. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part III. LNCS, vol. 6313, pp. 29–43. Springer, Heidelberg (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Xiao Liang
    • 1
    • 2
  • Xiang Ren
    • 2
  • Zhengdong Zhang
    • 2
  • Yi Ma
    • 2
    • 3
  1. 1.Tsinghua UniversityChina
  2. 2.Visual Computing GroupMicrosoft Research AsiaChina
  3. 3.Electrical and Computer EngineeringUniversity of Illinois at Urbana-ChampaignUSA

Personalised recommendations