Machine Vision and Applications

, Volume 26, Issue 7–8, pp 991–1005 | Cite as

A structural low rank regularization method for single image super-resolution

  • Jialin Peng
  • Benny Y. C. Hon
  • Dexing Kong
Original Paper


Example-learning-based algorithms such as those based on sparse coding or neighbor embedding have been popular for single image super-resolution in recent years. However, affected by several critical factors on the training data and example representation, their reconstructions are usually plagued by kinds of artifacts. The removing of these artifacts is one of the major tasks for these methods. Unlike most existing methods that employ more complicated training methods, in this paper we would like to recover a clear reconstruction by fusing several “dirty” coarse reconstructions which are outputs of one or several simple training methods with small training set. One underlying key observation is that although coarse reconstructions are corrupted by different artifacts, they refer to the same high-resolution image. This global structure information is captured by an image structure-based low rank regularization method. The advantage of our method is that it can remove not only small noises but also gross artifacts. Except sparsity and randomness of the large artifacts, no other knowledge about them is required. Experimental results show that the proposed method can not only dramatically improve coarse reconstructions but also achieve competitive results.


Super-resolution Low rank Fusion Regularization 



The work described in this paper is supported by the National Natural Science Foundation of China (NSFC) (Grant Nos. 11401231, 91330105), Science Foundation of Fujian Province (No. 2015J01254) and Science Technology Foundation for Middle-aged and Young Teacher of Fujian Province (No. JA14021).


  1. 1.
    Aharon, M., Elad, M., Bruckstein, A.: K-svd: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006)CrossRefGoogle Scholar
  2. 2.
    Bertsekas, D.: Constrained Optimization and Lagrange Multiplier Method. Cambridge University Press, Cambridge (2004)Google Scholar
  3. 3.
    Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)zbMATHCrossRefGoogle Scholar
  4. 4.
    Buades, A., Coll, B., Morel, J.M.: Nonlocal image and movie denoising. Int. J. Comput. Vis. 76(2), 123–139 (2008)CrossRefGoogle Scholar
  5. 5.
    Cai, J.F., Candès, E.J., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20(4), 1956–1982 (2008)CrossRefGoogle Scholar
  6. 6.
    Candès, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? IEEE Trans. Signal Process. 58(1), 1–37 (2009)Google Scholar
  7. 7.
    Chang, H., Yeung, D.Y., Xiong, Y.: Super-resolution through neighbor embedding. In: Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., pp. 275–282 (2004)Google Scholar
  8. 8.
    Chen, X., Qi, C.: Low-rank neighbor embedding for single image super-resolution. IEEE Signal Process. Lett. 21(1), 79–82 (2014)CrossRefGoogle Scholar
  9. 9.
    Dong, W., Zhang, L., Shi, G.: Centralized sparse representation for image restoration. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 1259–1266. IEEE (2011)Google Scholar
  10. 10.
    Dong, W., Zhang, L., Shi, G., Wu, X.: Image deblurring and supper-resolution by adaptive sparse domain selection and adaptive regularization. IEEE Trans. Image Process. 20(7), 1838–1857 (2011)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Elad, M., Figueiredo, M.A.T., Ma, Y.: On the role of sparse and redundant representations in image processing. IEEE Proc. Spec. Issue Appl. Sparse Represent. Compress. Sens. 98, 972–982 (2010)Google Scholar
  12. 12.
    Freeman, W.T., Jones, T.R., Pasztor, E.C.: Example-based superresolution. IEEE Comput. Graph. Appl. 22(2), 56–65 (2002)CrossRefGoogle Scholar
  13. 13.
    Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976)zbMATHCrossRefGoogle Scholar
  14. 14.
    Gao, X.B., Zhang, K.B., Tao, D.C., Li, X.: Image super-resolution with sparse neighbor embedding. IEEE Trans. Image Process. 21(7), 3194–3205 (2012)Google Scholar
  15. 15.
    Hale, E.T., Yin W., Zhang, Y.: A fixed-point continuation method for l1-regularized minimization with applications to compressed sensing. CAAM TR07-07, Rice University, 43:44 (2007)Google Scholar
  16. 16.
    Hale, E.T., Yin, W., Zhang, Y.: Fixed-point continuation for l1-minimization: methodology and convergence. SIAM J. Optim. 19(3), 1107–1130 (2008)zbMATHMathSciNetCrossRefGoogle Scholar
  17. 17.
    He, L., Qi, H., Zaretzki, R.: Beta process joint dictionary learning for coupled feature spaces with application to single image super-resolution. Comput. Vis. Pattern Recogn. 1, 345–353 (2013)Google Scholar
  18. 18.
    Huang, T.S., Tsai, R.Y.: Multi-frame image restoration and registration. Adv. Comput. Vis. Image Process. 1(2), 317–339 (1984)Google Scholar
  19. 19.
    Irani, M., Peleg, S.: Motion analysis for image enhancement: resolution, occlusion and transparency. J. Vis. Commun. Image Represent. 4(4), 324–335 (1993)CrossRefGoogle Scholar
  20. 20.
    Izenman, A.J.: Modern Multivariate Statistical Techniques: Regression, Classiffication, and Manifold Learning. Springer, New York (2008)CrossRefGoogle Scholar
  21. 21.
    Kim, C., Choi, K., Ra, J.B.: Example-based super-resolution via structure analysis of patches. IEEE Signal Process. Lett. 20(4), 407–410 (2013)CrossRefGoogle Scholar
  22. 22.
    Kim, K.I., Kwon, Y.: Single-image super-resolution using sparse regression and natural image prior. IEEE Trans. Pattern Anal. Mach. Intell. 32(6), 1127–1133 (2010)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Larsen, R.M.: Propack-software for large and sparse svd calculations (2005). Accessed 12 APR 2014
  24. 24.
    Li, X., Orchard, M.T.: New edge-directed interpolation. IEEE Trans. Image Process. 10(10), 1521–1527 (2001)CrossRefGoogle Scholar
  25. 25.
    Lin, Z., He, J., Tang, X., Tang, C.K.: Limits of learning-based super resolution algorithms. Int. J. Comput. Vis. 80(3), 406–420 (2008)CrossRefGoogle Scholar
  26. 26.
    Lin, Z., Chen, M., Wu, L., Ma, Y.: The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. Technical report, University of Illinois at Urbana-Champaign, Urbana-Champaign, IL (2010)Google Scholar
  27. 27.
    Lin, Z., Liu, R., Su, Z.: Linearized alternating direction method with adaptive penalty for low-rank representation. In: Advances in Neural Information Processing Systems, pp. 612–620 (2011)Google Scholar
  28. 28.
    Lu, X., Yuan, H., Yan, P., Yuan, Y., Li, X.: Geometry constrained sparse coding for single image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1648–1655 (2012)Google Scholar
  29. 29.
    Marquina, A., Osher, S.J.: Image super-resolution by tv-regularization and bregman iteration. J. Sci. Comput. 37(3), 367–382 (2008)zbMATHMathSciNetCrossRefGoogle Scholar
  30. 30.
    Park, S., Park, M., Kang, M.G.: Super-resolution image reconstruction: a technical overview. IEEE Signal Process. Mag. 20(3), 21–36 (2003)CrossRefGoogle Scholar
  31. 31.
    Pati, Y.C., Rezaifar, R., Krishnaprasad, P.S.: Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition. In: Proc. 27th Asilomar Conf. Signals, Syst. Comput., pp. 40–44 (1993)Google Scholar
  32. 32.
    Peyré, G., Bougleux, S., Cohen, L.: Non-local regularization of inverse problems. In: Proc. the 10th European Conference on Computer Vision, vol. III, pp. 57–68 (2008)Google Scholar
  33. 33.
    Protter, M., Elad, M., Takeda, H., Milanfar, P.: Generalizing the nonlocal-means to super-resolution reconstruction. IEEE Trans. Image Process. 18(1), 36–51 (2009)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Recht, B., Fazel, M., Parrilo, P.: Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Rev. 52(3), 471–501 (2010)zbMATHMathSciNetCrossRefGoogle Scholar
  35. 35.
    Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)CrossRefGoogle Scholar
  36. 36.
    Sun, J., Sun, J., Xu, Z., Shum, H.: Image super-resolution using gradient profile prior. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 1–8 (2008)Google Scholar
  37. 37.
    Takeda, H., Farsiu, S., Milanfar, P.: Kernel regression for image processing and reconstruction. IEEE Trans. Image Process. 16(2), 349–366 (2007)MathSciNetCrossRefGoogle Scholar
  38. 38.
    Wang, Z., Bovik, A.C., Sheikh, H.R.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRefGoogle Scholar
  39. 39.
    Xu, H., Caramanis, C., Mannor, S.: Sparse algorithms are not stable: a no-free-lunch theorem. IEEE Trans. Pattern Anal. Mach. Intell. 34(1), 187–193 (2012)MathSciNetCrossRefGoogle Scholar
  40. 40.
    Xu, H., Zhai, G., Yang, X.: Single image super-resolution with detail enhancement based on local fractal analysis of gradient. IEEE Trans. Circuits Syst. Video Technol. 23(10), 1740–1754 (2013)CrossRefGoogle Scholar
  41. 41.
    Yang, J., Lin, Z., Cohen, S.: Fast image super-resolution based on in-place example regression. In: Computer Vision and Pattern Recognition (CVPR), pp. 1059–1066. IEEE (2013)Google Scholar
  42. 42.
    Yang, J.C., Wright, J., Huang, T., Ma, Y.: Image super-resolution as sparse representation of raw image patches. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 1–8 (2008)Google Scholar
  43. 43.
    Yang, J.C., Wright, J., Huang, T., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010)MathSciNetCrossRefGoogle Scholar
  44. 44.
    Yang, J.C., Wang, Z.W., Lin, Z., Cohen, S., Huang, T.: Coupled dictionary training for image super-resolution. IEEE Trans. Image Process. 21(8), 3467–3478 (2011)MathSciNetCrossRefGoogle Scholar
  45. 45.
    Yang, S., Wang, M., Chen, Y., Sun, Y.: Single-image super-resolution reconstruction via learned geometric dictionaries and clustered sparse coding. IEEE Trans. Image Process. 21(9), 4016–4028 (2012)MathSciNetCrossRefGoogle Scholar
  46. 46.
    Yang, M., Wang, Y.: A self-learning approach to single image super-resolution. IEEE Trans. Multimed. 15(3), 498–508 (2013)CrossRefGoogle Scholar
  47. 47.
    Yu, J., Gao, X., Tao, D., Li, X., Zhang, K.: A unified learning framework for single image super-resolution. IEEE Trans. Neural Netw. Learn. Syst. 20(4), 407–410 (2013)Google Scholar
  48. 48.
    Zhang, H., Yang, J., Zhang, Y., Huang, T.S.: Non-local kernel regression for image and video restoration. In: Computer Vision-ECCV 2010, pp. 566–579. Springer, Berlin (2010)Google Scholar
  49. 49.
    Zhang, L., Wu, X.: An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Trans. Image Process. 15(8), 2226–2238 (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.The School of Computer Science and TechnologyHuaqiao UniversityXiamenChina
  2. 2.Department of MathematicsCity University of Hong KongHong KongChina
  3. 3.Department of mathematicsZhejiang UniversityHangzhouChina

Personalised recommendations