Advertisement

Multimedia Tools and Applications

, Volume 77, Issue 7, pp 7883–7907 | Cite as

Bidirectionally aligned sparse representation for single image super-resolution

  • Chao Xie
  • Weili Zeng
  • Shengqin Jiang
  • Xiaobo Lu
Article

Abstract

It has been demonstrated that the sparse representation based framework is one of the most popular and promising ways to handle the single image super-resolution (SISR) issue. However, due to the complexity of image degradation and inevitable existence of noise, the coding coefficients produced by imposing sparse prior only are not precise enough for faithful reconstructions. In order to overcome it, we present an improved SISR reconstruction method based on the proposed bidirectionally aligned sparse representation (BASR) model. In our model, the bidirectional similarities are first modeled and constructed to form a complementary pair of regularization terms. The raw sparse coefficients are additionally aligned to this pair of standards to restrain sparse coding noise and therefore result in better recoveries. On the basis of fast iterative shrinkage-thresholding algorithm, a well-designed mathematic implementation is introduced for solving the proposed BASR model efficiently. Thorough experimental results indicate that the proposed method performs effectively and efficiently, and outperforms many recently published baselines in terms of both objective evaluation and visual fidelity.

Keywords

Single image super-resolution Sparse representation Sparse coefficient alignment Bidirectional similarities 

Notes

Acknowledgments

The authors would like to thank the associate editor and anonymous reviewers for their constructive and precious comments, which helped us a lot in improving the presentation of this work.

This work was supported by the National Natural Science Foundation of China (No.61374194, No.61403081), the National Key Science & Technology Pillar Program of China (No.2014BAG01B03), the Key Research and Development Program of Jiangsu Province (No. BE2016739), and a Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions.

References

  1. 1.
    Aguena ML, Mascarenhas ND (2006) Multispectral image data fusion using POCS and super-resolution. Comput Vis Image Underst 102(2):178–187CrossRefGoogle Scholar
  2. 2.
    Baker S, Kanade T (2002) Limits on super-resolution and how to break them. IEEE Trans Pattern Anal Mach Intell 24(9):1167–1183CrossRefGoogle Scholar
  3. 3.
    Beck A, Teboulle M (2009) A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J Imag Sci 2(1):183–202MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Bioucas-Dias JM, Figueiredo MAT (2007) A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Trans Image Process 16(12):2992–3004MathSciNetCrossRefGoogle Scholar
  5. 5.
    Bose NK, Kim HC, Valenzuela HM (1993) Recursive implementation of total least squares algorithm for image reconstruction from noisy, undersampled multiframes. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 269–272Google Scholar
  6. 6.
    Buades A, Coll B, Morel J (2005) A non-local algorithm for image denoising. In: IEEE computer society conference on Computer Vision and Pattern Recognition (CVPR), 60–65Google Scholar
  7. 7.
    Candè EJ, Wakin MB (2008) An introduction to compressive sampling. IEEE Signal Process Mag 25(2):21–30CrossRefGoogle Scholar
  8. 8.
    Candes EJ, Romberg JK, Tao T (2006) Stable signal recovery from incomplete and inaccurate measurements. Commun Pure Appl Math 59(8):1207–1223MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Candes EJ, Wakin MB, Boyd SP (2008) Enhancing sparsity by reweighted ℓ 1 minimization. J Fourier Anal Appl 14(5–6):877–905MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Chang H, Yeung DY, Xiong Y (2004) Super-resolution through neighbor embedding. In: IEEE computer society conference on Computer Vision and Pattern Recognition (CVPR), I-IGoogle Scholar
  11. 11.
    Daubechies I, Defrise M, De Mol C (2004) An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun Pure Appl Math 57(11):1413–1457MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Dong W, Zhang L, Shi G, Wu X (2011) Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization. IEEE Trans Image Process 20(7):1838–1857MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Dong W, Zhang L, Shi G (2011) Centralized sparse representation for image restoration. In: IEEE International Conference on Computer Vision (ICCV), 1259–1266Google Scholar
  14. 14.
    Dong W, Zhang L, Shi G, Li X (2013) Nonlocally centralized sparse representation for image restoration. IEEE Trans Image Process 22(4):1620–1630MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307CrossRefGoogle Scholar
  16. 16.
    Donoho DL (2006) For most large underdetermined Systems of Linear Equations the minimal 1-norm solution is also the sparsest solution. Commun Pure Appl Math 59(6):797–829MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Elad M, Aharon M (2006) Image Denoising via sparse and redundant representations over learned dictionaries. IEEE Trans Image Process 15(12):3736–3745MathSciNetCrossRefGoogle Scholar
  18. 18.
    Elad M, Feuer A (1997) Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Trans Image Process 6(12):1646–1658CrossRefGoogle Scholar
  19. 19.
    Elad M, Yavneh I (2009) A plurality of sparse representations is better than the sparsest one alone. IEEE Trans Inf Theory 55(10):4701–4714MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Farsiu S, Robinson MD, Elad M, Milanfar P (2004) Fast and robust multiframe super resolution. IEEE Trans Image Process 13(10):1327–1344CrossRefGoogle Scholar
  21. 21.
    Farsiu S, Robinson D, Elad M, Milanfar P (2004) Advances and challenges in super-resolution. Int J Imaging Syst Technol 14(2):47–57CrossRefGoogle Scholar
  22. 22.
    Freeman WT, Jones TR, Pasztor EC (2002) Example-based super-resolution. IEEE Comput Graph Appl 22(2):56–65CrossRefGoogle Scholar
  23. 23.
    He Y, Yap K, Chen L, Chau L (2009) A soft MAP framework for blind super-resolution image reconstruction. Image Vis Comput 27(4):364–373CrossRefGoogle Scholar
  24. 24.
    Kim SP, Su W (1993) Recursive high-resolution reconstruction of blurred multiframe images. IEEE Trans Image Process 2(4):534–539CrossRefGoogle Scholar
  25. 25.
    Kim SP, Bose NK, Valenzuela HM (1990) Recursive reconstruction of high resolution image from noisy undersampled multiframes. IEEE Trans Acoust Speech Signal Process 38(6):1013–1027CrossRefGoogle Scholar
  26. 26.
    Li X, Hu Y, Gao X, Tao D, Ning B (2010) A multi-frame image super-resolution method. Signal Process 90(2):405–414CrossRefzbMATHGoogle Scholar
  27. 27.
    Lin ZC, Shum HY (2004) Fundamental limits of reconstruction-based superresolution algorithms under local translation. IEEE Trans Pattern Anal Mach Intell 26(1):83–97CrossRefGoogle Scholar
  28. 28.
    Lu X, Yuan H, Yan P, Yuan Y, Li X (2012) Geometry constrained sparse coding for single image super-resolution. In: IEEE conference on Computer Vision and Pattern Recognition (CVPR), 1648–1655Google Scholar
  29. 29.
    Mishra D, Majhi B, Sa PK, Dash R (2016) Development of robust neighbor embedding based super-resolution scheme. Neurocomputing 202:49–66CrossRefGoogle Scholar
  30. 30.
    Nasrollahi K, Moeslund TB (2014) Super-resolution: a comprehensive survey. Mach Vis Appl 25(6):1423–1468CrossRefGoogle Scholar
  31. 31.
    Nguyen N, Milanfar P, Golub G (2001) A computationally efficient superresolution image reconstruction algorithm. IEEE Trans Image Process 10(4):573–583CrossRefzbMATHGoogle Scholar
  32. 32.
    Omer OA, Tanaka T (2011) Region-based weighted-norm with adaptive regularization for resolution enhancement. Digital Signal Process 21(4):508–516CrossRefGoogle Scholar
  33. 33.
    Park SC, Park MK, Kang MG (2003) Super-resolution image reconstruction: a technical overview. IEEE Signal Process Mag 20(3):21–36CrossRefGoogle Scholar
  34. 34.
    Peleg T, Elad M (2014) A statistical prediction model based on sparse representations for single image super-resolution. IEEE Trans Image Process 23(6):2569–2582MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Purkait P, Pal NR, Chanda B (2014) A fuzzy-rule-based approach for single frame super resolution. IEEE Trans Image Process 23(5):2277–2290MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Qin F, He X, Chen W, Yang X, Wu W (2009) Video superresolution reconstruction based on subpixel registration and iterative back projection. J Electron Imaging 18(1):13007CrossRefGoogle Scholar
  37. 37.
    Stark H, Oskoui P (1989) High-resolution image recovery from image-plane arrays, using convex projections. J Opt Soc Am A 6(11):1715–1726CrossRefGoogle Scholar
  38. 38.
    Tian J, Ma K (2011) A survey on super-resolution imaging. SIViP 5(3):329–342CrossRefGoogle Scholar
  39. 39.
    Timofte R, De Smet V, Van Gool L (2013) Anchored neighborhood regression for fast example-based super-resolution. In: IEEE International Conference on Computer Vision (ICCV), 1920–1927Google Scholar
  40. 40.
    Timofte R, De Smet V, Van Gool L (2014) A+: adjusted anchored neighborhood regression for fast super-resolution. In: Asian Conference on Computer Vision (ACCV), 111–126Google Scholar
  41. 41.
    Timofte R, Rothe R, Van Gool L (2016) Seven ways to improve example-based single image super resolution. In: IEEE conference on Computer Vision and Pattern Recognition (CVPR), 1865–1873Google Scholar
  42. 42.
    Tsai RY, Huang TS (1984) Multiframe image restoration and registration. Advances in computer vision and Image Processing 1(2):317–339Google Scholar
  43. 43.
    Vrigkas M, Nikou C, Kondi LP (2013) Accurate image registration for MAP image super-resolution. Signal Process Image Commun 28(5):494–508CrossRefGoogle Scholar
  44. 44.
    Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612CrossRefGoogle Scholar
  45. 45.
    Yan C, Zhang Y, Dai F, Wang X, Li L, Dai Q (2014) Parallel deblocking filter for HEVC on many-core processor. Electron Lett 50(5):367–368CrossRefGoogle Scholar
  46. 46.
    Yan C, Zhang Y, Dai F, Zhang J, Li L, Dai Q (2014) Efficient parallel HEVC intra-prediction on many-core processor. Electron Lett 50(11):805–806CrossRefGoogle Scholar
  47. 47.
    Yan C, Zhang Y, Xu J, Dai F, Li L, Dai Q, Wu F (2014) A highly parallel framework for HEVC coding unit partitioning tree decision on many-core processors. IEEE Signal Process Lett 21(5):573–576CrossRefGoogle Scholar
  48. 48.
    Yan C, Zhang Y, Xu J, Dai F, Zhang J, Dai Q, Wu F (2014) Efficient parallel framework for HEVC motion estimation on many-core processors. IEEE Trans Circuits Syst Video Technol 24(12):2077–2089CrossRefGoogle Scholar
  49. 49.
    Yang J, Wright J, Huang T, Ma Y (2008) Image super-resolution as sparse representation of raw image patches. In: IEEE conference on Computer Vision and Pattern Recognition (CVPR), 1–8Google Scholar
  50. 50.
    Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873MathSciNetCrossRefzbMATHGoogle Scholar
  51. 51.
    Yang S, Liu Z, Wang M, Sun F, Jiao L (2011) Multitask dictionary learning and sparse representation based single-image super-resolution reconstruction. Neurocomputing 74(17):3193–3203CrossRefGoogle Scholar
  52. 52.
    Yue L, Shen H, Yuan Q, Zhang L (2014) A locally adaptive L1− L2 norm for multi-frame super-resolution of images with mixed noise and outliers. Signal Process 105:156–174CrossRefGoogle Scholar
  53. 53.
    Zeng W, Lu X (2012) A generalized DAMRF image modeling for superresolution of license plates. IEEE Trans Intell Transp Syst 13(2):828–837CrossRefGoogle Scholar
  54. 54.
    Zeng W, Lu X, Fei S (2015) Image super-resolution employing a spatial adaptive prior model. Neurocomputing 162:218–233CrossRefGoogle Scholar
  55. 55.
    Zeyde R, Elad M, Protter M (2012) On single image scale-up using sparse-representations. Curves and Surfaces (Springer), 711–730.Google Scholar
  56. 56.
    Zhang X, Wu X (2008) Image interpolation by adaptive 2-D autoregressive modeling and soft-decision estimation. IEEE Trans Image Process 17(6):887–896MathSciNetCrossRefGoogle Scholar
  57. 57.
    Zhang X, Feng X, Wang W (2013) Two-direction nonlocal model for image Denoising. IEEE Trans Image Process 22(1):408–412MathSciNetCrossRefzbMATHGoogle Scholar
  58. 58.
    Zhou L, Lu X, Yang L (2014) A local structure adaptive super-resolution reconstruction method based on BTV regularization. Multimedia Tools and Applications 71(3):1879–1892CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  • Chao Xie
    • 1
    • 2
  • Weili Zeng
    • 3
  • Shengqin Jiang
    • 1
    • 2
  • Xiaobo Lu
    • 1
    • 2
  1. 1.School of AutomationSoutheast UniversityNanjing 210096China
  2. 2.Key Laboratory of Measurement and Control of Complex Systems of EngineeringMinistry of EducationNanjing 210096China
  3. 3.College of Civil AviationNanjing University of Aeronautics and AstronauticsNanjing 210016China

Personalised recommendations