Advertisement

Super-Resolution Using Sub-Band Self-Similarity

  • Abhishek SinghEmail author
  • Narendra Ahuja
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9004)

Abstract

A popular approach for single image super-resolution (SR) is to use scaled down versions of the given image to build an internal training dictionary of pairs of low resolution (LR) and high resolution (HR) image patches, which is then used to predict the HR image. This self-similarity approach has the advantage of not requiring a separate external training database. However, due to their limited size, internal dictionaries are often inadequate for finding good matches for patches containing complex structures such as textures. Furthermore, the quality of matches found are quite sensitive to factors like patch size (larger patches contain structures of greater complexity and may be difficult to match), and dimensions of the given image (smaller images yield smaller internal dictionaries). In this paper we propose a self-similarity based SR algorithm that addresses the abovementioned drawbacks. Instead of seeking similar patches directly in the image domain, we use the self-similarity principle independently on each of a set of different sub-band images, obtained using a bank of orientation selective band-pass filters. Therefore, we allow the different directional frequency components of a patch to find matches independently, which may be in different image locations. Essentially, we decompose local image structure into component patches defined by different sub-bands, with the following advantages: (1) The sub-band image patches are simpler and therefore easier to find matches, than for the more complex textural patches from the original image. (2) The size of the dictionary defined by patches from the sub-band images is exponential in the number of sub-bands used, thus increasing the effective size of the internal dictionary. (3) As a result, our algorithm exhibits a greater degree of invariance to parameters like patch size and the dimensions of the LR image. We demonstrate these advantages and show that our results are richer in textural content and appear more natural than several state-of-the-art methods.

Keywords

Patch Size Image Patch Textural Detail Internal Dictionary Steerable Pyramid 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Freeman, W., Pasztor, E.: Learning low-level vision. In: ICCV (1999)Google Scholar
  2. 2.
    Yang, J., Wright, J., Huang, T., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Proc. 98, 1031–1044 (2010)MathSciNetGoogle Scholar
  3. 3.
    Yang, J., Wright, J., Huang, T., Ma, Y.: Image super-resolution as sparse representation of raw image patches. In: CVPR (2008)Google Scholar
  4. 4.
    Sun, J., Zheng, N.N., Tao, H., Shum, H.Y.: Image hallucination with primal sketch priors. In: CVPR (2003)Google Scholar
  5. 5.
    Yeung, D.Y., Yeung, D.Y., Xiong, Y.: Super-resolution through neighbor embedding. In: CVPR(2004)Google Scholar
  6. 6.
    Glasner, D., Bagon, S., Irani, M.: Super-resolution from a single image. In: ICCV (2009)Google Scholar
  7. 7.
    Freeman, W.T., Pasztor, E.C.: Learning low-level vision. IJCV 40, 25–47 (2000)CrossRefzbMATHGoogle Scholar
  8. 8.
    Wang, S., Zhang, D., Liang, Y., Pan, Q.: Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis. In: CVPR (2012)Google Scholar
  9. 9.
    Yang, C.Y., Yang, M.H.: Fast direct super-resolution by simple functions. In: ICCV (2013)Google Scholar
  10. 10.
    Ebrahimi, M., Vrscay, E.R.: Solving the inverse problem of image zooming using “Self-Examples”. In: Kamel, M.S., Campilho, A. (eds.) ICIAR 2007. LNCS, vol. 4633, pp. 117–130. Springer, Heidelberg (2007) CrossRefGoogle Scholar
  11. 11.
    Freedman, G., Fattal, R.: Image and video upscaling from local self-examples. ACM Trans. Graph. 28, 1–10 (2010)Google Scholar
  12. 12.
    Yang, C.-Y., Huang, J.-B., Yang, M.-H.: Exploiting self-similarities for single frame super-resolution. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part III. LNCS, vol. 6494, pp. 497–510. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  13. 13.
    Singh, A., Ahuja, N.: Sub-band energy constraints for self-similarity based super-resolution. In: ICPR (2014)Google Scholar
  14. 14.
    Singh, A., Porikli, F., Ahuja, N.: Super-resolving noisy images. In: CVPR (2014)Google Scholar
  15. 15.
    Barnsley, M.: Fractals Everywhere. Academic Press Professional, Inc., San Diego (1988) zbMATHGoogle Scholar
  16. 16.
    Polidori, E., luc Dugelay, J.: Zooming using iterated function systems (1995)Google Scholar
  17. 17.
    Zontak, M., Irani, M.: Internal statistics of a single natural image. In: CVPR (2011)Google Scholar
  18. 18.
    Freeman, W., Jones, T., Pasztor, E.: Example-based super-resolution. Comput. Graph. Appl. 22, 56–65 (2002). IEEECrossRefGoogle Scholar
  19. 19.
    Yang, J., Wang, Z., Lin, Z., Cohen, S., Huang, T.: Coupled dictionary training for image super-resolution. IEEE Trans. Image Proc. 21, 3467–3478 (2012)CrossRefMathSciNetGoogle Scholar
  20. 20.
    Yang, J., Lin, Z., Cohen, S.: Fast image super-resolution based on in-place example regression. In: CVPR (2013)Google Scholar
  21. 21.
    Fattal, R.: Image upsampling via imposed edge statistics. ACM Trans. Graph. 26(3), 95 (2007)CrossRefGoogle Scholar
  22. 22.
    Sun, J., Sun, J., Xu, Z., Shum, H.Y.: Gradient profile prior and its applications in image super-resolution and enhancement. IEEE Trans. Image Proc. 20, 1529–1542 (2011)CrossRefMathSciNetGoogle Scholar
  23. 23.
    Sun, J., Sun, J., Xu, Z., Shum, H.Y.: Image super-resolution using gradient profile prior. In: CVPR (2008)Google Scholar
  24. 24.
    Jiji, C.V., Chaudhuri, S.: Single-frame image super-resolution through contourlet learning. EURASIP J. Appl. Signal Process. 2006, 235 (2006)CrossRefGoogle Scholar
  25. 25.
    Gajjar, P., Joshi, M.: New learning based super-resolution: Use of dwt and igmrf prior. IEEE Trans. Image Proc. 19, 1201–1213 (2010)CrossRefMathSciNetGoogle Scholar
  26. 26.
    Sun, J., Zhu, J., Tappen, M.: Context-constrained hallucination for image super-resolution. In: CVPR (2010)Google Scholar
  27. 27.
    HaCohen, Y., Fattal, R., Lischinski, D.: Image upsampling via texture hallucination. In: ICCP (2010)Google Scholar
  28. 28.
    Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: CVPR (2005)Google Scholar
  29. 29.
    Simoncelli, E., Freeman, W.: The steerable pyramid: a flexible architecture for multi-scale derivative computation. In: ICIP (1995)Google Scholar
  30. 30.
    Simoncelli, E., Freeman, W., Adelson, E., Heeger, D.: Shiftable multiscale transforms. IEEE Trans. Info. Theory 38, 587–607 (1992)CrossRefMathSciNetGoogle Scholar
  31. 31.
    Do, M., Vetterli, M.: The contourlet transform: an efficient directional multiresolution image representation. IEEE Trans. Image Proc. 14, 2091–2106 (2005)CrossRefMathSciNetGoogle Scholar
  32. 32.
    Irani, M., Peleg, S.: Improving resolution by image registration. CVGIP 53, 231–239 (1991)Google Scholar
  33. 33.
    Kim, K., Kwon, Y.: Single-image super-resolution using sparse regression and natural image prior. IEEE TPAMI 32, 1127–1133 (2010)CrossRefMathSciNetGoogle Scholar
  34. 34.
    Shan, Q., Li, Z., Jia, J., Tang, C.K.: Fast image/video upsampling. ACM Trans. Graph. 27, 1–7 (2008)Google Scholar
  35. 35.
    Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Proc. 13, 600–612 (2004)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.University of Illinois at Urbana-Champaign ChampaignUSA

Personalised recommendations