Skip to main content
SpringerLink
Log in

LAPUSKA: Fast Image Super-Resolution via LAPlacian UpScale Knowledge Alignment

  • Conference paper
  • First Online:
Advances in Computational Intelligence (IWANN 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14135))

Included in the following conference series:

  • 589 Accesses

Abstract

Single image super-resolution is important part of computer vision open problems. Recently, deep neural networks have demonstrated excellent performance in this problem. In this work, several cutting edge methods for super-resolution problem using deep neural networks will be considered. Comparison of their effectiveness and evaluation of neural networks architectures with respect to different metrics is one of our main goals for this research. Modern deep learning methods often require large computational cost and load a lot of computer memory, which affects the ease of use of neural networks and the time of generation super-resolution results. In addition to the existing models, we propose a new architecture of neural networks based on best properties of considered architectures and designed to eliminate their shortcomings. Furthermore, we compare the quality of all considered deep learning methods with baseline method of bicubic interpolation.

The work of Ilya Makarov was made in the framework of the strategic project “Digital Business” within the Strategic Academic Leadership Program “Priority 2030" at NUST MISiS.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Agustsson, E., Timofte, R.: NTIRE 2017 challenge on single image super-resolution: dataset and study. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (2017)

    Google Scholar 

  2. Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2011)

    Article  PubMed  Google Scholar 

  3. Bevilacqua, M., Roumy, A., Guillemot, C., Alberi-Morel, M.L.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Proceedings of BMVC 2012 (2012)

    Google Scholar 

  4. Cheng, X., Fu, Z., Yang, J.: Zero-shot image super-resolution with depth guided internal degradation learning. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12362, pp. 265–280. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58520-4_16

    Chapter  Google Scholar 

  5. Diebel, J., Thrun, S.: An application of Markov random fields to range sensing. In: Advances in Neural Information Processing Systems, pp. 291–298 (2006)

    Google Scholar 

  6. Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10593-2_13

    Chapter  Google Scholar 

  7. Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 391–407. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_25

    Chapter  Google Scholar 

  8. Fattal, R.: Image upsampling via imposed edge statistics. ACM Trans. Graphics (TOG) 26(3), 95 (2007)

    Article  Google Scholar 

  9. Freeman, W.T., Jones, T.R., Pasztor, E.C.: Example-based super-resolution. IEEE Comput. Graphics Appl. 22(2), 56–65 (2002)

    Article  Google Scholar 

  10. Glasner, D., Bagon, S., Irani, M.: Super-resolution from a single image. In: ICCV, pp. 349–356. IEEE (2009)

    Google Scholar 

  11. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)

    Google Scholar 

  12. He, K., Sun, J., Tang, X.: Guided image filtering. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6311, pp. 1–14. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15549-9_1

    Chapter  Google Scholar 

  13. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)

    Google Scholar 

  14. Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: CVPR, pp. 5197–5206 (2015)

    Google Scholar 

  15. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)

  16. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43

    Chapter  Google Scholar 

  17. Karpov, A., Makarov, I.: Exploring efficiency of vision transformers for self-supervised monocular depth estimation. In: 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 711–719. IEEE (2022)

    Google Scholar 

  18. Kim, J., Kwon Lee, J., Mu Lee, K.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1646–1654 (2016)

    Google Scholar 

  19. Kim, J., Kwon Lee, J., Mu Lee, K.: Deeply-recursive convolutional network for image super-resolution. In: CVPR, pp. 1637–1645 (2016)

    Google Scholar 

  20. Kim, K.I., Kwon, Y.: Example-based learning for single-image super-resolution. In: Rigoll, G. (ed.) DAGM 2008. LNCS, vol. 5096, pp. 456–465. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-69321-5_46

    Chapter  Google Scholar 

  21. Kopf, J., Cohen, M.F., Lischinski, D., Uyttendaele, M.: Joint bilateral upsampling. ACM Trans. Graph. (ToG) 26(3), 96 (2007)

    Article  Google Scholar 

  22. Korinevskaya, A., Makarov, I.: Fast depth map super-resolution using deep neural network. In: ISMAR2018, pp. 117–122. IEEE, New York, USA (2018)

    Google Scholar 

  23. Lai, W.S., Huang, J.B., Ahuja, N., Yang, M.H.: Fast and accurate image super-resolution with deep Laplacian pyramid networks. arXiv:1710.01992 (2017)

  24. Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. arXiv preprint arXiv:1609.04802 (2016)

  25. Li, Y., Xue, T., Sun, L., Liu, J.: Joint example-based depth map super-resolution. In: ICME, pp. 152–157. IEEE (2012)

    Google Scholar 

  26. Makarov, I., Aliev, V., Gerasimova, O.: Semi-dense depth interpolation using deep convolutional neural networks. In: ACM Multimedia (MM2017), pp. 1407–1415. ACM, New York, USA (2017)

    Google Scholar 

  27. Makarov, I., Bakhanova, M., Nikolenko, S., Gerasimova, O.: Self-supervised recurrent depth estimation with attention mechanisms. PeerJ Comput. Sci. 8(e865), 1–25 (2022)

    Google Scholar 

  28. Makarov, I., Borisenko, G.: Depth inpainting via vision transformer. In: ISMAR2021, pp. 286–291. IEEE, New York, USA (2021)

    Google Scholar 

  29. Makarov, I., Korinevskaya, A., Aliev, V.: Fast semi-dense depth map estimation. In: Proceedings of the ACM Workshop on Multimedia for Real Estate Tech (RETech2018), pp. 18–21. ACM, New York, USA (2018)

    Google Scholar 

  30. Makarov, I., Korinevskaya, A., Aliev, V.: Sparse depth map interpolation using deep convolutional neural networks. In: TSP2018, pp. 1–5. IEEE (2018)

    Google Scholar 

  31. Makarov, I., et al.: On reproducing semi-dense depth map reconstruction using deep convolutional neural networks with perceptual loss. In: ACM Multimedia (MM2019), pp. 1080–1084. ACM, New York, USA (2019)

    Google Scholar 

  32. Maslov, D., Makarov, I.: Online supervised attention-based recurrent depth estimation from monocular video. PeerJ Comput. Sci. 6(e317), 1–22 (2020)

    Google Scholar 

  33. Maslov, D., Makarov, I.: Fast depth reconstruction using deep convolutional neural networks. In: Rojas, I., Joya, G., Català, A. (eds.) IWANN 2021. LNCS, vol. 12861, pp. 456–467. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85030-2_38

    Chapter  Google Scholar 

  34. Savchenko, A.V.: Fast inference in convolutional neural networks based on sequential three-way decisions. Inf. Sci. 560, 370–385 (2021)

    Article  Google Scholar 

  35. Savchenko, A.V.: MT-EmotiEffNet for multi-task human affective behavior analysis and learning from synthetic data. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds.) Computer Vision – ECCV 2022 Workshops. ECCV 2022. LNCS, vol. 13806. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-25075-0_4

  36. Savchenko, A.V., Belova, N.S.: Statistical testing of segment homogeneity in classification of piecewise-regular objects. Int. J. Appl. Math. Comput. Sci. 25(4), 915–925 (2015)

    Article  Google Scholar 

  37. Savchenko, A.V., Savchenko, L.V.: Towards the creation of reliable voice control system based on a fuzzy approach. Pattern Recogn. Lett. 65, 145–151 (2015)

    Article  Google Scholar 

  38. Savchenko, A., Khokhlova, Y.I.: About neural-network algorithms application in viseme classification problem with face video in audiovisual speech recognition systems. Optical Mem. Neural Netw. 23(1), 34–42 (2014)

    Article  Google Scholar 

  39. Shi, W., et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: CVPR, pp. 1874–1883 (2016)

    Google Scholar 

  40. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  41. Sokolova, A.D., Kharchevnikova, A.S., Savchenko, A.V.: Organizing multimedia data in video surveillance systems based on face verification with convolutional neural networks. In: van der Aalst, W.M.P., et al. (eds.) AIST 2017. LNCS, vol. 10716, pp. 223–230. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-73013-4_20

    Chapter  Google Scholar 

  42. Wang, Y., Wang, L., Wang, H., Li, P.: End-to-end image super-resolution via deep and shallow convolutional networks. arXiv preprint arXiv:1607.07680 (2016)

  43. Wang, Z., Bovik, A.C.: Mean squared error: love it or leave it? A new look at signal fidelity measures. IEEE Signal Process. Mag. 26(1), 98–117 (2009)

    Article  Google Scholar 

  44. Yue, L., Shen, H., Li, J., Yuan, Q., Zhang, H., Zhang, L.: Image super-resolution: the techniques, applications, and future. Signal Process. 128, 389–408 (2016). https://doi.org/10.1016/j.sigpro.2016.05.002

    Article  Google Scholar 

  45. Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: Boissonnat, J.-D., et al. (eds.) Curves and Surfaces 2010. LNCS, vol. 6920, pp. 711–730. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-27413-8_47

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. HSE University, Moscow, Russia

    Aleksei Pokoev

  2. AI Center, NUST MISiS, Moscow, Russia

    Ilya Makarov

  3. Artificial Intelligence Research Institute (AIRI), Moscow, Russia

    Ilya Makarov

Authors
  1. Aleksei Pokoev
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ilya Makarov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ilya Makarov .

Editor information

Editors and Affiliations

  1. University of Granada, Granada, Spain

    Ignacio Rojas

  2. University of Malaga, Málaga, Spain

    Gonzalo Joya

  3. Polytechnic University of Catalonia, Vilanova i la Geltrú, Spain

    Andreu Catala

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pokoev, A., Makarov, I. (2023). LAPUSKA: Fast Image Super-Resolution via LAPlacian UpScale Knowledge Alignment. In: Rojas, I., Joya, G., Catala, A. (eds) Advances in Computational Intelligence. IWANN 2023. Lecture Notes in Computer Science, vol 14135. Springer, Cham. https://doi.org/10.1007/978-3-031-43078-7_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43078-7_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43077-0

  • Online ISBN: 978-3-031-43078-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation