Abstract
Recently, software architectures applied to physical agents have become a boost from the emerging Artificial Intelligence(AI). In these smart physical agents, simultaneously image information processing appears vital in particular. Single Image Super Resolution(SISR) serves as the foundation of the image process, presenting its prospects driven by deep learning(DL) methods. In these DL methods, convolutional layers are stacked to implement a mapping between the original low resolution(LR) image and the high resolution(HR) image. Despite improved performances, convolution neural networks(CNN) based methods with large parameters are so complex and time-consuming, which is difficult to apply in mobile devices. To tackle the above issue, we propose Distillation Information Block(DIB) and Feature Information Refined Block(FIRB). In our proposed Lightweight Refined Networks for single image Super-Resolution(LRSR), to lengthen our network with fewer parameters increased, the proposed DIB grasps more information by using a large receptive field. To enhance the information utilization, we build FIRB to refine advanced features and recover more details. Furthermore, with the compact structure, the execution time can be comparatively reduced with higher performance. We conduct extensive experiments on different datasets, which demonstrate that the proposed method performs comparatively better compared with the state-of-the-art lightweight methods.
Similar content being viewed by others
References
Bevilacqua M, Roumy A, Guillemot C, Alberi-Morel ML (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. BMVC, Surrey, UK, p 1–10
Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition 1251–1258
Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In David J. Fleet, Tom as Pajdla, Bernt Schiele, and Tinne Tuytelaars, editors, Computer Vision - ECCV 2014 -13th European Conference, Zurich, Switzerland. Proceedings, Part IV, volume 8692 of Lecture Notes in Computer Science, p 184-199
Ellery A (2016) Autonomous navigation-self-localization and mapping (slam). Springer, Berlin Heidelberg
Foley JD, Dam AV, Feiner SK, Hughes JF (1990) Computer graphics, principle and practice, 2nd edition
Forsyth DA, Ponce J (2012) Computer Vision: A Modern Approach. Computer Vision: A Modern Approach, Second Edition. Pitman
Franois R (2010) A non-local approach for image super-resolution using intermodality priors. Medical image analysis 14(4):594–605
Glasner D, Bagon S, Irani M (2009) Super-resolution from a single image. In IEEE 12th International Conference on Computer Vision, ICCV 2009, Kyoto, Japan, p 349–356
Han K, Wang Y, Tian Q, Guo J, Xu C, Xu C (2020) Ghostnet: More features from cheap operations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) p 1577–1586
Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR (2012) Improving neural networks by preventing co-adaptation of feature detectors. Comput Sci 3(4):212–223
Hu J, Shen L, Albanie S, Sun G, Wu E (2017) Squeeze-and-excitation networks. CVPR p 7132-7141
Huang J-B, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In Proceedings of the IEEE conference on computer vision and pattern recognition 5197–5206
Hui Z, Wang X, Gao X (2018) Fast and accurate single image super-resolution via information distillation network. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. CVPR p 723-731
Isobe T, Li S, Jia X, Yuan S, Slabaugh G, Xu C, Li Y-L, Wang S, Tian Q (2020) Video super-resolution with temporal group attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. CVPR p 8005–8014
Keys RG (2003) Cubic convolution interpolation for digital image processing. IEEE Transactions on Acoustics, Speech, and Signal Processing 29
Kim J, Kwon Lee J, Mu Lee K (2016) Accurate image super-resolution using very deep convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. CVPR 1646–1654
Kim J, Kwon Lee J, Mu Lee K (2016) Deeply-recursive convolutional network for image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition. CVPR 1637–1645
Kim SK, Duh H, Sarhan NJ, Hahanov V (2013) Real-time multimedia computing. Multimed Tools Appl 65(2):181–186
Kingma DP, Ba J (2015) Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun, ICLR
Lai WS, Huang JB, Ahuja N, Yang MH (2017) Deep laplacian pyramid networks for fast and accurate super-resolution 5835–5843
Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang ZA (2016) Photo-realistic single image super-resolution using a generative adversarial network
Levine S, Pastor P, Krizhevsky A, Ibarz J, Quillen D (2016) Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. Int J Robot Res 37(4–5):421–436
Li Z, Yang J, Liu Z, Yang X, Jeon G, Wu W (2019) Feedback network for image super-resolution. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Lim B, Son S, Kim H, Nah S, Mu Lee K (2017) Enhanced deep residual networks for single image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops 136–144
Liu F, Yu Q, Chen L, Jeon G, Albertini MK, Yang X (2021) Aerial image super-resolution based on deep recursive dense network for disaster area surveillance. Pers Ubiquit Comput 1–10
Ma N, Zhang X, Zheng H-T, Sun J (2018) Shufflenet v2: Practical guidelines for efficient cnn architecture design. In Proceedings of the European Conference on Computer Vision (ECCV)
Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 2:416–423. IEEE
Matsui Y, Ito K, Aramaki Y, Fujimoto A, Ogawa T, Yamasaki T, Aizawa K (2017) Sketch-based manga retrieval using manga109 dataset. Multimed Tools Appl 76(20):21811–21838
Mattmann CA (2013) Computing: A vision for data science. Nature 493(7433):473–5
Molchanov P, Tyree S, Karras T, Aila T, Kautz J (2016) Pruning convolutional neural networks for resource efficient inference. OpenReview.net
Namhyuk A, Kang B, Sohn K-A (2018) Fast, accurate, and lightweight super-resolution with cascading residual network. In Proceedings of the European Conference on Computer Vision (ECCV) 252–268
Shi W, Caballero J, Huszár F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition 1874–1883
Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In IEEE Conference on Computer Vision and Pattern Recognition CVPR p 2798–2798
Tai Y, Yang J, Liu X, Xu C (2017) Memnet: A persistent memory network for image restoration. ICCV p 4549–4557
Thurnhofer-Hemsi K, Lopez-Rubio E, Roe-Vellve N, Molina-Cabello MA (2020) Multiobjective optimization of deep neural networks with combinations of lp-norm cost functions for 3d medical image super-resolution. Integrated Computer-Aided Engineering 27(3):233–251
Timofte R, Agustsson E, Gool LV, Yang MH, Zhang L, Lim B et al (2017) Ntire 2017 challenge on single image super-resolution: Methods and results. In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) p 852–863
Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. In International conference on curves and surfaces 711–730. Springer
Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. In Proceedings of the European Conference on Computer Vision (ECCV) 286–301
Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018) Residual dense network for image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition. CVPR 2472–2481
Zhou W, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612
Acknowledgements
This work was partially supported by the funding from Sichuan University under grant 2020SCUNG205.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Tong, J., Dou, Q., Yang, H. et al. Lightweight refined networks for single image super-resolution. Multimed Tools Appl 81, 3439–3458 (2022). https://doi.org/10.1007/s11042-021-11318-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-021-11318-9