Abstract
Super-resolution (SR) has received extensive attention in recent years for satellite image processing in a wide range of application scenarios, such as land classification, identification of changes, the discovery of resources, etc. Satellite images from satellite sensors are mostly low-resolution (LR) images, so they do not completely fulfill object detection and analysis criteria. SR has multiple residual network frameworks in deep learning that have improved performance and can extend thousands of layers in the system. However, each layer improves accuracy by doubling the number of layers, although training thousands of layers are too expensive, the process is slow, and there are functional recovery issues. We proposed a transferred wide residual Single Image Super-Resolution (SISR) remote sensing deep neural network model (WRSR). By increasing the width and reducing the residual network depth, the proposed approach has dramatically reduced memory costs. As a result, our model reduced memory costs by 21% in Enhanced Deep Residual Super-Resolution (EDSR) and 34% in SRResNet as a direct consequence of the in-depth reduction. The proposed architecture improves the efficiency of training loss by performing weight normalization instead of augmentation technology. We compared our method to five recent existing super-resolution (SR) deep neural network methods, tested over three public satellite image datasets and a standard reference (PRIM) dataset. Experiment analysis is evaluated in peak to signal noise ratio (PSNR) and structural similarity index measure (SSIM).
Similar content being viewed by others
Availability of Data and Material
All necessary data available in the manuscript, provided that any additional data needed is available upon request.
Code Availability
We have code for these proposed results that we do not currently shared.
References
Huo, X., Tang, R., Ma, L., Shao, K., & Yang, Y. (2019). A novel neural network for super-resolution remote sensing image reconstruction. International Journal of Remote Sensing, 40(5–6), 2375–2385. https://doi.org/10.1080/01431161.2018.1516319.
Deeba, F., Kun, S., Ali Dharejo, F., & Zhou, Y. (2020). Wavelet-based enhanced medical image super resolution. IEEE Access, 8, 37035–37044. https://doi.org/10.1109/ACCESS.2020.2974278.
Deeba, F., She, K., Zhou, Y., & Ali, F. (2020). Sparse representation based computed tomography images reconstruction by coupled dictionary learning algorithm. IET Image Processing. https://doi.org/10.1049/iet-ipr.2019.1312.
Jiang, K., Wang, Z., Yi, P., Wang, G., Lu, T., & Jiang, J. (2019). Edge-enhanced GAN for remote sensing image superresolution. IEEE Transactions on Geoscience and Remote Sensing, 57(8), 5799–5812. https://doi.org/10.1109/TGRS.2019.2902431.
Deeba, F., Kun, S., Wang, W., Ahmed, J., & Qadir, B. (2019). Wavelet integrated residual dictionary training for single image super-resolution. Multimedia Tools and Applications, 78(19), 27683–27701. https://doi.org/10.1007/s11042-019-07850-4.
Fernandez-Beltran, R., Latorre-Carmona, P., & Pla, F. (2017). Single-frame super-resolution in remote sensing: a practical overview. International Journal of Remote Sensing, 38(1), 314–354. https://doi.org/10.1080/01431161.2016.1264027.
Dharejo, F. A., Zhou, Y., Deeba, F., Jatoi, M. A., Du, Y., & Wang, X. (2020). A remote‐sensing image enhancement algorithm based on patch‐wise dark channel prior and histogram equalisation with colour correction. IET Image Processing, ipr2.12004. https://doi.org/10.1049/ipr2.12004
Dharejo, F. A., Zhou, Y., Deeba, F., & Du, Y. (2020). A color enhancement scene estimation approach for single image haze removal. IEEE Geoscience and Remote Sensing Letters. https://doi.org/10.1109/LGRS.2019.2951626.
Merino, M. T., & Nunez, J. (2007). Super-resolution of remotely sensed images with variable-pixel linear reconstruction. IEEE Transactions on Geoscience and Remote Sensing, 45(5), 1446–1457. https://doi.org/10.1109/TGRS.2007.893271.
Li, F., Jia, X., & Fraser, D. (2008). Universal HMT based super resolution for remote sensing images. In 2008 15th IEEE International Conference on Image Processing (pp. 333–336). IEEE. https://doi.org/10.1109/ICIP.2008.4711759
Yang, S., Sun, F., Wang, M., Liu, Z., & Jiao, L. (2011). Novel super resolution restoration of remote sensing images based on compressive sensing and example patches-aided dictionary learning. In 2011 International Workshop on Multi-Platform/Multi-Sensor Remote Sensing and Mapping (pp. 1–6). IEEE. https://doi.org/10.1109/M2RSM.2011.5697375
Yue, L., Shen, H., Li, J., Yuan, Q., Zhang, H., & Zhang, L. (2016). Image super-resolution: The techniques, applications, and future. Signal Processing, 128, 389–408. https://doi.org/10.1016/j.sigpro.2016.05.002.
Pan, Z., Ma, W., Guo, J., & Lei, B. (2019). Super-Resolution of Single Remote Sensing Image Based on Residual Dense Backprojection Networks. IEEE Transactions on Geoscience and Remote Sensing, 57(10), 7918–7933. https://doi.org/10.1109/TGRS.2019.2917427.
Lu, T., Wang, J., Zhang, Y., Wang, Z., & Jiang, J. (2019). Satellite image super-resolution via multi-scale residual deep neural network. Remote Sensing, 11(13), 1588. https://doi.org/10.3390/rs11131588.
Yu, J., Fan, Y., Yang, J., Xu, N., Wang, Z., Wang, X., & Huang, T. (2018). Wide activation for efficient and accurate image super-resolution. Retrieved from http://arxiv.org/abs/1808.08718
Zagoruyko, S., & Komodakis, N. (2016). Wide residual networks. In Procedings of the British Machine Vision Conference 2016 (pp. 87.1–87.12). British Machine Vision Association. https://doi.org/10.5244/C.30.87
Wang, Z., Chen, J., & Hoi, S. C. H. (2020). Deep learning for image super-resolution: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/TPAMI.2020.2982166.
Kim, J., Lee, J. K., & Lee, K. M. (2016). Accurate image super-resolution using very deep convolutional networks. In 2016 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 1646–1654). IEEE. https://doi.org/10.1109/CVPR.2016.182
Dong, C., Loy, C. C., He, K., & Tang, X. (2014). Learning a deep convolutional network for image super-resolution (pp. 184–199). https://doi.org/10.1007/978-3-319-10593-2_13
Tong, T., Li, G., Liu, X., & Gao, Q. (2017). Image super-resolution using dense skip connections. In 2017 IEEE international conference on computer vision (ICCV) (pp. 4809–4817). IEEE. https://doi.org/10.1109/ICCV.2017.514
Tai, Y., Yang, J., Liu, X., & Xu, C. (2017). MemNet: A persistent memory network for image restoration. In 2017 IEEE international conference on computer vision (ICCV) (pp. 4549–4557). IEEE. https://doi.org/10.1109/ICCV.2017.486
Dong, C., Loy, C. C., & Tang, X. (2016). Accelerating the super-resolution convolutional neural network. Lecture notes in computer science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 9906 LNCS, 391–407. https://doi.org/10.1007/978-3-319-46475-6_25
Shi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A. P., Bishop, R., Wang, Z. (2016). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In 2016 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 1874–1883). IEEE. https://doi.org/10.1109/CVPR.2016.207
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Shi, W. (2017). Photo-realistic single image super-resolution using a generative adversarial network. In 2017 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 105–114). IEEE. https://doi.org/10.1109/CVPR.2017.19
Lim, B., Son, S., Kim, H., Nah, S., & Lee, K. M. (2017). Enhanced deep residual networks for single image super-resolution. In IEEE computer society conference on computer vision and pattern recognition workshops, 2017-July (pp. 1132–1140). https://doi.org/10.1109/CVPRW.2017.151
Irani, M., & Peleg, S. (1991). Improving resolution by image registration. CVGIP: Graphical Models and Image Processing, 53(3), 231–239. https://doi.org/10.1016/1049-9652(91)90045-L.
Haris, M., Shakhnarovich, G., & Ukita, N. (2018). Deep back-projection networks for super-resolution. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 1664–1673). IEEE. https://doi.org/10.1109/CVPR.2018.00179
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017). Densely connected convolutional networks. In 2017 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 2261–2269). IEEE. https://doi.org/10.1109/CVPR.2017.243
Wang, Z., Yi, P., Jiang, K., Jiang, J., Han, Z., Lu, T., & Ma, J. (2019). Multi-memory convolutional neural network for video super-resolution. IEEE Transactions on Image Processing, 28(5), 2530–2544. https://doi.org/10.1109/TIP.2018.2887017.
Zhang, Y., Tian, Y., Kong, Y., Zhong, B., & Fu, Y. (2018). Residual Dense Network for Image Super-Resolution. In 2018 IEEE/CVF conference on computer vision and pattern recognition (pp. 2472–2481). IEEE. https://doi.org/10.1109/CVPR.2018.00262
Liu, B., & Ait-Boudaoud, D. (2020). Effective image super resolution via hierarchical convolutional neural network. Neurocomputing, 374, 109–116. https://doi.org/10.1016/j.neucom.2019.09.035.
Timofte, R., Agustsson, E., Gool, L. Van, Yang, M.-H., Zhang, L., Lim, B., Guo, Q. (2017). NTIRE 2017 Challenge on single image super-resolution: Methods and results. In 2017 IEEE conference on computer vision and pattern recognition workshops (CVPRW) (pp. 1110–1121). IEEE. https://doi.org/10.1109/CVPRW.2017.149
Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. In 32nd international conference on machine learning, ICML 2015, 1, (pp. 48–456).
Wang, Z., Liu, D., Yang, J., Han, W., & Huang, T. (2015). Deep networks for image super-resolution with sparse prior. In 2015 IEEE international conference on computer vision (ICCV) (pp. 370–378). IEEE. https://doi.org/10.1109/ICCV.2015.50
Kim, J., Lee, J. K., & Lee, K. M. (2016). Deeply-recursive convolutional network for image super-resolution. In 2016 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 1637–1645). IEEE. https://doi.org/10.1109/CVPR.2016.181
Timofte, R., De Smet, V., & Van Gool, L. (2015). A+: Adjusted anchored neighborhood regression for fast super-resolution (pp. 111–126). https://doi.org/10.1007/978-3-319-16817-3_8
Su, H., Wei, S., Yan, M., Wang, C., Shi, J., & Zhang, X. (2019). Object detection and instance segmentation in remote sensing imagery based on precise mask R-CNN. In IGARSS 2019–2019 IEEE international geoscience and remote sensing symposium (pp. 1454–1457). IEEE. https://doi.org/10.1109/IGARSS.2019.8898573
Blau, Y., Mechrez, R., Timofte, R., Michaeli, T., & Zelnik-Manor, L. (2019). The 2018 PIRM challenge on perceptual image super-resolution (pp. 334–355). https://doi.org/10.1007/978-3-030-11021-5_21
Funding
This work was supported in part by the Key Research Program of Frontier Sciences, CAS, and Grant number ZDBS-LY-DQC016, Beijing Natural Science Foundation under Grant No. 4212030, Beijing Nova Program of Science and Technology under Grant No. Z191100001119090, Natural Science Foundation of China under Grant No. 61836013 and, Youth Innovation Promotion Association CAS.
Author information
Authors and Affiliations
Contributions
Farah Deeba proposed the super-resolution (SR) method and has written the manuscript; Xuezhi Wang and Y. Zhou provided a useful guide to the SR method; FA dharejo and She Kun gave guidance on the experimental problem and some details; Yi Du compiled the experimental image data and polished the Language of paper.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Deeba, F., Zhou, Y., Dharejo, F.A. et al. Multi-scale Single Image Super-Resolution with Remote-Sensing Application Using Transferred Wide Residual Network. Wireless Pers Commun 120, 323–342 (2021). https://doi.org/10.1007/s11277-021-08460-w
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11277-021-08460-w