Advertisement

Towards Robust Evaluation of Super-Resolution Satellite Image Reconstruction

  • Michał Kawulok
  • Paweł Benecki
  • Jakub Nalepa
  • Daniel Kostrzewa
  • Łukasz Skonieczny
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10751)

Abstract

Super-resolution reconstruction (SRR) consists in processing an image or a bunch of images to generate a new image of higher spatial resolution. This problem has been intensively studied, but seldom is SRR applied in practice for satellite data. In this paper, we briefly review the state of the art on SRR algorithms and we argue that commonly adopted strategies for their evaluation do not reflect the operational conditions. We report our study on assessing the SRR outcome, relying on new quantitative measures. The obtained results allow us to outline the most important research pathways to improve the performance of SRR.

Keywords

Super-resolution Image processing Similarity measures 

Notes

Acknowledgments

The reported work is a part of the SISPARE project run by Future Processing and funded by European Space Agency. The authors were partially supported by Institute of Informatics funds no. BK-230/RAu2/2017 (MK) and BKM-509/RAu2/2017 (JN, DK).

References

  1. 1.
    Ahrens, B.: Genetic algorithm optimization of superresolution parameters. In: Proceedings of the GECCO, pp. 2083–2088. ACM (2005)Google Scholar
  2. 2.
    Akgun, T., Altunbasak, Y., Mersereau, R.M.: Super-resolution reconstruction of hyperspectral images. IEEE Trans. Image Process. 14(11), 1860–1875 (2005)CrossRefGoogle Scholar
  3. 3.
    Capel, D., Zisserman, A.: Super-resolution enhancement of text image sequences. In: Proceedings of the IEEE ICPR, vol. 1, pp. 600–605 (2000)Google Scholar
  4. 4.
    Cheng, M.H., Hwang, K.S., Jeng, J.H., Lin, N.W.: PSO-based fusion method for video super-resolution. J. Signal Process. Syst. 73(1), 25–42 (2013)CrossRefGoogle Scholar
  5. 5.
    Del Gallego, N.P., Ilao, J.: Multiple-image super-resolution on mobile devices: an image warping approach. EURASIP J. Image Video Process. 2017(1), 1–15 (2017)Google Scholar
  6. 6.
    Demirel, H., Anbarjafari, G.: Image resolution enhancement by using discrete and stationary wavelet decomposition. IEEE Trans. Image Process. 20(5), 1458–1460 (2011)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)CrossRefGoogle Scholar
  8. 8.
    Ducournau, A., Fablet, R.: Deep learning for ocean remote sensing: an application of convolutional neural networks for super-resolution on satellite-derived SST data. In: Proceedings of the IAPR WPRRS, pp. 1–6 (2016)Google Scholar
  9. 9.
    Farsiu, S., Robinson, M.D., Elad, M., Milanfar, P.: Fast and robust multiframe super resolution. IEEE Trans. Image Process. 13(10), 1327–1344 (2004)CrossRefGoogle Scholar
  10. 10.
    González-Audícana, M., Saleta, J.L., Catalán, R.G., García, R.: Fusion of multispectral and panchromatic images using improved IHS and PCA mergers based on wavelet decomposition. IEEE Trans. Geosci. Remote Sens. 42(6), 1291–1299 (2004)CrossRefGoogle Scholar
  11. 11.
    Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE CVPR, pp. 5197–5206 (2015)Google Scholar
  12. 12.
    Jiang, J., Hu, R., Wang, Z., Han, Z.: Face super-resolution via multilayer locality-constrained iterative neighbor embedding and intermediate dictionary learning. IEEE Trans. Image Process. 23(10), 4220–4231 (2014)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Li, F., Jia, X., Fraser, D.: Universal HMT based super resolution for remote sensing images. In: Proceedings of the IEEE ICIP, pp. 333–336 (2008)Google Scholar
  14. 14.
    Liebel, L., Körner, M.: Single-image super resolution for multispectral remote sensing data using convolutional neural networks. In: Proceedings of the ISPRS Congress, pp. 883–890 (2016)Google Scholar
  15. 15.
    Lorenzo, P.R., Nalepa, J., Kawulok, M., Ramos, L.S., Pastor, J.R.: Particle swarm optimization for hyper-parameter selection in deep neural networks. In: Proceedings of the GECCO, pp. 481–488. ACM, New York (2017)Google Scholar
  16. 16.
    Lukinavičius, G., Umezawa, K., Olivier, N., Honigmann, A., Yang, G., Plass, T., et al.: A near-infrared fluorophore for live-cell super-resolution microscopy of cellular proteins. Nat. Chem. 5(2), 132–139 (2013)CrossRefGoogle Scholar
  17. 17.
    Miravet, C., Rodrıguez, F.B.: A two-step neural-network based algorithm for fast image super-resolution. Image Vis. Comput. 25(9), 1449–1473 (2007)CrossRefGoogle Scholar
  18. 18.
    Molina, R., Vega, M., Mateos, J., Katsaggelos, A.K.: Variational posterior distribution approximation in Bayesian super resolution reconstruction of multispectral images. Appl. Comput. Harmon. Anal. 24(2), 251–267 (2008)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Nasrollahi, K., Moeslund, T.B.: Super-resolution: a comprehensive survey. Mach. Vis. Appl. 25(6), 1423–1468 (2014)CrossRefGoogle Scholar
  20. 20.
    Panagiotopoulou, A., Anastassopoulos, V.: Super-resolution image reconstruction techniques: trade-offs between the data-fidelity and regularization terms. Inf. Fusion 13(3), 185–195 (2012)CrossRefGoogle Scholar
  21. 21.
    Qian, S.E., Chen, G.: Enhancing spatial resolution of hyperspectral imagery using sensor’s intrinsic keystone distortion. IEEE Trans. Geosci. Remote Sens. 50(12), 5033–5048 (2012)CrossRefGoogle Scholar
  22. 22.
    Rubert, C., Fonseca, L., Velho, L.: Learning based super-resolution using YUV model for remote sensing images. In: Proceedings of the SIBGRAPI (2005)Google Scholar
  23. 23.
    Schultz, R.R., Stevenson, R.L.: Extraction of high-resolution frames from video sequences. IEEE Trans. Image Process. 5(6), 996–1011 (1996)CrossRefGoogle Scholar
  24. 24.
    Sheikh, H.R., Bovik, A.C.: Image information and visual quality. IEEE Trans. Image Process. 15(2), 430–444 (2006)CrossRefGoogle Scholar
  25. 25.
    Sheikh, H.R., Bovik, A.C., De Veciana, G.: An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Trans. Image Process. 14(12), 2117–2128 (2005)CrossRefGoogle Scholar
  26. 26.
    Sun, L., Hays, J.: Super-resolution from Internet-scale scene matching. In: Proceedings of the IEEE ICCP (2012)Google Scholar
  27. 27.
    Timofte, R., De Smet, V., Van Gool, L.: A+: adjusted anchored neighborhood regression for fast super-resolution. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9006, pp. 111–126. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-16817-3_8 Google Scholar
  28. 28.
    Wang, Y., Fevig, R., Schultz, R.R.: Super-resolution mosaicking of UAV surveillance video. In: Proceedings of the IEEE ICIP, pp. 345–348. IEEE (2008)Google Scholar
  29. 29.
    Wang, Z., Bovik, A.: A universal image quality index. IEEE Signal Process. Lett. 9(3), 81–84 (2002)CrossRefGoogle Scholar
  30. 30.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004)CrossRefGoogle Scholar
  31. 31.
    Wu, B., Li, C., Zhan, X.: Integrating spatial structure in super-resolution mapping of hyper-spectral image. Procedia Eng. 29, 1957–1962 (2012)CrossRefGoogle Scholar
  32. 32.
    Yang, F., Chen, Y., Wang, R., Zhang, Q.: Super-resolution microwave imaging: time-domain tomography using highly accurate evolutionary optimization method. In: Proceedings of the EuCAP, pp. 1–4. IEEE (2015)Google Scholar
  33. 33.
    Yue, L., Shen, H., Li, J., Yuan, Q., Zhang, H., Zhang, L.: Image super-resolution: the techniques, applications, and future. Signal Process. 128, 389–408 (2016)CrossRefGoogle Scholar
  34. 34.
    Zhang, H., Zhang, L., Shen, H.: A super-resolution reconstruction algorithm for hyperspectral images. Signal Process. 92(9), 2082–2096 (2012)CrossRefGoogle Scholar
  35. 35.
    Zhang, Y.: Problems in the fusion of commercial high-resolution satelitte as well as Landsat 7 images and initial solutions. In: Proceedings of the GTPA, pp. 1–6 (2002)Google Scholar
  36. 36.
    Zhong, Y., Zhang, L.: Remote sensing image subpixel mapping based on adaptive differential evolution. IEEE Trans. Syst. Man Cybern. Part B 42(5), 1306–1329 (2012)CrossRefGoogle Scholar
  37. 37.
    Zhu, H., Song, W., Tan, H., Wang, J., Jia, D.: Super resolution reconstruction based on adaptive detail enhancement for ZY-3 satellite images. In: Proceedings of the ISPRS, pp. 213–217 (2016)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Future ProcessingGliwicePoland
  2. 2.Institute of InformaticsSilesian University of TechnologyGliwicePoland

Personalised recommendations