Super-resolution of PROBA-V images using convolutional neural networks
- 170 Downloads
European Space Aqency (ESA)’s PROBA-V Earth observation (EO) satellite enables us to monitor our planet at a large scale to study the interaction between vegetation and climate, and provides guidance for important decisions on our common global future. However, the interval at which high-resolution images are recorded spans over several days, in contrast to the availability of lower-resolution images which is often daily. We collect an extensive dataset of both high- and low-resolution images taken by PROBA-V instruments during monthly periods to investigate Multi Image Super-resolution, a technique to merge several low-resolution images into one image of higher quality. We propose a convolutional neural network (CNN) that is able to cope with changes in illumination, cloud coverage, and landscape features which are introduced by the fact that the different images are taken over successive satellite passages at the same region. Given a bicubic upscaling of low resolution images taken under optimal conditions, we find the Peak Signal to Noise Ratio of the reconstructed image of the network to be higher for a large majority of different scenes. This shows that applied machine learning has the potential to enhance large amounts of previously collected EO data during multiple satellite passes.
Keywordsdeep learning convolutional neural network (CNN) super-resolution imaging remote sensing Earth observation (EO)
The authors would like to thank the anonymous reviewers for their extremely detailed comments and critical questions, which helped to make the presentation of these results clearer and increased the overall quality of this work.
- Shi, W. Z., Caballero, J., Huszar, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., Wang, Z. H. Realtime single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016.Google Scholar
- Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z. H. et al. Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017.Google Scholar
- Yeh, R. A., Chen, C., Lim, T. Y., Schwing, A. G., Hasegawa-Johnson, M., Do, M. N. Semantic image inpainting with deep generative models. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 5485–5493.Google Scholar
- Timofte, R., Gu, S. H., van Gool, L., Zhang, L., Yang, M. H. NTIRE 2018 challenge on single image super-resolution: Methods and results. In: Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2018: 852–863.Google Scholar
- Timofte, R., Agustsson, E., Van Gool, L., Yang, M. H., Zhang, L., Lim, B., Son, S., Kim, H., Nah, S., Lee, K. M., et al. Ntire 2017 challenge on single image super-resolution: Methods and results. In: Proceedings of Computer Vision and Pattern Recognition Workshops, 2017: 1110–1121.Google Scholar
- Huang, G., Liu, Z., van der Maaten, L., Weinberger, K. Q. Densely connected convolutional networks. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4700–4708.Google Scholar
- Mitzel, D., Pock, T., Schoenemann, T., Cremers, D. Video super resolution using duality based TV-L1 optical flow. In: Lecture Notes in Computer Science. Denzler, J., Notni, G., Süße, H., Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2009: 432–441.Google Scholar
- Murthy, K., Shearn, M., Smiley, B. D., Chau, A. H., Levine, J., Robinson, M. D. SkySat-1: very high-resolution imagery from a small satellite. In: Proceedings of SPIE 9241,Sensors, Systems, and Next-Generation Satellites XVIII, 2014: 92411E.Google Scholar
- Wolters, E., Dierckx, W., Iordache, M. D., Swinnen, E. PROBA-V Products User Manual, 2014.Google Scholar