Advertisement

PIRM2018 Challenge on Spectral Image Super-Resolution: Dataset and Study

  • Mehrdad ShoeibyEmail author
  • Antonio Robles-Kelly
  • Ran Wei
  • Radu Timofte
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11133)

Abstract

This paper introduces a newly collected and novel dataset (StereoMSI) for example-based single and colour-guided spectral image super-resolution. The dataset was first released and promoted during the PIRM2018 spectral image super-resolution challenge. To the best of our knowledge, the dataset is the first of its kind, comprising 350 registered colour-spectral image pairs. The dataset has been used for the two tracks of the challenge and, for each of these, we have provided a split into training, validation and testing. This arrangement is a result of the challenge structure and phases, with the first track focusing on example-based spectral image super-resolution and the second one aiming at exploiting the registered stereo colour imagery to improve the resolution of the spectral images. Each of the tracks and splits has been selected to be consistent across a number of image quality metrics. The dataset is quite general in nature and can be used for a wide variety of applications in addition to the development of spectral image super-resolution methods.

Keywords

Super-resolution Hyperspectral Multispectral RGB Stereo 

Notes

Acknowledgemnts

The PIRM2018 challenge was sponsored by CSIRO’s DATA61, Deakin University, ETH Zurich, HUAWEI, and MediaTek.

References

  1. 1.
    Robles-Kelly, A., Huynh, C.P.: Imaging Spectroscopy for Scene Analysis. Springer, London (2012).  https://doi.org/10.1007/978-1-4471-4652-0CrossRefGoogle Scholar
  2. 2.
    Goetz, A.F.: Three decades of hyperspectral remote sensing of the earth: a personal view. Remote Sens. Environ. 113, S5–S16 (2009)CrossRefGoogle Scholar
  3. 3.
    Hasan, M., Jia, X., Robles-Kelly, A., Zhou, J., Pickering, M.R.: Multi-spectral remote sensing image registration via spatial relationship analysis on sift keypoints. In: 2010 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), pp. 1011–1014. IEEE (2010)Google Scholar
  4. 4.
    Lu, G., Fei, B.: Medical hyperspectral imaging: a review. J. Biomed. Opt. 19(1), 010901 (2014)CrossRefGoogle Scholar
  5. 5.
    Feng, Y.Z., Sun, D.W.: Application of hyperspectral imaging in food safety inspection and control: a review. Crit. Rev. Food Sci. Nutr. 52(11), 1039–1058 (2012)CrossRefGoogle Scholar
  6. 6.
    Elarab, M., Ticlavilca, A.M., Torres-Rua, A.F., Maslova, I., McKee, M.: Estimating chlorophyll with thermal and broadband multispectral high resolution imagery from an unmanned aerial system using relevance vector machines for precision agriculture. Int. J. Appl. Earth Obs. Geoinf. 43, 32–42 (2015)CrossRefGoogle Scholar
  7. 7.
    Liang, H.: Advances in multispectral and hyperspectral imaging for archaeology and art conservation. Appl. Phys. A 106(2), 309–323 (2012)CrossRefGoogle Scholar
  8. 8.
    Bell, J.F., et al.: Multispectral imaging of mars from the mars science laboratory mastcam instruments: spectral properties and mineralogic implications along the gale crater traverse. In: AAS/Division for Planetary Sciences Meeting Abstracts, vol. 48 (2016)Google Scholar
  9. 9.
    Xie, Z., Jiang, P., Zhang, S., Xiong, J.: Hyperspectral face recognition based on spatio-spectral fusion and local binary pattern. In: AOPC 2017: Optical Sensing and Imaging Technology and Applications, vol. 10462, p. 104620C. International Society for Optics and Photonics (2017)Google Scholar
  10. 10.
    Wu, D., Sun, D.W.: Advanced applications of hyperspectral imaging technology for food quality and safety analysis and assessment: a reviewpart i: fundamentals. Innov. Food Sci. Emerg. Technol. 19, 1–14 (2013)CrossRefGoogle Scholar
  11. 11.
    Bigas, M., Cabruja, E., Forest, J., Salvi, J.: Review of CMOS image sensors. Microelectron. J. 37(5), 433–451 (2006)CrossRefGoogle Scholar
  12. 12.
    Timofte, R., et al.: Ntire 2017 challenge on single image super-resolution: methods and results. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1110–1121. IEEE (2017)Google Scholar
  13. 13.
    Kim, J., Kwon Lee, J., Mu Lee, K.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 1646–1654 (2016)Google Scholar
  14. 14.
    Lai, W.S., Huang, J.B., Ahuja, N., Yang, M.H.: Deep laplacian pyramid networks for fast and accurate superresolution. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, p. 5 (2017)Google Scholar
  15. 15.
    Timofte, R., et al.: Ntire 2018 challenge on single image super-resolution: methods and results. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2018Google Scholar
  16. 16.
    Timofte, R., De Smet, V., Van Gool, L.: A+: adjusted anchored neighborhood regression for fast super-resolution. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9006, pp. 111–126. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-16817-3_8CrossRefGoogle Scholar
  17. 17.
    Timofte, R., De Smet, V., Van Gool, L.: Anchored neighborhood regression for fast example-based super-resolution. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1920–1927 (2013)Google Scholar
  18. 18.
    Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of Eighth IEEE International Conference on Computer Vision, ICCV 2001, vol. 2, pp. 416–423. IEEE (2001)Google Scholar
  19. 19.
    Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: Boissonnat, J.-D., Chenin, P., Cohen, A., Gout, C., Lyche, T., Mazure, M.-L., Schumaker, L. (eds.) Curves and Surfaces 2010. LNCS, vol. 6920, pp. 711–730. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-27413-8_47CrossRefGoogle Scholar
  21. 21.
    Bevilacqua, M., Roumy, A., Guillemot, C., Alberi-Morel, M.L.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding (2012)Google Scholar
  22. 22.
    Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5197–5206 (2015)Google Scholar
  23. 23.
    Agustsson, E., Timofte, R.: Ntire 2017 challenge on single image super-resolution: dataset and study. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, vol. 3, p. 2 (2017)Google Scholar
  24. 24.
    Li, Y., Hu, J., Zhao, X., Xie, W., Li, J.: Hyperspectral image super-resolution using deep convolutional neural network. Neurocomputing 266, 29–41 (2017)CrossRefGoogle Scholar
  25. 25.
    Yasuma, F., Mitsunaga, T., Iso, D., Nayar, S.K.: Generalized assorted pixel camera: postcapture control of resolution, dynamic range, and spectrum. IEEE Trans. Image Process. 19(9), 2241–2253 (2010)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Chakrabarti, A., Zickler, T.: Statistics of real-world hyperspectral images. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 193–200. IEEE (2011)Google Scholar
  27. 27.
    Foster, D.H., Nascimento, S.M., Amano, K.: Information limits on neural identification of colored surfaces in natural scenes. Vis. Neurosci. 21(3), 331–336 (2004)CrossRefGoogle Scholar
  28. 28.
    Arad, B., Ben-Shahar, O., Timofte, R.: Ntire 2018 challenge on spectral reconstruction from RGB images. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2018Google Scholar
  29. 29.
    Loncan, L., et al.: Hyperspectral pansharpening: a review. arXiv preprint arXiv:1504.04531 (2015)
  30. 30.
    Lanaras, C., Baltsavias, E., Schindler, K.: Hyperspectral super-resolution by coupled spectral unmixing. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3586–3594 (2015)Google Scholar
  31. 31.
    Kawakami, R., Matsushita, Y., Wright, J., Ben-Ezra, M., Tai, Y.W., Ikeuchi, K.: High-resolution hyperspectral imaging via matrix factorization. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2329–2336. IEEE (2011)Google Scholar
  32. 32.
    Brown, M., Süsstrunk, S.: Multi-spectral sift for scene category recognition. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 177–184. IEEE (2011)Google Scholar
  33. 33.
    Zhi, T., Pires, B.R., Hebert, M., Narasimhan, S.G.: Deep material-aware cross-spectral stereo matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1916–1925 (2018)Google Scholar
  34. 34.
    Shoeiby, M., et al.: PIRM2018 challenge on spectral image super-resolution: methods and results. In: European Conference on Computer Vision Workshops (ECCVW) (2018)Google Scholar
  35. 35.
    Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., Brox, T.: Flownet 2.0: evolution of optical flow estimation with deep networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, p. 6 (2017)Google Scholar
  36. 36.
    Lahoud, F., Zhou, R., Süsstrunk, S.: Multi-modal spectral image super-resolution. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018 Workshops. LNCS, vol. 11133, pp. 35–50. Springer, Cham (2018)Google Scholar
  37. 37.
    Shi, Z., Chen, C., Xiong, Z., Liu, D., Zha, Z.J., Wu, F.: Deep residual attention network for spectral image super-resolution. In: European Conference on Computer Vision Workshops (ECCVW) (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Mehrdad Shoeiby
    • 1
    Email author
  • Antonio Robles-Kelly
    • 2
  • Ran Wei
    • 1
  • Radu Timofte
    • 3
  1. 1.DATA61 - CSIRO, Black Mountain LaboratoriesCanberraAustralia
  2. 2.Faculty of Science, Engineering and Built EnvironmentDeakin UniversityBurwoodAustralia
  3. 3.Computer Vision Laboratory, D-ITETETH ZurichZürichSwitzerland

Personalised recommendations