Skip to main content
SpringerLink
Log in

Separable-spectral convolution and inception network for hyperspectral image super-resolution

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Due to the limitation of the imaging system, it is hard to get Hyperspectral Image (HSI) with very high spatial resolution. Super-Resolution (SR) is a handling missing data technology to restore high-frequency information from the low-resolution image, can be used to solve this problem. Recently, Deep Learning (DL) has achieved great performance in computer vision, including SR. However, most DL-based HSI SR methods neglect the spectral disorder caused by normal 2D convolution. This paper proposes a novel end–end deep learning-based network named Separable-Spectral and Inception Network (SSIN) for HSI SR. In SSIN, the feature extraction module independently extracts features of each band image, and then these features are fused together to further exploit residual image by using feature fusion module. In reconstruction module, a multi-path connection is built to obtain features of different levels to restore high spatial resolution image in a coarse-to-fine manner. Experiments are implemented on two datasets include both indoor and airborne HSIs, and the performances of SSIN are evaluated in different conditions. Experimental results show that adding several separable spectral convolutions and multi-path connection in a deep network can greatly improve the SR performance, and SSIN achieves higher accuracy and better visualization compare with other methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Li W, Wu G, Zhang F, Du Q (2017) Hyperspectral image classification using deep pixel-pair features. IEEE Trans Geosci Remote Sens 55(2):844–853

    Article  Google Scholar 

  2. Yu H, Gao L, Li J et al. (2016) Spectral-spatial hyperspectral image classification using subspace-based support vector machines and adaptive markov random fields. Remote Sens 8(5):355

    Article  Google Scholar 

  3. Li W, Du Q, Zhang F, Hu W (2016) Hyperspectral image classification by fusing collaborative and sparse representations. IEEE J Sel Top Appl Earth Obs Remote Sens 9(9):4178–4187

    Article  Google Scholar 

  4. Du B, Zhang M, Zhang L, Hu R, Tao D (2017) PLTD: patch-based low-rank tensor decomposition for hyperspectral images. IEEE Trans Multimed 19(1):67–79

    Article  Google Scholar 

  5. Li W, Wu G, Du Q (2017) Transferred deep learning for anomaly detection in hyperspectral imagery. IEEE Geosci Remote Sens Lett 14(5):597–601

    Article  Google Scholar 

  6. Du B, Zhang Y, Zhang L, Tao D (2016) Beyond the sparsity-based target detector: a hybrid sparsity and statistics based detector for hyperspectral images. IEEE Trans Image Process 25(11):5345–5357

    Article  MathSciNet  MATH  Google Scholar 

  7. Zhang H, Zhang L, Shen H (2012) A super-resolution reconstruction algorithm for hyperspectral images. Sig Process 92(9):2082–2096

    Article  Google Scholar 

  8. Fernandez-Beltran R, Latorre-Carmona P, Pla F (2017) Single-frame super-resolution in remote sensing: a practical overview. Int J Remote Sens 38(1):314–354

    Article  Google Scholar 

  9. Eismann MT, Hardie RC (2005) Hyperspectral resolution enhancement using high-resolution multispectral imagery with arbitrary response functions. IEEE Trans Geosci Remote Sens 43(3):455–465

    Article  Google Scholar 

  10. Dong W, Fu F, Shi G et al. (2016) Hyperspectral image super-resolution via non-negative structured sparse representation. IEEE Trans Image Process 25(5):2337–2352

    Article  MathSciNet  MATH  Google Scholar 

  11. Akhtar N, Shafait F, Mian A (2015) Bayesian sparse representation for hyperspectral image super resolution. IEEE Conf Comput Vis Pattern Recognit. 2015:3631–3640

    Google Scholar 

  12. Akhtar N, Shafait F, Mian A (2014) Sparse spatio-spectral representation for hyperspectral image super-resolution. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T (eds) Computer Vision – ECCV 2014. Springer, Cham, pp 63–78

    Chapter  Google Scholar 

  13. Wei H, Liang X, Liu H et al. (2015) Hyperspectral imagery super-resolution by compressive sensing inspired dictionary learning and spatial-spectral regularization. Sensors 15(1):2041

    Article  Google Scholar 

  14. Yang J, Wright J, Huang T et al (2008) Image super-resolution as sparse representation of raw image patches. IEEE conf comput vision pattern recogn 2008(1):1–8

    Google Scholar 

  15. Yang J, Wright J, Huang T et al. (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  MATH  Google Scholar 

  16. Li F, Xin L, Guo Y et al. (2017) A framework of mixed sparse representations for remote sensing images. IEEE Trans Geosci Remote Sens PP(99):1–12

    Google Scholar 

  17. Timofte R, De V, Gool LV (2013) Anchored neighborhood regression for fast example-based super-resolution. IEEE international conference on computer vision. pp 1920–1927

  18. Timofte R, Smet VD, Gool LV (2014) A+: adjusted anchored neighborhood regression for fast super-resolution. Asian Conf Comput Vis 2014(9006):111–126

    Google Scholar 

  19. Timofte R, Rothe R, Gool LV (2016) Seven ways to improve example-based single image super resolution. IEEE conference on computer vision and pattern recognition. pp 1865–1873

  20. Huang JB, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. IEEE conference on computer vision and pattern recognition. pp 5197–5206

  21. Dong C, Chen CL, He K et al. (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307

    Article  Google Scholar 

  22. Wang Z, Liu D, Yang J et al. (2016) Deep networks for image super-resolution with sparse prior. IEEE international conference on computer vision. pp 370–378

  23. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. IEEE conference on computer vision and pattern recognition. pp 1646–1654

  24. Kim J, Lee JK, Lee KM (2016) Deeply-recursive convolutional network for image super-resolution. IEEE conference on computer vision and pattern recognition. 1637–1645

  25. Lim B, Son S, Kim H et al. (2017) Enhanced deep residual networks for single image super-resolution. IEEE conference on computer vision and pattern recognition. 1132–1140

  26. Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(2):2012

    Google Scholar 

  27. He K, Zhang X, Ren S et al. (2016) Deep residual learning for image recognition. IEEE conference on computer vision and pattern recognition. 770–778

  28. Du B, Xiong W, Wu J et al (2017) Stacked convolutional denoising auto-encoders for feature representation. IEEE Trans Cybern 47(4):1017–1027

    Article  Google Scholar 

  29. Ren S, Girshick R, Girshick R et al (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137

    Article  Google Scholar 

  30. Redmon J, Divvala S, Girshick R et al. (2016) You only look once: unified, real-time object detection. IEEE conference on computer vision and pattern recognition. 779–788

  31. Liu W, Anguelov D, Erhan D et al. (2016) SSD: single shot multibox detector. European conference on computer vision. pp 21–37

    Chapter  Google Scholar 

  32. He K, Gkioxari G, Dollár P et al. (2017) Mask R-CNN. IEEE international conference on computer vision

  33. Shi W, Caballero J, Huszar F et al. (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. IEEE conference on computer vision and pattern recognition. pp 1874–1883

  34. Ledig C, Theis L, Huszar F et al (2016) Photo-realistic single image super-resolution using a generative adversarial network. IEEE conference on computer vision and pattern recognition. pp 105–114

  35. Lim B, Son S, Kim H et al (2017) Enhanced deep residual networks for single image super-resolution. IEEE conference on computer vision and pattern recognition workshops. pp 1132–1140

  36. Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. IEEE conference on computer vision and pattern recognition. pp 2790–2798

  37. Liebel L, Körner M (2016) Single-image super resolution for multispectral remote sensing data using convolutional neural networks. Int Arch Photogramm Remote Sens S XLI-B3. 2016:883–890

    Article  Google Scholar 

  38. Li Y, Hu J, Zhao X et al. (2017) Hyperspectral image super-resolution using deep convolutional neural network. Neurocomputing 266(29):29–41

    Google Scholar 

  39. Hu J, Li Y, Xie W (2017) Hyperspectral image super-resolution by spectral difference learning and spatial error correction. IEEE Geosci Remote Sens Lett 14(10):1825–1829

    Article  Google Scholar 

  40. Galliani S, Lanars C, Marmanis D et al. (2017) Learned spectral super-resolution. arXiv preprint arXiv:1703.09470

  41. Lei S, Shi Z, Zou Z (2017) Super-Resolution for remote sensing images via local-global combined network. IEEE Geosci Remote Sens Lett PP(99):1–5

    Google Scholar 

  42. Yuan Y, Zheng X, Lu X (2017) Hyperspectral image superresolution by transfer learning. IEEE J Sel Top Appl Earth Obs Remote Sensing 10(5):1963–1974

    Article  Google Scholar 

  43. Mei S, Yuan X, Ji J et al (2017) Hyperspectral image spatial super-resolution via 3D full convolutional neural network. Remote Sens 9(11):1139

    Article  Google Scholar 

  44. Jonas A, Ji QW, Radu T (2017) In defense of shallow learned spectral reconstruction from rgb images. IEEE conference on computer vision and pattern recognition. pp 471–479

  45. Chang H, Yeung DY, Xiong Y (2004) Super-resolution through neighbor embedding. IEEE Conf Comput Vis Pattern Recognit. 1:275–282

    Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant No. 91638201, No. 61501017, and No. 41722108.

Author information

Authors and Affiliations

  1. College of Geoscience and Surveying Engineering, China University of Mining and Technology (Beijing), Beijing, 100083, China

    Ke Zheng & Ximin Cui

  2. Key Laboratory of Digital Earth Science, Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing, 100094, China

    Ke Zheng, Lianru Gao & Bing Zhang

  3. College of Computer Science and Software Engineering, Computer Vision Research Institute, Shenzhen University, Shenzhen, 518060, China

    Lianru Gao & Sen Jia

  4. College of Information Science and Technology, Beijing University of Chemical Technology, Beijing, 100029, China

    Qiong Ran

  5. University of Chinese Academy of Sciences, Beijing, 100049, China

    Bing Zhang

  6. Department Telecommunications and Information Processing, Ghent University, 9000, Ghent, Belgium

    Wenzhi Liao

Authors
  1. Ke Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lianru Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiong Ran
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ximin Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bing Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Wenzhi Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Sen Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lianru Gao.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zheng, K., Gao, L., Ran, Q. et al. Separable-spectral convolution and inception network for hyperspectral image super-resolution. Int. J. Mach. Learn. & Cyber. 10, 2593–2607 (2019). https://doi.org/10.1007/s13042-018-00911-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-018-00911-4

Keywords

Search

Navigation