Hyperspectral image super-resolution using multi-scale decomposition and convolutional neural network based on relation type between low- and high-resolution images

Abstract

Hyperspectral images (HSIs) usually have high spectral resolution and low spatial resolution. 3D convolutional neural networks (3D-CNNs) can successfully extract joint spectral and spatial information. But these networks do not provide high reconstruction accuracy in the spatial domain. To solve this problem, a different structure of 3D-CNN is suggested in this work. At first, different levels of decomposition dependent on the image types, i.e., low-resolution (LR) or high-resolution (HR) training images, are performed on the images by using the non-subsampled contourlet transform. To do this optimally, a similarity criterion is defined to determine the appropriate order of the decomposition levels. The defined similarity criterion is also used to select the appropriate training data from among the decomposed components at the network input, for the corresponding output. Then, according to the sparsity value of the decomposed components of LR and HR images, proper 3D-CNNs are individually designed. Finally, the networks are trained based on types of relations between the LR and HR image components. The proposed method is experimented on three different datasets. The results show superiority of the proposed method with respect to some competitors in terms of several measures.