Skip to main content
SpringerLink
Log in

Guided Hyperspectral Image Denoising with Realistic Data

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

The hyperspectral image (HSI) denoising has been widely utilized to improve HSI qualities. Recently, learning-based HSI denoising methods have shown their effectiveness, but most of them are based on synthetic dataset and lack the generalization capability on real testing HSI. Moreover, there is still no public paired real HSI denoising dataset to learn HSI denoising network and quantitatively evaluate HSI methods. In this paper, we mainly focus on how to produce realistic dataset for learning and evaluating HSI denoising network. On the one hand, we collect a paired real HSI denoising dataset, which consists of short-exposure noisy HSIs and the corresponding long-exposure clean HSIs. On the other hand, we propose an accurate HSI noise model which matches the distribution of real data well and can be employed to synthesize realistic dataset. On the basis of the noise model, we present an approach to calibrate the noise parameters of the given hyperspectral camera. Besides, on the basis of observation of high signal-to-noise ratio of mean image of all spectral bands, we propose a guided HSI denoising network with guided dynamic nonlocal attention, which calculates dynamic nonlocal correlation on the guidance information, i.e., mean image of spectral bands, and adaptively aggregates spatial nonlocal features for all spectral bands. The extensive experimental results show that a network learned with only synthetic data generated by our noise model performs as well as it is learned with paired real data, and our guided HSI denoising network outperforms state-of-the-art methods under both quantitative metrics and visual quality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  • Abdelhamed, A., Lin, S., & Brown, MS. (2018). A high-quality denoising dataset for smartphone cameras. In: Proc. of Conference on Computer Vision and Pattern Recognition, pp 1692–1700.

  • Acito, N., Diani, M., & Corsini, G. (2011). Signal-dependent noise modeling and model parameter estimation in hyperspectral images. IEEE Trans Geoscience and Remote Sensing, 49(8), 2957–2971.

    Article  Google Scholar 

  • Arad, B., & Ben-Shahar, O. (2016). Sparse recovery of hyperspectral signal from natural rgb images. In: Proc. of European Conference on Computer Vision, pp. 19–34.

  • Basedow, RW., Carmer, DC., & Anderson, ME. (1995). Hydice system: Implementation and performance. In: Proc. of SPIE’s Symposium on OE/Aerospace Sensing and Dual Use Photonics, pp. 258–267.

  • Bjorgan, A., Randeberg, & LL. (2015). Towards real-time medical diagnostics using hyperspectral imaging technology. In: Proc. of Clinical and Biomedical Spectroscopy and Imaging IV, p. 953712.

  • Borengasser, M., Hungate, W. S., & Watkins, R. (2007). Hyperspectral Remote Sensing: Principles and Applications. Remote Sensing Applications Series: CRC Press.

    Book  Google Scholar 

  • Cai, Y., Lin, J., Hu, X., Wang, H., Yuan, X., Zhang, Y., Timofte, R., & Van Gool, L. (2021). Mask-guided spectral-wise transformer for efficient hyperspectral image reconstruction. arXiv preprint arXiv:2111.07910

  • Cao, X., Zhou, F., Xu, L., Meng, D., Xu, Z., & Paisley, J. (2018). Hyperspectral image classification with markov random fields and a convolutional neural network. IEEE Trans Image Processing, 27(5), 2354–2367.

    Article  MathSciNet  Google Scholar 

  • Chakrabarti, A., & Zickler, TE. (2011). Statistics of real-world hyperspectral images. In: Proc. of Conference on Computer Vision and Pattern Recognition, pp. 193–200.

  • Chang, Y., Yan, L., & Zhong, S. (2017). Hyper-laplacian regularized unidirectional low-rank tensor recovery for multispectral image denoising. In: Proc. of Conference on Computer Vision and Pattern Recognition, pp. 4260–4268.

  • Chang, Y., Yan, L., Fang, H., Zhong, S., & Liao, W. (2018). Hsi-denet: Hyperspectral image restoration via convolutional neural network. IEEE Trans Geoscience and Remote Sensing, 57(2), 667–682.

    Article  Google Scholar 

  • Chang, Y., Yan, L., Zhao, X. L., Fang, H., Zhang, Z., & Zhong, S. (2020). Weighted low-rank tensor recovery for hyperspectral image restoration. IEEE Trans Cybernetics, 50(11), 4558–4572.

    Article  Google Scholar 

  • Charbonnier, P., Blanc-Feraud, L., Aubert, G., & Barlaud, M. (1994). Two deterministic half-quadratic regularization algorithms for computed imaging. In: Proc. of International Conference on Image Processing, 2, 168–172

  • Chen, C., Li, W., Tramel, E. W., Cui, M., Prasad, S., & Fowler, J. E. (2014). Spectral-spatial preprocessing using multihypothesis prediction for noise-robust hyperspectral image classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(4), 1047–1059.

    Article  Google Scholar 

  • Chen, C., Chen, Q., Xu, J., & Koltun, V. (2018). Learning to see in the dark. In: Proc. of Conference on Computer Vision and Pattern Recognition, pp. 3291–3300.

  • Chen, C., Chen, Q., Do, MN., & Koltun, V. (2019). Seeing motion in the dark. In: Proc. of International Conference on Computer Vision, pp. 3185–3194.

  • Chen, G., & Qian, S. E. (2010). Denoising of hyperspectral imagery using principal component analysis and wavelet shrinkage. IEEE Trans Geoscience and Remote Sensing, 49(3), 973–980.

    Article  Google Scholar 

  • Chen, Y., Cao, X., Zhao, Q., Meng, D., & Xu, Z. (2017). Denoising hyperspectral image with non-iid noise structure. IEEE Trans Cybernetics, 48(3), 1054–1066.

    Article  Google Scholar 

  • Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007). Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Trans Image Processing, 16(8), 2080–2095.

    Article  MathSciNet  Google Scholar 

  • Dong, W., Li, G., Shi, G., Li, X., & Ma, Y. (2015). Low-rank tensor approximation with laplacian scale mixture modeling for multiframe image denoising. In: Proc. of International Conference on Computer Vision, pp. 442–449.

  • Dong, W., Wang, H., Wu, F., Shi, G., & Li, X. (2019). Deep spatial-spectral representation learning for hyperspectral image denoising. IEEE Trans Computational Imaging, 5(4), 635–648.

    Article  Google Scholar 

  • Fu, Y., Lam, A., Sato, I., & Sato, Y. (2017). Adaptive spatial-spectral dictionary learning for hyperspectral image restoration. International Journal of Computer Vision, 122(2), 228–245.

    Article  MathSciNet  MATH  Google Scholar 

  • Fu, Y., Zheng, Y., Huang, H., Sato, I., & Sato, Y. (2018). Hyperspectral image super-resolution with a mosaic rgb image. IEEE Trans Image Processing, 27(11), 5539–5552.

    Article  MathSciNet  Google Scholar 

  • Fu, Y., Zhang, T., Zheng, Y., Zhang, D., & Huang, H. (2019). Hyperspectral image super-resolution with optimized rgb guidance. In: Proc. of Conference on Computer Vision and Pattern Recognition, pp. 11661–11670.

  • Gu, S., Li, Y., Gool, LV., & Timofte, R. (2019). Self-guided network for fast image denoising. In: Proc. of International Conference on Computer Vision, pp. 2511–2520.

  • Guo, S., Liang, Z., & Zhang, L. (2021). Joint denoising and demosaicking with green channel prior for real-world burst images. IEEE Trans Image Processing, 30, 6930–6942.

    Article  MathSciNet  Google Scholar 

  • He, C., Sun, L., Huang, W., Zhang, J., Zheng, Y., & Jeon, B. (2021). Tslrln: Tensor subspace low-rank learning with non-local prior for hyperspectral image mixed denoising. Signal Processing, 184, 108060.

    Article  Google Scholar 

  • He, K., Sun, J., & Tang, X. (2012). Guided image filtering. IEEE Trans Pattern Analysis and Machine Intelligence, 35(6), 1397–1409.

    Article  Google Scholar 

  • He, W., Zhang, H., Zhang, L., & Shen, H. (2015). Total-variation-regularized low-rank matrix factorization for hyperspectral image restoration. IEEE Trans Geoscience and Remote Sensing, 54(1), 178–188.

    Article  Google Scholar 

  • He, W., Yao, Q., Li, C., Yokoya, N., & Zhao, Q. (2019). Non-local meets global: An integrated paradigm for hyperspectral denoising. In: Proc. of Conference on Computer Vision and Pattern Recognition, pp. 6868–6877.

  • He, W., Yao, Q., Li, C., Yokoya, N., Zhao, Q., Zhang, H., & Zhang, L. (2020). Non-local meets global: An integrated paradigm for hyperspectral image restoration. IEEE Trans Pattern Analysis and Machine Intelligence Early Access.

  • Healey, G. E., & Kondepudy, R. (1994). Radiometric ccd camera calibration and noise estimation. IEEE Trans Pattern Analysis and Machine Intelligence, 16(3), 267–276.

    Article  Google Scholar 

  • Holst, G.C. (1998). Ccd arrays, cameras, and displays.

  • Hui, TW., Loy, CC., & Tang, X. (2016). Depth map super-resolution by deep multi-scale guidance. In: Proc. of European Conference on Computer Vision, pp. 353–369.

  • Jiang, H., & Zheng, Y. (2019) Learning to see moving objects in the dark. In: Proc. of International Conference on Computer Vision, pp. 7324–7333.

  • Kawakami, R., Zhao, H., Tan, R. T., & Ikeuchi, K. (2013). Camera spectral sensitivity and white balance estimation from sky images. International Journal of Computer Vision, 105(3), 187–204.

    Article  MATH  Google Scholar 

  • Kingma, D.P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

  • Kruse, F. A., Lefkoff, A., Boardman, J., Heidebrecht, K., Shapiro, A., Barloon, P., & Goetz, A. (1993). The spectral image processing system (sips) interactive visualization and analysis of imaging spectrometer data. Remote Sensing of Environment, 44(2–3), 145–163.

    Article  Google Scholar 

  • Kullback, S., & Leibler, R. A. (1951). On information and sufficiency. The Annals of Mathematical Statistics, 22(1), 79–86.

    Article  MathSciNet  MATH  Google Scholar 

  • Kwon, H., & Nasrabadi, N. M. (2007). Kernel spectral matched filter for hyperspectral imagery. International Journal of Computer Vision, 71(2), 127–141.

    Article  Google Scholar 

  • Li, Y., Huang, JB., Ahuja, N., & Yang, MH. (2016). Deep joint image filtering. In: Proc. of European Conference on Computer Vision, pp. 154–169.

  • Lin, B., Tao, X., & Lu, J. (2019). Hyperspectral image denoising via matrix factorization and deep prior regularization. IEEE Trans Image Processing, 29, 565–578.

    Article  MathSciNet  MATH  Google Scholar 

  • Liu, L., Jia, X., Liu, J., & Tian, Q. (2020). Joint demosaicing and denoising with self guidance. In: Proc. of Conference on Computer Vision and Pattern Recognition, pp. 2240–2249.

  • Liu, X., Bourennane, S., & Fossati, C. (2012). Denoising of hyperspectral images using the parafac model and statistical performance analysis. IEEE Trans Geoscience and Remote Sensing, 50(10), 3717–3724.

    Article  Google Scholar 

  • Long, J., Shelhamer, E., & Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In: Proc. of Conference on Computer Vision and Pattern Recognition, pp. 3431–3440.

  • Lu, G., & Fei, B. (2014). Medical hyperspectral imaging: A review. Journal of Biomedical Optics, 19(1), 010901.

    Article  Google Scholar 

  • Ma, C., Cao, X., Tong, X., Dai, Q., & Lin, S. (2014). Acquisition of high spatial and spectral resolution video with a hybrid camera system. International Journal of Computer Vision, 110(2), 141–155.

    Article  Google Scholar 

  • Maggioni, M., Katkovnik, V., Egiazarian, K., & Foi, A. (2012). Nonlocal transform-domain filter for volumetric data denoising and reconstruction. IEEE Trans Image Processing, 22(1), 119–133.

    Article  MathSciNet  MATH  Google Scholar 

  • Miao, Y. C., Zhao, X. L., Fu, X., Wang, J. L., & Zheng, Y. B. (2021). Hyperspectral denoising using unsupervised disentangled spatiospectral deep priors. IEEE Trans Geoscience and Remote Sensing, 60, 1–16.

    Google Scholar 

  • Monno, Y., Kiku, D., Tanaka, M., & Okutomi, M. (2015). Adaptive residual interpolation for color image demosaicking. In: Proc. of International Conference on Image Processing, pp. 3861–3865.

  • Morgan, E. C., Lackner, M., Vogel, R. M., & Baise, L. G. (2011). Probability distributions for offshore wind speeds. Energy Conversion and Management, 52(1), 15–26.

    Article  Google Scholar 

  • Nair, V., & Hinton, GE. (2010). Rectified linear units improve restricted boltzmann machines. In: Proc. of International Conference on Machine Learning.

  • Ojha, L., Wilhelm, M. B., Murchie, S. L., McEwen, A. S., Wray, J. J., Hanley, J., et al. (2015). Spectral evidence for hydrated salts in recurring slope lineae on Mars. Nature Geoscience, 8(11), 829–832.

    Article  Google Scholar 

  • Paszke, A., Gross, S,. Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L. et al. (2019). Pytorch: An imperative style, high-performance deep learning library. arXiv preprint arXiv:1912.01703

  • Peng, Y., Meng, D., Xu, Z., Gao, C., Yang, Y., & Zhang, B. (2014). Decomposable nonlocal tensor dictionary learning for multispectral image denoising. In: Proc. of Conference on Computer Vision and Pattern Recognition, pp. 2949–2956.

  • Plotz, T., & Roth, S. (2017). Benchmarking denoising algorithms with real photographs. In: Proc. of Conference on Computer Vision and Pattern Recognition, pp. 1586–1595.

  • Porter, WM., & Enmark, HT. (1987). A system overview of the airborne visible/infrared imaging spectrometer (aviris). In: Proc. of Annual Technical Symposium, pp. 22–31.

  • Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In: Proc. of International Conference on Medical image computing and computer-assisted intervention, pp. 234–241.

  • Schott, JR. (2007). Remote sensing: the image chain approach. Oxford University Press on Demand.

  • Shi, Q., Tang, X., Yang, T., Liu, R., & Zhang, L. (2021). Hyperspectral image denoising using a 3-d attention denoising network. IEEE Trans Geoscience and Remote Sensing, 59(12), 10348–10363.

    Article  Google Scholar 

  • Wald, L. (2000). Quality of high resolution synthesised images: Is there a simple criterion? In: Proc. of Conference on Fusion of Earth Data, pp. 99–103.

  • Wang, L., Zhang, S., & Huang, H. (2021). Adaptive dimension-discriminative low-rank tensor recovery for computational hyperspectral imaging. International Journal of Computer Vision, 129(10), 2907–2926.

    Article  MATH  Google Scholar 

  • Wang, X., Yu, K., Dong, C., & Loy, CC. (2018). Recovering realistic texture in image super-resolution by deep spatial feature transform. In: Proc. of Conference on Computer Vision and Pattern Recognition, pp. 606–615.

  • Wang, Y., Peng, J., Zhao, Q., Leung, Y., Zhao, X. L., & Meng, D. (2017). Hyperspectral image restoration via total variation regularized low-rank tensor decomposition. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(4), 1227–1243.

    Article  Google Scholar 

  • Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Trans Image Processing, 13(4), 600–612.

    Article  Google Scholar 

  • Wei, K., Fu, Y., & Huang, H. (2021). 3-d quasi-recurrent neural network for hyperspectral image denoising. IEEE Trans Neural Networks and Learning Systems, 32(1), 363–375.

    Article  Google Scholar 

  • Xie, Q., Zhao, Q., Meng, D., Xu, Z., Gu, S., Zuo, W., & Zhang, L. (2016a). Multispectral images denoising by intrinsic tensor sparsity regularization. In: Proc. of Conference on Computer Vision and Pattern Recognition, pp. 1692–1700.

  • Xie, Y., Qu, Y., Tao, D., Wu, W., Yuan, Q., & Zhang, W. (2016). Hyperspectral image restoration via iteratively regularized weighted schatten \( p \)-norm minimization. IEEE Trans Geoscience and Remote Sensing, 54(8), 4642–4659.

    Article  Google Scholar 

  • Xiong, F., Zhou, J., Zhao, Q., Lu, J., & Qian, Y. (2021). Mac-net: Model aided nonlocal neural network for hyperspectral image denoising. IEEE Trans Geoscience and Remote Sensing.

  • Xu, L., Ren, J., Yan, Q., Liao, R., & Jia, J. (2015). Deep edge-aware filters. In: Proc. of International Conference on Machine Learning, pp. 1669–1678.

  • Yasuma, F., Mitsunaga, T., Iso, D., & Nayar, S. K. (2010). Generalized assorted pixel camera: Postcapture control of resolution, dynamic range and spectrum. IEEE Trans Image Processing, 19(9), 2241–2253.

    Article  MathSciNet  MATH  Google Scholar 

  • Yuan, Q., Zhang, L., & Shen, H. (2012). Hyperspectral image denoising employing a spectral-spatial adaptive total variation model. IEEE Trans Geoscience and Remote Sensing, 50(10), 3660–3677.

    Article  Google Scholar 

  • Yuan, Q., Zhang, Q., Li, J., Shen, H., & Zhang, L. (2018). Hyperspectral image denoising employing a spatial-spectral deep residual convolutional neural network. IEEE Trans Geoscience and Remote Sensing, 57(2), 1205–1218.

    Article  Google Scholar 

  • Zhang, H., He, W., Zhang, L., Shen, H., & Yuan, Q. (2013). Hyperspectral image restoration using low-rank matrix recovery. IEEE Trans Geoscience and Remote Sensing, 52(8), 4729–4743.

    Article  Google Scholar 

  • Zhang, L., Wei, W., Zhang, Y., Shen, C., Avd, Hengel, & Shi, Q. (2018). Cluster sparsity field: An internal hyperspectral imagery prior for reconstruction. International Journal of Computer Vision, 126(8), 797–821.

    Article  MATH  Google Scholar 

  • Zhang, T., Fu, Y., & Li, C. (2021). Hyperspectral image denoising with realistic data. In: Proc. of International Conference on Computer Vision, pp. 2248–2257.

  • Zhao, B., Ulfarsson, M. O., Sveinsson, J. R., & Chanussot, J. (2022). Hyperspectral image denoising using spectral-spatial transform-based sparse and low-rank representations. IEEE Trans Geoscience and Remote Sensing, 60, 1–25.

    Google Scholar 

  • Zheng, H., Ji, M., Wang, H., Liu, Y., & Fang, L. (2018). Crossnet: An end-to-end reference-based super resolution network using cross-scale warping. In: Proc. of European Conference on Computer Vision, pp. 88–104.

  • Zhou, Y., Wu, G., Fu, Y., Li, K., & Liu, Y. (2021). Cross-mpi: Cross-scale stereo for image super-resolution using multiplane images. In: Proc. of Conference on Computer Vision and Pattern Recognition, pp. 14842–14851.

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grants No. 62171038, No. 61827901 and No. 62088101.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Beijing Institute of Technology, Beijing, 100081, China

    Tao Zhang & Ying Fu

  2. Advanced Research Institute of Multidisciplinary Science, Beijing Institute of Technology, Beijing, 100081, China

    Ying Fu & Jun Zhang

Authors
  1. Tao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ying Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ying Fu.

Additional information

Communicated by Boxin Shi.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, T., Fu, Y. & Zhang, J. Guided Hyperspectral Image Denoising with Realistic Data. Int J Comput Vis 130, 2885–2901 (2022). https://doi.org/10.1007/s11263-022-01660-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-022-01660-2

Keywords

Search

Navigation