Skip to main content
SpringerLink
Log in

Comprehensive review on fusion techniques for spatial information enhancement in hyperspectral imagery

  • Published:
Multidimensional Systems and Signal Processing Aims and scope Submit manuscript

Abstract

The volume of data grows with the advent of multiple types of remote sensing sensors and in order to extract the most useful information there is a need to combine the data gathered from the different sources. The widely used panchromatic and multispectral imageries in many applications offer decimetric and metric spatial resolution. However, the spectral resolution of these images is poor. Hyperspectral imaging has unique characteristics of providing very fine spectral resolution in a large number of bands with decametric spatial resolution and found to be highly useful for a wide span of application areas that requires high spectral resolution. The fusion of spectral and spatial information provides an effective way of enhancing the spatial quality of hyperspectral imagery as well as a method for preserving spectral quality. This fusion process is not a trivial task as always there has been a tradeoff between the preservation of spatial and spectral quality information as in the original sources of fusion. In this paper, a review on hyperspectral pansharpening and hyperspectral multispectral fusion based approaches has been reported. The widely adopted quantitative and qualitative performance measures to verify the fusion results are highlighted. In addition, the challenges in existing fusion techniques have also been discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. AVIRIS data is available from http://aviris.jpl.nasa.gov.

  2. Hyperion data is available from http://eo1.usgs.gov.

  3. Pavia dataset is available from http://ehu.eus/ccwinto/index.php?title=hyperspectral-remote-sensing-scenes.

  4. LANDAST data is available from http://glovis.usgs.gov.

References

  • Aiazzi, B., Alparone, L., Baronti, S., & Garzelli, A. (2002). Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis. IEEE Transactions on Geoscience and Remote Sensing, 40, 2300–2312.

    Article  Google Scholar 

  • Aiazzi, B., Baronti, S., & Selva, M. (2007). Improving component substitution pansharpening through multivariate regression of MS+Pan data. IEEE Transactions on Geoscience and Remote Sensing, 45(10), 3230–3239.

    Article  Google Scholar 

  • Alparone, L., Aiazzi, B., Baronti, S. & Garzelli, A. (2003). Sharpening of very high resolution images with spectral distortion minimization. In Proceedings of IEEE international conference geoscience and remote sensing (IGARSS), pp. 458–460.

  • Amro, I., Mateos, J., Vega, M., Molina, R., & Katsaggelos, A. K. (2011). A survey of classical methods and new trends in pansharpening of multispectral images. EURASIP Journal on Advances in Signal Processing, 79, 1–22.

  • Bieniarz, J., Cerra, D., Avbelj, J., Reinartz, P., & Miller, R. (2011). Hyperspectral image resolution enhancement based on spectral unmixing and information fusion. In ISPRS Hannover Workshop 2011: High-resolution earth imaging for geospatial information.

  • Chaudhuri, S., & Kotwal, K. (2013). Hyperspectral image fusion. New York: Springer.

    Book  Google Scholar 

  • Chembe, C., Engels, J., Hahn, M., & Gülch, E. (2012). Pansharpening of hyperspectral images in urban areas. In Proceedings on international archives of the photogrammetry, remote sensing and spatial information sciences, XXII ISPRS Congress, XXXIX-B7, pp. 387–392.

  • Cheng, G., Han, J., Guo, L., Liu, Z., Shuhui, B., & Ren, J. (2015). Effective and efficient midlevel visual elements-oriented land-use classification using VHR remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 53(8), 4238–4249.

    Article  Google Scholar 

  • Cheng, G., Han, J., Guo, L., Qian, X., Zhou, P., Yao, X., et al. (2013). Object detection in remote sensing imagery using a discriminatively trained mixture model. ISPRS Journal of Photogrammetry and Remote Sensing, 85, 32–43.

    Article  Google Scholar 

  • Cheng, G., Han, J., Zhou, P., & Guo, L. (2014). Multi-class geospatial object detection and geographic image classification based on collection of part detectors. ISPRS Journal of Photogrammetry and Remote Sensing, 98, 119–132.

    Article  Google Scholar 

  • Choi, M. (2006). A new intensity hue saturation fusion approach to image fusion with a tradeoff parameter. IEEE Transactions on Geoscience and Remote Sensing, 44(6), 1672–1682.

    Article  Google Scholar 

  • Choi, Y., Sharifahmadian, E., & Latifi, S. (2013). Performance analysis of contourlet based hyperspectral image fusion methods. International Journal on Information Theory, 2(1–2), 1–14.

    Article  Google Scholar 

  • Do, M. N., & Vetterli, M. (2005). The Contourlet transform: An efficient directional multiresolution image representation. IEEE Transactions on Image Processing, 14(12), 2091–2106.

    Article  MathSciNet  Google Scholar 

  • Du, Q., Younan, N. H., King, R., & Shah, V. P. (2007). On the performance evaluation of pan-sharpening techniques. IEEE Geoscience and Remote Sensing Letters, 4(4), 518–522.

    Article  Google Scholar 

  • Eismann, M. T., & Hardie, R. C. (2005). Hyperspectral resolution enhancement using high-resolution multispectral imagery with arbitrary response functions. IEEE Transactions on Geoscience and Remote Sensing, 43(3), 455–465.

    Article  Google Scholar 

  • Fauvel, M., Benediktsson, J. A., Chanussot, J., & Sveinsson, J. R. (2008). Spectral and spatial classification of hyperspectral data using SVMs and morphological profiles. IEEE Transactions on Geoscience and Remote Sensing, 46(11), 3804–3814.

    Article  Google Scholar 

  • Fasbneder, D., Radoux, J., & Bogaert, P. (2008). Bayesian data fusion for adaptable image pansharpening. IEEE Transactions on Geoscience and Remote Sensing, 46(6), 1847–1857.

    Article  Google Scholar 

  • Fonseca, L., Namikawal, L., Castejon, E., Carvalho, L., Pinho, C., & Pagamisse, A. (2011). Image fusion for remote sensing applications. In Y. Zheng (Ed.), Image fusion and its applications. InTech. http://www.intechopen.com/books/image-fusion-and-its-applications/image-fusion-for-remote-sensing-applications.

  • Garzell, A., Aiazzi, B., Barronti, S., Selva, M., & Alparone, L. (2010). Hyperspectral image fusion. In Proceedings of the hyperspectral 2010 workshop, pp. 17–19.

  • Ghamisi, P., Mura, M. D., & Benediktsson, J. A. (2015). A survey on spectral spatial classification techniques based on attribute profiles. IEEE Transactions on Geoscience and Remote Sensing, 53(5), 2335–2353.

    Article  Google Scholar 

  • González-Audícana, M., Saleta, J. L., Catalán, R. G., & García, R. (2004). Fusion of multispectral and panchromatic images using improved IHS and PCA mergers based on wavelet decomposition. IEEE Transactions on Geoscience and Remote Sensing, 42(6), 1291–1299.

    Article  Google Scholar 

  • Gu, Y., Zhang, Y., & Zhang, J. (2008). Integration of spatial-spectral information for resolution enhancement in hyperspectral images. IEEE Transactions on Geoscience and Remote Sensing, 46(5), 1347–1358.

    Article  MathSciNet  Google Scholar 

  • Grohnfeldt, C., Burns, T. M., & Zhu, X. X. (2015). Dynamic dictionary learning strategies for sparse representation based hyperspectral image enhancement. http://www.researchgate.net/publication/277279536.

  • Han, S. S., Li, H. T., & Gu, H. Y. (2008). The study on image fusion for high spatial resolution remote sensing images. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 37, 1159–1163.

    Google Scholar 

  • Hardie, R. C., Eismann, M. T., & Wilson, G. L. (2004). MAP estimation for hyperspectral image resolution enhancement using an auxiliary sensor. IEEE Transactions on Image Processing, 13(9), 1174–1184.

    Article  Google Scholar 

  • Hoyer, P. O. (2002). Non negative sparse coding. In Proceedings of IEEE workshop neural network signal processing, pp. 557–565.

  • Hsu, S. M., & Burke, H-h K. (2003). Multisensor fusion with hyperspectral imaging data: Detection and classification. Lincoln Laboratory Journal, 14(1), 145–159.

    Google Scholar 

  • Idrees, M. O., Saiedi, V., Pradhan, I., & Shafri, H. Z. M. (2015). Maximising urban features extraction from multi-sensor data with Dempster–Shafer theory and HSI data fusion techniques. Asian Journal of Applied Sciences, 3(2), 218–228.

    Google Scholar 

  • Kaplan, N. H., Erer, I., & Elibol, F. (2012). Fusion of multispectral and panchromatic images by combining bilateral filter and HIS transform. In 20th European signal processing conference, pp. 2501–2505.

  • Khan, M. M., Chanussot, J., Condat, L., & Montanvert, A. (2008). Indusion: Fusion of multispectral and panchromatic images using the induction scaling technique. IEEE Geoscience and Remote Sensing Letters, 5(1), 98–102.

    Article  Google Scholar 

  • Khandelwal, A., & Rajan, K. S. (2011). Hyperspectral image enhancement based on sensor simulation and vector decomposition. In 2011 proceedings of the 14th international conference on information fusion (FUSION), pp. 1–6.

  • Kotwal, K., & Chaudhuri, S. (2010). A fast approach for fusion of hyperspectral images through redundancy elimination. In Proceedings of the seventh Indian conference on computer vision, graphics and Image processing, pp. 506–511. doi:10.1145/1924559.1924627.

  • Laben, C., & Brower, B. (2000). Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening. U.S. Patent No. 6,011,875. Washington, DC: U.S. Patent and Trademark Office.

  • Lanaras, L., Baltsavias, E., & Schindler, K. (2015a). Advances in hyperspectral and multispectral image fusion and spectral unmixing. In The international archives of photogrammetry, remote sensing and spatial information sciences, Vol. XL-3, pp. 451–458. doi:10.5194/isprsarchives-XL-3-W3-451-2015.

  • Lanaras, C., Baltsavias, E., & Schindler, K. (2015b). Hyperspectral image fusion. In 36th Asian conference on remote sensing (ACRS), Philippines.

  • Lee, D. D., & Seung, H. S. (1999). Learning the parts of objects by non negative matrix factorization. Nature, 401, 788–791.

    Article  Google Scholar 

  • Liao, W., Pizurica, A., Bellens, R., Gautama, S., & Philips, W. (2015). Generalized graph-based fusion of hyperspectral and LiDAR data using morphological features. IEEE Geoscience and Remote Sensing Letters, 12(3), 552–556.

    Article  Google Scholar 

  • Licciardi, G., Khan, M. M., Chanussot, J., Montanvert, A., Condat, L., & Jutten, C. (2012). Fusion of hyperspectral and panchromatic images using multiresolution analysis and nonlinear PCA band reduction. EURASIP Journal on Advances in Signal processing, 1, 1–17.

    Google Scholar 

  • Licciardi, G., Veganzones, M. A., Vivone, G., Loncan, L., & Chanussot, J. (2015). Impact of hybrid pansharpening approaches applied to hyperspectral images. https://hal.archives-ouverts.fr/hal-0121191.

  • Li, H., Manjunath, B. S., & Mitra, S. K. (1994). Multi-sensor image fusion using the wavelet transforms. IEEE Transactions in Signal Processing, 2012(207), 51–55.

    Google Scholar 

  • Lin, H., Zhang, A., & Yang, S. (2015). Comparison of several hyperspectral image fusion methods for visualization (pp. 646–651). In International conference on advances in mechanical engineering and industrial informatics: Atlantis Press.

    Google Scholar 

  • Liu, J. G. (2000). Smoothing filter based intensity modulation: A spectral preserve image fusion technique for improving spatial details. International Journal of Remote Sensing, 21(18), 3461–3472.

    Article  Google Scholar 

  • Loncan, L., Almeida, L. B., Bioucas-Dias, J. M., Briottet, X., Chanussot, J., Dobigeon, N., Fabre, S., et al. (2015). Hyperspectral pansharpening: A review. arXiv preprint arXiv:1504.04531.

  • Matsuoka, M. (2012). The influence of spectral wavelength on the quality of pansharpened image simulated using hyperspectral data. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, 1, 297–302.

    Article  Google Scholar 

  • Mianji, F. A., Zhang, Y., & Babakhani, A. (2009a). Optimum method selection for resolution enhancement of hyperspectral imagery. Information Technology Journal, 8(3), 263–274.

    Article  Google Scholar 

  • Mianji, F. A., Zhang, Y., & Babakhani, A. (2010). Key information retrieval in hyperspectral imagery through spatial-spectral data fusion. Radio Engineering, 19(4), 734–744.

    Google Scholar 

  • Mianji, F. A., Zhang, Y., Gu, Y., & Babakhani, A. (2009b). Spatial-spectral data fusion for resolution enhancement of hyperspectral imagery. IEEE IGRASS, III, 1011–1014.

    Google Scholar 

  • Mianji, F. A., Zhang, Y., & Sulehria, H. K. (2008). Super resolution challenges in hyperspectral Imagery. Information Technology Journal, 7(7), 1030–1036.

    Article  Google Scholar 

  • Moeller, M., Wittman, T., & Bertozzi, A. L. (2009). A variational approach to hyperspectral Image fusion. In SPIE defense, security, and sensing, International Society for Optics and Photonics, pp. 73341E–73341E.

  • Naidu, V. P. S., & Raol, J. R. (2008). Pixel-level image fusion using wavelets and principal component analysis. Defense Science Journal, 58(3), 338–352.

    Article  Google Scholar 

  • Nguyen, M. Q., Atkinson, P. M., & Lewis, H. G. (2006). Superresolution mapping using a Hopfield neural network with fused images. IEEE Transactions on Geoscience and Remote Sensing, 44(3), 736–749.

    Article  Google Scholar 

  • Pande, H., & Tiwari, P. S. (2013). High-resolution and hyperspectral data fusion for classification. In Q. Miao (Ed.), New advances in image fusion. InTech. http://www.intechopen.com/books/new-advances-in-image-fusion/high-resolution-and-hyperspectral-data-fusion-for-classification.

  • Rahmani, S., Strait, M., Merkurjev, D., Moeller, M., & Wittman, T. (2010). An adaptive IHS pan-sharpening method. IEEE Geoscience and Remote Sensing Letters, 7(4), 746–750.

    Article  Google Scholar 

  • Ranchin, T., & Wald, L. (2000). Fusion of high spatial and spectral resolution images: The ARSIS concept and its implementation. Photogrammetric Engineering and Remote Sensing, 66(1), 49–61.

    Google Scholar 

  • Sadjadi, F. (2005). Comparative image fusion analysis. In IEEE Computer Society conference on computer vision and pattern recognition-workshops (CVPR), p. 8, IEEE.

  • Villa, A., Chanussot, J., Benediktsson, J. A., & Jutten, C. (2011). Spectral unmixing for the classification of hyperspectral images at a finer spatial resolution. IEEE Journal of Selected Topics in Signal Processing, 5(3), 521–533.

    Article  Google Scholar 

  • Vivone, G., Alpharone, L., Chanussot, J., Mura, M. D., Garzelli, A., Giorgio, A., et al. (2015). A critical comparison among pansharpening algorithms. IEEE Transactions on Geoscience and Remote Sensing, 53(5), 2565–2586.

    Article  Google Scholar 

  • Vivone, G., Restaino, R., Licciardi, G., Dalla Mura, M., & Chanussot, J. (2014). Multiresolution analysis and component substitution techniques for hyperspectral pansharpening. In Geoscience and remote sensing symposium (IGARSS), 2014 IEEE international, pp. 2649–2652. IEEE.

  • Wang, Q., Shen, Y., Zhang, Y., & Zhang, J. Q. (2003). A quantitative method for evaluating the performances of hyperspectral image fusion. IEEE Transactions on Instrumentation and Measurement, 52(4), 1041–1047.

    Article  Google Scholar 

  • Wei, Q., Bioucas Dias, J. M., Dobigeon, N., & Tourneret, J. Y. (2015a). Hyperspectral and multispectral image fusion based on a sparse representation. IEEE Transactions on Geoscience and Remote Sensing, 53(7), 3658–3668.

    Article  Google Scholar 

  • Wei, Q., Dobigeon, N., & Tourneret, J.-Y. (2014). Bayesian fusion of hyperspectral and multispectral images. In Proceedings of IEEE international conference on acoustics, speech, and signal processing (ICASSP), pp. 3176–3180.

  • Wei, Q., Dobigeon, N., & Tourneret, J.-Y. (2015b). Bayesian fusion of multispectral and hyperspectral images using a block coordinate descent method. In Proceedings of IEEE GRSS workshop hyperspectral image signal process evolution in remote sensing (WHISPERS), Tokyo, Japan.

  • Yokoya, N., Yairi, T., & Iwasaki, A. (2011). Hyperspectral, multispectral, and panchromatic data fusion based on coupled non-negative matrix factorization. In 2011 3rd workshop on hyperspectral image and signal processing: Evolution in remote sensing (WHISPERS), pp. 1–4. IEEE.

  • Zhang, Y. (2008). Understanding image fusion. Photogrammetric Engineering and Remote Sensing, 70(6), 657–661.

    Google Scholar 

  • Zhang, Z., & Shi, Z. (2013). Nonnegative matrix factorization-based hyperspectral and panchromatic image fusion. Neural Computing and Applications, 23(3–4), 895–905.

    Article  Google Scholar 

  • Zhang, H., Zhang, L., & Shen, H. (2012). A super-resolution reconstruction algorithm for hyperspectral images. Signal Processing, 92, 2082–2096.

    Article  Google Scholar 

  • Zhao, Y.-Q., Zhang, L., & Kong, S. G. (2010). Band subset based clustering and fusion for hyperspectral imagery classification. IEEE Transactions on Geoscience and Remote Sensing, 49(2), 747–756.

    Article  Google Scholar 

  • Zheng, Y. (2011). Image fusion and its applications. Rijeka: Intech.

    Book  Google Scholar 

  • Zhukov, O., Oertel, D., Lanzl, F., & Reinhackel, G. (1999). Unmixing based multisensory multiresolution image fusion. IEEE Transactions on Geoscience and Remote Sensing, 37(3), 1212–1226.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of ECE, University College of Engineering, Thoothukudi, India

    A. Mookambiga

  2. Department of Computer Science and Engineering, National Engineering College (an Autonomous Institution), Kovilpatti, India

    V. Gomathi

Authors
  1. A. Mookambiga
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. V. Gomathi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. Mookambiga.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mookambiga, A., Gomathi, V. Comprehensive review on fusion techniques for spatial information enhancement in hyperspectral imagery. Multidim Syst Sign Process 27, 863–889 (2016). https://doi.org/10.1007/s11045-016-0415-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11045-016-0415-2

Keywords

Search

Navigation