Abstract
The availability of multi-sensor and multi-resolution images obtained through earth observation satellites has given rise to a greater need for fusion technology. Fusion can provide comprehensive information about a particular scene or area by bringing the information of two or more images into a single plane. The technique has been in use for the past three decades, and several methodologies have been introduced in the literature. However, with the invention of new sensors and technology, the number of challenges has increased. While several review and survey papers have explored different aspects of fusion, this paper provides a comprehensive discussion of the tools available and the implications of their use for the fusion of remote sensing images. Tools based on multi-resolution analysis (MRA) and multi-geometric analysis (MGA) are widely used in the field of image fusion. We provide a detailed study of MRA- and MGA-based tools, their effectiveness, and the impacts of the corresponding fusion schema in retaining the desired information.
Similar content being viewed by others
References
B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis. IEEE Trans. Geosc. Remote Sens. 40(10), 2300–2312 (2002)
K. Amolins, Y. Zhang, P. Dare, Wavelet based image fusion techniques – an introduction, review and comparison. ISPRS J. Photogramm. Remote Sens. 62(4), 249–263 (2007)
C. Beeri, Y. Kanza, E. Safra, Y. Sagiv, Object fusion in geographic information systems, in Proceedings of the Thirtieth International Conference on Very Large Data Bases, vol. 30 (VLDB Endowment, 2004), pp. 816–827
U. Benz, E. Pottier, Object based analysis of polarimetric SAR data in alpha-entropy-anisotropy decomposition using fuzzy classification by ecognition, in IEEE 2001 International Geoscience and Remote Sensing Symposium, 2001. IGARSS’01, vol. 3 (IEEE, 2001), pp. 1427–1429
T. Blaschke, Object based image analysis for remote sensing. ISPRS J. Photogramm. Remote Sens. 65(1), 2–16 (2010)
T. Blaschke, S. Lang, G. Hay, Object-based Image Analysis: Spatial Concepts for Knowledge-driven Remote Sensing Applications (Springer, Berlin, 2008)
I. Bloch, Information combination operators for data fusion: a comparative review with classification. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 26(1), 52–67 (1996)
R.S. Blum, Z. Liu, Multi-sensor Image Fusion and Its Applications (CRC Press, Boca Raton, 2005)
B. Bouchon-Meunier, Aggregation and Fusion of Imperfect Information, vol. 12 (Physica-Verlag Heidelberg, 1998)
T.F. Chan, J. Shen, Image Processing and Analysis: Variational, PDE, Wavelet, and Stochastic Methods (SIAM, Philadelphia, 2005)
X. Chang, L. Jiao, F. Liu, F. Xin, Multicontourlet-based adaptive fusion of infrared and visible remote sensing images. IEEE Geosci. Remote Sens. Lett. 7(3), 549–553 (2010)
C.H. Chen, Information Processing for Remote Sensing (World Scientific, Singapore, 1999)
S. Cheng, M. Qiguang, X. Pengfei, A novel algorithm of remote sensing image fusion based on shearlets and PCNN. Neurocomputing 117, 47–53 (2013)
L.J. Chipman, T.M. Orr, L.N. Graham, Wavelets and image fusion, in International Conference on Image Processing, 1995. Proceedings, vol. 3 (IEEE, 1995), pp. 248–251
H. Chu, D.G. Teng, M.G. Wang, Fusion of remotely sensed images based on subsampled contourlet transform and spectral response, in Urban Remote Sensing Event, 2009 Joint (IEEE, 2009), pp. 1–5
C.K. Chui, Wavelets: a tutorial in theory and applications, in Wavelet Analysis and its Applications, ed. by C.K. Chui, vol. 1 (Academic Press, San Diego, CA, 1992)
S. Constantinos, M.S. Pattichis, E. Micheli-Tzanakou, Medical imaging fusion applications: an overview, in Conference Record of the Thirty-Fifth Asilomar Conference on Signals, Systems and Computers, 2001, vol. 2 (IEEE, 2001), pp. 1263–1267
A.L. Da Cunha, J. Zhou, M.N. Do, The nonsubsampled contourlet transform: theory, design, and applications. IEEE Trans. Image Process. 15(10), 3089–3101 (2006)
I. Daubechies, Orthonormal bases of compactly supported wavelets. Commun. Pure Appl. Math. 41(7), 909–996 (1988)
C. Deng, S. Wang, X. Chen, Remote sensing images fusion algorithm based on shearlet transform, in International Conference on Environmental Science and Information Application Technology, 2009. ESIAT 2009, vol. 3 (IEEE, 2009), pp. 451–454
M.N. Do, M. Vetterli, The contourlet transform: an efficient directional multiresolution image representation. IEEE Trans. Image Process. 14(12), 2091–2106 (2005)
J. Dong, D. Zhuang, Y. Huang, J. Fu, Advances in multi-sensor data fusion: algorithms and applications. Sensors 9(10), 7771–7784 (2009)
J.P. Donnay, M.J. Barnsley, P.A. Longley, Remote Sensing and Urban Analysis: GISDATA 9 (CRC Press, Boca Raton, 2003)
D.L. Donoho, M.R. Duncan, Digital curvelet transform: strategy, implementation, and experiments, in AeroSense 2000 (International Society for Optics and Photonics, 2000), pp. 12–30
C. Duan, Q.H. Huang, X.G. Wang, S. Wang, Remote image fusion based on PCA and dual tree compactly supported shearlet transform. J. Inf. Hiding Multimed Signal Process. 5(3), 485–496 (2014)
M. Ehlers, Multisensor image fusion techniques in remote sensing. ISPRS J. Photogramm. Remote Sens. 46(1), 19–30 (1991)
M.C. El-Mezouar, K. Kpalma, N. Taleb, J. Ronsin, A pan-sharpening based on the non-subsampled contourlet transform: application to worldview-2 imagery. IEEE J. Select. Top. Appl. Earth Obs. Remote Sens. 7(5), 1806–1815 (2014)
J. Esteban, A. Starr, R. Willetts, P. Hannah, P. Bryanston-Cross, A review of data fusion models and architectures: towards engineering guidelines. Neural Comput. Appl. 14(4), 273–281 (2005)
A. Fanelli, A. Leo, M. Ferri, Remote sensing images data fusion: A wavelet transform approach for urban analysis, in Remote Sensing and Data Fusion over Urban Areas, IEEE/ISPRS Joint Workshop 2001 (IEEE, 2001), pp. 112–116
F. Fang, F. Li, C. Shen, G. Zhang, A variational approach for pan-sharpening. IEEE Trans. Image Process. 22(7), 2822–2834 (2013)
A. Garg, S. Agarwal, T.S. Huang, Fusion of global and local information for object detection, in 16th International Conference on Pattern Recognition, 2002. Proceedings, vol. 3 (IEEE, 2002), pp. 723–726
A. Garzelli, F. Nencini, Interband structure modeling for pan-sharpening of very high-resolution multispectral images. Inf. Fusion 6(3), 213–224 (2005)
M. González-Audícana, X. Otazu, O. Fors, A. Seco, Comparison between Mallat’s and the à trous discrete wavelet transform based algorithms for the fusion of multispectral and panchromatic images. Int. J. Remote Sens. 26(3), 595–614 (2005)
K. Grochenig, W.R. Madych, Multiresolution analysis. haar bases, and self-similar tilings of r/sup n. IEEE Trans. Inf. Theory 38(2), 556–568 (1992)
X. Gros, NDT Data Fusion (Elsevier, Amsterdam, 1996)
D.L. Hall, J. Llinas, Multisensor Data Fusion (CRC Press, Boca Raton, 2001)
S. Ibrahim, M. Wirth, Visible and ir data fusion technique using the contourlet transform, in International Conference on Computational Science and Engineering, 2009. CSE’09, vol. 2 (IEEE, 2009), pp. 42–47
S. Ioannidou, V. Karathanassi, Investigation of the dual-tree complex and shift-invariant discrete wavelet transforms on QuickBird image fusion. IEEE Geosci. Remote Sens. Lett. 4(1), 166–170 (2007)
J.R. Jensen, Introductory digital image processing: a remote sensing perspective. Technical reports, University of South Carolina, Columbus (1986)
D. Jiang, D. Zhuang, J. Fu, Y. Huang, Survey of Multispectral Image Fusion Techniques in Remote Sensing Applications (INTECH Open Access Publisher, London, 2011)
B. Khaleghi, A. Khamis, F.O. Karray, S.N. Razavi, Multisensor data fusion: a review of the state-of-the-art. Inf. Fusion 14(1), 28–44 (2013)
Y. Kim, C. Lee, D. Han, Y. Kim, Y. Kim, Improved additive-wavelet image fusion. IEEE Geosci. Remote Sens. Lett. 8(2), 263–267 (2011)
R.L. King, J. Wang, A wavelet based algorithm for pan sharpening Landsat 7 imagery, in IEEE 2001 International Geoscience and Remote Sensing Symposium, 2001. IGARSS’01, vol. 2 (IEEE, 2001), pp. 849–851
L. Kun, G. Lei, L. Huihui, C. Jingsong, Fusion of infrared and visible light images based on region segmentation. Chin. J. Aeronaut. 22(1), 75–80 (2009)
D. Li, Remote sensing image fusion based on nonsubsampled contourlet transform and pca, in International Conference on Computer Technology and Development, 2009. ICCTD’09, vol. 1 (IEEE, 2009), pp. 165–168
H. Li, B. Manjunath, S.K. Mitra, Multi-sensor image fusion using the wavelet transform, in IEEE International Conference on Image Processing, 1994. Proceedings. ICIP-94, vol. 1 (IEEE, 1994), pp. 51–55
M. Liggins II, D. Hall, J. Llinas, Handbook of Multisensor Data Fusion: Theory and Practice (CRC Press, Boca Raton, 2017)
W.Q. Lim, The discrete shearlet transform: a new directional transform and compactly supported shearlet frames. IEEE Trans. Image Process. 19(5), 1166–1180 (2010)
J. Llinas, D.L. Hall, An introduction to multi-sensor data fusion, in Proceedings of the 1998 IEEE International Symposium on Circuits and Systems, 1998. ISCAS’98, vol. 6 (IEEE, 1998), pp. 537–540
L. Loncan, L.B. de Almeida, J.M. Bioucas-Dias, X. Briottet, J. Chanussot, N. Dobigeon, S. Fabre, W. Liao, G.A. Licciardi, M. Simoes et al., Hyperspectral pansharpening: a review. IEEE Geosci. Remote Sens. Mag. 3(3), 27–46 (2015)
H. Lu, X. Hu, L. Zhang, S. Yang, S. Serikawa, Local energy based image fusion in sharp frequency localized contourlet transform. J. Comput. Inf. Syst. 6(12), 3997–4005 (2010)
R.C. Luo, C.C. Yih, K.L. Su, Multisensor fusion and integration: approaches, applications, and future research directions. IEEE Sens. J. 2(2), 107–119 (2002)
A.G. Mahyari, M. Yazdi, Panchromatic and multispectral image fusion based on maximization of both spectral and spatial similarities. IEEE Trans. Geosci. Remote Sens. 49(6), 1976–1985 (2011)
H. Maitre, I. Bloch, Image fusion. Vistas Astron. 41(3), 329–335 (1997)
S.G. Mallat, A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans. Pattern Anal. Mach. Intell. 11(7), 674–693 (1989)
P. Mangalraj, A. Agrawal, Fusion of multi-sensor satellite images using non-subsampled contourlet transform. Proc. Comput. Sci. 54, 713–720 (2015)
Q. Miao, B. Wang, The contourlet transform for image fusion, in Defense and Security Symposium (International Society for Optics and Photonics, 2006), pp. 62,420Z–62,420Z
Qg Miao, C. Shi, Pf Xu, M. Yang, Yb Shi, A novel algorithm of image fusion using shearlets. Opt. Commun. 284(6), 1540–1547 (2011)
H.B. Mitchell, Image Fusion: Theories, Techniques and Applications (Springer, Berlin, 2010)
N. Mitianoudis, T. Stathaki, Pixel-based and region-based image fusion schemes using ica bases. Inf. Fusion 8(2), 131–142 (2007)
A. Mitiche, J. Aggarwal, Multiple sensor integration/fusion through image processing: a review. Opt. Eng. 25(3), 253,380–253,380 (1986)
S. Mukhopadhyay, B. Chanda, Fusion of 2d grayscale images using multiscale morphology. Pattern Recognit. 34(10), 1939–1949 (2001)
S. Nikolov, P. Hill, D. Bull, N. Canagarajah, Wavelets for image fusion, in Wavelets in Signal and Image Analysis, ed. by A.A. Petrosian, F.G. Meyer (Springer, Dordrecht, 2001), pp. 213–241
J. Nunez, X. Otazu, O. Fors, A. Prades, V. Pala, R. Arbiol, Multiresolution-based image fusion with additive wavelet decomposition. IEEE Trans. Geosci. Remote Sens. 37(3), 1204–1211 (1999)
G. Pajares, J.M. De La Cruz, A wavelet-based image fusion tutorial. Pattern Recognit. 37(9), 1855–1872 (2004)
J.H. Park, K.O. Kim, Y.K. Yang, Image fusion using multiresolution analysis, in IEEE 2001 International Geoscience and Remote Sensing Symposium, 2001. IGARSS’01, vol. 2 (IEEE, 2001), pp. 864–866
D. Picone, R. Restaino, G. Vivone, P. Addesso, J. Chanussot, Pansharpening of hyperspectral images: exploiting data acquired by multiple platforms, in 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS) (IEEE, 2016), pp. 7220–7223
G. Piella, A general framework for multiresolution image fusion: from pixels to regions. Inf. Fusion 4(4), 259–280 (2003)
C. Polh, J. Van Genderen, Multisensor image fusion in remote sensing: concepts, methods and applications. Int. J. Remote Sens. 19(5), 823–854 (1998)
M. Poobalasubramanian, A. Agrawal, Fusion of pan and multispectral remote sensing images in shearlet domain by considering regional metrics. J. Appl. Remote Sens. 10(4), 045,003–045,003 (2016)
M. Qiguang, W. Baoshu, A novel image fusion method using contourlet transform, in 2006 International Conference on Communications, Circuits and Systems Proceedings, vol. 1 (IEEE, 2006), pp. 548–552
T. Ranchin, B. Aiazzi, L. Alparone, S. Baronti, L. Wald, Image fusion the ARSIS concept and some successful implementation schemes. ISPRS J. Photogramm. Remote Sens. 58(1), 4–18 (2003)
T. Ranchin, L. Wald, Fusion of high spatial and spectral resolution images: the ARSIS concept and its implementation. Photogramm. Eng. Remote Sens. 66(1), 49–61 (2000)
J. Salerno, Information fusion: a high-level architecture overview, in Proceedings of the Fifth International Conference on Information Fusion, 2002, vol. 1 (IEEE 2002), pp. 680–686
R.A. Schowengerdt, Remote Sensing: Models and Methods for Image Processing (Academic press, London, 2006)
A.K. Shackelford, C.H. Davis, A combined fuzzy pixel-based and object-based approach for classification of high-resolution multispectral data over urban areas. IEEE Trans. GeoSci. Remote Sens. 41(10), 2354–2363 (2003)
V.P. Shah, N.H. Younan, R.L. King, An efficient pan-sharpening method via a combined adaptive pca approach and contourlets. IEEE Trans. GeoSci. Remote Sens. 46(5), 1323–1335 (2008)
C. Shi, F. Liu, Q. Miao, Pan-sharpening via regional division and nsst. Multimed. Tools Appl. 74(18), 7843–7857 (2015)
G. Simone, A. Farina, F.C. Morabito, S.B. Serpico, L. Bruzzone, Image fusion techniques for remote sensing applications. Inf. Fusion 3(1), 3–15 (2002)
M.I. Smith, J.P. Heather, A review of image fusion technology in 2005, in Defense and Security (International Society for Optics and Photonics, 2005), pp. 29–45
H. Song, S. Yu, L. Song, X. Yang, Fusion of multispectral and panchromatic satellite images based on contourlet transform and local average gradient. Opt. Eng. 46(2), 020,502–020,502 (2007)
M. Song, X. Chen, P. Guo, A fusion method for multispectral and panchromatic images based on HSI and contourlet transformation, in 10th Workshop on Image Analysis for Multimedia Interactive Services, 2009. WIAMIS’09. (IEEE, 2009), pp. 77–80
T. Stathaki, Image Fusion: Algorithms and Applications (Academic Press, London, 2011)
L. Tang, F. Zhao, Z.G. Zhao, The nonsubsampled contourlet transform for image fusion, in International Conference on Wavelet Analysis and Pattern Recognition, 2007. ICWAPR’07, vol. 1 (IEEE, 2007), pp. 305–310
L. Tang, Z.G. Zhao, The wavelet-based contourlet transform for image fusion, in Eighth ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing, 2007. SNPD 2007, vol. 2 (IEEE, 2007), pp. 59–64
K.P. Upla, P.P. Gajjar, M.V. Joshi, Multiresolution fusion using contourlet transform based edge learning, in 2011 IEEE International Geoscience and Remote Sensing Symposium (IGARSS) (IEEE, 2011), pp. 523–526
G. Vivone, R. Restaino, G. Licciardi, M. Dalla Mura, J. Chanussot, Multiresolution analysis and component substitution techniques for hyperspectral pansharpening, in 2014 IEEE international Geoscience and remote sensing symposium (IGARSS) (IEEE, 2014), pp. 2649–2652
L. Wald, A European proposal for terms of reference in data fusion, in Commission VII Symposium“ Resource and Environmental Monitoring”, vol. 32 (1998), pp. 651–654
L. Wald, Data Fusion: Definitions and Architectures: Fusion of Images of Different Spatial Resolutions (Presses des MINES, Paris, 2002)
V. Walter, Object-based classification of remote sensing data for change detection. ISPRS J. Photogramm. Remote Sens. 58(3), 225–238 (2004)
H. Wang, J. Peng, W. Wu, Fusion algorithm for multisensor images based on discrete multiwavelet transform. IEE Proc. Vis. Image Signal Process. 149(5), 283–289 (2002)
H. Wang, Q. Yang, R. Li, Tunable-Q contourlet-based multi-sensor image fusion. Signal Process. 93(7), 1879–1891 (2013)
Z. Wang, D. Ziou, C. Armenakis, D. Li, Q. Li, A comparative analysis of image fusion methods. IEEE Trans. Geosci. Remote Sens. 43(6), 1391–1402 (2005)
X.L. Wei, Y.A. Zheng, Z.Z. Cui, Q.L. Wang, Multi-band SAR images fusion using the em algorithm in contourlet domain, in Fourth International Conference on Fuzzy Systems and Knowledge Discovery, 2007. FSKD 2007, vol. 1 (IEEE, 2007), pp. 502–506
Q. Weng, Remote Sensing of Impervious Surfaces (CRC Press, Boca Raton, 2007)
Y. Wu, C. Wu, S. Wu, Fusion of remote sensing images based on nonsubsampled contourlet transform and region segmentation. J. Shanghai Jiaotong Univers. (Sci.) 16(6), 722–727 (2011)
Y. Xiao-Hui, J. Li-Cheng, Fusion algorithm for remote sensing images based on nonsubsampled contourlet transform. Acta Autom. Sin. 34(3), 274–281 (2008)
B. Yang, Zl Jing, Ht Zhao, Review of pixel-level image fusion. J. Shanghai Jiaotong Univers. (Sci.) 15(1), 6–12 (2010)
S. Yang, M. Wang, L. Jiao, Contourlet hidden markov tree and clarity-saliency driven PCNN based remote sensing images fusion. Appl. Soft Comput. 12(1), 228–237 (2012)
S. Yang, M. Wang, L. Jiao, R. Wu, Z. Wang, Image fusion based on a new contourlet packet. Inf. Fusion 11(2), 78–84 (2010)
S. Yang, M. Wang, Y. Lu, W. Qi, L. Jiao, Fusion of multiparametric SAR images based on SW-nonsubsampled contourlet and PCNN. Signal Process. 89(12), 2596–2608 (2009)
J. Zhang, Multi-source remote sensing data fusion: status and trends. Int. J. Image Data Fusion 1(1), 5–24 (2010)
Y. Zhang, Problems in the fusion of commercial high-resolution satelitte as well as Landsat 7 images and initial solutions. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 34(4), 587–592 (2002)
Y. Zhang, Understanding image fusion. Photogramm. Eng. Remote Sens. 70(6), 657–661 (2004)
Y. Zhang, Methods for image fusion quality assessment—a review, comparison and analysis. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 37(PART B7), 1101–1109 (2008)
Y. Zhang, G. Hong, An IHS and wavelet integrated approach to improve pan-sharpening visual quality of natural colour IKONOS and QuickBird images. Inf. Fusion 6(3), 225–234 (2005)
Y. Zhang, R.K. Mishra, A review and comparison of commercially available pan-sharpening techniques for high resolution satellite image fusion, in 2012 IEEE International Geoscience and Remote Sensing Symposium (IGARSS) (IEEE, 2012), pp. 182–185
Y.A. Zheng, C. Zhu, J. Song, X. Zhao, Fusion of multi-band SAR images based on contourlet transform, in 2006 IEEE International Conference on Information Acquisition (IEEE, 2006), pp. 420–424
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Mangalraj, P., Sivakumar, V., Karthick, S. et al. A Review of Multi-resolution Analysis (MRA) and Multi-geometric Analysis (MGA) Tools Used in the Fusion of Remote Sensing Images. Circuits Syst Signal Process 39, 3145–3172 (2020). https://doi.org/10.1007/s00034-019-01316-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00034-019-01316-6