Abstract
Recent advances in remote sensing technology have led to an increased availability of a multitude of satellite and airborne data sources, with increasing resolution. The term resolution here includes spatial and spectral resolutions. Additionally, at lower altitudes, airplanes and Unmanned Aerial Vehicles (UAVs) can deliver very high-resolution data from targeted locations. Remote sensing acquisitions employ both passive (optical and thermal range, multispectral, and hyperspectral) and active devices such as Synthetic Aperture Radar (SAR) and Light Detection and Ranging (LiDAR). Diverse information of the Earth’s surface can be obtained from these multiple imaging sources. Optical and SAR characterize the surface of the ground, LiDAR provides the elevation, while multispectral and hyperspectral sensors reveal the material composition. These multisource remote sensing images, once combined/fused together, provide a more comprehensive interpretation of land cover/use (urban and climatic changes), natural disasters (floods, hurricanes, and earthquakes), and potential exploitation (oil fields and minerals). However, automatic interpretation of remote sensing data remains challenging. Two fundamental problems in data fusion of multisource remote sensing images are (1) differences in resolution hamper the ability to fastly interpret multisource remote sensing images and (2) there is no clear methodology yet on combining the diverse information of different data sources. In this chapter, we will introduce our recent solutions for these two problems, with an introduction on signal-level fusion (hyperspectral image pansharpening) first, followed by feature-level fusion (graph-based fusion model for multisource data classification).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
- 3.
With faster algorithms (e.g., K-D trees) than direct nearest neighbours searching, the complexity can be reduced. More details on efficient representation and search techniques for large data sets can be found in Chap. 2.
References
Hall, L., Llinas, J.: An introduction to multisensor data fusion. Proc. IEEE 85, 6–23 (1997)
Pohl, C., Van Genderen, J.L.: Multisensor image fusion in remote sensing: concepts, methods and applications. Int. J. Remote Sens. 19, 823–854 (1998)
Zhang, J.: Multi-source remote sensing data fusion: status and trends. Int. J. Image Data Fus. 1(1), 5–24 (2010)
Gamba, P.: Image and data fusion in remote sensing of urban areas: status issues and research trends. Int. J. Image Data Fus. 5(1), 2–12 (2014)
Dalla Mura, M., Prasad, S., Pacifici, F., Gamba, P., Chanussot, J., Benediktsson, J.: Challenges and opportunities of multimodality and data fusion in remote sensing. Proc. IEEE 103(9), 1585–1601 (2015)
Lahat, D., Adali, T., Jutten, C.: Multimodal data fusion: an overview of methods, challenges and prospects. Proc. IEEE 103(9), 1449–1477 (2015)
Gomez-Chova, L., Tuia, D., Moser, G., Camps-Valls, G.: Multimodal classification of remote sensing images: a review and future directions. Proc. IEEE 103(9), 1560–1584 (2015)
Luo, R.C., Kay, M.G.: A tutorial on multisensor integration and fusion. In:16 Annual Conference of IEEE Industrial Electronics Society, 1990, pp. 707–722 (1990)
Loncan, L., Almeida, L.B., Bioucas Dias, J., et al.: Hyperspectral pansharpening: a review. IEEE Geosci. Remote Sens. Mag. 3(3), 27–46 (2015)
Carper, W., Lillesand, T.M., Kiefer, P.W.: The use of intensity- hue-saturation transformations for merging SPOT panchromatic and multispectral image data. Photogramm. Eng. Remote Sens. 56(4), 459–467 (1990)
Tu, T.M., Su, S.C., Shyu, H.C., Huang, P.S.: A new look at IHS-like image fusion methods. Inf. Fus. 2(3), 117–186 (2001)
Shettigara, V.: A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set. Photogramm. Eng. Remote Sens. 58(5), 561–567 (1992)
Shah, V., Younan, N., King, R.: An efficient pan-sharpening method via a combined adaptive PCA approach and contourlets. IEEE Trans. Geosci. Remote Sens. 46(5), 1323–1335 (2008)
Mallat, S.: A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans. Pattern Anal. Mach. Intell. 11(7), 674–693 (1989)
Nason, G.P., Silverman, B.W.: The stationary wavelet transform and some statistical applications in Wavelets and Statistics. In: A. Antoniadis, G.Oppenheim (ed.), vol. 103, pp. 281–299 Springer, New York (1995)
Burt, P.J., Adelson, E.H.: The Laplacian pyramid as a compact image code. IEEE Trans. Commun. 31(4), 532–540 (1983)
Do, M.N., Vetterli, M.: The contourlet transform: an efficient directional multiresolution image representation. IEEE Trans. Image Process. 14(12), 2091–2106 (2005)
Starck, J., Murtagh, F.: The undecimated wavelet decomposition and its reconstruction. IEEE Trans. Image Process. 16(2), 297–309 (2007)
Ballester, C., Caselles, V., Igual, L., et al.: A variational model for P\(+\)XS image fusion. Int. J. Comput. Vis. 5969(1), 43–58 (2006)
Palsson, F., Sveinsson, J., Ulfarsson, M., Benediktsson, J.: A new pansharpening algorithm based on total variation. IEEE Geosci. Remote Sens. Lett. 11(1), 318–322 (2014)
He, X., Condat, L., Bioucas Dias, J., et al.: A new pansharpening method based on spatial and spectral sparsity priors. IEEE Trans. Image Process. 23(9), 4160–4174 (2014)
Moeller, M., Wittman, T., Bertozzi, A.: A variational approach to hyperspectral image fusion. In: SPIE Defense, Security, and Sensing (2009)
Vivone, G., Alparone, L., Chanussot, J., et al.: Multi-resolution analysis and component substitution techniques for hyperspectral pansharpening. In: 2014 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), 2649–2652 (2014)
Yokoya, N., Yairi, T., Iwasaki, A.: Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion. IEEE Trans. Geosci. Remote Sens. 50(2), 528–537 (2012)
Wei, Q., Dobigeon, N., Tourneret, J.Y.: Bayesian fusion of multi- band images. IEEE J. Select. Top. Signal Process. 9(6), 1117–1127 (2015)
Simoes, M., Bioucas-Dias, J., Almeida, L., Chanussot, J.: A convex formulation for hyperspectral image superresolution via subspace-based regularization. IEEE Trans. Geosci. Remote Sens. 53(6), 3373–3387 (2015)
Wei, Q., Bioucas-Dias, J., Dobigeon, N., Tourneret, J.Y.: Hyperspectral and multispectral image fusion based on a sparse representation. IEEE Trans. Geosci. Remote Sens. 53(7), 3658–3668 (2015)
Liao, W., Huang, X., Coillie, F., et al.: Processing of Multiresolution Thermal Hyperspectral and Digital Color Data: Outcome of the 2014 IEEE GRSS Data Fusion Contest. IEEE J. Select. Top. Appl. Earth Observ. Remote Sens. 8(6), 2984–2996 (2015)
Liao, W., Huang, X., Coillie, F., Guy, T., Scheunders, P., Pizurica, A., Philips, W. Two-stage fusion of thermal hyperspectral and visible RGB image by PCA and Guided filter. In: 7th workshop on hyperspectral image and signal processing: evolution in remote sensing (WHISPERS 2015), Tokyo, Japan (2015)
Zhu, X., Grohnfeldt, C., Bamler, R.: Exploiting joint sparsity for pan-sharpening: the J-sparse FI algorithm. IEEE Trans. Geosci. Remote Sens. 54(5), 2664–2681 (2016)
He, K., Sun, J., Tang, X.: Guided image filtering. IEEE Trans. Pattern Anal. Mach. Intell. 35(6), 1397–1409 (2013)
Burges, C.J.C.: A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Discov. 2, 121–167 (1998)
Chang, C.C., Lin, C.J.: (2001). LIBSVM: A Library for Support Vector Machines. ACM Trans. Intell. Syst. Technol., 2(3), 27:1–27:27, 2011. http://www.csie.ntu.edu.tw/~cjlin/libsvm
Fauvel, M., Benediktsson, J.A., Chanussot, J., Sveinsson, J.R.: Spectral and spatial classification of hyperspectral data using SVMs and morphological profile. IEEE Trans. Geosci. Remote Sens. 46(11), 3804–3814 (2008)
Swatantrana, A., Dubayaha, R., Robertsb, D., Hoftona, M., Blairc, J.B.: Mapping biomass and stress in the Sierra Nevada using lidar and hyperspectral data fusion. Remote Sens. Environ. 115(11), 2917–2930 (2011)
Koetz, B., Sun, G., Morsdorf, F., Ranson, K.J., et al.: Fusion of imaging spectrometer and LIDAR data over combined radiative transfer models for forest canopy characterization. Remote Sens. Environ. 106(4), 449–459 (2007)
Dalponte, M., Bruzzone, L., Gianelle, D.: Fusion of Hyperspectral and LIDAR Remote Sensing Data for Classification of Complex Forest Areas. IEEE Trans. Geosci. Remote Sens. 46(5), 1416–1427 (2008)
Naidooa, L., Choa, M.A., Mathieua, R., Asnerb, G.: Classification of savanna tree species, in the Greater Kruger National Park region, by integrating hyperspectral and LiDAR data in a Random Forest data mining environment. ISPRS J. Photogramm. Remote Sens. 69, 167–179 (2012)
Pedergnana, M., Reddy Marpu, P., Dalla Mura, M., Benediktsson, J.A., Bruzzone, L.: Classification of remote sensing optical and LiDAR data using extended attribute profiles. IEEE J. Select. Top. Signal Process. 6(7), 856–865 (2012)
Khodadadzadeh, M., Li, J., Prasad, M., Plaza, A.: Fusion of Hyperspectral and LiDAR Remote Sensing Data Using Multiple Feature Learning. IEEE J. Select. Top. Appl. Earth Observ. Remote Sens. 8(6), 2971–2983 (2015)
Zhang, Y., Prasad, S.: Multisource geospatial data fusion via local joint sparse representation. IEEE Trans. Geosci. Remote Sens. 54(6), 3265–3276 (2016)
Liao, W., Bellens, R., Pizurica, A., Gautama, S., Philips, W.: Generalized graph-based fusion of hyperspectral and LiDAR data using morphological features. IEEE Geosci. Remote Sens. Lett. 12(3), 552–556 (2015)
Camps-Valls, G., Gomez-Chova, L., Munoz-Mari, J., Vila-Frances, J., Calpe-Maravilla, J.: Composite kernels for hyperspectral image classification. IEEE Geosci. Remote Sens. Lett. 3(1), 93–97 (2006)
Fauvel, M., Chanussot, J., Benediktsson, J.: A spatial-spectral kernel-based approach for the classification of remote-sensing images. Pattern Recognit. 45(1), 381–392 (2012)
Li, J., Marpu, P.R., Plaza, A., Bioucas-Dias, J.M., Benediktsson, J.: Generalized composite Kernel framework for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 51(9), 4816–4829 (2013)
Voisin, A., Krylov, V.A., Moser, G., Serpico, S.B., Zerubia, J.: Supervised Classification of Multisensor and Multiresolution Remote Sensing Images with A Hierarchical Copula-based Approach. IEEE Trans. Geosci. Remote Sens. 52(6), 3346–3358 (2014)
Tuia, D., Volpi, M., Trolliet, M., Camps-Valls, G.: Semisupervised manifold alignment of multimodal remote sensing images. IEEE Trans. Geosci. Remote Sens. 52(12), 7708–7720 (2014)
Fang, L., Li, S., Kang, X., Benediktsson, J.: Spectral-spatial hyperspectral image classification via multiscale adaptive sparse representation. IEEE Trans. Geosci. Remote Sens. 52(12), 7738–7749 (2014)
Gunatilaka, A.H., Baertlein, B.A.: Feature-level and decision-level fusion of noncoincidently sampled sensors for land mine detection. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 577–589 (2001)
Calhoun, V.D., Adali, T., Pearlson, G.D., Kiehl, K.A.: Neuronal chronometry of target detection: fusion of hemodynamic and event-related potential data. NeuroImage 30(2), 544–553 (2006)
Calhoun, V.D., Adali, T., Liu, J.: A feature-based approach to combine functional MRI, structural MRI, and EEG brain imaging data. In: 2006 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS), New York (2006)
Correa, N.M., Li, Y.O., Adali, T., Calhoun, V.D.: Canonical correlation analysis for feature-based fusion of biomedical imaging modalities and its application to detection of associative networks in schizophrenia. IEEE J. Select. Top. Signal Process. 2(6), 998–1007 (2008)
Jagadeesan, A., Thillaikkarasi, T., Duraiswamy, K.: Protected bio-cryptography key invention from multimodal modalities: feature level fusion of fingerprint and Iris. Eur. J. Sci. Res. 49(4), 484–502 (2011)
Conti, V., Militello, C., Sorbello, F., Vitabile, S.: A frequency-based approach for features fusion in fingerprint and iris multi-modal biometric identification systems. IEEE Trans. Syst. Man Cybern. C 40(4), 384–395 (2010)
Nagar, A., Nandakumar, K., Jain, A.K.: Multibiometric cryptosystems based on feature-level fusion. IEEE Trans. Inf. Forensics Secur. 7(1), 255–268 (2012)
Camps-Valls, G., Gomez-Chova, L., et al.: Kernel-based framework for multitemporal and multisource remote sensing data classification and change detection. IEEE Trans. Geosci. Remote Sens. 46(6), 1822–1835 (2008)
Tuia, D., Ratle, F., Pozdnoukhov, A., Camps-Valls, G.: Multi-source composite kernels for urban image classification. IEEE Geosci. Remote Sens. Lett. 7(1), 88–92 (2010)
Tuia, D., Camps-Valls, G., Matasci, G., Kanevski, M.: Learning relevant image features with multiple kernel classification. IEEE Trans. Geosci. Remote Sens. 48(10), 3780–3791 (2010)
Gomez-Chova, L., Camps-Valls, G., Bruzzone, L., Calpe-Maravilla, J.: Mean map kernel methods for semisupervised cloud classification. IEEE Trans. Geosci. Remote Sens. 48(1), 207–220 (2010)
Volpi, M., Camps-Valls, G., Tuia, D.: Spectral alignment of cross-sensor images with automated kernel canonical correlation analysis. ISPRS J. Photogramm. Remote Sens. 107, 50–63 (2015)
Dalla Mura, M., Benediktsson, J.A., Waske, B., Bruzzone, L.: Extended profiles with morphological attribute filters for the analysis of hyperspectral data. Int. J. Remote Sens. 31(22), 5975–5991 (2010)
Dalla Mura, M., Villa, A., Benediktsson, J.A., Chanussot, J., Bruzzone, L.: Classification of hyperspectral images by using extended morphological attribute profiles and independent component analysis. IEEE Geosci. Remote Sens. Lett. 8(3), 541–545 (2011)
Liao, W., Dalla Mura, M., Chanussot, J., Pizurica, A.: Fusion of Spectral and Spatial Information for Classification of Hyperspectral Remote Sensed Imagery by Local Graph. IEEE J. Select. Top. Appl. Earth Observ. Remote Sens. 9(2), 583–594 (2016)
Blaschke, T.: Object based Image Analysis for Remote Sensing. ISPRS J. Photogramm. Remote Sens. 65(1), 2–16 (2010)
Soille, P.: Morphological Image Analysis, Principles and Applications, 2nd edn. Springer, Berlin (2003)
Benediktsson, J., Palmason, J., Sveinsson, J.R.: Classification of hyperspectral data from urban areas based on extended morphological profiles. IEEE Trans. Geosci. Remote Sens. 43(3), 480–491 (2005)
Dalla Mura, M., Benediktsson, J., Waske, B., Bruzzone, L.: Morphological attribute profiles for the analysis of very high resolution images. IEEE Trans. Geosci. Remote Sens. 48(10), 3747–3762 (2010)
Huang, X., Liu, H., Zhang, L.: Spatiotemporal detection and analysis of urban villages in mega city regions of china using high-resolution remotely sensed imagery. IEEE Trans. Geosci. Remote Sens. 53(7), 3639–3657 (2015)
Bruzzone, L., Bovolo, F.: A novel framework for the design of change-detection systems for very-high-resolution remote sensing images. Proc. IEEE 101(3), 609–630 (2013)
Braun, A.C., Rojas, C., et al.: Design of a Spectral-Spatial Pattern Recognition Framework for Risk Assessments Using Landsat Data-A Case Study in Chile. IEEE J. Select. Top. Appl. Earth Observ. Remote Sens. 7(3), 917–928 (2014)
Liao, W., Bellens, R., Pižurica, A., Philips, W., Pi, Y.: Classification of Hyperspectral Data Over Urban Areas Using Directional Morphological Profiles and Semi-Supervised Feature Extraction. IEEE J. Select. Top. Appl. Earth Observ. Remote Sens. 5(4), 1177–1190 (2012)
Bellens, R., Gautama, S., Martinez-Fonte, L., Philips, W., Chan, J.C.-W., Canters, F.: Improved classification of VHR images of urban areas using directional morphological profiles. IEEE Trans. Geosci. Remote Sens. 46(10), 2803–2812 (2008)
Scholkopf, B., Smola, A.J., Muller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 10, 1299–1319 (1998)
Belkin, M., Niyogi, P.: Laplacia Eigenmaps and Spectral Techniques for Embedding and Clustering. In: Advances in Neural Information Processing Systems 14, 585–591, MIT Press, British Columbia, Canada (2002)
He, X.F., Niyogi, P.: Locality preserving projections. In: Advances in Neural Information Processing Systems 16, pp. 153–160. MIT Press, Cambridge (2004)
Debes, C., Merentitis, A., Heremans, R., et al.: Hyperspectral and LiDAR Data Fusion: Outcome of the 2013 GRSS Data Fusion Contest. IEEE J. Select. Top. Appl. Earth Observ. Remote Sens. 7(6), 2405–2418 (2014)
Kuo, B.C., Landgrebe, D.A.: Nonparametric weighted feature extraction for classification. IEEE Trans. Geosci. Remote Sens. 42(5), 1096–1105 (2004)
Tarabalka, Y., Benediktsson, J.A., Chanussot, J.: Spectral-spatial classification of hyperspectral imagery based on partitional clustering techniques. IEEE Trans. Geosci. Remote Sens. 47(8), 2973–2987 (2009)
Li, J., Bioucas-Dias, J.M., Plaza, A.: Spectral-spatial hyperspectral image segmentation using subspace multinomial logistic regression and Markov random field. IEEE Trans. Geosci. Remote Sens. 50(3), 809–823 (2012)
Camps-Valls, G., Shervashidze, N., Borgwardt, K.M.: Spatio-spectral remote sensing image classification with graph Kernels. IEEE Geosci. Remote Sens. Lett. 7(4), 741–745 (2010)
Chen, G., Qian, S.E.: Dimensionality reduction of hyperspectral imagery using improved locally linear embedding. J. Appl. Remote Sens. 1, 1–10 (2007)
Jimenez, M.D., Prelcic, N.: Linear boundary extensions for einite length signals and paraunitary two-channel filterbanks. IEEE Trans. Signal Process 52(11), 3213–3226 (2004)
Chen, G., Bui, T.D., Krzyzak, A.: Image denoising with neighbour dependency and customized wavelet and threshold. Pattern Recogn. 38(1), 115–124 (2005)
Acknowledgements
The authors would like to thank Telops Inc. (Québec, Canada), the Hyperspectral Image Analysis group and the NSF Funded Center for Airborne Laser Mapping (NCALM) at the University of Houston, and Prof. Paolo Gamba from the University of Pavia (Italy) for providing the data sets used in this Chapter. This work was supported by the Fund for Scientific Research in Flanders (FWO) project G037115N “Data Fusion for Image Analysis in Remote Sensing.” Wenzhi Liao is a postdoctoral fellow of the Research Foundation Flanders (FWO-Vlaanderen) and acknowledges its support.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this chapter
Cite this chapter
Liao, W., Chanussot, J., Philips, W. (2018). Remote Sensing Data Fusion: Guided Filter-Based Hyperspectral Pansharpening and Graph-Based Feature-Level Fusion. In: Moser, G., Zerubia, J. (eds) Mathematical Models for Remote Sensing Image Processing. Signals and Communication Technology. Springer, Cham. https://doi.org/10.1007/978-3-319-66330-2_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-66330-2_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-66328-9
Online ISBN: 978-3-319-66330-2
eBook Packages: EngineeringEngineering (R0)