Special issue on multimodal data fusion for multidimensional signal processing
- 1.1k Downloads
As one of the most important research topics, multidimensional signal processing is particularly emphasised for effective data analytics and interpretation. This has been successfully applied in a number of applications such as data compression, content extraction and retrieval, data mining, and classification, especially in video, hyperspectral imaging (HSI) and medical applications.
Recently, it has become a new trend to apply multimodal data fusion in multidimensional signal processing. This proves to be useful in not only effective feature extraction and data reduction but also data classification and decision making. Motivated by these strong interests, this Special Issue aims to report the progress in this research hotspot, where researchers can share their findings and exchange their approaches and results.
In this Special Issue, 11 papers are included which can be grouped into two thematic areas, i.e. fusion for denoising and enhancement of signal, and fusion for detection and classification as detailed below.
Data fusion for denoising and enhancement of the signal
In this group we have 5 papers, where the topics covered include pansharpening in multispectral and hyperspectral imagines, underwater video dehazing and compressed sensing for secure signal processing. By applying spatial-temporal and spatial-spectral fusion of multi-sourced data, the quality of the images is successfully improved.
The paper by Licciardi et al. (2015) presented an effective solution for pansharpening in hyperspectral imaging. By analyzing the drawbacks of two commonly used approaches, component substitution (CS) and multi-resolution analysis (MRA), a hybrid approach is proposed. A non-linear principal component analysis is also employed for dimensionality reduction. These have helped to improve the spatial resolution while preserving as much as possible of the original spectral information as well as reduction in the computational time.
Hallabia et al. (2016) present a novel pansharpening method by using low-pass and high-pass filter banks for decomposition and reconstruction processing. The low-pass approximation of the pansharpened multispectral image is used to preserve the spectral quality whereas the high-pass filter facilitates obtaining the high-frequency panchromatic image details. Extensive experiments on Pléaides and GeoEye-1 data and comparisons with previous methods demonstrate the superiority of the proposed method.
In the paper from Mookambiga and Gomathi (2016), a comprehensive review on fusion techniques for spatial information enhancement in hyperspectral imagery is presented. These have been categorized into two classes, i.e. hyperspectral pansharpening and hyperspectral image fusion with multispectral images. In addition to component substitution (CS) and multi-resolution analysis (MRA), matrix factorization and Bayesian approaches are also covered. Discussions on some available datasets and relevant measures for quantitative and qualitative assessment are also given. Some summarized challenges will also provide directions for future investigation.
Zhao et al. (2015) propose a novel integrated secure data processing framework for multi-sourced data fusion and processing. In the proposed framework, the distributed compressed sensing is adopted for outsourced data processing and recovering service. The generation of the pseudo-random sensing matrix can facilitate the data encryption in distributed compressed sensing, where joint recovery of multiparty data at legal users’ side is achieved. Experimental results can demonstrate the proposed framework is effective and requires fewer measurements than other traditional methods. The proposed work can also be applied to other applications such as cloud-based collaborative secure signal processing and data mining.
In the paper proposed by Qing et al. (2016), dehazing of underwater videos is focused to restore/enhance video degraded by medium scattering and light absorption. A spatial–temporal information fusion method is proposed, which include correlation based transmission estimation for color consistency and background light estimation to keep consistent atmospheric light values in a video. Comprehensive results have been demonstrated to validate its efficacy in terms of haze removing and color balancing capabilities.
Data and feature fusion for improved detection and classification
There are six papers contained in this group, where the topics covered include target detection and change detection in remote sensing, color image retrieval, image forgery detection, pedestrian detection and faked painting identification. In addition to conventional supervised learning such as support vector machine, new machine learning approaches such as deep learning, active learning and weakly supervised learning are presented. Adaptive feature fusion is also introduced to avoid empirically parameter settings. These will provide useful insights for readers to further explore relevant techniques and applications.
The paper by Zhou et al. (2015) proposed an inpainting-based depth-image-based rendering algorithm for virtual view synthesis. Simulation results show that the synthesized virtual images for the proposed DIBR system have better quality than those obtained using other works (about 4–5 dB improvement). The parallel computation reduced runtime by 51 times and 25 times for generating nine and two views, respectively, compared to a single threaded approach. The method is realized on the compute unified device architecture parallel computing platform which runs on a graphics processing unit.
Yan et al. (2016) propose an adaptive fusion of color and spatial features for content-based retrieval of logo/trademark images. This is particularly important when these two features are combined in such applications, where weighting of these features can be optimized according to the image content. First, dominant colors are extracted via color quantization and k-means clustering. For each extracted dominant color, a component-based spatial descriptor is derived as local features. The significance of color features is used to measure the image quality and adaptive feature fusion. Experimental results have shown improved retrieval precision even with added Gaussian, salt and pepper, and speckle noise.
In the paper by Gao et al. (2016), a novel semisupervised support vector machine classifier is proposed to solve the problem of insufficient labeled samples, using active learning (AL) and context information. First, AL is employed to select unlabeled samples as semilabled samples, which are then relabelled by exploiting the context information. The relabelled samples are combined with labeled ones for training, along with a new query function to enhance the reliability of the classification and a fusion method to further enhance the robustness. Experimental results on change detection from real remote sensing images have shown significantly improved detection accuracy and efficiency.
Zheng et al. (2016) present a new approach for image forgery detection, using adaptive fusion of keypoint-based and block-based features. First, the image is segmented into non-overlapped regions, from which feature points are extracted using the scale invariant feature transform (SIFT). The ratio between the number of keypoints and the total number of pixels is used to classify a region into smooth or non-smooth (keypoints) regions. Accordingly, Zernike moments and SIFT features are respectively applied for effective forgery detection. Experimental results show that the proposed approach outperforms the keypoint-based method in reliability of detection and the block-based method in efficiency.
The paper by Ren et al. (2016) propose to apply multi-view and multi-plane data fusion for the robust detection of pedestrians in intelligent video surveillance. With the estimated homography, foreground regions are projected from multiple camera views to a reference view. The homographic transformations for a set of parallel planes, from the head plane to the ground, are applied to identify false-positive detections caused by foreground intersections of non-corresponding objects. Multiple features including occupancy information and color cues are also extracted from such planes for joint decision-making. Experimental results on real world sequences have validated the good performance of the proposed approach.
Wang et al. (2016) propose to leverage the hyperspectral image for identifying of fake Chinese paintings. Chinese paintings are scanned by a visual band hyperspectral camera with the spectral frequency ranging from 400 to 900 nm. Afterwards, the spectral and spatial features are extracted based on principal component analysis and convolution neural network, respectively. Finally, the extracted spectral features and spatial features are combined to train a support vector machines for classification. Extensive evaluations can demonstrate the effectiveness of the proposed approach.
Finally, we would like to thank all the authors for their valuable contributions, the reviewers for their constructive suggestions, and the then Editor-in-Chief (EIC) while preparing this Special Issue, Dr Zhiping Lin, and the now EIC, Prof Eric Rogers, and other working staff for the journal, for their kind help, valuable support and encouragement.
- Gao, F., Lv, W., Zhang, Y., Sun, J., Wang, J., & Yang, E. (2016). A novel semisupervised support vector machine classifier based on active learning and context information. Multidimensional Systems and Signal Processing. doi:10.1007/s11045-016-0396-1.
- Hallabia, H., Kallel, A., Ben Hamida, A., & Le Hégarat-Mascle, S. (2016). High spectral quality pansharpening approach based on MTF-matched filter banks. Multidimensional Systems and Signal Processing. doi:10.1007/s11045-016-0421-4.
- Licciardi, G., Vivone, G., Dalla Mura, M., Restaino, R., & Chanussot, J. (2015). Multi-resolution analysis techniques and nonlinear PCA for hybrid pansharpening applications. Multidimensional Systems and Signal Processing. doi:10.1007/s11045-015-0359-y.
- Mookambiga, A., & Gomathi, V. (2016). Comprehensive review on fusion techniques for spatial information enhancement in hyperspectral imagery. Multidimensional Systems and Signal Processing. doi:10.1007/s11045-016-0415-2.
- Qing, C., Yu, F ., Xu, X., Huang, W., & Jin, J. (2016). Underwater video dehazing based on spatial-temporal information fusion. Multidimensional Systems and Signal Processing. doi:10.1007/s11045-016-0407-2.
- Ren, J., Xu, M., Smith, J. S., & Cheng, S. (2016). Multi-view and multi-plane data fusion for effective pedestrian detection in intelligent visual surveillance. Multidimensional Systems and Signal Processing. doi:10.1007/s11045-016-0428-x.
- Wang, Z., Lu, D., Zhang, D., Sun, M., & Zhou, Y. (2016). Fake modern Chinese painting identification based on spectral-spatial feature fusion on hyperspectral image. Multidimensional Systems and Signal Processing. doi:10.1007/s11045-016-0429-9.
- Yan, Y., Ren, J., Li, Y., Windmill, J. F. C., Ijomah, W., & Chao, K.-M. (2016). Adaptive fusion of color and spatial features for noise-robust retrieval of colored logo and trademark images. Multidimensional Systems and Signal Processing. doi:10.1007/s11045-016-0382-7.
- Zhao, H., Wei, W., Cai, J., Lei, F., & Luo, J. (2015). Distributed compressed sensing for multi-sourced fusion and secure signal processing in private cloud. Multidimensional Systems and Signal Processing. doi:10.1007/s11045-015-0371-2.
- Zheng, J., Liu, Y., Ren, J., Zhu, T., Yan, Y., & Yang, H. (2016). Fusion of block and keypoints based approaches for effective copy-move image forgery detection. Multidimensional Systems and Signal Processing. doi:10.1007/s11045-016-0416-1.
- Zhou, P., Cheng, G., Liu, Z., Bu, S., & Hu, X. (2015). Weakly supervised target detection in remote sensing images based on transferred deep features and negative bootstrapping. Multidimensional Systems and Signal Processing. doi:10.1007/s11045-015-0370-3.