Data compression in ViSAR sensor networks using non-linear adaptive weighting
- 176 Downloads
Nowadays, industrial video synthetic aperture radars (ViSARs) are widely used for aerial remote sensing and surveillance systems in smart cities. A main challenge of a group of networked ViSAR sensors in an IoT-based environment is low bandwidth of wireless links for communicating big video data. In this research, we propose a non-linear statistical estimator for adaptive reconstruction of compressed ViSAR data. Our proposed reconstruction filter is based on an adaptively generated non-linear weight mask of spatial observations. It can strongly outperform several conventional and well-known reconstruction filters for three different video samples.
KeywordsVideo synthetic aperture radar (ViSAR) Non-linear reconstruction filter Adaptive weighting Data compression Interpolation Internet of Things (IoT)
Internet of Things
Linear minimum mean square error
Markov random field
SDTV to HDTV data conversion
Video synthetic aperture radars
The interpolation process is one of the most common processes in remote sensing image and video analysis. Some applications of interpolation in order to estimate unknown pixels are image compression, high-rate video transmission, image and video watermarking, image reconstruction, restoration, and magnification. For instance in , a modified scheme was proposed for converting standard-definition television (SDTV) frames to high-definition television (HDTV) standard  to be used in video transmission technologies such as DVB-T. Researches on interpolation algorithms include a wide range of research on which some details of them are reviewed as follows. Two most famous interpolators are bi-cubic convolution (mainly abbreviated as BC) and bi-linear (BL) . Today, BC and BL are classified into non-adaptive techniques in terms of local edge computation and indeed provide two linear reconstruction filters . Another main point about them is to use both methods in image processing software tools for remote sensing such as ENVI and ERDAS. However, we wish to focus on newer and efficient types of interpolators entitled edge-guided interpolation algorithms. Edge-guided methods are often applicable in image and video reconstruction problems . In , a technique has been represented which estimates anything based on an assumption that every image can be modeled as a locally stationary Gaussian process. Based on this assumption, the local covariance coefficients in low-resolution (LR) images are estimated, and then, the interpolation process is performed based on geometric duality between the LR and the high-resolution (HR) covariance. A key issue of this method that makes it unsuitable for ViSAR frames is to consider some statistical assumptions which do not exist in practice. In , a new scheme was proposed which uses tensor tool for interpolation in order to realize the edge-guided interpolation. This method could outperform some existing methods.
In this research, we want to propose a new edge-guided interpolator based on statistical estimation. Our purposed method uses an adaptive weighting mechanism which makes it edge-guided, non-linear, and fully greedy. Our scheme is an extension for the method discussed in [7, 8, 9] for remote sensing applications. In , a basic edge-guided interpolation based on linear minimum mean square error estimation (LMMSE) was introduced for benchmark images such that some evaluations about it have been done in . LMMSE includes two phases of directional filtering using a pre-interpolator and data fusion of two orthogonal directions. LMMSE scheme for remote sensing images has been discussed in . This interpolator is a relatively adaptive scheme needing a pre-interpolator based on linear filtering, e.g., linear or cubic interpolation, for directional filtering. In the current work, we are going to propose a full-adaptive version of LMMSE for remote sensing data of ViSAR whereas our proposed technique does not need any pre-interpolator. In fact, if LMMSE can outperform some linear methods like BL or BC, it is completely natural because they have been used as pre-interpolator in LMMSE structure (although as two one-dimensional components), but our proposed method which is named adaptive LMMSE (ALMMSE) can fuse directional observation without need to any linear pre-interpolator and however outperforms the linear interpolators. Our experiments show that it is winner against five conventional techniques among the most popular non-adaptive/linear reconstruction filters.
We can also use the proposed approach for magnifying some multispectral images such as IKONOS and QuickBird images or images related to high-resolution optical remote sensing sensors [11, 12, 13]. In addition, there are many other applications for interpolation algorithms, e.g., data hiding [14, 15, 16, 17, 18], interpolation-based image denoising and demosaicking [19, 20, 21], SDTV to HDTV conversion (SD2HD)  in video processing, color processing , information fusion [8, 9], and shadow detection  which can be assisted by ALMMSE algorithm. As we mentioned, the main focus of this research is towards interpolation-based image/video compression [10, 24]. For compression, we firstly down-sample video frames to reduce the information size at the sender side and then reconstruct them using an interpolator at the receiver side. Consequently, ALMMSE can be used in different processes of remote sensing images.
The rest of this paper is organized as follows: in Section 2, we review LMMSE details and some of its applications in digital image processing; then in Section 3, we present our proposed scheme (ALMMSE); and finally, we evaluate it in Section 4. Evaluations show that the use of a locally adaptive estimation in ViSAR frames creates better quality compared to many conventional techniques. The last section is a dedicated conclusion on the work.
2 Related work
The main shortcoming in the design of LMMSE interpolator is to select equal weights for two corresponding pixels which are in the same direction. According to logic of greedy algorithms, LMMSE is not classified into full-adaptive algorithms because it considers a general assumption about generality of images in its computations. However, we can consider it as a partially-adaptive interpolator compared to linear interpolators such as BL and BC. In our study, the aim is to create a new LMMSE-based interpolator for reconstruction of a kind of compressed remote sensing data without need to any pre-interpolation step.
In , the authors have proposed an LMMSE-based interpolator for color demosaicking. Demosaicking is a certain type of interpolation which is commonly applicable in some color images, for example Kodak dataset . The demosaicking algorithm in  is based on LMMSE and strongly outperforms BL interpolator . Another application of LMMSE is noise reduction. A denoising algorithm is practically a low pass filter which filters high frequency variations of images, or in the other words, it reduces/removes the noises.
Most of the interpolators have mechanisms based on averaging process which is equal to low pass filtering. Therefore, LMMSE-based interpolators can be used in the noise reduction problems. For example in , an LMMSE-based denoising algorithm has been proposed for a wide range of digital images. Quality of the scheme is observable. Interpolation in spatial domain is also a way for image compression, for more details about LMMSE-based image compression refer to . Thus, in addition to direct applications of interpolators such as magnification, interpolation is widely used for image restoration and reconstruction. In other researches, interpolation-based data hiding [8, 14, 15, 18, 25] and multispectral image fusion (pan-sharpening)  are carried out using it. For example in , LMMSE has been applied as a magnifier for achieving better quality in pan-sharpening process of Landsat-8 images compared to a linear interpolator. Another application for LMMSE is to do denoising for improving classification accuracy in digital images, because noise reduces accuracy of classifiers (a pre-process based on noise reduction algorithms is normally essential before classification).
3 Proposed method
In the proposed scheme, an interpolation without any assumption regarding estimation weights is applied to reconstruct compressed frames . In fact, there are no default weights, and all of them are computed adaptively. Our proposed scheme adaptively estimates non-existing pixels to keep edge information in the best way. In this section, we discuss our ALMMSE interpolation method which is an edge-guided scheme and uses four nearest neighbors from two orthogonal directions to estimate targeted pixels; thus, it has suitability for Markov random field (MRF)-based neighborhood systems with order of 1 or 2 such as many remote sensing images. An important point about ALMMSE is to be a full-adaptive non-linear approach that does not need any pre-interpolation compared to linear schemes using polynomials (non-adaptive methods) and traditional LMMSE (with a pre-interpolator).
In order to compress ViSAR frames using the proposed method, we should make down-sampled versions from HR frames (to create LR frames) with lower resolution and then reconstruct the LR frames using our interpolator. To do this, for example, we estimate 75% of the removed pixels through compression (with a down-sampling algorithm like Algorithm 1) with using only 25% of the remaining cases (sample pixels). In such experiments, we can reduce the video size to be one fourth of the original version. Therefore, we use ALMMSE as a regular interpolation for a quartered template.
Note that computing of the targeted pixels is firstly based on four nearest neighbors, but in some positions due to the existence of two estimated pixel among four nearest neighbors, practically, the estimation procedure has been performed by six neighbors of which four of these six pixels are not within MRF neighborhood. In the next section, the proposed scheme is evaluated. We will see that the proposed scheme is effective on ViSAR dataset.
Moreover, evaluation is performed based on objective and subjective quality assessment metrics. In addition to the proposed method, a pre-processing step before doing the re-sampling process exists which contains two blocks of down-sampling and up-sampling. Suppose that an input image is a typical M × N matrix (for simplicity, M and N are even). Algorithm 1 and Algorithm 2 describe these two blocks.
Qualitative descriptions of the interpolation methods
Different features in video compression
Quartered (Q)/non-quartered (NQ)
Suitability for SAR videos
Pixel location-based adaptivity
Gray level-based adaptivity (edge-guided)
Q (not flexible)
Q (not flexible)
Yes (for NQ)
Yes (for NQ)
Results for video 1
Results for video 2
Results for video 3
In the recent years, data processing for IoT became an interesting topic of research [27, 28, 29, 30]. In our study, we proposed ALMMSE interpolation algorithm for the remote sensing ViSAR frames captured by imaging radars in an IoT-enabled radar networks of drones and airplanes . This scheme is a new edge-guided interpolator based on non-linear statistical estimation which has no assumption on local weights and also does not need any pre-interpolator. The main feature of the proposed method is to use the most adaptation in comparison to another edge-guided interpolator and conventional interpolation techniques. We compared it with several linear interpolators which do not need any pre-interpolator too. All experiments illustrate a clear consequence about superiority of the proposed method. As a future work, we can go ahead to propose a more accurate version of ALMMSE with lower computational complexity. Evaluation of this proposed method for other remote sensing devices may determine some future directions.
We would like to thank Sandia National Laboratory for ViSAR data used as dataset in this research.
MK participated in the mathematical design of the proposed method and its computer implementation. SS coordinated the industrial application and raw data preparation, and helped out for the study. MK and SS have completed the first draft of this paper. All authors have read and approved the manuscript.
The authors declare that they have no competing interests.
- 3.Gonzalez, R.C.; Woods, R.E. Digital image processing, 3rd edition, 2008, Prentice Hall, New Jersey.Google Scholar
- 4.Khosravi, M.R. et al. A new statistical technique for interpolation of landsat images. ICAUCAE 2016, SID Conference Publications, 2016.Google Scholar
- 10.Khosravi, M.R.; Salari, R.; Rostami, H. Analysis of modeling error in digital image spatial estimation. International Congress of Electrical Engineering, Computer Science and Information Technology (IT2015), No. 3, SBU, 2015, pp. 363-371.Google Scholar
- 11.M. Huang, C. He, Q. Zhu, Qin, J (Maize seed variety classification using the integration of spectral and image features combined with feature transformation based on hyperspectral imaging, Applied Sciences, 2016)Google Scholar
- 12.D. Song, L. Song, Y. Sun, P. Hu, K. Tu, L. Pan, H. Yang, Huang, M (Black heart detection in white radish by hyperspectral transmittance imaging combined with chemometric analysis and a successive projections algorithm, Applied Sciences, 2016)Google Scholar
- 14.M.R. Khosravi, S. Samadi, Modified data aggregation for aerial ViSAR sensor networks in transform domain. The 25th Int'l Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA'19), 2019; Las Vegas. USA.Google Scholar
- 16.M. Vaidelys, J. Ragulskiene, P. Ziaukas, Ragulskis (M. Image hiding scheme based on the atrial fibrillation model, Applied Sciences, 2015)Google Scholar
- 17.W. Kuo, S. Kuo, Wuu, L (Multi-bit data hiding scheme for compressing secret messages, Applied Sciences, 2015)Google Scholar
- 20.Getreuer, P. Zhang-Wu Directional LMMSE image demosaicking. Image Processing On Line (IPOL Journal). 2011.Google Scholar
- 22.M.R. Khosravi, S. Samadi, R. Mohseni, Spatial interpolators for intra-frame resampling of SAR videos: a comparative study using real-time HD, medical and radar data. Current Signal Transduction Therapy (2019)Google Scholar
- 27.R. Zhang, P. Xie, C. Wang, G. Liu, S. Wan, Classifying transportation mode and speed from trajectory data via deep multi-scale learning, Computer Networks 162, 106861 (2019)Google Scholar
- 28.Z Gao, HZ Xuan, H Zhang, S Wan, KKR Choo, Adaptive fusion and category-level dictionary learning model for multi-view human action recognition, IEEE Internet of Things Journal, 2019.Google Scholar
- 29.S. Wan, Y. Zhao, T. Wang, Z. Gu, Q.H. Abbasi, K.K.R. Choo, Multi-dimensional data indexing and range query processing via Voronoi diagram for internet of things, Future Generation Computer Systems 91, 382-391 (2019)Google Scholar
- 31.B. Bahri-Aliabadi, M. Khosravi, S. Samadi, Frame rate computing in video SAR using geometrical analysis. The 24th Int'l Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA'18), pp. 165-167, 2018; Las Vegas. USA.Google Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.