# Visual sensor image enhancement based on non-sub-sampled shearlet transform and phase stretch transform

- 92 Downloads

**Part of the following topical collections:**

## Abstract

Acquiring clear images is a requisite in visual sensor networks. Image enhancement is an effective way to improve image quality. In this paper, non-sub-sampled shearlet transform (NSST) multi-scale analysis is combined with phase stretch transform (PST) to nonlinearly enhance the images captured by visual sensors. The components of different scales after NSST multi-scale decomposition are processed by nonlinear models with different thresholds. The thresholds of the enhanced model are determined by the local standard deviation of PST feature map. The noise is well suppressed, and the detail features are enhanced obviously. Experiments show that the proposed algorithm can improve image distortion, clear details, and enhance image contrast effectively.

## Keywords

Image enhancement Image processing Non-sub-sampled shearlet transform Phase stretch transform## Abbreviations

- CII
Contrast Improvement Index

- DSBP
Dynamic Stretching-based Brightness Preservation

- EPI
Edge Preserved Index

- Fuzzy_INT
Tuned Fuzzy Intensification operators

- LSD
Local standard deviation

- NSCT
Non-sub-sampled contourlet transform

- NSST
Non-sub-sampled shearlet transform

- PST
Phase stretch transform

## 1 Introduction

Wireless sensor network technology has developed rapidly in recent years [1, 2, 3]. It has been widely used in military, intelligent transportation, medical care, and other fields [4, 5]. Visual sensor is a kind of commonly used sensor in wireless sensor networks. Visual sensor is an instrument that uses optical elements to obtain image information of external environment. A visual sensor network is constructed by optical sensor devices distributed in space. Visual sensor networks are very helpful in applications involving environmental monitoring, area detection, and trail tracking. The purpose of computer vision is to extract meaningful descriptive information from images or image sequences. So the images obtained by sensor networks must be clear and easy to be identified. While in the process of acquiring digital images in visual sensor networks, it often leads to blurred images, low contrast, and difficult identification due to some objective factors, such as illumination, camera angles, and imaging characteristics of sensors.

Image enhancement is a crucial pre-treatment step for many image processing applications, because it is an effective way to improve the image quality. Image enhancement refers to the use of certain methods that can highlight the interested features in an image according to some specific needs, meanwhile weakening or removing irrelevant information. By this way, it can transform the image into a more suitable style for human eye recognition and computer perception.

Image enhancement algorithms can be roughly grouped into two kinds: space domain image enhancement and frequency domain image enhancement. Space domain image enhancement algorithm is mainly based on grayscale transformation, histogram processing [6], mean filtering, and other methods. Its basic principle is to adjust the gray value of images in order to achieve local contrast enhancement and noise suppression. Frequency domain image enhancement is also called transform domain image enhancement. Its basic idea is to use multi-resolution analysis tools to modify the transform coefficients at different scales of the image so as to indirectly remove noise and enhance details of some interested features. Nowadays, the transform domain image enhancement algorithm has gradually become a research hotspot. Common multi-scale analysis tools for image enhancement include wavelet transform [7], contourlet transform, and shearlet transform. Literature [8] proposed an algorithm for image enhancement based on non-sub-sampled contourlet transform (NSCT) and adaptive threshold. Literature [9] combined contrast limited adaptive histogram equalization and wavelet transform to enhance the local details of an image. Shearlet transform appeared in recent years. It has good results in image processing applications, such as image fusion, enhancement, and denoising. In our early research, we have applied non-sub-sampled shearlet transform (NSST) to enhance images adaptively and fuse images in compressive domain [10, 11].

Image phase information plays a crucial role in the procedure of human perception of images. Phase information can reflect the space information of images and has a strong ability to interpret images. Phase correlation is often applied to image registration [12, 13]. Literature [14] proposed a phase information-based contrast enhancement scheme that can transform phase changes into amplitude variations so as to reveal the inhibited structural details in high-frequency components.

From the above analysis, it is supposed that if we can combine the accuracy of multi-scale analysis with the stability of phase analysis in the process of image enhancement, we will get a more ideal enhancement performance. In this paper, we propose an image enhancement method which takes the advantages of NSST and phase stretch transform (PST) to nonlinearly enhance the images captured by visual sensors. Experiments show that the enhanced images processed by our algorithm have clear details and contrast without any artifacts.

## 2 Design of the algorithm

### 2.1 NSST multiscale analysis

NSST is a multi-scale and multi-directional analysis tool. It is an optimal approximation, which is a “true” sparse representation of the image in all directions and at various scales. In addition, unlike traditional shearlet transform, NSST does not perform down-sampling when performing multi-scale decomposition of an image. It combines a non-down-sampling tower transform with a non-sub-sampled directional filter bank and transforms. The size of the directional sub-bands of each scale is the same as that of the original image. The transform has shift invariance and can overcome the pseudo-Gibbs phenomenon when the image is reconstructed.

### 2.2 Feature extraction with PST

*n*and

*m*are the space domain variables,

*A*[

*n*,

*m*] is the output phase image,

*B*[

*n*,

*m*] is the input image, FFT2 is the two-dimensional fast Fourier transform, IFFT2 is the two-dimensional inverse fast Fourier transform.

*p*and

*q*are the frequency domain variables. The function \( \tilde{K}\left[p,q\right] \) is the warped phase kernel, and \( \tilde{L}\left[p,q\right] \) is the frequency response of the localization kernel.

### 2.3 Nonlinear enhancement model

*sigm*is defined as\( sigm(x)=\frac{1}{1+{e}^{-x}} \)。.

In this function, there are two parameters of *b* and *c*, where *b* is used to control the enhancement range, and its value is generally between 0 and 1. *c* is used to control the enhancement strength, and it usually uses the fixed value between 20 and 50. However, our algorithm does not pre-set fixed values for *b* and *c*. Instead, we set two thresholds of the function to determine the shape of the function curve, and then obtain the values of *b* and *c*. After the procedure of NSST decomposition, the image has been decomposed into different scales and directions, so we can set various thresholds for different scales of the coefficients and enhance them respectively. Therefore, the enhancement process would be more refined.

### 2.4 Algorithm flow and steps

NSST transform is applied to decompose the original image. After that, the NSST coefficients of different scales and directions can be obtained. Since image detail information often exists in high-frequency coefficients, we choose high scale coefficients for processing, while keep the low frequency coefficients unchanged. The purpose of image enhancement is to enhance detail information and suppress noise. Therefore, we need to use the appropriate method to distinguish high-frequency signals.

_{n}(

*x*,

*y*) is the mean value of the pixels in the local window centered at

*p*(

*x*,

*y*) and the size of the local window is (2

*n*+ 1) ∗ (2

*n*+ 1).

*y*=

*x*. It is the threshold that characterizes the feature coefficients from noise coefficients. It is called as T1. If the abscissa value of a certain point on the function curve is less than T1, then the value of its ordinate will be reduced nonlinearly, that is suppression (applicable to noise information); while if the abscissa value of a certain point on the function curve is greater than T1, the value of its ordinate will be nonlinearly stretched, that is enhancement (applicable to detail information). The other threshold is the first intersection of the enhancement function and the line of

*y*= 1. It is the threshold that determines the extent of enhancement. We call it T2. If the abscissa of a certain point on the enhancement function curve is larger than T2, it is considered as strong edge, so the ordinate value of this part will be set to 1 after enhancement.

*k*1 times of the minimum value of the LSD as T1 and

*k*2 times of the maximum as T2. The values of

*k*1 and

*k*2 depend on the quality of the original image and the decomposition scales. In our algorithm,

*k*1 is an integer proportional to the decomposition level

*n*, and

*k*2 is an integer inversely proportional to the decomposition level, as is shown in (4).

*k*. If the image contains noise, the value of

*k*should be enlarged properly. And smaller

*k*will be more suitable for low contrast image. When the two thresholds are determined, the values of

*b*and

*c*can be obtained by solving the nonlinear equation group (5). Hence, the enhancement function is formed.

After that, if we bring the pixel’s LSD of the PST feature map into the enhancement function, the enhancement ratio at the pixel can be read in the enhancement curve. It is the ratio of the ordinate value to the abscissa value. The NSST coefficients of the each scale can be multiplied with the enhancement ratio, and then the new coefficients after the enhancement processing can be obtained. Finally, the enhanced image is formed by NSST inverse transform.

- 1)
Decompose the original image with NSST. The source image can be decomposed into a low-frequency sub-band and a series of high-frequency sub-bands. In our experiment, the image is decomposed into three scales and the coefficients of the higher two layers will be enhanced.

- 2)
The original image is processed by PST, and the image feature map is obtained.

- 3)
Calculate the LSD of each pixel in the feature map. The value represents the complexity of image detail, and it can be used as the reference of the enhancement extent. That is, the enhancement extent of NSST coefficient is determined according to the LSD of each pixel in the PST feature map.

- 4)
The two thresholds of the nonlinear enhancement function are set up according to the maximum and minimum value of the LSD in the feature map. Different

*k*values are set for different scales of NSST coefficients, and then the enhancement models are constructed for different decomposition scales. - 5)
The LSD of each pixel in the feature map is taken into the enhancement model, and the enhancement ratio of each coefficient can be obtained. Multiply the NSST coefficients of each layer with the enhancement ratios, so as to obtain the enhanced coefficients.

- 6)
The NSST inverse transformation is performed on the enhanced higher two-scale coefficients and the unprocessed coefficients of the first scale to reconstruct the enhanced image.

## 3 Experiments and simulation

### 3.1 Experiment settings

The experiments are carried out on four images. They are standard images Lena and Barbara, image Car shot by optical sensor, and a local screenshot of a radar image. Five representative algorithms including the proposed method are selected to enhance the images respectively. All experiments are carried out on Matlab R2016b platform.

### 3.2 Simulation and evaluation

Generally, it is difficult for human beings to perceive the mild differences among the enhancement results of various algorithms. Hence, the paper adopts two widely used objective evaluation parameters, Edge Preserved Index (EPI) [18] and Contrast Improvement Index (CII) [19] to evaluate the image enhancement effects. EPI measures the edge preserving ability of the enhanced image. CII is used to measure the contrast of the processed image, so as to reflect the details of the image and the enhancement effect. The greater values of EPI and CII indicate the better quality of the image.

## 4 Results and discussion

By subjective evaluation, it can be seen that our algorithm has significant effects on detail and edge enhancement. The enhanced images have no pseudo-Gibbs phenomenon, and the details are clear without obvious distortion. It has better enhancement effects on images with complex details.

Objective evaluation parameters for Lena image set

Lena | DSBP | Fuzzy_INT | NSCT | NSST | Proposed |
---|---|---|---|---|---|

EPI | 1.1704 | 1.0968 | 2.0942 | 2.3440 | 2.5283 |

CII | 1.0688 | 1.0777 | 1.0639 | 1.0734 | 1.0823 |

Objective evaluation parameters for Barbara image set

Barbara | DSBP | Fuzzy_INT | NSCT | NSST | Proposed |
---|---|---|---|---|---|

EPI | 1.0103 | 1.0033 | 1.8084 | 2.0732 | 3.1209 |

CII | 1.0909 | 1.0978 | 1.1463 | 1.1961 | 1.3666 |

Objective evaluation parameters for car image set

Car | DSBP | Fuzzy_INT | NSCT | NSST | Proposed |
---|---|---|---|---|---|

EPI | 1.0041 | 1.0133 | 1.1538 | 2.1276 | 2.6462 |

CII | 1.0178 | 1.0021 | 1.0956 | 1.0959 | 1.2940 |

Objective evaluation parameters for the SAR image set

Radar | DSBP | Fuzzy_INT | NSCT | NSST | Proposed |
---|---|---|---|---|---|

EPI | 1.0152 | 1.0183 | 1.6543 | 1.8928 | 2.4206 |

CII | 1.0149 | 1.0170 | 1.2109 | 1.2299 | 1.3678 |

The proposed algorithm is suitable for the enhancement of optical images captured by visual sensors. And it is also applicable for radar image enhancement. We have tried to extend our method to improve images captured by infrared sensors, but we have not achieved good performance until now. Our algorithm depends largely on the detail features of images, so it is less effective in infrared images enhancement which are not rich in detail textures.

## 5 Conclusion

This paper describes an approach for accomplishing an image enhancement algorithm, which can improve the clarity of the gray images captured by visual sensors. The proposed method combines NSST and PST and takes the advantages of multi-scale analysis and image phase analysis to perform nonlinear enhancement of images. The algorithm can enhance image details and edge features without amplifying noise. And there are no ringing artifacts near edges. Experiments on Matlab platform prove that the algorithm is better than the other algorithms in terms of accurate edge preservation and adequate contrast enhancement.

## Notes

### Acknowledgements

The author would like to thank Tianjin Key Laboratory of Wireless Mobile Communications and Power Transmission for the support.

### Funding

This work has been partly supported by National Natural Science Foundation of China (61701344), Tianjin Normal University Application Development Foundation (52XK1601), Tianjin Normal University Doctoral Foundation (52XB1603, 52XB1713) and Tianjin Higher Education Creative Team Funds Program in China.

### Availability of data and materials

The datasets supporting the conclusions of this article are included within the article (and its additional file(s)).

### Author’s contributions

YT conceived and designed the study. The author read and approved the final manuscript.

### Author’s information

Ying Tong was born in Tianjin, China, in 1982. She received the B.S. and M.S. degree from Tianjin Normal University, the Ph. D degree from Tianjin University in 2004, 2007 and 2015 respectively. Since 2007, she has been working at Tianjin Normal University in China. She is a lecturer of Tianjin Key Laboratory of Wireless Mobile Communications and Power Transmission. Her research interests include computer vision and digital signal processing.

### Competing interests

The author declares that she has no competing interests.

### Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## References

- 1.W. Wang, Y. Jiang, D. Wang, Through wall human detection based on stacked denoising autoencoder algorithm. J. Tianjin Normal Univ. (Natural Sci. Ed.)
**37**(5), 50–54 (2017)MathSciNetGoogle Scholar - 2.S.L. Ma, Y. Wang, Design and implementation of an ultra-wideband high-accuracy ranging system. J. Tianjin Normal Univ. (Natural Sci. Ed.)
**37**(6), 55–57 (2017)Google Scholar - 3.J.C. Wu, C.C. Zeng, J.R. Sun, Research and application of wireless intelligent network monitoring smog system based on STM32F407. J. Tianjin Normal Univ. (Natural Sci. Ed.)
**37**(6), 62–66 (2017)Google Scholar - 4.Z. Liu, J. Chen, Y. Tong, et al., Research and implementation of digital baseband signal transmission. Journal of Tianjin Normal University (Natural Science Edition)
**38**(1), 51–55 (2018)Google Scholar - 5.M.Z. Shi, X.W. Gong, Parameters identification method via cepstrum analysis for mix blurred image restoration. J. Tianjin Normal Univ. (Natural Sci. Ed)
**37**(5), 60–65 (2017)Google Scholar - 6.S.Y. Lee, D. Kim, C. Kim, Ramp distribution-based image enhancement techniques for infrared images. IEEE Signal Process. Lett.
**25**(7), 931–935 (2018)CrossRefGoogle Scholar - 7.K. Kim, S. Kim, K.S. Kim, Effective image enhancement techniques for fog-affected indoor and outdoor images. IET Image Process.
**12**(4), 465–471 (2018)CrossRefGoogle Scholar - 8.L. Dong, Y. Bing, Y. Mei, et al., Image enhancement based on the nonsubsampled Contourlet transform and adaptive threshold. Acta Electron. Sin.
**36**(3), 527–530 (2008)Google Scholar - 9.L. Huang, W. Zhao, J. Wang, et al., Combination of contrast limited adaptive histogram equalisation and discrete wavelet transform for image enhancement. Image Process. IET
**9**(10), 908–915 (2015)CrossRefGoogle Scholar - 10.Y. Tong, J. Chen, Nonlinear adaptive image enhancement in wireless sensor networks based on non-sub-sampled shearlet transform. EURASIP J. Wirel. Commun. Netw.
**2017**(1), 46 (2017)CrossRefGoogle Scholar - 11.Y. Tong, J. Chen, Compressive sensing image fusion in heterogeneous sensor networks based on Shearlet and wavelet transform. EURASIP J. Wirel. Commun. Netw.
**2017**(1), 52 (2017)CrossRefGoogle Scholar - 12.Y. Dong, T. Long, W. Jiao, et al., A novel image registration method based on phase correlation using low-rank matrix factorization with mixture of Gaussian. IEEE Trans. Geosci. Remote Sens. (Basel)
**56**(1), 446–460 (2017)CrossRefGoogle Scholar - 13.J. Fan, Y. Wu, F. Wang, et al., SAR image registration using phase congruency and nonlinear diffusion-based SIFT. IEEE Geosci. Remote Sens. Lett.
**12**(3), 562–566 (2014)Google Scholar - 14.S. Cakir, D.C. Kahraman, R. Cetin-Atalay, et al., Contrast enhancement of microscopy images using image phase information. IEEE Access
**2018**(6), 3839–3850 (2018)CrossRefGoogle Scholar - 15.H. Asghari, B. Jalali, Edge detection in digital images using dispersive phase stretch transform. Int J Biomed Imaging
**2015**, 687819 (2015)CrossRefGoogle Scholar - 16.M.A. Rahman, S. Liu, S.C.F. Lin, C.Y. Wong, G. Jiang, N. Kwok, Image contrast enhancement for brightness preservation based on dynamic stretching. Intl J. Image Proc.
**9**(4), 241–253 (2015)Google Scholar - 17.Z. Al-Ameen, Visibility enhancement for images captured in dusty weather via tuned tri-threshold fuzzy intensification operators. Int. J. Intelligent Syst Appl.
**8**(8), 10–17 (2016)Google Scholar - 18.F. Sattar, L. Floreby, G. Salomonsson, B. Lovstrom, et al., Image enhancement based on a nonlinear multiscale method. IEEE Trans. Image Process.
**6**(6), 888–895 (1997)CrossRefGoogle Scholar - 19.B.S. Khehra, A.P.S. Pharwaha, Digital mammogram enhancement using Kapur measure of entropy and mathematical morphology. Biomed Eng. Appl. Basis Commun.
**25**(3), 1350029 (2013)CrossRefGoogle Scholar

## Copyright information

**Open Access**This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.