# Single image dehazing via an improved atmospheric scattering model

- 3.1k Downloads
- 8 Citations

## Abstract

Under foggy or hazy weather conditions, the visibility and color fidelity of outdoor images are prone to degradation. Hazy images can be the cause of serious errors in many computer vision systems. Consequently, image haze removal has practical significance for real-world applications. In this study, we first analyze the inherent weaknesses of the atmospheric scattering model and propose an improvement to address those weaknesses. Then, we present a fast image haze removal algorithm based on the improved model. In our proposed method, the input image is partitioned into several scenes based on the haze thickness. Next, averaging and erosion operations calculate the rough scene luminance map in a scene-wise manner. We obtain the rough scene transmission map by maximizing the contrast in each scene and then develop a way to gently remove the haze using an adaptive method for adjusting scene transmission based on scene features. In addition, we propose a guided total variation model for edge optimization, so as to prevent from the block effect as well as to eliminate the negative effect from the wrong scene segmentation results. The experimental results demonstrate that our method is effective in solving a series of common problems, including uneven illuminance, overenhanced and oversaturated images, and so forth. Moreover, our method outperforms most current dehazing algorithms in terms of visual effects, universality, and processing speed.

## Keywords

Atmospheric scattering model Single image dehazing Scene segmentation Guided total variation model## 1 Introduction

Often, when outdoor images are acquired under poor weather conditions, such as haze and fog, the visibility of the captured scene is prone to significant degradation (see Fig. 1a). Narasimhan [1] exploited the interactions of light with particles suspended in the atmosphere (scattering, absorption, and emission) that result in reduced contrast, faded colors, and low saturation. Many computer vision applications rely on the assumption that the input image is haze free; consequently, degraded images may cause catastrophic errors. Hence, research on image dehazing is of practical significance, and the search for effective haze removal methods has attracted increased attention in recent years.

Early studies adopted image enhancement techniques to increase the visibility of hazy images. Among these, the Retinex [2] and Choi [3] image processing techniques are typical examples. However, because these techniques do not take the spatial distribution of haze into account and because they ignore the fact that the haze thickness is dependent on the scene depth, their dehazing effect is not visually compelling.

Therefore, subsequent research work focused mainly on haze removal based on the atmospheric scattering model, which has proved to be more attractive than using traditional image enhancement techniques. When using an atmospheric scattering model, it is critical to estimate the scene depth accurately. The literature [4, 5, 6] proposed using multiple images or external information to derive the scene depth map; however, this requirement is difficult to fulfill in many real-world applications.

More recently, single-image haze removal methods have attracted the most research attention—and remarkable progress has been made. Generally, these methods take advantage of strong prior knowledge or assumptions to produce the depth map. For example, by assuming that clear images possess higher local contrast than hazy ones, Tan [7] proposed deriving the transmission map based on a Markov random field (MRF) model and removing haze by maximizing the local contrast. However, Tan’s results tend to be oversaturated because, in spirit, it is similar to contrast stretching. Nishino [8] exploited the statistical properties hidden in images by adopting the Bayesian posterior probability model to remove haze. The results show its superiority for heavily hazed images, but after restoration, the image color tends to be overenhanced for misty images. Fattal [9] assumed that transmission and surface shading are locally uncorrelated and removed haze on the basis of color statistics. However, Fattal’s algorithm does not work for heavily hazed images. Tarel [10] used the median filter to estimate the dissipation function. However, because the median filter shows poor edge preserving performance, the method left small amounts of mist around depth changes in the dehazed image. To solve this problem, Xiao [11] proposed a guided joint bilateral filter for haze removal. Meng [12] estimated a rough transmission map using a boundary constraint and proposed a regularization method to blur the map. Although this method is fast, it tends to distort the color fidelity when dealing with white objects. He [13] obtained a rough estimate of transmission via dark channel prior and adopted soft matting for transmission refinement. Although the dehazing results are almost perfect in their visual effect, He’s algorithm is not applicable for real-time systems, because the soft matting operation incurs expensive computation and memory consumption overhead. To solve this problem, He [14] replaced the soft matting with a guided filter, which proved to be more efficient, but only at the cost of degrading the visual effect. Gibson [15] presented the median dark channel prior method based on [13], which accelerates the haze removal process to some extent, because it requires no refinement of the transmission map. Nevertheless, this method fails to achieve good visual results. In particular, it is prone to leaving dark spots in the dehazed image. Li [16] exploited the detail change prior to estimate the airlight. However, because the result contained excessive texture details, this form of haze removal is unsatisfactory. Zhu [17] created a linear model-to-model scene depth under the color attenuation prior and learned the parameters of the model with a supervised learning method. However, because the scattering coefficient in the atmospheric scattering model cannot actually be regarded as a constant, Zhu’s method proved to be unstable in its haze removal performance.

## 2 Analysis of and improvement on the atmospheric scattering model

*I*is the observed image, \(\rho \) is the scene reflectance,

*A*is the global atmospheric light (regarded as constant in the input image),

*t*denotes the transmission and—if we assume that the haze is homogenous—

*t*can be expressed as follows:

*d*represents the scene depth. Evidently, it is an ill-posed problem to estimate

*A*and

*t*from a single input image. In recent years, many studies have exploited stronger priors or used assumptions as constraints to solve this challenging problem. Although significant progress has been made, the visual effect after restoration is still less than satisfactory.

Figure 2 shows various restoration results under different global atmospheric light levels. Clearly, for a smaller values of *A*, the local contrast in the dark tree shadow is enhanced, but a large number of the detail structures are lost in the bright region. As the Retinex theory states [2], scene reflectance is an intrinsic feature of objects, and it is independent of the incident light. One problem is that we cannot recover the ideal scene reflectance regardless of the value of *A*, which is a result of the assumption that the atmospheric light level is constant in Eq. 1. However, that assumption is not always true in the real world. The intensity of atmospheric light may vary among different regions. As shown in Fig. 2a, the light intensity tends to be zero in the shadowed area, but it approaches one at the distant horizon. Thus, assuming that *A* is constant has obvious limitations. In addition, the estimation of the transmission map involves considerable redundant computation, because the transmission map is estimated in a pixelwise manner, while in reality, the depth changes relatively smoothly in the same scene.

*t*, and the atmospheric light,

*A*, can be redefined as the scene transmission map,

*T*, and the scene luminance map,

*L*, respectively. Therefore, the redefined model can be expressed as follows:

*L*(

*i*),

*T*(

*i*) refer to the scene luminance and scene transmission that are constant in the \(i{\mathrm{th}}\) scene, respectively. The redefined model significantly simplifies the estimation of transmission, because the scene luminance and scene transmission need to be estimated for only a limited number of scenes (see Fig. 3).

## 3 A single image dehazing method

From the redefined model in Sect. 2, it can be inferred that all the scenes in the input image should be recognized first. Then, the scene luminance and scene transmission need to be estimated for each scene separately based on the results of scene segmentation. To eliminate negative effects caused by this scene-wise operation, it is also necessary to refine the scene transmission map and scene luminance map with the goal of preserving the essential depth structure while achieving local smoothness. Figure 4 shows a flowchart of our method.

### 3.1 Scene segmentation

*i*denotes the patch index, and \(\varphi \) and \(\phi \) are the mean and standard deviation functions, respectively. The haze distribution map

*V*is constructed after all the patches have been traversed in the hazy image.

*V*is divided into

*k*scenes, the pixel (

*x*,

*y*) belongs to the following scene:

*C*is the scene segmentation map, \(V_{\mathrm{sort}}\) is a vector in an ascending order of haze thickness coefficients of pixels, and

*l*denotes the image resolution.

*k*values result in more elaborate scene segmentation results; however, they may also cause the subsequent estimation procedure to be more complicated. We set

*k*to 15 throughout our experiments by taking both computational complexity and partition accuracy into account.

### 3.2 The rough estimate of scene luminance

### 3.3 The rough estimate of scene transmission

*V*can reflect the haze thickness. In the same manner, we are able to measure the haze thickness of a scene by averaging

*V*(

*x*,

*y*) of all the pixels in a scene, as follows:

### 3.4 Edge optimization based on a guided total variation model

As described in Sect. 3.1, scene partition is inherently a patchwise process that will blur the edges in the scene transmission map \((\tilde{T})\) as well as in the three scene luminance maps \((L^{R},\, L^{G},\, L^{B})\). Thus, it will thus produce halo artifacts in the dehazing result. At the same time, the accuracy of the estimates for scene transmission and scene luminance may suffer from erroneous scene segmentation. Moreover, both the scene transmission map and the scene luminance map should possess the characteristic of local spatial smoothing, because excessive texture details may have a negative impact on the dehazing effect [11]. Intuitively, adopting a filter with a guiding function is a good choice for solving this problem [14, 22]. Such filters include the joint bilateral filter, guided filter, etc. However, these methods are extremely sensitive to parameter values and different parameter selections can greatly affect the filtering results.

*G*. The weight

*W*is defined as:

*W*increases as the gradient increases. This means that the importance of the second term is reduced, but the third term becomes more important, thus achieving the goal of both blurring the texture details and preserving the edges around areas with sudden depth changes.

*G*. According to the Euler–Lagrange equation, Eq. (17) satisfies:

*T*0 refine to \(T_{\mathrm{refine}}^0 =\tilde{T}\). The guiding image,

*G*, is the gray component of the hazy input image. After repeated testing, we have found that this approach can achieve good dehazing results when \({\alpha }=3,\, {\beta }=3\cdot \) (Iter\(-1\)), \({\gamma }=4\), and \(\hbox {r}=\hbox {max}(l_{h},l_{w})/15\), where \((l_{h}, l_{w})\) are the height and width of the image, respectively. As Fig. 10 shows, the GTV model can achieve fast convergence. Moreover, after only a few iterations, the output scene transmission map is effective at highlighting the depth structure while blurring a large amount of the texture details (in fact, the outcome of the \(1{\mathrm{st}}\) iteration is capable of preserving the depth details consistently to the original hazy image. As the subsequent iterations proceed, the texture details become increasingly blurred). The three scene luminance maps \(L^{R}\), \(L^{G}\), and \(L^{B}\) can be refined in the same way.

### 3.5 Image restoration

*Lc*refine are known, we can derive the scene reflectance \(\rho \) by Eq. 3. For convenience, we rewrite Eq. 3 as follows:

## 4 Experiments

In this section, we compare the quality of image haze removal using our proposed method with other typical dehazing algorithms. In the following experiments, our algorithms are implemented in MATLAB on a computer with an Intel (R) Core(Tm) i5-4210U CPU and 8.00 GB of RAM. All the parameters of our proposed method are set as described in Sect. 3.

### 4.1 The visual effect

### 4.2 Comprehensive comparison

Next, we show the haze removal results of both our method and several other representative algorithms. (The test images in Fig. 14 were downloaded from the Internet, and the test images in Figs. 12, 13, and 15 originate from Fattal’s website: http://www.cs.huji.ac.il/~raananf/). Figure 12 shows the results obtained by Tan [7], Kratz [26] and our method, respectively. As Fig. 12 shows, both algorithms proposed by Tan and Kratz work well for contrast enhancement but result in oversaturated color and halo effects near areas with discontinuous depths. Comparatively, our algorithm performs better in terms of color fidelity, and the visual results seem more natural.

From left to right, the panels in Fig. 13 show the input image, the results obtained by Choi [3], Kopf [5], Fattal [9], and our method, respectively. Clearly, the results of all those algorithms except ours lose some information in the sky region, causing an unsatisfactory visual effect. Our method performs well in preserving the information in the sky region, and exhibits clearer visibility after restoration.

In Fig. 14, we compare our method with the algorithms presented by Taral [10], Meng [12], He [13], Gibson [15], Zhu [17], and Qi [27]. Obviously, the sky color is overenhanced in the results of Taral, Meng, He, Gibson and Qi, while it is not in our method and Zhu’s; however, our method outperforms Zhu’s algorithm in the dehazing visual effect.

### 4.3 The objective assessment

*e*recommended by [28] and the structure similarity

*f*proposed by Wang [29] to assess our approach quantitatively. The measures

*e*and

*f*are defined as follows:

*I*and the restored image

*R*, respectively, \(\mu _{\hat{{B}}_i}\) and \(\sigma _{\hat{{B}}_i}^2\) denote the mean and the variance of \(\hat{{B}}_i\), respectively, \(\mu _{\tilde{B}_i}\) and \(\sigma _{\tilde{B}_i}^2\) denote the mean and the variance of \(\tilde{B}_i\), respectively, and \(\sigma _{\hat{{B}}_i \tilde{B}_i}\) is the covariance between \(\hat{{B}}_i\) and \(\tilde{B}_i \). The constants \(c_{1}\) and \(c_{2}\) are included to avoid instability.

For the sake of fairness, we test the most up-to-date dehazing algorithms on two benchmark images from Fattal’s website. The visual results for all the algorithms are shown in Fig. 16a, c; the corresponding quantitative comparison results are listed in Table 1. As Table 1 shows, Tarel [10] achieves the maximum *e* value, followed by Meng [12], Gibson [15], and He [13]. However, this does not mean these algorithms are superior to our method, because the number of visible edges can increase when excessive dehazing leads to noise amplification in the image. This problem can be solved by the approaches described in [30, 31]. The *f* values listed in Table 1 demonstrate that our method achieves the maximum similarity in structure, which indicates that the depth structure of our result conforms better to the original image.

The objective assessment

Image | Index | Choi | Kopf | Tan | Nishino | Tarel | Meng | He | Gibson | Zhu | Fattal | Our |
---|---|---|---|---|---|---|---|---|---|---|---|---|

y16 | | 0.1527 | 0.0036 | \(-\)0.0164 | 0.0777 | | 0.0951 | 0.1432 | 0.0853 | 0.1104 | \(-\)0.0750 | 0.1127 |

| 0.6828 | 0.9279 | 0.7452 | 0.7928 | 0.8043 | 0.8335 | 0.9016 | 0.6648 | 0.8238 | 0.8761 | | |

ny17 | | \(-\)0.0004 | 0.0285 | \(-\)0.0411 | \(-\)0.0716 | | 0.2204 | 0.0427 | 0.2075 | 0.0309 | \(-\)0.1129 | 0.0082 |

| 0.6002 | 0.8772 | 0.7379 | 0.7813 | 0.8055 | 0.7841 | 0.8488 | 0.6178 | 0.8285 | 0.7877 | |

Processing speed is also important when evaluating algorithmic performance. Algorithms such as Tan [7], Nishino [8] and He [13] involve complex operations (e.g., MRF or soft matting) that greatly reduce the speed of haze removal. Therefore, we compare our method only with algorithms with less complexity. Figure 17 shows the curve for the computation time consumed in processing images under different resolutions. As the results show, our method is only slightly slower than Gibson [15], and it is faster than the others. From aspects of both dehazing effect and computational efficiency, our proposed algorithm is better suited for dehazing applications.

### 4.4 Situations not suited to our method

### 4.5 Conclusion and future work

In this study, we propose a single image haze removal approach based on an improved atmospheric scattering model. We first analyze the weaknesses of the atmospheric scattering model and propose an improvement to it. Then, based on the improved model as the starting point, we develop methods to automatically partition scenes and perform estimates of the scene luminance and scene transmission maps in a scene-wise manner. Finally, we present a GTV model to achieve edge optimization. The experimental results demonstrate that our approach outperforms most up-to-date algorithms in terms of both visual effect and processing speed.

It is possible to further accelerate the procedure of haze removal; for example, we can reduce the number of scenes segmented for images with smooth changes in depth. Therefore, our future work will focus on the following two aspects: (1) adaptively setting the number of scenes based on the features of the image, and (2) investigating improvements to the atmospheric scattering model, which we expect to be more applicable in inhomogeneous atmospheric conditions.

## Notes

### Acknowledgments

The authors wish to thank Dr. Pengfei Wu and Dr. Zhenfei Gu for their help with proofreading. We would also like to thank the reviewers for their valuable comments. This work is supported by the National Natural Science Foundations of P. R. China (Grant No. 61571241), the Jiangsu Province Graduate Research and Innovation Project (Grant No. CXZZ130476), and the Science Research Fund of NUPT (Grant No. NY215169).

## References

- 1.Narasimhan, S.G., Nayar, S.K.: Vision and the atmosphere. Int. J. Comput. Vis.
**48**, 233–254 (2002)CrossRefzbMATHGoogle Scholar - 2.Land, E.H., Mccann, J.J.: Lightness and retinex theory. J. Opt. Soc. Am.
**61**, 1–11 (1971)CrossRefGoogle Scholar - 3.Choi, L.K., You, J., Bovik, A.C.: Referenceless prediction of perceptual fog density and perceptual image defogging. IEEE Trans. Image Process.
**24**, 3888–3901 (2015)CrossRefMathSciNetGoogle Scholar - 4.Shwartz, S., Namer, E., Schechner, Y.Y.: Blind haze separation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 1984–1991. IEEE Computer Society, New York (2006)Google Scholar
- 5.Kopf, J., Neubert, B., Chen, B., Cohen, M., Cohen-Or, D., Deussen, O., Uyttendaele, M., Lischinski, D.: Deep photo. ACM Trans. Graph.
**27**, 1–9 (2008)CrossRefGoogle Scholar - 6.Schechner, Y.Y., Narasimhan, S.G., Nayar, S.K.: Polarization-based vision through haze. Appl. Opt.
**42**, 511–525 (2003)CrossRefGoogle Scholar - 7.Tan, R.T.: Visibility in bad weather from a single image. In: IEEE Conference on Computer Vision and Pattern Recognition 2008, pp. 1–8. IEEE, Piscataway (2008)Google Scholar
- 8.Nishino, K., Kratz, L., Lombardi, S.: Bayesian defogging. Int. J. Comput. Vis.
**98**, 263–278 (2012)CrossRefMathSciNetGoogle Scholar - 9.Fattal, R.: Single image dehazing. ACM Trans. Graph.
**27**, 72 (2008)CrossRefGoogle Scholar - 10.Tarel, J.P., Hautiere, N.: Fast visibility restoration from a single color or gray level image. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 2201–2208. IEEE, Piscataway (2009)Google Scholar
- 11.Xiao, C., Gan, J.: Fast image dehazing using guided joint bilateral filter. Vis. Comput.
**28**, 713–721 (2012)CrossRefGoogle Scholar - 12.Meng, G., Wang, Y., Duan, J., Xiang, S. Pan, C.: Efficient image dehazing with boundary constraint and contextual regularization. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp. 617–624. IEEE, Piscataway (2013)Google Scholar
- 13.He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell.
**33**, 2341–2353 (2011)CrossRefGoogle Scholar - 14.He, K., Jian, S., Tang, X.: Guided image filtering. IEEE Trans. Pattern Anal. Mach. Intell.
**35**, 1397–1409 (2013)CrossRefGoogle Scholar - 15.Gibson, K.B., Vo, D.T., Nguyen, T.Q.: An investigation of dehazing effects on image and video coding. IEEE Trans. Image Process.
**21**, 662–673 (2011)CrossRefzbMATHMathSciNetGoogle Scholar - 16.Li, J., Zhang, H., Yuan, D., Sun, M.: Single image dehazing using the change of detail prior. Neurocomputing
**156**, 1–11 (2015)CrossRefGoogle Scholar - 17.Zhu, Q., Mai, J., Shao, L.: A fast single image haze removal algorithm using color attenuation prior. IEEE Trans. Image Process.
**24**, 3522–3533 (2015)Google Scholar - 18.Li, B., Wang, S., Zheng, J., Zheng, L.: Single image haze removal using content-adaptive dark channel and post enhancement. IET Comput. Vis.
**8**, 131–140 (2013)CrossRefGoogle Scholar - 19.Ding, K., Chen, W., Wu, X.: Optimum inpainting for depth map based on L0 total variation. Vis. Comput.
**30**, 1311–1320 (2013)CrossRefGoogle Scholar - 20.Liu, X., Zeng, F., Huang, Z. Ji, Y.: Single color image dehazing based on digital total variation filter with color transfer. In: IEEE International Conference on Image Processing, pp. 909–913. IEEE, Piscataway (2013)Google Scholar
- 21.Qian, L., Chen, M.Y., Zhou, D.H.: Single image haze removal via depth-based contrast stretching transform. Sci. China Inf. Sci.
**58**, 1–17 (2015)CrossRefGoogle Scholar - 22.Liu, C., Zhao, J., Shen, Y., Zhou, Y., Wang, X., Ouyang, Y.: Texture filtering based physically plausible image dehazing. Vis. Comput.
**32**, 911–920 (2016)CrossRefGoogle Scholar - 23.Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D Nonlinear Phenom.
**60**, 259–268 (1992)CrossRefzbMATHMathSciNetGoogle Scholar - 24.Dong, N.A.N., Du-yan, B.I., Shi-ping, M.A., Lin-yuan, H.E., Xiao-long, L.O.U.: Single image dehazing method based on scene depth constraint. Chin. J. Electron.
**43**, 500–504 (2015)Google Scholar - 25.Fattal, R.: Dehazing using color-lines. ACM Trans. Graph.
**34**, 1–14 (2014)CrossRefGoogle Scholar - 26.Kratz, L., Nishino, K.: Factorizing scene albedo and depth from a single foggy image. In: IEEE International Conference on Computer Vision, pp. 1701–1708 (2009)Google Scholar
- 27.Qi, M., Hao, Q., Guan, Q., Kong, J., Zhang, Y.: Image dehazing based on structure preserving. Optik Int. J. Light Electron Opt.
**27**, 21153–21160 (2015)Google Scholar - 28.Hautiere, N., Tarel, J.P., Aubert, D., Dumont, E.: Blind contrast enhancement assessment by gradient ratioing at visible edges. Image Anal. Stereol.
**27**, 87–95 (2008)Google Scholar - 29.Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process.
**13**, 600–612 (2004)CrossRefGoogle Scholar - 30.Khmag, A., Ramli, A.R., bin Hashim, S.J., Al-Haddad, S.A.R.: Additive noise reduction in natural images using second-generation wavelet transform hidden Markov models. IEEE J. Trans. Electr. Electron. Eng.
**11**, 339–347 (2016)Google Scholar - 31.Shao, L., Yan, R., Li, X., Liu, Y.: From heuristic optimization to dictionary learning: a review and comprehensive comparison of image denoising algorithms. IEEE Trans. Cybern.
**44**, 1001–1013 (2014)CrossRefGoogle Scholar - 32.Aydin, T.O., Mantiuk, R., Myszkowski, K., Seidel, H.P.: Dynamic range independentimage quality assessment. ACM. Trans. Graph.
**27**, 1–69 (2008)CrossRefGoogle Scholar - 33.Shi, Z., Long, J., Tang, W., Zhang, C.: Single image dehazing in inhomogeneous atmosphere. Optik Int. J. Light Electron Opt.
**125**, 3868–3875 (2014)CrossRefGoogle Scholar

## Copyright information

**Open Access**This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.