Mutual-Structure for Joint Filtering

Abstract

Previous joint/guided filters directly transfer structural information from the reference to the target image. In this paper, we analyze the major drawback—that is, there may be completely different edges in the two images. Simply considering all patterns could introduce significant errors. To address this issue, we propose the concept of mutual-structure, which refers to the structural information that is contained in both images and thus can be safely enhanced by joint filtering. We also use an untraditional objective function that can be efficiently optimized to yield mutual structure. Our method results in important edge preserving property, which greatly benefits depth completion, optical flow estimation, image enhancement, stereo matching, to name a few.

Appendix: Proof of Claim 4.1

Proof

In Eq. (5), $e(I_p,G_p)$ reaches the minimum when setting the derivatives $\frac{\partial e(I_p,G_p)^2}{\partial a_p^1}$ and $\frac{\partial e(I_p,G_p)^2}{\partial a_p^0}$ to zeros, which yields

$$\begin{aligned} a_p^1 = \frac{cov(I_p,G_p)}{\sigma (I_p)}, \quad a_p^0=\overline{G}_p-a_p^1\overline{I}_p, \end{aligned}$$

(26)

where $\overline{I}_p$ and $\overline{G}_p$ are the mean intensities of patches centered at p on I and G respectively. By simply substituting $a_p^1$ and $a_p^0$ into Eq. (5) and arranging it according to Eq. (1), we obtain

$$\begin{aligned} e(I_p,G_p) = \sigma (G_p)(1-\rho (I_p,G_p)^2), \end{aligned}$$

(27)

as shown in Eq. (6). $\square$