Windsurf: the best way to SURF

Abstract

Despite their popularity, approaches based on salient point descriptors have yet to be proven effective for content-based image retrieval. In this paper, we show how the Windsurf library can be effectively exploited to assess a fair comparison among the existing alternative approaches based on salient points, which can be contrasted on aspects of both effectiveness and efficiency. Our extensive experimental evaluation, performed on four different image benchmarks, indeed, shows that techniques based on salient point descriptors have effectiveness not better than other existing techniques and are less amenable to be indexed, and thus, their efficiency remains questionable.

Appendices

Appendix A: The Windsurf system

This appendix provides a brief description of Windsurf; additional details can be found in [4]. Windsurf fragments each image (document) into a set of regions (elements) whose pixels are similar in color and texture. To this end, each image is analyzed using (the third level of the LL sub-band of) a 2-D Haar Discrete Wavelet Transform (DWT) in the HSV color space. The K-means clustering algorithm is then used to group wavelet coefficients (these are compared using their Mahalanobis distance). To discover the “right” number of regions for each image, the K-means algorithm is iterated with $K \in [2,10]$, and the “optimal” K value is chosen as the one that minimizes a validity function. Each of the so-obtained clusters corresponds to an image region $R_i$, that is represented, for each of the 4 sub-bands of the third level DWT, by the centroid and the covariance matrix of the wavelet coefficients in the corresponding cluster: for band $B$, these are denoted, respectively, as $\mu _{R_i}^B$ and $\mathbf {C}_{R_i}^{B}$. Intuitively, the centroid summarizes color information, while the covariance matrix captures the texture of pixels in the cluster.

To compare two regions (elements) $R_i$ and $R_j$, the element distance function $\delta$ takes into account differences between wavelet descriptors over the four frequency sub-bands:

$$\begin{aligned} \delta \left( R_i,R_j \right) = \sqrt{\frac{1}{4} \sum _{B=1}^4 \delta _{B} \left( R_i,R_j \right) ^2}. \end{aligned}$$

(6)

The distance $\delta _{B} \left( R_i,R_j \right)$ on each band $B$ is computed as the Bhattacharyya metric coefficient [25]:

$$\begin{aligned} \delta _{B} \left( R_i,R_j \right) ^2 = 1 - e^{-\rho _{B}\left( R_i,R_j \right) }, \end{aligned}$$

(7)

where $\rho _{B}\left( R_i,R_j \right)$ (the Bhattacharyya distance) is expressed as follows:

$$\rho _{B}\left( R_i,R_j \right) = \frac{1}{2} \ln \left( \frac{ \left| \frac{\mathbf {C}_{R_i}^{B} + \mathbf {C}_{R_j}^{B}}{2} \right| }{\left| \mathbf {C}_{R_i}^{B} \right| ^{\frac{1}{2}}\cdot \left| \mathbf {C}_{R_j}^{B} \right| ^{\frac{1}{2}}}\right)+\frac{1}{8} \left[ \left( \mu _{R_i}^B- \mu _{R_j}^B\right) ^{T} \left( \frac{\mathbf {C}_{R_i}^{B} + \mathbf {C}_{R_j}^{B}}{2} \right) ^{-1} \left( \mu _{R_i}^B- \mu _{R_j}^B\right) \right]$$

(8)

in which $|\mathbf {A}|$ denotes the determinant of matrix $\mathbf {A}$. The definition of $\rho _{B}\left( R_i,R_j \right)$ consists of two terms: the first one is used to compare the covariance matrices of the two regions, while the second term is the Mahalanobis distance between regions centroids, where an average covariance matrix is used. Note that if the regions have the same centroid, the second term vanishes, and thus, the first term measures how similar the two 3-D ellipsoids are (this is the case of regions with similar colors but different texture).

We finally note that the element distance $\delta$ defined in Eq. 6 is a metric, since the Bhattacharyya metric coefficient (Eq. 7) is a metric [25] and the square root of a (weighted) sum of squared metrics is also a metric.

Appendix B: Distance distribution of the Hausdorff metric

This appendix shows how it is possible to compute the distance distribution for the Hausdorff metric ${d}_H$, given the distance distribution for the ground distance $\delta$ (see Eq. 1).

Formally, given a distance function $\delta$ over a domain $\Omega$, its distance distribution F(x) equals the probability that the distance between two random points $\mathbf {p}$ and $\mathbf {q}$ drawn from $\Omega$ is not higher than x, $F(x) = \Pr \{ \delta (\mathbf {p}, \mathbf {q}) \le x \}$ [16].

Given the ground distance $\delta$ with its distance distribution F(x), we now want to compute the distance distribution G(x) of ${d}_H$, supposing that the two sets, $\mathbf {S}_1$ and $\mathbf {S}_2$, contain random points drawn from $\Omega$ with the same distribution (so that the distance between any point of $\mathbf {S}_1$ and any point of $\mathbf {S}_2$ is distributed according to F(x)).

We first assume that the two sets have different cardinality, $N_1$ and $N_2$. Given any point $\mathbf {p}_i$ in $\mathbf {S}_1$, the distribution of distance to its NN in $\mathbf {S}_2$ can be computed as follows [16]:

$$\begin{aligned} \Pr \{ \min _j \delta (\mathbf {p}_i, \mathbf {q}_j) \le x \} = 1 - \left( 1 - F(x) \right) ^{N_2}. \end{aligned}$$

The (asymmetric) distance from $\mathbf {S}_1$ to $\mathbf {S}_2$ is computed as the maximum of NN distances for all points in $\mathbf {S}_1$, whose distribution is:

$$\begin{aligned} \Pr \{ \max _i \min _j \delta (\mathbf {p}_i, \mathbf {q}_j) \le x \} = \left( 1 - \left( 1 - F(x) \right) ^{N_2} \right) ^{N_1}. \end{aligned}$$

The same can be computed for the opposite distance obtaining:

$$\Pr \{ {d}_H (\mathbf {S}_1, \mathbf {S}_2) \le x \} = \min\left\{ \left( 1 - \left( 1 - F(x) \right) ^{N_2} \right) ^{N_1}, \left( 1 - \left( 1 - F(x) \right) ^{N_1} \right) ^{N_2} \right\}$$

which, for the case $N_1=N_2$, becomes:

$$\begin{aligned} \Pr \{ {d}_H (\mathbf {S}_1, \mathbf {S}_2) \le x \} = \left( 1 - \left( 1 - F(x) \right) ^N \right) ^N. \end{aligned}$$