Hybrid Diffusion: Spectral-Temporal Graph Filtering for Manifold Ranking

Abstract

State of the art image retrieval performance is achieved with CNN features and manifold ranking using a k-NN similarity graph that is pre-computed off-line. The two most successful existing approaches are temporal filtering, where manifold ranking amounts to solving a sparse linear system online, and spectral filtering, where eigen-decomposition of the adjacency matrix is performed off-line and then manifold ranking amounts to dot-product search online. The former suffers from expensive queries and the latter from significant space overhead. Here we introduce a novel, theoretically well-founded hybrid filtering approach allowing full control of the space-time trade-off between these two extremes. Experimentally, we verify that our hybrid method delivers results on par with the state of the art, with lower memory demands compared to spectral filtering approaches and faster compared to temporal filtering.

A General Derivation

The derivation of our algorithm in Sect. 4.1 applies only to the particular function (filter) $h_\alpha$ (3). Here, as in [11], we generalize to a much larger class of functions, that is, any function h that has a series expansion

$$\begin{aligned} h(A) = \sum _{i=0}^\infty c_i A^i. \end{aligned}$$

(24)

We begin with the same eigenvalue decomposition (6) of $\mathcal {W}$ and, assuming that $h(\mathcal {W})$ converges absolutely, its corresponding decomposition

$$\begin{aligned} h(\mathcal {W}) = U_1 h(\varLambda _1) U_1^{\!\top }+ U_2 h(\varLambda _2) U_2^{\!\top }, \end{aligned}$$

(25)

similar to (8), where $U_1$, $U_2$ have the same orthogonality properties (5).

Again, the first term is exactly the low-rank approximation that is used by spectral filtering, and the second is the approximation error

$$\begin{aligned} e_\alpha (\mathcal {W})&\mathrel {{\text {:=}}}U_2 h(\varLambda _2) U_2^{\!\top } \end{aligned}$$

(26)

$$\begin{aligned}&= \sum _{i=0}^\infty c_i U_2 \varLambda _2^i U_2^{\!\top } \end{aligned}$$

(27)

$$\begin{aligned}&= \sum _{i=0}^\infty c_i \left( U_2 \varLambda _2 U_2^{\!\top }\right) ^i - c_0 U_1 U_1^{\!\top } \end{aligned}$$

(28)

$$\begin{aligned}&= h(U_2 \varLambda _2 U_2^{\!\top }) - h(0) U_1 U_1^{\!\top }. \end{aligned}$$

(29)

Again, we have used the series expansion (24) of h in (27) and (29). Now, Eq. (28) is due to the fact that

$$\begin{aligned} (U_2 \varLambda _2 U_2^{\!\top })^i = U_2 \varLambda _2^i U_2^{\!\top } \end{aligned}$$

(30)

for $i \ge 1$, as can be verified by induction, while for $i = 0$,

$$\begin{aligned} U_2 \varLambda _2^0 U_2^{\!\top }= U_2 U_2^{\!\top }= I_n - U_1 U_1^{\!\top }= (U_2 \varLambda _2 U_2^{\!\top })^0 - U_1 U_1^{\!\top }. \end{aligned}$$

(31)

In both (30) and (31) we have used the orthogonality properties (5).

Finally, combining (25), (29) and (6), we have proved the following.

Theorem 2

Assuming the series expansion (24) of transfer function h and the eigenvalue decomposition (6) of the symmetrically normalized adjacency matrix $\mathcal {W}$, and given that $h(\mathcal {W})$ converges absolutely, it is decomposed as

$$\begin{aligned} h(\mathcal {W}) = U_1 g(\varLambda _1) U_1^{\!\top }+ h(\mathcal {W}- U_1 \varLambda _1 U_1^{\!\top }), \end{aligned}$$

(32)

where

$$\begin{aligned} g(A) \mathrel {{\text {:=}}}h(A) - h(\mathbf {O}) \end{aligned}$$

(33)

for $n \times n$ real symmetric matrix A. For $h = h_\alpha$ and for $x \in [-1,1]$ in particular, $g_\alpha (x) \mathrel {{\text {:=}}}h_\alpha (x) - h_\alpha (0) = (1 - \alpha ) \alpha x / (1 - \alpha x)$.

This general derivation explains where the general definition of function g (33) is coming from in (16) corresponding to our treatment of $h_\alpha$ in Sect. 4.1.