Accelerating directed densest subgraph queries with software and hardware approaches

Abstract

Given a directed graph G, the directed densest subgraph (DDS) problem refers to finding a subgraph from G, whose density is the highest among all subgraphs of G. The DDS problem is fundamental to a wide range of applications, such as fake follower detection and community mining. Theoretically, the DDS problem closely connects to other essential graph problems, such as network flow and bipartite matching. However, existing DDS solutions suffer from efficiency and scalability issues. In this paper, we develop a convex-programming-based solution by transforming the DDS problem into a set of linear programs. Based on the duality of linear programs, we develop efficient exact and approximation algorithms. Particularly, our approximation algorithm can support flexible parameterized approximation guarantees. We further investigate using GPU to speed up the solution of convex programs in parallel and achieve hundreds of times speedup compared to the original Frank–Wolfe computation. We have performed an extensive empirical evaluation of our approaches on eight real large datasets. The results show that our proposed algorithms are up to five orders of magnitude faster than the state of the art.

Appendix

1.1 Convergence rate of   Frank-Wolfe-DDS

To perform the convergence analysis of (Algorithm 1), it would be easier if the objective function is differentiable [27], which, however, is not the case for $\Vert \textbf{r} \Vert _{\infty }$ in ${\textsf{D}}{\textsf{P}}(c)$ ((4.6)). Hence, we construct a convex program with a differentiable objective function, which shares the same optimal solution and minimizer of the linearization of the objective function at a specific position ((4.8)) with ${\textsf{D}}{\textsf{P}}(c)$.

$$\begin{aligned} \begin{aligned} {\textsf{C}}{\textsf{P}}(c)\text { } \min{} & {} f(\alpha , \beta )=\frac{1}{4\sqrt{c}}\sum _{u\in V} r_{\alpha }(u)^{2} + \frac{\sqrt{c}}{4}\sum _{v\in V}r_{\beta }(v)^{2}&\\ \text {s.t.}{} & {} \alpha , \beta , \textbf{r} \text { satisfy the constraints in } {\textsf{D}}{\textsf{P}}(c). \end{aligned} \end{aligned}$$

(9.1)

We can verify that (4.8) is also the minimizer of the linear function given by $\partial f(\alpha , \beta )$. Hence, (Algorithm 1) applies to both ${\textsf{D}}{\textsf{P}}(c)$ and ${\textsf{C}}{\textsf{P}}(c)$. Further, the following two lemmas indicate that the optimal solution of ${\textsf{C}}{\textsf{P}}(c)$ induces the c-biased DDS, which is also the objective of ${\textsf{D}}{\textsf{P}}(c)$.

Lemma 9.1

Suppose that an optimal solution $(\alpha , \beta )$ of ${\textsf{C}}{\textsf{P}}(c)$ induces the density vector $\textbf{r}\in {\mathbb {R}}_{+}^{2|V|}$. Then, we have

• 1. $\exists (u,v) \in E, r_{\alpha }(u)>r_{\beta }(v) \Rightarrow \alpha _{u,v} = 0, \beta _{v,u} = 1$;

• 2. $\exists (u,v) \in E, r_{\alpha }(u)<r_{\beta }(v) \Rightarrow \beta _{v,u} = 0, \alpha _{u,v} = 1$.

Proof

We prove the lemma by contradiction. For (1), suppose $\alpha _{u,v} > 0$. There exists $\epsilon > 0$ such that we could decrease $\alpha _{u,v}$ by $\epsilon$ and increase $\beta _{v,u}$ by $\epsilon$ to strictly decrease the objective function because $\frac{\partial f}{\partial \alpha _{u,v}}=r_{\alpha }(u) > \frac{\partial f}{\partial \beta _{v,u}}=r_{\beta }(v)$. This contradicts the optimal assumption. Similarly, we can also prove (2). $\square$

To simplify the notations, we denote ${\mathcal {D}}_{c}$ as the feasible set of ${\textsf{D}}{\textsf{P}}(c)$ and ${\textsf{C}}{\textsf{P}}(c)$, as ${\textsf{D}}{\textsf{P}}(c)$ and ${\textsf{C}}{\textsf{P}}(c)$ share the same constraints.

Lemma 9.2

Suppose a non-empty subset pair (S, T), where $S, T \subseteq V$, is stable with respect to a pair $(\alpha , \beta , \textbf{r}) \in {\mathcal {D}}_{c}$. Suppose that $\exists \rho _{c}^{*} \in {\mathbb {R}}$ such that $\forall u \in S, r_{\alpha }(u)=\rho _{c}^{*}$ and $\forall v \in T, r_{\beta }(v)=\rho _{c}^{*}$. Then, G[S, T] is the c-biased DDS and has c-biased density $\rho _{c}^{*}$.

Proof

As $(\alpha , \beta , \textbf{r})$ is a feasible solution of ${\textsf{D}}{\textsf{P}}(c)$ and (S, T) is stable, the objective value of $(\alpha , \beta , \textbf{r})$ is $\Vert \textbf{r} \Vert _{\infty }=\rho _{c}^{*}$. Moreover, since (S, T) is stable, $\rho _{c}(S,T)=\frac{2\sqrt{c c'}}{c+c'}\cdot \frac{|E(S,T)|}{\sqrt{|S||T|}}=\rho _{c}^{*}$, where $c'=\frac{|S|}{|T|}$. This comes from $|E(S, T)|= (\frac{|S|}{2\sqrt{c}}+\frac{\sqrt{c}|T|}{2})\rho _{c}(S, T)$. By Lemma 4.1, (S, T) gives a feasible primal solution in ${\textsf{L}}{\textsf{P}}(c)$ with objective value $\rho _{c}^{*}$. Hence, $\rho _{c}^{*}$ is the optimal value for both ${\textsf{L}}{\textsf{P}}(c)$ and ${\textsf{D}}{\textsf{P}}(c)$, which means that G[S, T] is the c-biased DDS. $\square$

Lemmas 9.1,9.2 imply that an optimal solution $(\alpha , \beta , \textbf{r})$ of ${\textsf{C}}{\textsf{P}}(c)$ induces the c-biased DDS $G[S_{c}^{*}, T_{c}^{*}]$ in G, where $S_{c}^{*}=\{u | r_{\alpha }(u)=\Vert \textbf{r} \Vert _{\infty }\}$ and $T_{c}^{*}=\{v | r_{\beta }(v)=\Vert \textbf{r} \Vert _{\infty }\}$.

Hence, we can confirm that (Algorithm 1) applies to both ${\textsf{D}}{\textsf{P}}(c)$ and ${\textsf{C}}{\textsf{P}}(c)$ and the optimal solutions of both programs induce the c-biased DDS. Hence, we use ${\textsf{C}}{\textsf{P}}(c)$ to analyze the convergence rate of . According to the previous convergence analysis of the Frank–Wolfe-based algorithms in [13, 27], the convergence rate of our algorithm can be described by a value related to the graph, $Q_{c}=\frac{1}{2}\textsf{Diam}({\mathcal {D}}_{c})^{2}\sup _{(\alpha ,\beta )\in {\mathcal {D}}_{c}}\Vert \nabla ^{2}f(\alpha ,\beta ) \Vert _{2}$, where $\textsf{Diam}({\mathcal {D}}_{c})$ is the diameter of ${\mathcal {D}}_{c}$, $\nabla ^{2}f(\alpha ,\beta )$ is the Hessian, and $\Vert \cdot \Vert _{2}$ is the spectral norm of a matrix.

Theorem 9.1

(Convergence Rate of Frank–Wolfe [27]) Suppose $(\alpha ^{*}, \beta ^{*}) \in {\mathcal {D}}_{c}$ is an optimal solution of ${\textsf{C}}{\textsf{P}}(c)$. Then, for all $i\ge 1$, $f(\alpha ^{(i)}, \beta ^{(i)}) - f(\alpha ^{*}, \beta ^{*}) \le \frac{2Q_{c}}{i+2}$.

Lemma 9.3

(Bounding $Q_{c}$) Given a directed graph $G=(V,E)$ with maximum outdegree $d^{+}_{\max }$ and maximum indegree $d^{-}_{\max }$ and a given c, we have that $Q_{c} \le 2|E|\max \{\sqrt{c}d^{+}_{\max },\frac{1}{\sqrt{c}}d^{-}_{\max })\}$.

Proof

First, we have $\textsf{Diam}({\mathcal {D}}_{c})=\sqrt{2|E|}$. The Hessian of $f(\alpha ,\beta )$ is irrelevant to the value of $(\alpha , \beta )$, and it is a nonnegative symmetric matrix. Therefore, $\sup _{(\alpha ,\beta )\in {\mathcal {D}}_{c}}\Vert \nabla ^{2}f(\alpha ,\beta ) \Vert _{2}$ is the maximum singular value of $\nabla ^{2}f(\alpha ,\beta )$. Let $A=\nabla ^{2}f(\alpha ,\beta )$, $\lambda _{1}$ be the maximum singular value (also the maximum eigenvalue) of A, x be the eigenvector associated with $\lambda _{1}$, and p be the component in which x has maximum absolute value. Without loss of generality, we assume $x_{p}$ is positive. We have

$$\begin{aligned} \lambda _{1} x_{p} = (Ax)_{p}=\sum _{q=1}^{2n}A_{p,q} x_{q}&\le \sum _{q=1}^{2n}A_{p,q}x_{p} \\&\le x_{p} \max \{2\sqrt{c}d^{+}_{\max },\frac{2}{\sqrt{c}}d^{-}_{\max } \}. \end{aligned}$$

Therefore, $Q_{c} \le 2|E|\max \{\sqrt{c}d^{+}_{\max },\frac{1}{\sqrt{c}}d^{-}_{\max })\}$. $\square$

Lemma 9.4

Suppose $(\alpha , \beta , r) \in {\mathcal {D}}_{c}$ such that $\varepsilon := \Vert \textbf{r} \Vert _{\infty } - \rho _{c}^{*}$, where $\rho _{c}^{*}=\Vert \textbf{r}^{*}\Vert _{\infty }$ and $(\alpha ^{*}, \beta ^{*}, \textbf{r}^{*})$ is the optimal solution of ${\textsf{D}}{\textsf{P}}(c)$. Then, we have that $(4\sqrt{c}+\frac{4}{\sqrt{c}})\cdot \left( f(\alpha , \beta ) - f(\alpha ^{*}, \beta ^{*}) \right) \ge \varepsilon ^{2}$.

Proof

First, we have $f(\alpha , \beta )-f(\alpha ^{*}, \beta ^{*}) \ge f(\alpha - \alpha ^{*}, \beta - \beta ^{*})$, because $f(\alpha , \beta ) - f(\alpha ^{*}, \beta ^{*}) - f(\alpha - \alpha ^{*}, \beta - \beta ^{*})$ is an affine function on ${\mathcal {D}}_{c}$ and obtains its minimum value 0 at $(\alpha ^{*}, \beta ^{*})$. Second, $f(\alpha - \alpha ^{*}, \beta - \beta ^{*})$ can be bounded by the $l^{2}$-norm of $\textbf{r}-\textbf{r}^{*}$, i.e., $(4\sqrt{c} + \frac{4}{\sqrt{c}})f(\alpha - \alpha ^{*}, \beta - \beta ^{*}) \ge \Vert \textbf{r} - \textbf{r}^{*} \Vert _{2}^{2}$. For the infinity norm and the $l^{2}$-norm, we have $\Vert \textbf{r} \Vert _{\infty } - \rho _{c}^{*} \le \Vert \textbf{r} - \textbf{r}^{*} \Vert _{\infty } \le \Vert \textbf{r} - \textbf{r}^{*} \Vert _{2}$. Combining the above inequalities, we will have the lemma. $\square$

Corollary 9.1

(Convergence of Algorithm 1) Suppose $d^{+}_{\max }$ (resp. $d^{-}_{\max }$) is the maximum outdegree (resp. indegree) of G and c is fixed. In Algorithm 1, for $i > 16(\sqrt{c} + \frac{1}{\sqrt{c}}) \frac{|E|\max \{\sqrt{c}d^{+}_{\max },\frac{1}{\sqrt{c}}d^{-}_{\max })\}}{\varepsilon ^{2}}$, we have $\Vert \textbf{r}^{(i)} \Vert _{\infty } - \rho _{c}^{*} \le \varepsilon$.

1.2 Proofs

Proof of Lemma 4.1

We prove the lemma by showing a feasible solution (x, s, t, a, b) of ${\textsf{L}}{\textsf{P}}(c)$. Let $a=\frac{2c'}{c+c'}$ and $b=\frac{2c}{c+c'}$. For each vertex $u\in P$, set $s_{u}=\frac{a\sqrt{c}}{|P|}=\frac{2c'\sqrt{c}}{(c+c')|P|}.$ For each vertex $v\in Q$, set $t_{v}=\frac{b}{\sqrt{c}|Q|}=\frac{2c}{(c+c')\sqrt{c}|Q|}=\frac{2c'\sqrt{c}}{(c+c')|P|}$. For each edge $(u,v)\in E(P,Q)$, set $x_{u,v}=s_{u}=t_{v}$. All the remaining variables are set to 0. Now, $\sum _{u\in V}s_{u}=a\sqrt{c}$ and $\sum _{v\in V}t_{v}=\frac{b}{\sqrt{c}}$. Hence, this is a feasible solution to ${\textsf{L}}{\textsf{P}}(c)$. The value of this solution is

$$\begin{aligned}{} & {} \frac{2c'\sqrt{c}}{(c+c')|P|}|E(P,Q)|=\frac{2\sqrt{c}c'\sqrt{|Q|}}{(c+c')\sqrt{|P|}}\frac{|E(P,Q)|}{\sqrt{|P||Q|}}\\{} & {} \quad =\frac{2\sqrt{c}\sqrt{c'}}{c+c'}\rho (P,Q). \end{aligned}$$

Thus, the lemma holds. $\square$

Proof of Lemma 4.2

Without loss of generality, we can assume that for each $(u,v)\in E$, $x_{u,v}=\min \{s_{u}, t_{v}\}$. We define a collection of sets S, T indexed by a parameter $r\ge 0$. Let $S(r)=\{u|s_u \ge r\}$, $T(r)=\{v|t_{v}\ge r\}$, and $E(r)=\{(u,v)|x_{u,v}=\min \{s_{u}, t_{v}\}\}$. Hence, E(r) is precisely the set of edges that go from S(r) to T(r).

Now, $\int _{0}^{\infty }|S(r)|\text {d}r=\sum _{u\in V}s_{u}=a\sqrt{c}$. Similarly, $\int _{0}^{\infty }|T(r)|\text {d}r=\sum _{v\in V}t_{v}=\frac{b}{\sqrt{c}}$. By the Cauchy–Schwarz inequality,

$$\begin{aligned}{} & {} \int _{0}^{\infty }\sqrt{|S(r)||T(r)|}\text {d}r\\{} & {} \quad \le \sqrt{\left( \int _{0}^{\infty }|S(r)|\text {d}r\right) \left( \int _{0}^{\infty }|T(r)|\text {d}r\right) }=\sqrt{ab}. \end{aligned}$$

Note that $\int _{0}^{\infty }|E(r)|\text {d}r=\sum _{(u,v)\in E}x_{u,v}$. This is the objective function value of the solution. Let this value be $x_{\text {sum}}$.

We claim that there exists r such that $\frac{E(r)}{\sqrt{|S(r)||T(r)|}}\ge \frac{x_{\text {sum}}}{\sqrt{ab}}$. Suppose there was no such r. Then,

$$\begin{aligned} \int _{0}^{\infty }|E(r)|\text {d}r < \frac{x_{\text {sum}}}{\sqrt{ab}} \int _{0}^{\infty }\sqrt{|S(r)||T(r)|}\text {d}r \le x_{\text {sum}}. \end{aligned}$$

This gives a contradiction. Thus, the lemma holds. $\square$

Proof of Lemma5.3

As $G[S_{c}^{*}, T_{c}^{*}]$ is the c-biased DDS with c-biased density $\rho _{c}(S^{*}, T^{*})$, there must exist $u\in S$ satisfying $r_{\alpha }(u) \le \rho _{c}(S_{c}^{*}, T_{c}^{*})$, or $v\in T$ satisfying $r_{\beta }(v) \le \rho _{c}(S_{c}^{*}, T_{c}^{*})$. Otherwise, G[S, T] is a subgraph with a higher c-biased density than $G[S_{c}^{*}, T_{c}^{*}]$.

Now, we prove the lemma by contradiction. Assume $G[S_{c}^{*},T_{c}^{*}]$ is not contained in G[S, T]. Based on whether $G[S_{c}^{*},T_{c}^{*}]$ overlaps G[S, T], there are two cases.

• 1. $S_{c}^{*} \cap S = \emptyset$ and $T_{c}^{*} \cap T = \emptyset$. Since $|E(S_{c}^{*}, T_{c}^{*})|=(\frac{|S_{c}^{*}|}{\sqrt{c}}+\sqrt{c}|T_{c}^{*}|)\rho _{c}(S_{c}^{*}, T_{c}^{*})$, there exists $u\in S_{c}^{*}, r_{\alpha }(u) \ge \rho _{c}(S_{c}^{*}, T_{c}^{*})$ or $v\in T_{c}^{*}, r_{\beta }(v) \ge \rho _{c}(S_{c}^{*}, T_{c}^{*})$.

• 2. $S_{c}^{*} \cap S \ne \emptyset$ or $T_{c}^{*} \cap T \ne \emptyset$.

  $$\begin{aligned} \begin{aligned}&|E(S_c^{*}, T_{c}^{*})| \\ =&|E(S_c^{*}\cap S, T_{c}^{*}\cap T)| + |E(S_c^{*}, T_{c}^{*})\setminus E(S, T)| \\ =&\left( \frac{|S_{c}^{*}\cap S|}{\sqrt{c}}+\sqrt{c}|T_{c}^{*}\cap T|\right) \rho _{c}(S_c^{*}\cap S, T_{c}^{*}\cap T) \\&+ \left( \frac{|S_{c}^{*}\setminus S|}{\sqrt{c}}+\sqrt{c}|T_{c}^{*}\setminus T|\right) \rho '_{c}. \end{aligned} \end{aligned}$$

  Since $\rho _{c}(S_c^{*}\cap S, T_{c}^{*}\cap T) \le \rho _{c}(S_{c}^{*}, T_{c}^{*})$, we have $\rho '_{c} \ge \rho _{c}(S_{c}^{*}, T_{c}^{*})$. Thus, there exists $u\in S_{c}^{*}{\setminus } S, r_{\alpha }(u) \ge \rho _{c}(S_{c}^{*}, T_{c}^{*})$ or $v\in T_{c}^{*}{\setminus } T, r_{\beta }(v) \ge \rho _{c}(S_{c}^{*}, T_{c}^{*})$.

Consequently, for each case above, combining the inequalities will give us a contradiction to the first condition of the stable (S, T)-induced subgraph definition. Hence, the lemma holds. $\square$