Streaming Graph Computations with a Helpful Advisor

Abstract

Motivated by the trend to outsource work to commercial cloud computing services, we consider a variation of the streaming paradigm where a streaming algorithm can be assisted by a powerful helper that can provide annotations to the data stream. We extend previous work on such annotation models by considering a number of graph streaming problems. Without annotations, streaming algorithms for graph problems generally require significant memory; we show that for many standard problems, including all graph problems that can be expressed with totally unimodular integer programming formulations, only constant space (measured in words) is needed for single-pass algorithms given linear-sized annotations. We also obtain protocols achieving essentially optimal tradeoffs between annotation length and memory usage for several important problems, including integer matrix-vector multiplication, as well as shortest s-t path in small-diameter graphs. We also obtain non-trivial tradeoffs for minimum weight bipartite perfect matching and shortest s-t path in general graphs.

Appendices

Appendix A: Linear Programs with Rational Entries

One might hope that Theorem 3 extends naturally to linear programs with rational entries. The following simple variant of Example 1 demonstrates that with rational data it is no longer sufficient to assume all subdeterminants of the constraint matrix are bounded in absolute value.

Example 2

Consider the linear program of Example 1, with A replaced by the matrix

Observe that all subdeterminants of this matrix are bounded (above) in absolute value by 1. Exactly as in Example 1, the unique feasible point is the vector x∈ℤ^c with i’th coordinate equal to 2ⁱ, and the value of the linear program is Ω(2^c). $\mathcal {V}$ cannot manipulate quantities of such magnitude exactly with less than linear space.

However, context often provides an a priori bound on the value of the program and the length of the optima, especially in applications to combinatorial optimization. Still, without sufficiently strong assumptions we cannot guarantee that there is an optimal solution that can be exactly specified in small precision, and $\mathcal {H}$ must therefore send an approximate representation of the optimum. This rounded optimum is not guaranteed to be feasible for the exact LP, but will be essentially optimal for a very slightly perturbed LP.

Theorem 13

Suppose all entries of A, b, and c are rational numbers p/q for p,q∈ℤ. Assume the value of the linear program is finite, and there is a known polynomial upper bound on the length of an optimum. For any 0<ϵ ₁<1/|A|, there is an $\epsilon_{2} = g(\mathcal{A}) \epsilon_{1}$, with independent of ϵ ₁, such that the following is true. There is a valid $(|A| \frac{\log 1/\epsilon_{1}}{\log |A|}, \frac{\log 1/\epsilon_{1}}{\log |A|})$ protocol for obtaining an additive-ϵ ₂ approximation to the value of the perturbed primal LP max{c ^T x∣A x≤b+ϵ ₁ 1}. In particular, if , then we obtain an (|A|,1) protocol.

Proof

First, suppose all entries of A, b, and c are integer multiples of ϵ for to be determined; we remove this assumption later.

1.1 A.1 Protocol Specification

Notice that all entries of A, b and c are elements of a universe of size , and $\mathcal {V}$ can fingerprint all multisets as in Theorem 3 using a finite field $\mathbb{F}_{p}$ where . These fingerprints require $O(\frac{\log 1/\epsilon_{1}}{\log |A|})$ words of space.

Since the value of the LP is finite, there exist primal and dual optimal solutions x ^∗ and y ^∗, and by assumption all entries of x ^∗ and y ^∗ have absolute value polynomial in b and c. Let $\hat{\mathbf {x}}^{*}$ and $\hat{\mathbf {y}}^{*}$ denote the vectors x ^∗ and y ^∗, with all entries rounded up to integer multiples of ϵ where $\epsilon=\frac{\epsilon_{1}}{2u}$. Here, u∈ℤ is a strict upper bound on ∥A∥_∞, ∥c∥₁, ∥b∥₁, ∥x∥₁, and ∥y∥₁, where throughout, if A is a matrix then ∥A∥ _p denotes the operator norm of A induced by the p norm on vectors, and if x is a vector then ∥x∥ _p denotes the p-norm on vectors. Notice u is polynomial in b and c, does not depend on ϵ ₁, and can be determined by V while observing the stream.

The protocol is exactly as in Theorem 3, except $\mathcal {H}$ sends vectors $\hat{\mathbf {x}}$ and $\hat{\mathbf {y}}$ claimed to be $\hat{\mathbf {x}}^{*}$ and $\hat{\mathbf {y}}^{*}$. $\mathcal {V}$ checks that

• 1. ϵ ₁/ϵ≥2u. That is, $\hat{\mathbf{x}}$ and $\hat{\mathbf{y}}$ are represented at sufficiently high precision.

• 2. $\hat{\mathbf{x}}$ is feasible for the perturbed primal.

• 3. $|\mathbf{b}^{T} \hat{\mathbf{y}} - \mathbf{c}^{T} \hat{\mathbf{x}}| \leq \epsilon_{1}$.

• 4. $\hat{\mathbf{y}}$ is feasible for the LP

  

  This LP can be thought of as the perturbed dual, although it is not the dual of the perturbed primal.

If all checks pass, $\mathcal {V}$ outputs $\mathbf{c}^{T} \hat{\mathbf{x}}$.

1.2 A.2 Protocol Validity

Write $\mathbf{e}(\mathbf{x}^{*})= \hat{\mathbf{x}}^{*} - \mathbf{x}^{*}$ and $\mathbf{e}(\mathbf{y})= \hat{\mathbf{y}}^{*} - \mathbf{y}^{*}$. Notice that ∥e(x ^∗)∥_∞≤ϵ and ∥e(y ^∗)∥_∞≤ϵ for all i. Since x ^∗ and y ^∗ are primal and dual feasible, it is easy to see $\hat{\mathbf{x}}^{*}$ and $\hat{\mathbf{y}}^{*}$ are feasible for the perturbed LPs. Indeed,

$$\|A\mathbf {x}^*-A\hat{\mathbf {x}}^*\|_{\infty} = \|A\mathbf{e}(\mathbf{x}^*)\|_{\infty} \leq \|A\|_{\infty}\|\mathbf{e}(\mathbf{x}^*)\|_{\infty} \leq u \epsilon \leq \epsilon_1 ,$$

and a similar argument shows $\hat{\mathbf {y}}^{*}$ is feasible for the perturbed dual.

Since c ^T x=b ^T y, it follows that

$$|\mathbf{b}^T \hat{\mathbf{y}}^* - \mathbf{c}^T\hat{\mathbf{x}}^*|\leq |\mathbf{c}^T\mathbf{e}(\mathbf {x})| + |\mathbf{b}^T \mathbf{e}(\mathbf {y})| \leq 2\epsilon u \leq \epsilon_1 .$$

Therefore, if $\hat{\mathbf {x}}$ and $\hat{\mathbf {y}}$ are as claimed, all of $\mathcal {V}$’s checks will pass.

To prove the protocol is valid, it therefore suffices to show that if vectors x and y pass $\mathcal {V}$’s checks, then c ^T x is within an additive ϵ ₂=(u+2)ϵ ₁ of the true value of the perturbed primal LP. c ^T x is clearly a lower bound on the value of the perturbed primal, since x is feasible for the perturbed primal. We claim b ^T y+(u+1)ϵ ₁ is an upper bound on the value of the perturbed primal. Indeed, since y is feasible for the perturbed dual, ∥A ^T y−c∥_∞≤ϵ for all i, and thus for all feasible x of the perturbed primal,

where the third inequality holds because x is feasible for the perturbed primal and y≥0. Therefore, c ^T x is a lower bound on the value of the perturbed primal and b ^T y+(u+1)ϵ ₁≤c ^T x+(u+2)ϵ ₁ is an upper bound, which completes the proof of validity.

If the entries of A, b, and c are not integer multiples of ϵ, but are instead arbitrary rationals p/q for p,q∈ℤ, we run the above protocol on the derived stream in which each stream element is rounded to the nearest integer multiple of ϵ. (If $\mathcal {V}$ cannot determine the necessary precision ϵ in advance, we can afford for $\mathcal {V}$ to calculate it while observing the stream, and then have $\mathcal {H}$ replay the entire stream.) This introduces error into all of the above calculations, but we can obtain the same approximation guarantees by setting ϵ a polynomial factor smaller than above. This does not affect the asymptotic costs of the protocol.  □

Appendix B: Proof of Theorem 5

Proof

The dual function of any quadratic programming problem is given by

and q(μ)=−∞ otherwise. Strong duality always holds for quadratic programs. That is, for all μ, q(μ) is a lower bound on the value f ^∗ of the primal quadratic program, and moreover there exists a μ ^∗≥0 such that q(μ ^∗)=f ^∗ [3, Example 5.3.1]. Consequently, we can use a protocol akin to that of Theorem 13 and Corollary 4: essentially $\mathcal {H}$ proves optimality of a primal solution x by providing a dual-optimal solution μ, and proving to $\mathcal {V}$ that the values of x and μ are equal. But since $\mathcal {H}$ cannot afford to send an exact representation of x or μ, $\mathcal {H}$ instead sends vectors $\hat{\mathbf{x}}$ and $\hat{\mu}$ with all entries integer multiples of $\epsilon=\frac{\epsilon_{1}}{h(\mathcal{A})}$. Here h is some function polynomial in b and c and independent of ϵ ₁, to be specified later. These vectors are claimed to be rounded versions of the true optima.

Let f(x) denote the primal objective function evaluated at x. $\mathcal {V}$ checks that

• 1. $\epsilon_{1}/\epsilon \geq h(\mathcal{A})$. That is, x and μ are represented at sufficiently high precision.

• 2. $A\hat{\mathbf{x}} \leq \mathbf{b} + \epsilon_{1} \mathbf{1}$.

• 3. μ≥0.

• 4. .

  If all checks pass, $\mathcal {V}$ outputs $\mathbf{c}^{T} \hat{\mathbf{x}}$, otherwise $\mathcal {V}$ outputs ⊥.

The first and third checks are trivial to perform, and $\mathcal {V}$ can perform the second check with a single matrix-vector multiplication operation. To perform the fourth check, $\mathcal {V}$ first computes $f(\hat{\mathbf{x}})$ with a single matrix-vector multiplication and two inner product computations. $\mathcal {V}$ then computes q(μ) by verifying a constant number of matrix-vector multiplications and inner product computations as follows. First $\mathcal {V}$ computes in a sequence of four matrix-vector multiplications: (1) z ₁₁:=A ^T μ; (2) z ₁₂:=Q ⁻¹ z ₁₁ (computed by having $\mathcal {H}$ specify z ₁₂ and verifying that Q z ₁₂=z ₁₁); (3) z ₁₃:=A z ₁₂; and (4) . We stress that $\mathcal {V}$ need not explicitly invert Q or be provided with Q ⁻¹ to compute z ₁₂:=Q ⁻¹ z ₁₁; instead, it suffices to verify a single matrix-vector multiplication. Next, $\mathcal {V}$ computes z ₂:=μ ^T b with a single inner product computation, and computes z ₃:=μ ^T AQ ⁻¹ c and $\mathbf{z}_{4} := \frac{1}{2}\mathbf{c}^{T}Q^{-1}\mathbf{c}$ similarly to z ₁. Given z ₁,z ₂,z ₃, and z ₄, $\mathcal {V}$ may compute the value q(μ)=z ₁−z ₂−z ₃−z ₄.

It remains to argue that we can set $\epsilon =\epsilon_{1}/h(\mathcal{A})$ such that: if $\hat{\mathbf{x}}$ and are as claimed, then all checks will pass with probability 1, and if all checks pass then with high probability the value of the perturbed QP is $\mathbf{c}^{T} \hat{\mathbf{x}} \pm \epsilon$.

Suppose $\hat{\mathbf{x}}$ and μ are as claimed. Then first and third checks will pass, and the second check will pass by exactly the same argument as in Theorem 13. For the final check, let ; ∥e∥_∞≤ϵ. Then

By repeated application of the triangle inequality to each term in the above sum, it can be seen that the above expression is at most , where u is a known upper bound on ∥A∥_∞, ∥Q∥_∞, ∥Q ⁻¹∥_∞, ∥b∥₁, ∥c∥₁, ∥x∥₁ and ∥μ∥₁. Notice that ∥Q ⁻¹∥_∞ is polynomially bounded as long as Q is sufficiently positive definite, in the sense that all eigenvalues of Q are at least . All other quantities for which u is an upper bound are polynomial in b and c by assumption, and the above expression is therefore at most for some known polynomial in b and c that is independent of ϵ ₁.

Similarly, for some polynomial in b and c that is independent of ϵ ₁. Thus, is also for some polynomial in b and c that only depends on u and is independent of ϵ ₁. We can therefore choose such that the final check will pass if $\hat{\mathbf{x}}$ and are as claimed.

Finally, we argue that if all four checks pass, then f(x) is a $g(\mathcal{A})\epsilon_{1}$-approximation to the value of the perturbed primal for some independent of ϵ ₁. The argument in this case is simpler than that in Theorem 13, because if the third check passes, then μ is actually feasible for the dual of the perturbed primal, as the only constraint of the dual is that μ≥0. Indeed, let q _P denote the dual of the perturbed primal; it is easy to see that q _P (μ)=q(μ)−μ ^T(ϵ ₁ 1). By weak duality, for any x that is feasible for the perturbed primal, and any μ, f(x)≥q _P (μ)≥q(μ)−uϵ ₁. Since the fourth check passed, f(x)−q(μ)≤ϵ ₁, and therefore f(x)−q _P (μ)≤(u+1)ϵ ₁. We can let $g(\mathcal{A})=u+1$.

By invoking the matrix-vector multiplication protocols of Theorem 3 and Theorem 4 (handling the fact that the vectors are not integral as in Corollary 4) to compute all matrix-vector multiplications in the above description, we obtain the theorem. □