Skip to main content
Log in

A Combinatorial Cut-Toggling Algorithm for Solving Laplacian Linear Systems

Algorithmica Aims and scope Submit manuscript

Abstract

Over the last two decades, a significant line of work in theoretical algorithms has made progress in solving linear systems of the form \({{\textbf {L}}}{{\textbf {x}}} = {{\textbf {b}}}\), where \({{\textbf {L}}}\) is the Laplacian matrix of a weighted graph with weights \({w(i,j)}>{0}\) on the edges. The solution \({{\textbf {x}}}\) of the linear system can be interpreted as the potentials of an electrical flow in which the resistance on edge (ij) is 1/w(ij). Kelner et al. (in: Proceedings of the 45th Annual ACM Symposium on the Theory of Computing, pp 911–920, 2013. https://doi.org/10.1145/2488608.2488724) give a combinatorial, near-linear time algorithm that maintains the Kirchoff Current Law, and gradually enforces the Kirchoff Potential Law by updating flows around cycles (cycle toggling). In this paper, we consider a dual version of the algorithm that maintains the Kirchoff Potential Law, and gradually enforces the Kirchoff Current Law by cut toggling: each iteration updates all potentials on one side of a fundamental cut of a spanning tree by the same amount. We prove that this dual algorithm also runs in a near-linear number of iterations. We show, however, that if we abstract cut toggling as a natural data structure problem, this problem can be reduced to the online vector–matrix-vector problem, which has been conjectured to be difficult for dynamic algorithms (Henzinger et al., in: Proceedings of the 47th Annual ACM Symposium on the Theory of Computing, pp 21–30, 2015. https://doi.org/10.1145/2746539.2746609). The conjecture implies that the data structure does not have an \(O(n^{1-\epsilon })\) time algorithm for any \(\epsilon > 0\), and thus a straightforward implementation of the cut-toggling algorithm requires essentially linear time per iteration. To circumvent the lower bound, we batch update steps, and perform them simultaneously instead of sequentially. An appropriate choice of batching leads to an \({\widetilde{O}}(m^{1.5})\) time cut-toggling algorithm for solving Laplacian systems. Furthermore, we show that if we sparsify the graph and call our algorithm recursively on the Laplacian system implied by batching and sparsifying, we can reduce the running time to \(O(m^{1 + \epsilon })\) for any \(\epsilon > 0\). Thus, the dual cut-toggling algorithm can achieve (almost) the same running time as its primal cycle-toggling counterpart.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Algorithm 2
Algorithm 3
Algorithm 4

Notes

  1. To make matters somewhat confusing, the Laplacian solver literature treats the space of potentials as primal due to its origins in numerical analysis.

  2. In a personal communication, Sherman [15] said he also had worked out a dual version of the KOSZ algorithm, but was unable to solve the data structure problem for the updates to potentials. Our result explains why this might be difficult to do.

References

  1. Spielman, D.A., Teng, S.-H.: Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems. In: Proceedings of the 36th Annual ACM Symposium on the Theory of Computing, pp. 81–90 (2004)

  2. Chen, L., Kyng, R., Liu, Y.P., Peng, R., Gutenberg, M.P., Sachdeva, S.: Maximum Flow and Minimum-Cost Flow in Almost-Linear Time (2022). https://doi.org/10.48550/ARXIV.2203.00671. arXiv:2203.00671

  3. Christiano, P., Kelner, J.A., Mądry, A., Spielman, D., Teng, S.-H.: Electrical flows, Laplacian systems, and faster approximation of maximum flow in undirected graphs. In: Proceedings of the 42nd Annual ACM Symposium on the Theory of Computing, pp. 273–282 (2011)

  4. Kathuria, T., Liu, Y.P., Sidford, A.: Unit capacity maxflow in almost \(m^{4/3}\) time. SIAM J. Comput. (2022). https://doi.org/10.1137/20M1383525

    Article  MATH  Google Scholar 

  5. Mądry, A.: Computing maximum flow with augmenting electrical flows. In: Proceedings of the 57th IEEE Symposium on the Foundations of Computer Science, pp. 593–602 (2016)

  6. Gao, Y., Liu, Y.P., Peng, R.: Fully dynamic electrical flows: sparse maxflow faster than Goldberg-Rao. In: 2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS), pp. 516–527 (2022). https://doi.org/10.1109/FOCS52979.2021.00058

  7. Kelner, J.A., Orecchia, L., Sidford, A., Zhu, Z.A.: A simple, combinatorial algorithm for solving SDD systems in nearly-linear time. In: Proceedings of the 45th Annual ACM Symposium on the Theory of Computing, pp. 911–920 (2013). https://doi.org/10.1145/2488608.2488724

  8. Boman, E.G., Deweese, K., Gilbert, J.R.: Evaluating the potential of a dual randomized Kaczmarz Laplacian linear solver. Informatica 40, 95–107 (2016)

  9. Boman, E.G., Deweese, K., Gilbert, J.R.: An empirical comparison of graph Laplacian solvers. In: Proceedings of the 18th Workshop on Algorithm Engineering and Experiments, pp. 174–188 (2016)

  10. Deweese, K., Gilbert, J.R., Miller, G., Peng, R., Xu, H., Xu, S.: An empirical study of cycle toggling based Laplacian solvers. In: Proceedings of the 7th SIAM Workshop on Combinatorial Scientific Computing, pp. 33–41 (2016)

  11. Hoske, D., Lukarski, D., Meyerhenke, H., Wegner, M.: Engineering a combinatorial Laplacian solver: Lessons learned. Algorithms 9 (2016). Article 72

  12. Goldberg, A.V., Tarjan, R.E.: Finding minimum-cost circulations by canceling negative cycles. J. ACM 36, 873–886 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  13. Ervolina, T.R., McCormick, S.T.: Two strongly polynomial cut cancelling algorithms for minimum cost network flow. Discrete Appl. Math. 46, 133–165 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  14. Henzinger, M., Krinninger, S., Nanongkai, D., Saranurak, T.: Unifying and strengthening hardness for dynamic problems via the online matrix-vector multiplication conjecture. In: Proceedings of the 47th Annual ACM Symposium on the Theory of Computing, pp. 21–30 (2015). https://doi.org/10.1145/2746539.2746609

  15. Sherman, J.: Personal communication (2017)

  16. Kyng, R., Pachocki, J., Peng, R., Sachdeva, S.: A framework for analyzing resparsification algorithms. In: Proceedings of the 2017 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pp. 2032–2043 (2017). https://doi.org/10.1137/1.9781611974782.132

  17. Batson, J.D., Spielman, D.A., Srivastava, N., Teng, S.: Spectral sparsification of graphs: theory and algorithms. Commun. ACM 56(8), 87–94 (2013)

    Article  Google Scholar 

  18. Abraham, I., Neiman, O.: Using petal-decompositions to build a low stretch spanning tree. In: Proceedings of the 44th Symposium on the Theory of Computing, pp. 395–406 (2012). https://doi.org/10.1145/2213977.2214015

  19. Williamson, D.P.: Network Flow Algorithms. Cambridge University Press, Cambridge (2019)

    Book  MATH  Google Scholar 

  20. Sleator, D.D., Tarjan, R.E.: A data structure for dynamic trees. J. Comput. Syst. Sci. 26(3), 362–391 (1983). https://doi.org/10.1016/0022-0000(83)90006-5

    Article  MathSciNet  MATH  Google Scholar 

  21. Karger, D.R.: Minimum cuts in near-linear time. J. ACM 47(1), 46–76 (2000). https://doi.org/10.1145/331605.331608

    Article  MathSciNet  MATH  Google Scholar 

  22. Koutis, I., Miller, G.L., Peng, R.: A fast solver for a class of linear systems. Commun. ACM 55(10), 99–107 (2012). https://doi.org/10.1145/2347736.2347759

    Article  Google Scholar 

  23. Jambulapati, A., Sidford, A.: Ultrasparse ultrasparsifiers and faster laplacian system solvers. In: Marx, D. (ed.) Proceedings of the 2021 ACM-SIAM Symposium on Discrete Algorithms, SODA 2021, Virtual Conference, January 10–13, 2021, pp. 540–559. SIAM (2021). arXiv:2011.08806

  24. Cohen, M.B., Kyng, R., Miller, G.L., Pachocki, J.W., Peng, R., Rao, A.B., Xu, S.C.: Solving SDD linear systems in nearly \(m \log ^{1/2} n\) time. In: Proceedings of the Forty-Sixth Annual ACM Symposium on Theory of Computing, pp. 343–352 (2014)

  25. Koutis, I., Xu, S.C.: Simple parallel and distributed algorithms for spectral graph sparsification. ACM Trans. Parallel Comput. (2016). https://doi.org/10.1145/2948062

    Article  Google Scholar 

  26. Spielman, D.A., Teng, S.: Spectral sparsification of graphs. SIAM J. Comput. 40(4), 981–1025 (2011). arXiv:0808.4134

    Article  MathSciNet  MATH  Google Scholar 

  27. Kapralov, M., Panigrahy, R.: Spectral sparsification via random spanners. In: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference. ITCS ’12, pp. 393–398. Association for Computing Machinery, New York, NY, USA (2012). https://doi.org/10.1145/2090236.2090267

  28. Peng, R., Spielman, D.A.: An Efficient Parallel Solver for SDD Linear Systems (2013)

Download references

Acknowledgements

Monika Henzinger was supported by funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme Grant agreement No. 101019564 “The Design of Modern Fully Dynamic Data Structures (MoDynStruct)” and from the Austrian Science Fund (FWF) project “Fast Algorithms for a Reactive Network Layer (ReactNet)”, P 33775-N, with additional funding from the netidee SCIENCE Stiftung, 2020–2024. Billy Jin was Supported in part by NSERC fellowship PGSD3-532673-2019 and NSF grant CCF-2007009. Richard Peng was supported in part by an NSERC Discovery Grant and NSF grant CCF-1846218. David P. Williamson was supported in part by NSF grant CCF-2007009.

Author information

Authors and Affiliations

Authors

Contributions

All authors contributed to the paper.

Corresponding author

Correspondence to Billy Jin.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

A Omitted Proofs from Sect. 3

Lemma 1

Let \({{\textbf {x}}} \in {\mathbb {R}}^V\) be a vector of potentials and let \(C \subset V\). Let \({{\textbf {x}}}'\) be the potentials obtained from \({{\textbf {x}}}\) as in the algorithm (that is, by adding \(\Delta \) to the potential of every vertex in C so that flow conservation is satisfied across \(\delta (C)\)). Then

$$\begin{aligned} {\mathcal {B}}({{\textbf {x}}}') - {\mathcal {B}}({{\textbf {x}}}) = \frac{\Delta ^2}{2R(C)}. \end{aligned}$$

Proof

The way we update \({{\textbf {x}}}\) is by adding a constant \(\Delta \) to the potentials of every vertex in C, where

$$\begin{aligned} \Delta = (b(C) - f(C))\cdot R(C) \end{aligned}$$

Recall that f(C) is the net amount of flow going out of C in the flow induced by \({{\textbf {x}}}\). That is,

$$\begin{aligned} f(C) = \sum _{\begin{array}{c} ij \in E \\ i \in C,\, j \not \in C \end{array}} \frac{x(i) - x(j)}{r(i, j)} \end{aligned}$$

Note that the new potentials \({{\textbf {x}}}'\) can be expressed as \({{\textbf {x}}}' = {{\textbf {x}}} + \Delta \mathbb {1}_C\). We have

$$\begin{aligned} 2({\mathcal {B}}({{\textbf {x}}}') - {\mathcal {B}}({{\textbf {x}}}))&= 2{{\textbf {b}}}{^\top }{{\textbf {x}}}' - ({{\textbf {x}}}'){^\top }{{\textbf {L}}}{{\textbf {x}}}' - (2{{\textbf {b}}}{^\top }{{\textbf {x}}} - {{\textbf {x}}}{^\top }{{\textbf {L}}}{{\textbf {x}}}) \\&= 2{{\textbf {b}}}{^\top }({{\textbf {x}}} + \Delta \cdot \mathbb {1}_C) - 2{{\textbf {b}}}{^\top }{{\textbf {x}}} - ({{\textbf {x}}}'){^\top }{{\textbf {L}}}{{\textbf {x}}}' + {{\textbf {x}}}{^\top }{{\textbf {L}}}{{\textbf {x}}} \\&= 2\Delta \cdot {{\textbf {b}}}{^\top }\mathbb {1}_C - \sum _{ij \in E}\frac{1}{r(i, j)}\left[ (x'(i) - x'(j))^2 - (x(i) - x(j))^2 \right] \\&= 2\Delta \cdot {{\textbf {b}}}{^\top }\mathbb {1}_C - \sum _{ij \in \delta (C)}\frac{1}{r(i, j)}\left[ (x'(i) - x'(j))^2 - (x(i) - x(j))^2 \right] \\&= 2\Delta \cdot {{\textbf {b}}}{^\top }\mathbb {1}_C - \sum _{\begin{array}{c} i \in C, \, j \not \in C \\ ij \in \delta (C) \end{array}}\frac{1}{r(i, j)}\left[ (x(i)+\Delta - x(j))^2 - (x(i) - x(j))^2 \right] \\&= 2\Delta \cdot {{\textbf {b}}}{^\top }\mathbb {1}_C - \sum _{\begin{array}{c} i \in C, \, j \not \in C \\ ij \in \delta (C) \end{array}}\frac{1}{r(i,j)}\left[ 2\Delta \cdot (x(i)-x(j)) +\Delta ^2\right] \\&= 2\Delta \cdot {{\textbf {b}}}{^\top }\mathbb {1}_C - 2\Delta \cdot f(C) - \Delta ^2 \sum _{(i,j) \in \delta (C)} \frac{1}{r(i,j)} \\&= 2\Delta \cdot {{\textbf {b}}}{^\top }\mathbb {1}_C - 2\Delta \cdot f(C) - \Delta ^2\cdot R(C)^{-1} \\&= 2\Delta \cdot b(C) - 2\Delta \cdot f(C) - \Delta ^2 R(C)^{-1} \\&= 2\Delta ^2 R(C)^{-1} - \Delta ^2 R(C)^{-1} \\&= \Delta ^2/R(C). \end{aligned}$$

\(\square \)

Lemma 2

We have \({{\,\textrm{gap}\,}}({{\textbf {f}}}, {{\textbf {x}}}) = \frac{1}{2}\sum _{(i, j) \in \vec {E}} r(i, j) \left( f(i, j) - \frac{x(i)-x(j)}{r(i, j)}\right) ^2.\)

Proof

By definition, we have

$$\begin{aligned} 2{{\,\textrm{gap}\,}}({{\textbf {f}}}, {{\textbf {x}}})&= \sum _{e \in E} r(e)f(e)^2 - (2{{\textbf {b}}}{^\top }{{\textbf {x}}} - {{\textbf {x}}}{^\top }{{\textbf {L}}}{{\textbf {x}}}). \end{aligned}$$

Note that

$$\begin{aligned} {{\textbf {b}}}{^\top }{{\textbf {x}}}&= \sum _{i \in V}b(i)x(i) \\&= \sum _{i \in V} x(i) \left( \sum _{j: (i,j) \in \vec {E}} f(i,j) - \sum _{j:(j,i) \in \vec {E}} f(j,i)\right) \\&= \sum _{(i,j) \in \vec {E}}f(i,j)(x(i) - x(j)) \end{aligned}$$

and

$$\begin{aligned} {{\textbf {x}}}{^\top }{{\textbf {L}}}{{\textbf {x}}}&= \sum _{(i,j) \in \vec {E}} \frac{(x(i) - x(j))^2}{r(i,j)}. \end{aligned}$$

Plugging these into our expression for \({{\,\textrm{gap}\,}}({{\textbf {f}}}, {{\textbf {x}}})\), we obtain

$$\begin{aligned} 2{{\,\textrm{gap}\,}}({{\textbf {f}}}, {{\textbf {x}}})&= \sum _{(i,j) \in \vec {E}}\left[ r(i,j)f(i,j)^2 - 2f(i,j)(x(i) - x(j)) + \frac{(x(i) - x(j))^2}{r(i,j)}\right] \\&= \sum _{(i,j) \in \vec {E}} r(i,j)\left( f(i,j) - \frac{x(i)-x(j)}{r(i,j)}\right) ^2 \end{aligned}$$

which is what we wanted to show. \(\square \)

Lemma 3

Let T be a spanning tree, \({{\textbf {x}}}\) vertex potentials, and \({{\textbf {b}}}\) a supply vector. Let \({{\textbf {f}}}_{T, {{\textbf {x}}}}\) be the associated tree-defined flow. Then

$$\begin{aligned} {{\,\textrm{gap}\,}}({{\textbf {f}}}_{T, {{\textbf {x}}}}, {{\textbf {x}}}) = \frac{1}{2} \sum _{(i, j) \in T} r(i, j) \cdot \frac{\Delta (C(i, j))^2}{R(C(i, j))^2}. \end{aligned}$$

Proof

Recall that C(ij), \(\Delta (C(i,j))\) and R(C(ij)) were defined as follows:

  • C(ij) is the set of vertices on the side of the fundamental cut of T determined by (ij) containing i. In other words, C(ij) consists of the vertices in the component of \(T - ij\) with \(i \in C(i,j)\) and \(j \not \in C(i,j)\).

  • \(R(C(i,j)) = \left( \sum _{ij \in \delta (C)} \frac{1}{r(i,j)}\right) ^{-1}\).

  • \(\Delta (C(i,j)) = (b(C(i,j)) - f(C(i,j)))R(C(i,j))\), where

    • \(b(C(i,j)) = {{\textbf {b}}}{^\top }\mathbb {1}_{C(i,j)}\), and

    • \(f(C(i,j)) = \displaystyle \sum _{\begin{array}{c} k \in C(i,j), \, l \not \in C(i,j) \\ kl \in E \end{array}} \frac{x(k) - x(l)}{r(k, l)}\)

We have

$$\begin{aligned} 2{{\,\textrm{gap}\,}}({{\textbf {f}}}_{T, {{\textbf {x}}}}, {{\textbf {x}}})&= \sum _{(i, j) \in E} r(i, j) \left( f_{T,{{\textbf {x}}}}(i, j) - \frac{x(i)-x(j)}{r(i, j)}\right) ^2 \\&= \sum _{(i, j) \in T} r(i, j) \left( f_{T,{{\textbf {x}}}}(i, j) - \frac{x(i)-x(j)}{r(i, j)}\right) ^2 \\&= \sum _{(i, j) \in T} r(i, j) \left[ \left( b(C(i, j)) - \sum _{\begin{array}{c} k \in C(i, j), l \not \in C(i, j)\\ kl \in E - ij \end{array}} \frac{x(k) - x(l)}{r(k, l)}\right) - \frac{x(i) - x(j)}{r(i, j)} \right] ^2 \\&= \sum _{(i, j) \in T} r(i, j) \left[ b(C(i,j)) - \sum _{\begin{array}{c} k \in C(i, j), l \not \in C(i, j)\\ kl \in E \end{array}} \frac{x(k) - x(l)}{r(k, l)}\right] ^2 \\&= \sum _{(i, j) \in T} r(i, j) \left[ b(C(i,j)) - f(C(i, j))\right] ^2\\&= \sum _{(i, j) \in T} r(i, j)\cdot \frac{\Delta (C(i, j))^2}{R(C(i, j))^2} \end{aligned}$$

\(\square \)

Lemma 4

If each iteration of the algorithm samples an edge \((i, j) \in T\) according to the probabilities \(P_{ij} = \frac{1}{\tau } \cdot \frac{r(i, j)}{R(C(i, j))}\), then we have

$$\begin{aligned} {\mathcal {B}}({{\textbf {x}}}^*) - {\mathbb {E}}[{\mathcal {B}}({{\textbf {x}}}^{t+1})] \le \left( 1 - \frac{1}{\tau }\right) \left( {\mathcal {B}}({{\textbf {x}}}^*) - {\mathcal {B}}({{\textbf {x}}}^t)\right) . \end{aligned}$$

Proof

We know from the discussion above that

$$\begin{aligned} {\mathbb {E}}[{\mathcal {B}}({{\textbf {x}}}^{t+1})] - {\mathcal {B}}({{\textbf {x}}}^t) = \frac{1}{\tau }{{\,\textrm{gap}\,}}({{\textbf {f}}}_{T, {{\textbf {x}}}}, {{\textbf {x}}}^t), \end{aligned}$$

where \({{\textbf {f}}}_{T, {{\textbf {x}}}}\) is the tree-defined flow associated with potentials \({{\textbf {x}}}^t\). Since \({{\,\textrm{gap}\,}}({{\textbf {f}}}_{T, {{\textbf {x}}}}, {{\textbf {x}}}^t) \ge {\mathcal {B}}({{\textbf {x}}}^*) - {\mathcal {B}}({{\textbf {x}}}^t)\), we get

$$\begin{aligned} {\mathbb {E}}[{\mathcal {B}}({{\textbf {x}}}^{t+1})] - {\mathcal {B}}({{\textbf {x}}}^t) \ge \frac{1}{\tau }\left( {\mathcal {B}}({{\textbf {x}}}^*) - {\mathcal {B}}({{\textbf {x}}}^t)\right) . \end{aligned}$$

Rearranging gives

$$\begin{aligned} {\mathcal {B}}({{\textbf {x}}}^*) - {\mathbb {E}}[{\mathcal {B}}({{\textbf {x}}}^{t+1})] \le \left( 1 - \frac{1}{\tau }\right) \left( {\mathcal {B}}({{\textbf {x}}}^*) - {\mathcal {B}}({{\textbf {x}}}^t)\right) , \end{aligned}$$

as desired. \(\square \)

Corollary 1

After \(K = \tau \ln (\frac{1}{\epsilon })\) iterations, \({\mathcal {B}}({{\textbf {x}}}^*) - {\mathbb {E}}[{\mathcal {B}}({{\textbf {x}}}^K)] \le \epsilon \cdot {\mathcal {B}}({{\textbf {x}}}^*)\).

Proof

Define the random variable \(D_t:= {\mathcal {B}}({{\textbf {x}}}^*) - {\mathcal {B}}({{\textbf {x}}}^t)\). By Lemma 4, we know that

$$\begin{aligned} {\mathbb {E}}\left[ D^{t+1} \mid {{\textbf {x}}}^t\right] \le \left( 1 - \frac{1}{\tau }\right) {\mathbb {E}}\left[ D^t \mid {{\textbf {x}}}^t\right] \end{aligned}$$

for all possible vectors of potentials \({{\textbf {x}}}^t\). This implies that \({\mathbb {E}}\left[ D^{t+1}\right] \le \left( 1 - \frac{1}{\tau }\right) {\mathbb {E}}\left[ D^t \right] \) unconditionally.

By induction on t, it then follows that

$$\begin{aligned} {\mathbb {E}}\left[ D^K\right] \le \left( 1 - \frac{1}{\tau }\right) ^K{\mathbb {E}}\left[ D^0\right] =\left( 1 - \frac{1}{\tau }\right) ^K\left( {\mathcal {B}}({{\textbf {x}}}^*) - {\mathcal {B}}({{\textbf {x}}}^0)\right) = \left( 1 - \frac{1}{\tau }\right) ^K{\mathcal {B}}({{\textbf {x}}}^*). \end{aligned}$$

Thus,

$$\begin{aligned} {\mathcal {B}}({{\textbf {x}}}^*) - {\mathbb {E}}[{\mathcal {B}}({{\textbf {x}}}^K)] \le \left( 1 - \frac{1}{\tau }\right) ^K{\mathcal {B}}({{\textbf {x}}}^*). \end{aligned}$$

Using the inequality \(1-x \le e^{-x}\), we obtain

$$\begin{aligned} {\mathcal {B}}({{\textbf {x}}}^*) - {\mathbb {E}}[{\mathcal {B}}({{\textbf {x}}}^K)] \le e^{-K/\tau } {\mathcal {B}}({{\textbf {x}}}^*). \end{aligned}$$

Hence, if \(K \ge \tau \ln (\frac{1}{\epsilon })\), then we will have \({\mathcal {B}}({{\textbf {x}}}^*) - {\mathbb {E}}[{\mathcal {B}}({{\textbf {x}}}^K)] \le \epsilon \cdot {\mathcal {B}}({{\textbf {x}}}^*)\), as desired. \(\square \)

Theorem 1

Let \(\tau \) be the total stretch of T. After \(K = \tau \ln (\frac{\tau }{\epsilon })\) iterations, Dual KOSZ returns \({{\textbf {x}}}^K \in {\mathbb {R}}^V\) and \({{\textbf {f}}}^K \in {\mathbb {R}}^{\vec {E}}\) such that \({\mathbb {E}}\left\Vert {{{\textbf {x}}}^* - {{\textbf {x}}}^K}\right\Vert _{{\textbf {L}}}^2 \le \frac{\epsilon }{\tau }\left\Vert {{{\textbf {x}}}^*}\right\Vert _{{\textbf {L}}}^2\) and \({\mathbb {E}}[{\mathcal {E}}({{\textbf {f}}}^K)] \le (1+\epsilon ) {\mathcal {E}}({{\textbf {f}}}^*)\), for \({{\textbf {f}}}^*\) and \({{\textbf {x}}}^*\) optimal primal and dual solutions respectively.

Proof of Theorem 1

By Corollary 1, after \(K=\tau \ln (\frac{\tau }{\epsilon })\) iterations, the algorithm returns potentials \({{\textbf {x}}}^K\) such that \({\mathcal {B}}({{\textbf {x}}}^*) - {\mathbb {E}}[{\mathcal {B}}({{\textbf {x}}}^K)] \le \frac{\epsilon }{\tau } \cdot {\mathcal {B}}({{\textbf {x}}}^*)\). Combining with Lemma 13, we get that \({\mathbb {E}}\left\Vert {{{\textbf {x}}}^* - {{\textbf {x}}}^K}\right\Vert _{{\textbf {L}}}^2 \le \frac{\epsilon }{\tau }\left\Vert {{{\textbf {x}}}^*}\right\Vert _{{\textbf {L}}}^2\). Finally, Lemma 14 gives \({\mathbb {E}}\left[ {\mathcal {E}}({{\textbf {f}}}^K)\right] \le (1+\epsilon ){\mathcal {E}}({{\textbf {f}}}^*)\). \(\square \)

Lemma 12

We have \(\tau = \text {st}_T(G, {{\textbf {r}}})\).

Proof

We write out the definitions of \(\tau \) and \(\text {st}_T(G, {{\textbf {r}}})\):

$$\begin{aligned} \tau = \sum _{(i, j) \in T} \frac{r(i, j)}{R(C(i, j))} = \sum _{(i, j) \in T} r(i, j)\sum _{(k,l) \in \delta (C(i, j))} \frac{1}{r(k, l)} \end{aligned}$$

and

$$\begin{aligned} \text {st}_T(G, {{\textbf {r}}}) = \sum _{(i, j) \in \vec {E}} \text {st}_T((i, j), {{\textbf {r}}}) = \sum _{(i, j) \in \vec {E}} \frac{1}{r(i, j)}\sum _{(k, l) \in P(i,j)} r(k, l), \end{aligned}$$

where P(ij) is the unique path from i to j in T.

It turns out that the expressions for \(\tau \) and \(\text {st}_T(G)\) are summing exactly the same terms, just in different ways. Indeed, we have

$$\begin{aligned} \tau&= \sum _{(i, j) \in T} \sum _{(k,l) \in \delta (C(i, j))}\frac{r(i, j)}{r(k, l)} \\&= \sum _{(k, l) \in \vec {E}} \sum _{(i, j) \in P(k, l)} \frac{r(i, j)}{r(k, l)} \\&= \text {st}_T(G, {{\textbf {r}}}). \end{aligned}$$

To switch the order of summation from the first line to the second line, we used the fact that for an edge \((k, l) \in \vec {E}\), we have \((k,l) \in \delta (C(i, j))\) if and only if \((i, j) \in P(k, l)\). This is because T is a spanning tree. \(\square \)

By Corollary 1, we know that the potentials \({{\textbf {x}}}^t\) found by the algorithm satisfy the property that \({\mathcal {B}}({{\textbf {x}}}^t)\) converges to \({\mathcal {B}}({{\textbf {x}}}^*)\) at a linear rate, in expectation. The following lemma shows that if \({{\textbf {x}}}\) is a set of potentials such that \({\mathcal {B}}({{\textbf {x}}})\) is close to \({\mathcal {B}}({{\textbf {x}}}^*)\), then \({{\textbf {x}}}\) is close to \({{\textbf {x}}}^*\) as a vector (measured in the matrix norm defined by the Laplacian \({{\textbf {L}}}\)).

Lemma 13

Let \({{\textbf {x}}}\) be any vector of potentials. Then \(\frac{1}{2}\left\Vert {{{\textbf {x}}}^* - {{\textbf {x}}}}\right\Vert _{{\textbf {L}}}^2 = {\mathcal {B}}({{\textbf {x}}}^*) - {\mathcal {B}}({{\textbf {x}}}).\) In particular, if \({\mathcal {B}}({{\textbf {x}}}^*) - {\mathcal {B}}({{\textbf {x}}}) \le \epsilon \cdot {\mathcal {B}}({{\textbf {x}}}^*)\), then \(\left\Vert {{{\textbf {x}}}^* - {{\textbf {x}}}}\right\Vert _{{\textbf {L}}}^2 \le \epsilon \left\Vert {{{\textbf {x}}}^*}\right\Vert _{{\textbf {L}}}^2.\)

Proof

We have

$$\begin{aligned} \left\Vert {{{\textbf {x}}}^* - {{\textbf {x}}}}\right\Vert _L^2&= ({{\textbf {x}}}^* - {{\textbf {x}}}){^\top }{{\textbf {L}}} ({{\textbf {x}}}^* - {{\textbf {x}}}) \\&= ({{\textbf {x}}}^*){^\top }{{\textbf {L}}}{{\textbf {x}}}^* - 2{{\textbf {x}}}{^\top }{{\textbf {L}}}{{\textbf {x}}}^* + {{\textbf {x}}}{^\top }{{\textbf {L}}}{{\textbf {x}}} \\&= 2{\mathcal {B}}({{\textbf {x}}}^*) - 2{{\textbf {x}}}{^\top }{{\textbf {b}}} + {{\textbf {x}}}{^\top }{{\textbf {L}}}{{\textbf {x}}} \\&= 2{\mathcal {B}}({{\textbf {x}}}^*) - 2{\mathcal {B}}({{\textbf {x}}}). \end{aligned}$$

In particular, if \({\mathcal {B}}({{\textbf {x}}}^*) - {\mathcal {B}}({{\textbf {x}}}) \le \epsilon \cdot {\mathcal {B}}({{\textbf {x}}}^*)\), then \(\left\Vert {{{\textbf {x}}}^* - {{\textbf {x}}}}\right\Vert _{{\textbf {L}}}^2 \le 2\epsilon \cdot {\mathcal {B}}({{\textbf {x}}}^*) = \epsilon \left\Vert {{{\textbf {x}}}^*}\right\Vert _{{\textbf {L}}}^2\). This is because

$$\begin{aligned} 2{\mathcal {B}}({{\textbf {x}}}^*) = 2{{\textbf {b}}}{^\top }{{\textbf {x}}}^* - ({{\textbf {x}}}^*){^\top }{{\textbf {L}}}{{\textbf {x}}}^* = ({{\textbf {x}}}^*){^\top }{{\textbf {L}}}{{\textbf {x}}}^* = \left\Vert {{{\textbf {x}}}^*}\right\Vert _{{\textbf {L}}}^2. \end{aligned}$$

\(\square \)

Next, we show that if \({\mathcal {B}}({{\textbf {x}}})\) is sufficiently close to \({\mathcal {B}}({{\textbf {x}}}^*)\), then the associated tree-defined flow \({{\textbf {f}}}_{T, {{\textbf {x}}}}\) has energy sufficiently close to \({\mathcal {E}}({{\textbf {f}}}^*)\).

Lemma 14

For any distribution over \({{\textbf {x}}}\) such that \({\mathbb {E}}_{{{\textbf {x}}}}[{\mathcal {B}}({{\textbf {x}}})] \ge (1-\frac{\epsilon }{\tau }){\mathcal {B}}({{\textbf {x}}}^*)\), we have \({\mathbb {E}}_{{{\textbf {x}}}}[{\mathcal {E}}({{\textbf {f}}}_{T, {{\textbf {x}}}})] \le (1+\epsilon ){\mathcal {E}}({{\textbf {f}}}^*)\).

Proof

For ease of notation, in this proof let \({{\textbf {f}}}= {{\textbf {f}}}_{T, {{\textbf {x}}}}\). (Note that \({{\textbf {f}}}\) is a random vector that is a function of \({{\textbf {x}}}\).) We have \({\mathbb {E}}_{{{\textbf {x}}}}[{\mathcal {E}}({{\textbf {f}}}) - {\mathcal {E}}({{\textbf {f}}}^*)] = {\mathbb {E}}_{{\textbf {x}}}[{{\,\textrm{gap}\,}}({{\textbf {f}}}, {{\textbf {x}}}^*)]\).

For a fixed choice of \({{\textbf {x}}}\), consider running the algorithm for one more iteration starting from \({{\textbf {x}}}\) to obtain a vector \({{\textbf {x}}}'\). Then we have \({\mathbb {E}}[{\mathcal {B}}({{\textbf {x}}}')] - {\mathcal {B}}({{\textbf {x}}}) = \frac{1}{\tau }{{\,\textrm{gap}\,}}({{\textbf {f}}}, {{\textbf {x}}})\). This implies \({\mathcal {B}}({{\textbf {x}}}^*) - {\mathcal {B}}({{\textbf {x}}}) \ge \frac{1}{\tau }{{\,\textrm{gap}\,}}({{\textbf {f}}}, {{\textbf {x}}})\). Taking expectations with respect to \({{\textbf {x}}}\), we get \({\mathbb {E}}_{{{\textbf {x}}}}[{\mathcal {B}}({{\textbf {x}}}^*) - {\mathcal {B}}({{\textbf {x}}})] \ge \frac{1}{\tau }{\mathbb {E}}_{{{\textbf {x}}}}[ {{\,\textrm{gap}\,}}({{\textbf {f}}}, {{\textbf {x}}})]\). Thus,

$$\begin{aligned} {\mathbb {E}}_{{{\textbf {x}}}}[{\mathcal {E}}({{\textbf {f}}}) - {\mathcal {E}}({{\textbf {f}}}^*)]&= {\mathbb {E}}_{{{\textbf {x}}}}[{{\,\textrm{gap}\,}}({{\textbf {f}}}, {{\textbf {x}}}^*)] \\&= {\mathbb {E}}_{{{\textbf {x}}}}[{{\,\textrm{gap}\,}}({{\textbf {f}}}, {{\textbf {x}}}) - ({\mathcal {B}}({{\textbf {x}}}^*) - {\mathcal {B}}({{\textbf {x}}}))] \\&\le (\tau -1){\mathbb {E}}_{{\textbf {x}}}[{\mathcal {B}}({{\textbf {x}}}^*) - {\mathcal {B}}({{\textbf {x}}})] \\&\le \tau {\mathbb {E}}_{{\textbf {x}}}[{\mathcal {B}}({{\textbf {x}}}^*) - {\mathcal {B}}({{\textbf {x}}})] \\&\le \epsilon {\mathcal {B}}({{\textbf {x}}}^*) \\&= \epsilon {\mathcal {E}}({{\textbf {f}}}^*). \end{aligned}$$

\(\square \)

B Spectral Approximations

Let \({{\textbf {A}}}, {{\textbf {B}}}\) be \(n \times n\) symmetric, positive semidefinite matrices. We say that \({{\textbf {B}}}\) is a \(\gamma \)-spectral sparsifier of \({{\textbf {A}}}\) if

$$\begin{aligned} \left( 1 - \gamma \right) {{\textbf {A}}}\preceq {{\textbf {B}}}\preceq \left( 1 + \gamma \right) {{\textbf {A}}}. \end{aligned}$$

Proposition 1

(Spectral Approximations) Suppose \({{\textbf {A}}}, {{\textbf {B}}}\in {\mathbb {S}}^n_+\) and \(\left( 1 - \gamma \right) {{\textbf {A}}}\preceq {{\textbf {B}}}\preceq \left( 1 + \gamma \right) {{\textbf {A}}}\). Let \({{\textbf {x}}}, {{\textbf {y}}}, {{\textbf {z}}}, {{\textbf {b}}}\in {\mathbb {R}}^n\). Then the following hold:

  1. 1.

    \((1-\gamma ) \left\Vert {{{\textbf {x}}}}\right\Vert _{{{\textbf {A}}}}^2 \le \left\Vert {{{\textbf {x}}}}\right\Vert _{{{\textbf {B}}}}^2 \le (1+\gamma )\left\Vert {{{\textbf {x}}}}\right\Vert _{{{\textbf {A}}}}^2\)

  2. 2.

    If \({{\textbf {A}}}{{\textbf {x}}}= {{\textbf {b}}}\) and \({{\textbf {B}}}{{\textbf {y}}}= {{\textbf {b}}}\), then \(\left\Vert {{{\textbf {x}}}- {{\textbf {y}}}}\right\Vert _{{{\textbf {A}}}}^2 \le h(\gamma ) \left\Vert {{{\textbf {x}}}}\right\Vert _{{{\textbf {A}}}}^2\), where \(h(\gamma ) = \frac{\gamma ^2}{(1-\gamma )^2}\).

Proof

The first one is by definition of \(\left\Vert {{{\textbf {x}}}}\right\Vert _{{{\textbf {A}}}}^2 = {{\textbf {x}}}^{\top } {{\textbf {A}}}{{\textbf {x}}}\).

For the second one, first we claim that it is sufficient to prove that \(\left\Vert {{{\textbf {A}}}^\dag {{\textbf {b}}}- {{\textbf {B}}}^\dag {{\textbf {b}}}}\right\Vert _{{{\textbf {A}}}}^2 \le h(\gamma ) \left\Vert {{{\textbf {A}}}^\dag {{\textbf {b}}}}\right\Vert _{{{\textbf {A}}}}^2\). This is because in general, we have \({{\textbf {x}}}= {{\textbf {A}}}^\dag {{\textbf {b}}}+ {{\textbf {u}}}\), and \({{\textbf {y}}}= {{\textbf {B}}}^\dag {{\textbf {b}}}+ {{\textbf {v}}}\), for some \({{\textbf {u}}}\in {{\,\textrm{Null}\,}}({{\textbf {A}}})\) and \({{\textbf {v}}}\in {{\,\textrm{Null}\,}}({{\textbf {B}}})\). Moreover, the condition \( \left( 1 - \gamma \right) {{\textbf {A}}}\preceq {{\textbf {B}}}\preceq \left( 1 + \gamma \right) {{\textbf {A}}}\) implies that \({{\,\textrm{Null}\,}}({{\textbf {A}}}) = {{\,\textrm{Null}\,}}({{\textbf {B}}})\). Hence, \(\left\Vert {{{\textbf {x}}}- {{\textbf {y}}}}\right\Vert _{{{\textbf {A}}}}^2 = \left\Vert {{{\textbf {A}}}^\dag {{\textbf {b}}}- {{\textbf {B}}}^\dag {{\textbf {b}}}}\right\Vert _{{{\textbf {A}}}}^2\), and \(\left\Vert {{{\textbf {x}}}}\right\Vert _{{{\textbf {A}}}}^2 = \left\Vert {{{\textbf {A}}}^\dag {{\textbf {b}}}}\right\Vert _{{{\textbf {A}}}}^2\).

Next, we expand \(\left\Vert {{{\textbf {A}}}^\dag {{\textbf {b}}}- {{\textbf {B}}}^\dag {{\textbf {b}}}}\right\Vert _{{{\textbf {A}}}}^2 \le h(\gamma ) \left\Vert {{{\textbf {A}}}^\dag {{\textbf {b}}}}\right\Vert _{{{\textbf {A}}}}^2\) into

$$\begin{aligned} ({{\textbf {A}}}^\dag {{\textbf {b}}}- {{\textbf {B}}}^\dag {{\textbf {b}}})^T {{\textbf {A}}}({{\textbf {A}}}^\dag {{\textbf {b}}}- {{\textbf {B}}}^\dag {{\textbf {b}}}) \le h(\gamma ) {{\textbf {b}}}^T {{\textbf {A}}}^\dag {{\textbf {b}}}, \end{aligned}$$

or equivalently,

$$\begin{aligned} {{\textbf {b}}}^T \left( {{\textbf {A}}}^\dag - {{\textbf {B}}}^\dag \right) {{\textbf {A}}}\left( {{\textbf {A}}}^\dag - {{\textbf {B}}}^\dag \right) {{\textbf {b}}}\le h(\gamma ) {{\textbf {b}}}^T {{\textbf {A}}}^\dag {{\textbf {b}}}. \end{aligned}$$

To prove the above inequality, it suffices to prove that

$$\begin{aligned} \left( {{\textbf {A}}}^\dag - {{\textbf {B}}}^\dag \right) {{\textbf {A}}}\left( {{\textbf {A}}}^\dag - {{\textbf {B}}}^\dag \right) \preceq h(\gamma ) {{\textbf {A}}}^\dag . \end{aligned}$$
(3)

Multiplying the left and right sides of Eq. 3 by \({{\textbf {A}}}^{\frac{1}{2}}\), we get that (3) is implied by

$$\begin{aligned} {{\textbf {A}}}^{\frac{1}{2}}\left( {{\textbf {A}}}^\dag - {{\textbf {B}}}^\dag \right) {{\textbf {A}}}\left( {{\textbf {A}}}^\dag - {{\textbf {B}}}^\dag \right) {{\textbf {A}}}^{\frac{1}{2}} \preceq h(\gamma ) {{\textbf {A}}}^{\frac{1}{2}} {{\textbf {A}}}^\dag {{\textbf {A}}}^{\frac{1}{2}}. \end{aligned}$$
(4)

Let \(\Pi := {{\textbf {A}}}^{\frac{1}{2}} {{\textbf {A}}}^\dag {{\textbf {A}}}^{\frac{1}{2}}\) be the projection map onto the row space of \({{\textbf {A}}}\). Note that \(\Pi = {{\textbf {A}}}^\dag {{\textbf {A}}}= {{\textbf {A}}}{{\textbf {A}}}^\dag \). Also, \(\Pi = {{\textbf {A}}}^{\frac{\dag }{2}}{{\textbf {A}}}^{\frac{1}{2}} = {{\textbf {A}}}^{\frac{1}{2}}{{\textbf {A}}}^{\frac{\dag }{2}}\). These can be seen using the spectral decomposition. Now, the reason why Eq. 4 implies Eq. 3 is because if we can multiply both sides of (4) with one copy of \( {{\textbf {A}}}^{\frac{\dag }{2}}\) on the left and one copy of \({{\textbf {A}}}^{\frac{\dag }{2}}\) on the right. Then Eq. 4 becomes \(\Pi \left( {{\textbf {A}}}^\dag - {{\textbf {B}}}^\dag \right) {{\textbf {A}}}\left( {{\textbf {A}}}^\dag - {{\textbf {B}}}^\dag \right) \Pi \preceq h(\gamma ) \Pi {{\textbf {A}}}^\dag \Pi .\) We have \(\Pi ({{\textbf {A}}}^{\dag } - {{\textbf {B}}}^{\dag }) = ({{\textbf {A}}}^{\dag } - {{\textbf {B}}}^{\dag })\Pi = {{\textbf {A}}}^{\dag } - {{\textbf {B}}}^{\dag }\) because \({{\textbf {A}}}\) and \({{\textbf {B}}}\) have the same null space. Similarly, \(\Pi {{\textbf {A}}}^{\dag } = {{\textbf {A}}}^{\dag } \Pi = {{\textbf {A}}}^{\dag }\).

To prove (4), first rewrite it as

$$\begin{aligned} \left( {{\textbf {A}}}^{\frac{1}{2}}\left( {{\textbf {A}}}^\dag - {{\textbf {B}}}^\dag \right) {{\textbf {A}}}^{\frac{1}{2}} \right) ^2 \preceq h(\gamma ) \Pi , \end{aligned}$$

or equivalently

$$\begin{aligned} \left( \Pi - {{\textbf {A}}}^{\frac{1}{2}}{{\textbf {B}}}^\dag {{\textbf {A}}}^\frac{1}{2} \right) ^2 \preceq h(\gamma ) \Pi . \end{aligned}$$

From the spectral approximation \(\left( 1 - \gamma \right) {{\textbf {A}}}\preceq {{\textbf {B}}}\preceq \left( 1 + \gamma \right) {{\textbf {A}}}\), we deduce that

$$\begin{aligned} \frac{1}{1 + \gamma } {{\textbf {A}}}^\dag \preceq {{\textbf {B}}}^\dag \preceq \frac{1}{1-\gamma } {{\textbf {A}}}^\dag , \end{aligned}$$

which, when multiplying on the left and right by \({{\textbf {A}}}^{\frac{1}{2}}\), implies that

$$\begin{aligned} \frac{1}{1 + \gamma } \Pi \preceq {{\textbf {A}}}^{\frac{1}{2}}{{\textbf {B}}}^\dag {{\textbf {A}}}^{\frac{1}{2}} \preceq \frac{1}{1-\gamma } \Pi . \end{aligned}$$

This in turn gives

$$\begin{aligned} \frac{-\gamma }{1 + \gamma } \Pi \preceq \Pi - {{\textbf {A}}}^{\frac{1}{2}}{{\textbf {B}}}^\dag {{\textbf {A}}}^{\frac{1}{2}} \preceq \frac{\gamma }{1-\gamma } \Pi . \end{aligned}$$

Observe that any eigenvector of \(\Pi - {{\textbf {A}}}^{\frac{1}{2}}{{\textbf {B}}}^\dag {{\textbf {A}}}^{\frac{1}{2}}\) is also an eigenvector of \(\Pi \) (they share eigenspaces because \({{\textbf {A}}}\) and \({{\textbf {B}}}\) have the same null spaces). Moreover, the eigenvalues of \(\Pi \) are 0 or 1. This implies that the eigenvalues of \(\Pi - {{\textbf {A}}}^{\frac{1}{2}}{{\textbf {B}}}^\dag {{\textbf {A}}}^\frac{1}{2}\) are all between \(\frac{-\gamma }{1 + \gamma }\) and \(\frac{\gamma }{1-\gamma }\). Hence, the eigenvalues of \(\left( \Pi - {{\textbf {A}}}^{\frac{1}{2}}{{\textbf {B}}}^\dag {{\textbf {A}}}^\frac{1}{2} \right) ^2\) are all between 0 and \( \frac{\gamma ^2}{(1-\gamma )^2}\), and thus \(\left( \Pi - {{\textbf {A}}}^{\frac{1}{2}}{{\textbf {B}}}^\dag {{\textbf {A}}}^\frac{1}{2} \right) ^2 \preceq \frac{\gamma ^2}{(1-\gamma )^2} \Pi \). \(\square \)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Henzinger, M., Jin, B., Peng, R. et al. A Combinatorial Cut-Toggling Algorithm for Solving Laplacian Linear Systems. Algorithmica 85, 3680–3716 (2023). https://doi.org/10.1007/s00453-023-01154-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00453-023-01154-8

Keywords

Navigation