The Laplacian Paradigm: Emerging Algorithms for Massive Graphs

  • Shang-Hua Teng
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6108)


This presentation describes an emerging paradigm for the design of efficient algorithms for massive graphs. This paradigm, which we will refer to as the Laplacian Paradigm, is built on a recent suite of nearly-linear time primitives in spectral graph theory developed by Spielman and Teng, especially their solver for linear systems A x = b, where A is the Laplacian matrix of a weighted, undirected n-vertex graph and b is an n-place vector.

In the Laplacian Paradigm for solving a problem (on a massive graph), we reduce the optimization or computational problem to one or multiple linear algebraic problems that can be solved efficiently by applying the nearly-linear time Laplacian solver. So far, the Laplacian paradigm already has some successes. It has been applied to obtain nearly-linear-time algorithms for applications in semi-supervised learning, image process, web-spam detection, eigenvalue approximation, and for solving elliptic finite element systems. It has also been used to design faster algorithms for generalized lossy flow computation and for random sampling of spanning trees.

The goal of this presentation is to encourage more researchers to consider the use of the Laplacian Paradigm to develop faster algorithms for solving fundamental problems in combinatorial optimization (e.g., the computation of matchings, flows and cuts), in scientific computing (e.g., spectral approximation), in machine learning and data analysis (such as for web-spam detection and social network analysis), and in other applications that involve massive graphs.


Span Tree Weighted Graph Laplacian Matrix Spectral Graph Theory Nest Dissection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abraham, I., Bartal, Y., Neiman, O.: Nearly Tight Low Stretch Spanning Trees. In: FOCS 2008, pp. 781–790 (2008)Google Scholar
  2. 2.
    Alon, N., Karp, R.M., Peleg, D., West, D.: A graph-theoretic game and its application to the k-server problem. SIAM Journal on Computing 24(1), 78–100 (1995)zbMATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    Alpert, C.J., Yao, S.-Z.: Spectral partitioning: the more eigenvectors, the better. In: DAC 1995: Proceedings of the 32nd ACM/IEEE conference on Design automation, pp. 195–200. ACM, New York (1995)CrossRefGoogle Scholar
  4. 4.
    Andersen, R., Borgs, C., Chayes, J.T., Hopcroft, J.E., Jain, K., Mirrokni, V.S., Teng, S.-H.: Robust PageRank and locally computable spam detection features. In: Fourth International Workshop on Adversarial Information Retrieval on the Web. ACM International Conference Proceeding Series, pp. 69–76 (2008)Google Scholar
  5. 5.
    Andersen, R., Borgs, C., Chayes, J.T., Hopcraft, J.E., Mirrokni, V.S., Teng, S.-H.: Local computation of pagerank contributions. In: Bonato, A., Chung, F.R.K. (eds.) WAW 2007. LNCS, vol. 4863, pp. 150–165. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  6. 6.
    Andersen, R., Chung, F., Lang, K.: Local graph partitioning using pagerank vectors. In: Proceedings: 47th Annual Symposium on Foundations of Computer Science, pp. 475–486 (2006)Google Scholar
  7. 7.
    Andersen, R., Peres, Y.: Finding sparse cuts locally using evolving sets. In: STOC, pp. 235–244 (2009)Google Scholar
  8. 8.
    Arora, S., Hazan, E., Kale, S.: 0(sqrt (log n)) approximation to Sparsest Cut in õ(n2) time. In: FOCS: IEEE Symposium on Foundations of Computer Science (FOCS), pp. 238–247 (2004)Google Scholar
  9. 9.
    Arora, S., Kale, S.: A combinatorial, primal-dual approach to semidefinite programs. In: STOC 2007: Proceedings of the thirty-ninth annual ACM symposium on Theory of computing, pp. 227–236. ACM, New York (2007)CrossRefGoogle Scholar
  10. 10.
    Arora, S., Rao, S., Vazirani, U.: Expander flows, geometric embeddings and graph partitioning. In: STOC 2004: Proceedings of the thirty-sixth annual ACM symposium on Theory of computing, pp. 222–231. ACM, New York (2004)CrossRefGoogle Scholar
  11. 11.
    Batson, J., Spielman, D.A., Srivastava, N.: Twice-Ramanujan sparsifiers (2008),
  12. 12.
    Bebendorf, M., Hackbusch, W.: Existence of H-matrix approximants to the inverse FE-matrix of elliptic operators with L  ∞ -coefficients. Numerische Mathematik 95(1), 1–28 (2003)zbMATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Benczúr, A.A., Karger, D.R.: Approximating s-t minimum cuts in O(n2) time. In: Proceedings of The Twenty-Eighth Annual ACM Symposium on the Theory of Computing, STOC 1996 (1996)Google Scholar
  14. 14.
    Bern, M., Gilbert, J., Hendrickson, B., Nguyen, N., Toledo, S.: Support-graph preconditioners. SIAM J. Matrix Anal. and Appl. 27(4), 930–951 (2006)zbMATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Boman, E.G., Hendrickson, B.: Support theory for preconditioning. SIAM Journal on Matrix Analysis and Applications 25(3), 694–717 (2003)zbMATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Boman, E.G., Hendrickson, B., Vavasis, S.: Solving epplitic finite element systems in nearly-linear time with support preconditionersGoogle Scholar
  17. 17.
    Briggs, W.L., Henson, V.E., McCormick, S.F.: A Multigrid Tutorial, 2nd edn. SIAM, Philadelphia (2001)Google Scholar
  18. 18.
    Cheeger, J.: A lower bound for smallest eigenvalue of laplacian. In: Gunning, R.C. (ed.) Problems in Analysis, pp. 195–199. Princeton University Press, Princeton (1970)Google Scholar
  19. 19.
    Chew, P.: There is a planar graph almost as good as the complete graph. In: SCG 1986: Proceedings of the second annual symposium on Computational geometry, pp. 169–177. ACM, New York (1986)CrossRefGoogle Scholar
  20. 20.
    Cormen, T., Leiserson, C., Rivest, R., Stein, C.: Introduction to Algorithms, 3rd edn.Google Scholar
  21. 21.
    Daitch, S.I., Spielman, D.A.: Support-graph preconditioners for 2-dimensional trussesGoogle Scholar
  22. 22.
    Daitch, S.I., Spielman, D.A.: Faster approximate lossy generalized flow via interior point algorithms. In: Proceedings of the 40th Annual ACM Symposium on Theory of Computing, pp. 451–460 (2008)Google Scholar
  23. 23.
    Elkin, M., Emek, Y., Spielman, D.A., Teng, S.-H.: Lower-stretch spanning trees. SIAM Journal on Computing 32(2), 608–628 (2008)CrossRefMathSciNetGoogle Scholar
  24. 24.
    Frieze, A.M., Miller, G.L., Teng, S.-H.: Separator Based Parallel Divide and Conquer in Computational Geometry. In: SPAA 1992, pp. 420–429 (1992)Google Scholar
  25. 25.
    George, J.A.: Nested dissection of a regular finite element mesh. SIAM J. Numer. Anal. 10, 345–363 (1973)zbMATHCrossRefMathSciNetGoogle Scholar
  26. 26.
    Gilbert, J.R., Tarjan, R.E.: The analysis of a nested dissection algorithm. Numerische Mathematik 50(4), 377–404 (1987)zbMATHCrossRefMathSciNetGoogle Scholar
  27. 27.
    Golub, G.H., Overton, M.: The convergence of inexact Chebychev and Richardson iterative methods for solving linear systems. Numerische Mathematik 53, 571–594 (1988)zbMATHCrossRefMathSciNetGoogle Scholar
  28. 28.
    Gremban, K.: Combinatorial Preconditioners for Sparse, Symmetric, Diagonall y Dominant Linear Systems. PhD thesis, Carnegie Mellon University, CMU-CS-96-123 (1996)Google Scholar
  29. 29.
    Gulli, A., Signorini, A.: The indexable web is more than 11.5 billion pages. In: WWW 2005: Special interest tracks and posters of the 14th international conference on World Wide Web, pp. 902–903. ACM, New York (2005)CrossRefGoogle Scholar
  30. 30.
    Joshi, A.: Topics in Optimization and Sparse Linear Systems, Ph.D. thesis, UIUC (1997)Google Scholar
  31. 31.
    Khandekar, R., Rao, S., Vazirani, U.: Graph partitioning using single commodity flows. In: STOC 2006: Proceedings of the thirty-eighth annual ACM symposium on Theory of computing, pp. 385–390. ACM, New York (2006)CrossRefGoogle Scholar
  32. 32.
    Kolla, A., Makarychev, Y., Saberi, A., Teng, S.-H.: Subgraph Sparsification. In: STOC 2010 (2010)Google Scholar
  33. 33.
    Koutis, I., Miller, G., Peng, R.: Approaching optimality for solving SDD systemsGoogle Scholar
  34. 34.
    Koutis, I., Miller, G., Tolliver, D.: Combinatorial preconditioners and multilevel solvers for problems in computer vision and image processing. In: International Symp. of Visual Computing, pp. 1067–1078 (2009)Google Scholar
  35. 35.
    Lipton, R.J., Rose, D.J., Tarjan, R.E.: Generalized nested dissection. SIAM Journal on Numerical Analysis 16(2), 346–358 (1979)zbMATHCrossRefMathSciNetGoogle Scholar
  36. 36.
    Leighton, T., Rao, S.: Multicommodity max-flow min-cut theorems and their use in designing approximation algorithms. Journal of the ACM 46(6), 787–832 (1999)zbMATHCrossRefMathSciNetGoogle Scholar
  37. 37.
    Lovasz, L., Simonovits, M.: Random walks in a convex body and an improved volume algorithm. RSA: Random Structures & Algorithms 4, 359–412 (1993)zbMATHCrossRefMathSciNetGoogle Scholar
  38. 38.
    Maggs, B., Miller, G., Parekh, O., Ravi, R., Woo, S.M.: Finding effective support-tree preconditioners. In: ACM SPAA, pp. 176–185 (2005)Google Scholar
  39. 39.
    Madry, A., Kelner, J.: Faster generation of random spanning trees. In: FOCS (2009)Google Scholar
  40. 40.
    Miller, G.L., Teng, S.-H., Thurston, W.P., Vavasis, S.A.: Separators for sphere-packings and nearest neighbor graphs. J. ACM 44(1), 1–29 (1997)zbMATHCrossRefMathSciNetGoogle Scholar
  41. 41.
    Orecchia, L., Schulman, L.J., Vazirani, U.V., Vishnoi, N.K.: On partitioning graphs via single commodity flows. In: STOC 2008: Proceedings of the 40th annual ACM symposium on Theory of computing, pp. 461–470. ACM, New York (2008)CrossRefGoogle Scholar
  42. 42.
    Rief, J.: Efficient approximate solution of sparse linear systems. Computer and Mathematics with Applications 36(9), 37–58 (1998)CrossRefGoogle Scholar
  43. 43.
    Sharma, A., Liu, X., Miller, P., Nakano, A., Kalia, R.K., Vashishta, P., Zhao, W., Campbell, T.J., Haas, A.: Immersive and interactive exploration of billion-atom systems. In: VR 2002: Proceedings of the IEEE Virtual Reality Conference 2002, p. 217. IEEE Computer Society, Los Alamitos (2002)CrossRefGoogle Scholar
  44. 44.
    Shklarski, G., Toledo, S.: Rigidity in finite-element matrices: Sufficient conditions for the rigidity of structures and substructures. SIAM J. Matrix Anal. and Appl. 30(1), 7–40 (2008)zbMATHCrossRefMathSciNetGoogle Scholar
  45. 45.
    Spielman, D.: Graphs and networks: Random walks and spectral graph drawing. Computer Science, Yale (September 18, 2007),
  46. 46.
    Spielman, D.A., Srivastava, N.: Graph sparsification by effective resistances. In: Proceedings of the 40th annual ACM Symposium on Theory of Computing, pp. 563–568 (2008)Google Scholar
  47. 47.
    Spielman, D., Teng, S.-H.: Spectral partitioning works: planar graphs and finite element meshes. In: FOCS 1996: Proceedings of the 37th annual symposium on Foundations of Computer Science, pp. 96–105 (1996)Google Scholar
  48. 48.
    Spielman, D.A., Teng, S.-H.: Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems. In: Proceedings of the thirty-sixth annual ACM symposium on Theory of computing, pp. 81–90 (2003)Google Scholar
  49. 49.
    Spielman, D.A., Teng, S.-H.: A local clustering algorithm for massive graphs and its application to nearly-linear time graph partitioning. CoRR, abs/0809.3232 (2008),
  50. 50.
    Spielman, D.A., Teng, S.-H.: Nearly-linear time algorithms for preconditioning and solving symmetric, diagonally dominant linear systems. CoRR, abs/cs/0607105 (2008),
  51. 51.
    Spielman, D.A., Teng, S.-H.: Spectral sparsification of graphs. CoRR, abs/0808.4134 (2008),
  52. 52.
    Tolliver, D.A.: Spectral rounding and image segmentation. PhD thesis, Pittsburgh, PA, USA (2006); Adviser-Miller, G.L., Adviser-Collins, R.T.Google Scholar
  53. 53.
    Tolliver, D.A., Miller, G.L.: Graph partitioning by spectral rounding: Applications in image segmentation and clustering. In: CVPR 2006: Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1053–1060. IEEE Computer Society, Los Alamitos (2006)Google Scholar
  54. 54.
    Trefethen, L.N., Bau, D.: Numerical Linear Algebra. SIAM, Philadelphia (1997)zbMATHGoogle Scholar
  55. 55.
    Vaidya, P.: Solving linear equations with symmetric diagonally dominant matrices by constructing good preconditioners. An invited at IMA, U. Minnesota (October 1991)Google Scholar
  56. 56.
    Zhou, D., Huang, J., Schölkopf, B.: Learning from Labeled and Unlabeled Data on a Directed Graph. In: The 22nd International Conference on Machine Learning, pp. 1041–1048 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Shang-Hua Teng
    • 1
  1. 1.Computer Science DepartmentUniversity of Southern California 

Personalised recommendations