Multisection in the Stochastic Block Model Using Semidefinite Programming

Naman Agarwal; Afonso S. Bandeira; Konstantinos Koiliaris; Alexandra Kolla

doi:10.1007/978-3-319-69802-1_4

Compressed Sensing and its Applications pp 125-162 | Cite as

Multisection in the Stochastic Block Model Using Semidefinite Programming

Authors
Authors and affiliations

Naman Agarwal
Afonso S. Bandeira
Konstantinos Koiliaris
Alexandra Kolla

Chapter

First Online: 18 January 2018

2 Citations
902 Downloads

Part of the Applied and Numerical Harmonic Analysis book series (ANHA)

Abstract

We consider the problem of identifying underlying community-like structures in graphs. Toward this end, we study the stochastic block model (SBM) on k-clusters: a random model on n = km vertices, partitioned in k equal sized clusters, with edges sampled independently across clusters with probability q and within clusters with probability p, p > q. The goal is to recover the initial “hidden” partition of [n]. We study semidefinite programming (SDP)-based algorithms in this context. In the regime $p = \frac {\alpha \log (m)}{m}$ and $q = \frac {\beta \log (m)}{m}$, we show that a certain natural SDP-based algorithm solves the problem of exact recovery in the k-community SBM, with high probability, whenever $\sqrt {\alpha } - \sqrt {\beta } > \sqrt {1}$, as long as $k=o(\log n)$. This threshold is known to be the information theoretically optimal. We also study the case when $k=\theta (\log (n))$. In this case however, we achieve recovery guarantees that no longer match the optimal condition $\sqrt {\alpha } - \sqrt {\beta } > \sqrt {1}$, thus leaving achieving optimality for this range an open question.

Keywords

Graph partitioning Random models Stochastic block model Semidefinite programming Dual certificate

This is a preview of subscription content, to check access.

Notes

Acknowledgements

Most of the work presented in this paper was conducted while ASB was at Princeton University and partly conducted while ASB was at the Massachusetts Institute of Technology. ASB acknowledges support from AFOSR Grant No. FA9550-12-1-0317, NSF DMS-1317308, NSF DMS-1712730, and NSF DMS-1719545.

Appendix

Forms of Chernoff Bounds and Hoeffding Bounds Used in the Arguments

Theorem 7 (Chernoff)

Suppose X₁…X_n be independent random variables taking values in {0, 1}. Let X denote their sum and let $\mu = \mathbb {E}[X]$ be its expectation. Then for any δ > 0 it holds that

$$\displaystyle \begin{aligned} \mathbb{P}\left( X > (1 + \delta)\mu\right) < \left(\frac{e^{\delta}}{(1 + \delta)^{(1+\delta)}}\right)^{\mu}\:, \end{aligned} $$

(25)

$$\displaystyle \begin{aligned} \mathbb{P}\left( X < (1 - \delta)\mu\right) < \left(\frac{e^{-\delta}}{(1 - \delta)^{(1-\delta)}}\right)^{\mu} \:. \end{aligned} $$

(26)

A simplified form of the above bound is the following formula (for δ ≤ 1)

$$\displaystyle \begin{aligned}\mathbb{P}\left( X \geq (1 + \delta)\mu\right) \leq e^{-\frac{\delta^2 \mu}{3}}\:, \end{aligned}$$

$$\displaystyle \begin{aligned}\mathbb{P}\left( X \leq (1 - \delta)\mu\right) \leq e^{-\frac{\delta^2 \mu}{2}} \:.\end{aligned}$$

Theorem 8 (Bernstein)

Suppose X₁…X_n be independent random variables taking values in [−M, M]. Let X denote their sum and let $\mu = \mathbb {E}[X]$ be its expectation, then

$$\displaystyle \begin{aligned} \mathbb{P}\left( |X - \mu| \geq t \right) \leq \exp\left(-\frac{1}{2}\frac{t^2}{\sum_i \mathbb{E}[(X_i - \mathbb{E}[X_i])^2] + Mt/3}\right) \:. \end{aligned}$$

Corollary 1

Suppose X₁…X_n are i.i.d Bernoulli variables with parameter p. Let σ = σ(X_i) = p(1 − p); then we have that for any r ≥ 0

$$\displaystyle \begin{aligned}\mathbb{P}\left(X \geq \mu + \alpha\sigma\sqrt{n\log(r)}+ \alpha\log(r)\right) \leq e^{-\frac{\alpha\log(r)}{4}} \:.\end{aligned}$$

Proof

We have that nσ² = np(1 − p) and M = 1. We can now choose $t = \alpha \sigma \sqrt {n\log (r)} + \alpha \log (r)$. This implies that $\frac {n\sigma ^2 + t/3}{t^2} \leq \frac {1}{\log (r)}\left (1/\alpha ^2 + 1/3\alpha \right ) \leq \frac {2}{\alpha \log (r)} $ which implies from Theorem 8 that $\mathbb {P}\left (X > \mu + \alpha \sigma \sqrt {n\log (r)}+ \alpha \log (r)\right ) \leq e^{-\frac {\alpha \log (r)}{4}}.$ □

Theorem 9 (Hoeffding)

Let X₁…X_n be independent random variables. Assume that the X_i are bounded in the interval [a_i, b_i]. Define the empirical mean of these variables as

$$\displaystyle \begin{aligned} \bar{X} = \frac{\sum_i \bar{X_i}}{n} \:, \end{aligned}$$

then

$$\displaystyle \begin{aligned} \mathbb{P}\left( |\bar{X} - \mathbb{E}[\bar{X}]| \geq t \right) \leq 2\exp\left(- \frac{2n^2t^2}{\sum_{i = 1}^{n} (b_i - a_i)^2}\right) \:. \end{aligned} $$

(27)

References

1.
E. Abbe, C. Sandon, Community detection in general stochastic block models: fundamental limits and efficient recovery algorithms (2015). Available online at arXiv:1503.00609 [math.PR]Google Scholar
2.
E. Abbe, A.S. Bandeira, G. Hall, Exact recovery in the stochastic block model (2014). Available online at arXiv:1405.3267 [cs.SI]Google Scholar
3.
N. Alon, N. Kahale, A spectral technique for coloring random 3-colorable graphs. SIAM J. Comput. 26(6), 1733–1748 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
4.
N. Alon, M. Krivelevich, B. Sudakov, Finding a large hidden clique in a random graph, in Proceedings of the Ninth Annual ACM-SIAM Symposium on Discrete Algorithms, 25–27 January 1998, San Francisco, CA (1998), pp. 594–598zbMATHGoogle Scholar
5.
E. Arias-Castro, N. Verzelen, Community detection in random networks (2013). Available online at arXiv:1302.7099 [math.ST]Google Scholar
6.
P. Awasthi, A.S. Bandeira, M. Charikar, R. Krishnaswamy, S. Villar, R. Ward, Relax, no need to round: integrality of clustering formulations, in 6th Innovations in Theoretical Computer Science (ITCS 2015) (2015)CrossRefzbMATHGoogle Scholar
7.
A.S. Bandeira, Random Laplacian matrices and convex relaxations (2015). Available online at arXiv:1504.03987 [math.PR]Google Scholar
8.
A.S. Bandeira, R.V. Handel, Sharp nonasymptotic bounds on the norm of random matrices with independent entries. Ann. Probab. 44(4), 2479–2506 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
9.
A.S. Bandeira, M. Charikar, A. Singer, A. Zhu, Multireference alignment using semidefinite programming, in 5th Innovations in Theoretical Computer Science (ITCS 2014) (2014)zbMATHGoogle Scholar
10.
A.S. Bandeira, Y. Chen, A. Singer, Non-unique games over compact groups and orientation estimation in cryo-em (2015). Available at arXiv:1505.03840 [cs.CV]Google Scholar
11.
R.B. Boppana, Eigenvalues and graph bisection: an average-case analysis, in Proceedings of the 28th Annual Symposium on Foundations of Computer Science, SFCS ’87, Washington, DC (IEEE Computer Society, Washington, 1987), pp. 280–285Google Scholar
12.
T.N. Bui, S. Chaudhuri, F.T. Leighton, M. Sipser, Graph bisection algorithms with good average case behavior, in 25th Annual Symposium on Foundations of Computer Science, 24–26 October 1984, West Palm Beach, FL (1984), pp. 181–192Google Scholar
13.
M. Charikar, K. Makarychev, Y. Makarychev, Near-optimal algorithms for unique games, in Proceedings of the Thirty-eighth Annual ACM Symposium on Theory of Computing, STOC ’06, New York, NY (ACM, New York, 2006), pp. 205–214zbMATHGoogle Scholar
14.
Y. Chen, J. Xu, Statistical-computational tradeoffs in planted problems and submatrix localization with a growing number of clusters and submatrices (2014). Available online at arXiv:1402.1267 [stat.ML]Google Scholar
15.
P. Chin, A. Rao, V. Vu, Stochastic block model and community detection in the sparse graphs: A spectral algorithm with optimal rate of recovery (2015). Available online at: arXiv:1501.05021Google Scholar
16.
A. Condon, R.M. Karp, Algorithms for graph partitioning on the planted partition model. Random Struct. Algor. 18(2), 116–140 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
17.
A. Decelle, F. Krzakala, C. Moore, L. Zdeborová, Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications. Phys. Rev. E 84, 066106 (2011)CrossRefGoogle Scholar
18.
P. Erdös, A. Renyi, On random graphs. I. Publ. Math. 6, 290–297 (1959)zbMATHGoogle Scholar
19.
U. Feige, J. Kilian, Heuristics for semirandom graph problems. J. Comput. Syst. Sci. 63(4), 639–671 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
20.
A.M. Frieze, M. Jerrum, Improved approximation algorithms for max k-cut and max bisection, in Proceedings of the 4th International IPCO Conference on Integer Programming and Combinatorial Optimization (Springer-Verlag, London, 1995), pp. 1–13zbMATHGoogle Scholar
21.
B. Hajek, Y. Wu, J. Xu, Achieving exact cluster recovery threshold via semidefinite programming (2014). Available online at arXiv:1412.6156 [stat.ML]Google Scholar
22.
B. Hajek, Y. Wu, J. Xu, Achieving exact cluster recovery threshold via semidefinite programming: extensions (2015). Available online at arXiv:1502.07738 [stat.ML]Google Scholar
23.
R. Krauthgamer, J. Naor, R. Schwartz, Partitioning graphs into balanced components, in Proceedings of the Twentieth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA ’09 (Society for Industrial and Applied Mathematics, Philadelphia, PA, 2009), pp. 942–949CrossRefGoogle Scholar
24.
K. Makarychev, Y. Makarychev, A. Vijayaraghavan, Constant factor approximation for balanced cut in the PIE model, in Symposium on Theory of Computing, STOC 2014, New York, NY, May 31–June 03 (2014), pp. 41–49Google Scholar
25.
L. Massoulié, Community detection thresholds and the weak Ramanujan property, in Symposium on Theory of Computing, STOC 2014, New York, NY, May 31–June 03 (2014), pp. 694–703Google Scholar
26.
F. McSherry, Spectral partitioning of random graphs, in Proceedings of the 42nd IEEE Symposium on Foundations of Computer Science, FOCS ’01 (IEEE Computer SocietyWashington, DC, 2001), p. 529Google Scholar
27.
E. Mossel, J. Neeman, A. Sly, Stochastic block models and reconstruction (2012). Available online at arXiv:1202.1499Google Scholar
28.
E. Mossel, J. Neeman, A. Sly, A proof of the block model threshold conjecture (2013). Available online at arXiv:1311.4115Google Scholar
29.
E. Mossel, J. Neeman, A. Sly, Belief propagation, robust reconstruction and optimal recovery of block models, in Proceedings of The 27th Conference on Learning Theory, COLT 2014, Barcelona, June 13–15 (2014), pp. 356–370Google Scholar
30.
E. Mossel, J. Neeman, A. Sly, Consistency thresholds for binary symmetric block models (2014). Available online at arXiv: 1407.1591Google Scholar
31.
V. Vu, A simple SVD algorithm for finding hidden partitions. Available online at arXiv: 1404.3918 (2014)Google Scholar
32.
S.-Y. Yun, A. Proutiere, Accurate community detection in the stochastic block model via spectral algorithms (2014). Available online at arXiv: 1412.7335Google Scholar

Copyright information

Authors and Affiliations

Naman Agarwal
- 1
Afonso S. Bandeira
- 2
Email author
Konstantinos Koiliaris
- 3
Alexandra Kolla
- 3

1.Princeton UniversityPrincetonUSA
2.Department of Mathematics, Courant Institute of Mathematical Sciences and Center for Data ScienceNew York UniversityNew YorkUSA
3.University of Illinois Urbana - ChampaignUrbanaUSA

Multisection in the Stochastic Block Model Using Semidefinite Programming

Abstract

Keywords

Notes

Acknowledgements

References

Copyright information

Authors and Affiliations

Personalised recommendations

Cite chapter