Consistent Estimation of Mixed Memberships with Successive Projections

  • Maxim PanovEmail author
  • Konstantin Slavnov
  • Roman Ushakov
Conference paper
Part of the Studies in Computational Intelligence book series (SCI, volume 689)


This paper considers the parameter estimation problem in Mixed Membership Stochastic Block Model (MMSB), which is a quite general instance of random graph model allowing for overlapping community structure. We present the new algorithm successive projection overlapping clustering (SPOC) which combines the ideas of spectral clustering and geometric approach for separable non-negative matrix factorization. The proposed algorithm is provably consistent under MMSB with general conditions on the parameters of the model. SPOC is also shown to perform well experimentally in comparison to other algorithms.



The research was supported by the Russian Science Foundation grant (project 14-50-00150). The authors would like to thank Nikita Zhivotovskiy and Alexey Naumov for very insightful discussions on matrix concentration. The help of Emilie Kaufmann, who provided the code of SAAC algorithm, is especially acknowledged.


  1. 1.
    Airoldi, E.M., Blei, D.M., Fienberg, S.E., Xing, E.P.: Mixed membership stochastic blockmodels. J. Mach. Learn. Res. 9, 1981–2014 (2008)Google Scholar
  2. 2.
    Anandkumar, A., Ge, R., Hsu, D., Kakade, S.: A tensor spectral approach to learning mixed membership community models. In: Conference on Learning Theory, pp. 867–881 (2013)Google Scholar
  3. 3.
    Araujo, M.C.U., Saldanha, T.C.B., Galvao, R.K.H., Yoneyama, T., Chame, H.C., Visani, V.: The successive projections algorithm for variable selection in spectroscopic multicomponent analysis. Chemom. Intell. Lab. Syst. 57(2), 65–73 (2001).,
  4. 4.
    Arora, S., Ge, R., Kannan, R., Moitra, A.: Computing a nonnegative matrix factorization—provably. In: Proceedings of the Forty-fourth Annual ACM Symposium on Theory of Computing, STOC ’12, pp. 145–162. ACM, New York, NY, USA (2012).
  5. 5.
    Backstrom, L., Huttenlocher, D., Kleinberg, J., Lan, X.: Group formation in large social networks: membership, growth, and evolution. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 44–54. ACM. (2006)
  6. 6.
    Gillis, N., Vavasis, S.A.: Fast and robust recursive algorithmsfor separable nonnegative matrix factorization. IEEE Trans. Pattern Anal. Mach. Intell. 36(4), 698–714 (2014). CrossRefzbMATHGoogle Scholar
  7. 7.
    Gillis, N., Vavasis, S.A.: Semidefinite programming based preconditioning for more robust near-separable nonnegative matrix factorization. SIAM J. Optim. 25(1), 677–698 (2015). MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Girvan, M., Newman, M.E.: Community structure in social and biological networks. Proc. Natl. Acad. Sci. 99(12), 7821–7826. (2002)
  9. 9.
    Huang, K., Fu, X., Sidiropoulos, N.D.: Anchor-free correlated topic modeling: identifiability and algorithm. Adv. Neural Inf. Process. Syst. 1786–1794 (2016)Google Scholar
  10. 10.
    Kaufmann, E., Bonald, T., Lelarge, M.: A Spectral Algorithm with Additive Clustering for the Recovery of Overlapping Communities in Networks, pp. 355–370. Springer International Publishing, Cham (2016)Google Scholar
  11. 11.
    Leskovec, J., Mcauley, J.J.: Learning to discover social circles in ego networks. Adv. Neural Inf. Process. Syst. 539–547 (2012)Google Scholar
  12. 12.
    Li, Y., Liang, Y., Risteski, A.: Recovery guarantee of non-negative matrix factorization via alternating updates. ArXiv e-prints (2016)Google Scholar
  13. 13.
    Lu, Z., Sun, X., Wen, Y., Cao, G., La Porta, T.: Algorithms and applications for community detection in weighted networks. IEEE Trans. Parallel Distrib. Syst. 26(11), 2916–2926 (2015)CrossRefGoogle Scholar
  14. 14.
    Mao, X., Sarkar, P., Chakrabarti, D.: On mixed memberships and symmetric nonnegative matrix factorizations. In: Proceedings of the 34th International Conference on Machine Learning, vol. 70, pp. 2324–2333. PMLR, International Convention Centre, Sydney, Australia. (2017)
  15. 15.
    Mizutani, T.: Ellipsoidal rounding for nonnegative matrix factorization under noisy separability. J. Mach. Learn. Res. 15, 1011–1039. (2014)
  16. 16.
    Mizutani, T.: Robustness analysis of preconditioned successive projection algorithm for general form of separable NMF problem. Linear Algebr Appl 497, 1–22 (2016). MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Palla, G., Derényi, I., Farkas, I., Vicsek, T.: Uncovering the overlapping community structure of complex networks in nature and society. Nature 435, 9 (2005)CrossRefGoogle Scholar
  18. 18.
    Panov, M., Slavnov, K., Ushakov, R.: Consistent estimation of mixed memberships with successive projections (Supplement). ArXiv e-prints (2017)Google Scholar
  19. 19.
    Perozzi, B., Al-Rfou, R., Skiena, S.: Deepwalk: Online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’14, pp. 701–710. ACM, New York, NY, USA (2014).
  20. 20.
    Von Luxburg, U.: A tutorial on spectral clustering. Stat. Comput. 17(4), 395–416 (2007)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Yang, J., Leskovec, J.: Overlapping community detection at scale: a nonnegative matrix factorization approach. In: Proceedings of the sixth ACM international conference on Web search and data mining, pp. 587–596. ACM. (2013)
  22. 22.
    Zhang, Y., Levina, E., Zhu, J.: Detecting overlapping communities in networks using spectral methods. ArXiv e-prints (2014)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Maxim Panov
    • 1
    Email author
  • Konstantin Slavnov
    • 2
  • Roman Ushakov
    • 3
  1. 1.Skolkovo Institute of Science and Technology (Skoltech), Institute for Information Transmission Problems of RASMoscowRussia
  2. 2.Skolkovo Institute of Science and Technology (Skoltech)MoscowRussia
  3. 3.Moscow Institute of Physics and Technology, Institute for Information Transmission Problems of RASMoscowRussia

Personalised recommendations