Journal of Heuristics

, Volume 22, Issue 5, pp 759–782 | Cite as

Partitioning (hierarchically clustered) complex networks via size-constrained graph clustering

Article

Abstract

The most commonly used method to tackle the graph partitioning problem in practice is the multilevel metaheuristic. In this paper we introduce size-constrained label propagation (SCLaP) and show how it can be used to instantiate both the coarsening phase and the refinement phase of multilevel graph partitioning. We mainly target networks with highly irregular and hierarchically clustered structure (but other network types can be partitioned as well). Additionally, we augment the basic algorithm with several extensions to further improve its speed and/or solution quality. Depending on the configuration of the resulting partitioner using SCLaP, we are able to compute high-quality partitions outperforming all competitors, or instead, to compute similarly good partitions as the best competitor in terms of quality, hMetis, while being an order of magnitude faster. Our fastest configuration partitions the largest real-world graph in our study (it has 3.3 billion edges) with sequential code in about ten minutes while cutting less than half of the edges than the fastest competitor, kMetis.

Keywords

Multilevel graph partitioning Size-constrained graph clustering Heuristics Label propagation Aggressive graph coarsening 

References

  1. Abou-Rjeili, A., Karypis, G.: Multilevel Algorithms for Partitioning Power-Law Graphs. In: Proceedings of 20th International Parallel and Distributed Processing Symposium (2006)Google Scholar
  2. Bader, D.A., Meyerhenke, H., Sanders, P., Schulz, C., Kappes, A., Wagner, D.: Benchmarking for graph clustering and partitioning. In: Alhajj, R., Rokne, J. (eds.) Encyclopedia of Social Network Analysis and Mining, pp. 73–82. Springer, New York (2014)Google Scholar
  3. Bichot, C., Siarry, P. (eds.): Graph Partitioning. Wiley, Hoboken (2011)MATHGoogle Scholar
  4. Bui, T.N., Jones, C.: Finding good approximate vertex and edge partitions is NP-hard. Inf. Process. Lett. 42(3), 153–159 (1992)MathSciNetCrossRefMATHGoogle Scholar
  5. Buluç, A., Meyerhenke, H., Safro, I., Sanders, P., Schulz, C.: Recent advances in graph partitioning. In: Algorithm Engineering—Selected Topics, to app. ArXiv:1311.3144 (2014)
  6. Chevalier, C., Safro, I.: Comparison of coarsening schemes for multilevel graph partitioning. In: Proceedings of the 3rd International Conference on Learning and Intelligent Optimization of LNCS, vol. 5851, pp. 191–205 (2009)Google Scholar
  7. Costa, L.F., Oliveira Jr., O.N., Travieso, G., Rodrigues, F.A., Boas, P.R.V., Antiqueira, L., Viana, M.P., Rocha, L.E.C.: Analyzing and modeling real-world phenomena with complex networks: a survey of applications. Adv. Phys. 60(3), 329–412 (2011)CrossRefGoogle Scholar
  8. Dhillon, I.S., Guan, Y., Kulis, B.: Weighted graph cuts without eigenvectors: a multilevel approach. IEEE Trans. Pattern Anal. Mach. Intell. 29(11), 1944–1957 (2007)CrossRefGoogle Scholar
  9. Gilbert, J., Reinhardt, S., Shah, V.: High-Performance Graph Algorithms from Parallel Sparse Matrices. Springer, Berlin (2006)Google Scholar
  10. Hendrickson, B., Kolda, T.G.: Graph partitioning models for parallel computing. Parallel Comput. 26(12), 1519–1534 (2000)MathSciNetCrossRefMATHGoogle Scholar
  11. Holtgrewe, M., Sanders, P., Schulz, C.: Engineering a scalable high quality graph partitioner. In: Proceedings of the 24th International Parallel and Distributed Processing symposium, pp. 1–12 (2010)Google Scholar
  12. Karypis, G., Kumar, V.: A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM J. Sci. Comput. 20(1), 359–392 (1998)MathSciNetCrossRefMATHGoogle Scholar
  13. Karypis, G., Kumar, V.: Multilevel \(k\)-way hypergraph partitioning. In: Proceedings of the 36th ACM/IEEE Design Automation Conference, pp. 343–348. ACM, New York (1999)Google Scholar
  14. Kirmani, S., Raghavan, P.: Scalable parallel graph partitioning. In: International Conference for High Performance Computing, Networking, Storage and Analysis (SC’13), p. 51. ACM, New York (2013)Google Scholar
  15. Kothapalli, K., Pemmaraju, S., Sardeshmukh, V.: On the analysis of a label propagation algorithm for community detection. In: Frey, D., Raynal, M., Sarkar, S., Shyamasundar, R., Sinha, P. (eds.) Distributed Computing and Networking. Lecture Notes in Computer Science, vol. 7730, pp. 255–269. Springer, Berlin (2013)CrossRefGoogle Scholar
  16. Laboratory of Web Algorithms, University of Milano. Datasets, http://law.di.unimi.it/datasets.php (2016)
  17. Lancichinetti, A., Fortunato, S., Kertész, J.: Detecting the overlapping and hierarchical community structure in complex networks. New J. Phys. 11(3), 033015 (2009)CrossRefGoogle Scholar
  18. LaSalle, D., Karypis, G.: Multi-threaded modularity based graph clustering using the multilevel paradigm. J. Parallel Distrib. Comput. 76, 66–80 (2014). doi:10.1016/j.jpdc.2014.09.012. (To appear)
  19. Leskovec, J.: Stanford Network Analysis Package (SNAP). http://snap.stanford.edu/data/index.html (2016)
  20. Meyerhenke, H., Monien, B., Schamberger, S.: Accelerating shape optimizing load balancing for parallel FEM simulations by algebraic multigrid. In: Proceedings of 20th International Parallel and Distributed Processing Symposium (2006)Google Scholar
  21. Meyerhenke, H., Sanders, P., Schulz, C.: Partitioning complex networks via size-constrained clustering. In: Proceedings of 13th International Symposium on Experimental Algorithms (SEA 2014), LNCS, vol. 8504, pp. 351–363. Springer, New York (2014)Google Scholar
  22. Meyerhenke, H., Sanders, P., Schulz, C.: Parallel graph partitioning for complex networks. In: Proceedings of 29th IEEE International Parallel & Distributed Processing symposium (IPDPS 2015), IEEE (2015)Google Scholar
  23. Ovelgönne, M.: Distributed community detection in web-scale networks. In: 2013 International Conference on Advances in Social Networks Analysis and Mining, pp. 66–73 (2013)Google Scholar
  24. Ovelgönne, M., Geyer-Schulz, A.: An ensemble learning strategy for graph clustering. In: Graph Partitioning and Graph Clustering in Contemporary Mathematics, vol. 588. AMS and DIMACS (2013)Google Scholar
  25. Pellegrini, F.: Scotch Home Page. https://www.labri.fr/perso/pelegrin/scotch/ (2012a)
  26. Pellegrini, F.: Scotch and PT-scotch graph partitioning software: an overview. In: Naumann, U., Schenk, O. (eds.) Combinatorial Scientific Computing, pp. 373–406. CRC Press, Boca Raton (2012b)Google Scholar
  27. Raghavan, U.N., Albert, R., Kumara, S.: Near linear time algorithm to detect community structures in large-scale networks. Phys. Rev. E 76(3), 036106 (2007)CrossRefGoogle Scholar
  28. Rotta, R., Noack, A.: Multilevel local search algorithms for modularity clustering. J. Exp. Algorithmics 16, 2–3 (2011)MathSciNetCrossRefMATHGoogle Scholar
  29. Sanders, P., Schulz, C.: KaHIP—Karlsruhe high qualtity partitioning homepage. http://algo2.iti.kit.edu/documents/kahip/index.html (2016)
  30. Sanders, P., Schulz, C.: Engineering multilevel graph partitioning algorithms. In: Proceedings of the 19th European Symposium on Algorithms of LNCS, vol. 6942, pp. 469–480. Springer, New York (2011)Google Scholar
  31. Sanders, P., Schulz, C.: Think locally, act globally: highly balanced graph partitioning. In: Proceedings of the 12th International Symposium on Experimental Algorithms (SEA’12), LNCS. Springer, New York (2013)Google Scholar
  32. Schloegel, K., Karypis, G., Kumar, V.: Graph partitioning for high performance scientific simulations. In: The Sourcebook of Parallel Computing, pp. 491–541 (2003)Google Scholar
  33. Schulz, C.: High Quality Graph Partititioning. PhD thesis, KIT (2013)Google Scholar
  34. Slota, G., Madduri, K., Rajamanickam, S.: Pulp: Scalable multi-objective multi-constraint partitioning for small-world networks. In: IEEE international conference on big data (big data), pp. 481–490 (2014)Google Scholar
  35. Southwell, R.V.: Stress-calculation in frameworks by the method of “systematic relaxation of constraints”. Proc. R. Soc. Lond. Ser. A 151(872), 56–95 (1935)CrossRefMATHGoogle Scholar
  36. Staudt, C.L., Meyerhenke, H.: Engineering parallel algorithms for community detection in massive networks. IEEE Trans. Parallel Distrib. Syst. 27(1), 171–184 (2016)CrossRefGoogle Scholar
  37. Ugander, J., Backstrom, L.: Balanced label propagation for partitioning massive graphs. In: 6’th International Conference on Web Search and Data Mining (WSDM’13), pp. 507–516. ACM, New York (2013)Google Scholar
  38. Walshaw, C.: Multilevel refinement for combinatorial optimisation problems. Ann. Oper. Res. 131(1), 325–372 (2004)MathSciNetCrossRefMATHGoogle Scholar
  39. Walshaw, C., Cross, M.: Mesh partitioning: a multilevel balancing and refinement algorithm. SIAM J. Sci. Comput. 22(1), 63–80 (2000)MathSciNetCrossRefMATHGoogle Scholar
  40. Walshaw, C., Cross, M.: JOSTLE: parallel multilevel graph-partitioning software—an overview. Mesh Partit. Tech. Domain Decompos. Tech. pp. 27–58 (2007)Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Karlsruhe Institute of Technology (KIT), Institute of Theoretical InformaticsKarlsruheGermany

Personalised recommendations