Skip to main content
SpringerLink
Log in

Partitioning (hierarchically clustered) complex networks via size-constrained graph clustering

  • Published:
Journal of Heuristics Aims and scope Submit manuscript

Abstract

The most commonly used method to tackle the graph partitioning problem in practice is the multilevel metaheuristic. In this paper we introduce size-constrained label propagation (SCLaP) and show how it can be used to instantiate both the coarsening phase and the refinement phase of multilevel graph partitioning. We mainly target networks with highly irregular and hierarchically clustered structure (but other network types can be partitioned as well). Additionally, we augment the basic algorithm with several extensions to further improve its speed and/or solution quality. Depending on the configuration of the resulting partitioner using SCLaP, we are able to compute high-quality partitions outperforming all competitors, or instead, to compute similarly good partitions as the best competitor in terms of quality, hMetis, while being an order of magnitude faster. Our fastest configuration partitions the largest real-world graph in our study (it has 3.3 billion edges) with sequential code in about ten minutes while cutting less than half of the edges than the fastest competitor, kMetis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Abou-Rjeili, A., Karypis, G.: Multilevel Algorithms for Partitioning Power-Law Graphs. In: Proceedings of 20th International Parallel and Distributed Processing Symposium (2006)

  • Bader, D.A., Meyerhenke, H., Sanders, P., Schulz, C., Kappes, A., Wagner, D.: Benchmarking for graph clustering and partitioning. In: Alhajj, R., Rokne, J. (eds.) Encyclopedia of Social Network Analysis and Mining, pp. 73–82. Springer, New York (2014)

    Google Scholar 

  • Bichot, C., Siarry, P. (eds.): Graph Partitioning. Wiley, Hoboken (2011)

    MATH  Google Scholar 

  • Bui, T.N., Jones, C.: Finding good approximate vertex and edge partitions is NP-hard. Inf. Process. Lett. 42(3), 153–159 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  • Buluç, A., Meyerhenke, H., Safro, I., Sanders, P., Schulz, C.: Recent advances in graph partitioning. In: Algorithm Engineering—Selected Topics, to app. ArXiv:1311.3144 (2014)

  • Chevalier, C., Safro, I.: Comparison of coarsening schemes for multilevel graph partitioning. In: Proceedings of the 3rd International Conference on Learning and Intelligent Optimization of LNCS, vol. 5851, pp. 191–205 (2009)

  • Costa, L.F., Oliveira Jr., O.N., Travieso, G., Rodrigues, F.A., Boas, P.R.V., Antiqueira, L., Viana, M.P., Rocha, L.E.C.: Analyzing and modeling real-world phenomena with complex networks: a survey of applications. Adv. Phys. 60(3), 329–412 (2011)

    Article  Google Scholar 

  • Dhillon, I.S., Guan, Y., Kulis, B.: Weighted graph cuts without eigenvectors: a multilevel approach. IEEE Trans. Pattern Anal. Mach. Intell. 29(11), 1944–1957 (2007)

    Article  Google Scholar 

  • Gilbert, J., Reinhardt, S., Shah, V.: High-Performance Graph Algorithms from Parallel Sparse Matrices. Springer, Berlin (2006)

    Google Scholar 

  • Hendrickson, B., Kolda, T.G.: Graph partitioning models for parallel computing. Parallel Comput. 26(12), 1519–1534 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  • Holtgrewe, M., Sanders, P., Schulz, C.: Engineering a scalable high quality graph partitioner. In: Proceedings of the 24th International Parallel and Distributed Processing symposium, pp. 1–12 (2010)

  • Karypis, G., Kumar, V.: A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM J. Sci. Comput. 20(1), 359–392 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  • Karypis, G., Kumar, V.: Multilevel \(k\)-way hypergraph partitioning. In: Proceedings of the 36th ACM/IEEE Design Automation Conference, pp. 343–348. ACM, New York (1999)

  • Kirmani, S., Raghavan, P.: Scalable parallel graph partitioning. In: International Conference for High Performance Computing, Networking, Storage and Analysis (SC’13), p. 51. ACM, New York (2013)

  • Kothapalli, K., Pemmaraju, S., Sardeshmukh, V.: On the analysis of a label propagation algorithm for community detection. In: Frey, D., Raynal, M., Sarkar, S., Shyamasundar, R., Sinha, P. (eds.) Distributed Computing and Networking. Lecture Notes in Computer Science, vol. 7730, pp. 255–269. Springer, Berlin (2013)

    Chapter  Google Scholar 

  • Laboratory of Web Algorithms, University of Milano. Datasets, http://law.di.unimi.it/datasets.php (2016)

  • Lancichinetti, A., Fortunato, S., Kertész, J.: Detecting the overlapping and hierarchical community structure in complex networks. New J. Phys. 11(3), 033015 (2009)

    Article  Google Scholar 

  • LaSalle, D., Karypis, G.: Multi-threaded modularity based graph clustering using the multilevel paradigm. J. Parallel Distrib. Comput. 76, 66–80 (2014). doi:10.1016/j.jpdc.2014.09.012. (To appear)

  • Leskovec, J.: Stanford Network Analysis Package (SNAP). http://snap.stanford.edu/data/index.html (2016)

  • Meyerhenke, H., Monien, B., Schamberger, S.: Accelerating shape optimizing load balancing for parallel FEM simulations by algebraic multigrid. In: Proceedings of 20th International Parallel and Distributed Processing Symposium (2006)

  • Meyerhenke, H., Sanders, P., Schulz, C.: Partitioning complex networks via size-constrained clustering. In: Proceedings of 13th International Symposium on Experimental Algorithms (SEA 2014), LNCS, vol. 8504, pp. 351–363. Springer, New York (2014)

  • Meyerhenke, H., Sanders, P., Schulz, C.: Parallel graph partitioning for complex networks. In: Proceedings of 29th IEEE International Parallel & Distributed Processing symposium (IPDPS 2015), IEEE (2015)

  • Ovelgönne, M.: Distributed community detection in web-scale networks. In: 2013 International Conference on Advances in Social Networks Analysis and Mining, pp. 66–73 (2013)

  • Ovelgönne, M., Geyer-Schulz, A.: An ensemble learning strategy for graph clustering. In: Graph Partitioning and Graph Clustering in Contemporary Mathematics, vol. 588. AMS and DIMACS (2013)

  • Pellegrini, F.: Scotch Home Page. https://www.labri.fr/perso/pelegrin/scotch/ (2012a)

  • Pellegrini, F.: Scotch and PT-scotch graph partitioning software: an overview. In: Naumann, U., Schenk, O. (eds.) Combinatorial Scientific Computing, pp. 373–406. CRC Press, Boca Raton (2012b)

  • Raghavan, U.N., Albert, R., Kumara, S.: Near linear time algorithm to detect community structures in large-scale networks. Phys. Rev. E 76(3), 036106 (2007)

    Article  Google Scholar 

  • Rotta, R., Noack, A.: Multilevel local search algorithms for modularity clustering. J. Exp. Algorithmics 16, 2–3 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  • Sanders, P., Schulz, C.: KaHIP—Karlsruhe high qualtity partitioning homepage. http://algo2.iti.kit.edu/documents/kahip/index.html (2016)

  • Sanders, P., Schulz, C.: Engineering multilevel graph partitioning algorithms. In: Proceedings of the 19th European Symposium on Algorithms of LNCS, vol. 6942, pp. 469–480. Springer, New York (2011)

  • Sanders, P., Schulz, C.: Think locally, act globally: highly balanced graph partitioning. In: Proceedings of the 12th International Symposium on Experimental Algorithms (SEA’12), LNCS. Springer, New York (2013)

  • Schloegel, K., Karypis, G., Kumar, V.: Graph partitioning for high performance scientific simulations. In: The Sourcebook of Parallel Computing, pp. 491–541 (2003)

  • Schulz, C.: High Quality Graph Partititioning. PhD thesis, KIT (2013)

  • Slota, G., Madduri, K., Rajamanickam, S.: Pulp: Scalable multi-objective multi-constraint partitioning for small-world networks. In: IEEE international conference on big data (big data), pp. 481–490 (2014)

  • Southwell, R.V.: Stress-calculation in frameworks by the method of “systematic relaxation of constraints”. Proc. R. Soc. Lond. Ser. A 151(872), 56–95 (1935)

    Article  MATH  Google Scholar 

  • Staudt, C.L., Meyerhenke, H.: Engineering parallel algorithms for community detection in massive networks. IEEE Trans. Parallel Distrib. Syst. 27(1), 171–184 (2016)

    Article  Google Scholar 

  • Ugander, J., Backstrom, L.: Balanced label propagation for partitioning massive graphs. In: 6’th International Conference on Web Search and Data Mining (WSDM’13), pp. 507–516. ACM, New York (2013)

  • Walshaw, C.: Multilevel refinement for combinatorial optimisation problems. Ann. Oper. Res. 131(1), 325–372 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  • Walshaw, C., Cross, M.: Mesh partitioning: a multilevel balancing and refinement algorithm. SIAM J. Sci. Comput. 22(1), 63–80 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  • Walshaw, C., Cross, M.: JOSTLE: parallel multilevel graph-partitioning software—an overview. Mesh Partit. Tech. Domain Decompos. Tech. pp. 27–58 (2007)

Download references

Acknowledgments

This work was partially supported by Deutsche Forschungsgemeinschaft under Grants DFG SA 933/10-1 and DFG ME 3619/2-1

Author information

Authors and Affiliations

  1. Karlsruhe Institute of Technology (KIT), Institute of Theoretical Informatics, Am Fasanengarten 5, 76131, Karlsruhe, Germany

    Henning Meyerhenke, Peter Sanders & Christian Schulz

Authors
  1. Henning Meyerhenke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peter Sanders
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christian Schulz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Henning Meyerhenke or Christian Schulz.

Additional information

A preliminary shorter version of this paper appeared in Proceedings of 13th International Symposium on Experimental Algorithms (SEA 2014) (Meyerhenke et al. 2014).

Appendix

Appendix

See Tables 4 and 5.

Table 4 Average cuts for \(k\in \{2,16,64\}\) and a representative graph set and a number of configurations
Full size table
Table 5 Best cuts for \(k\in \{2,16,64\}\) and a representative graph set and a number of configurations
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Meyerhenke, H., Sanders, P. & Schulz, C. Partitioning (hierarchically clustered) complex networks via size-constrained graph clustering. J Heuristics 22, 759–782 (2016). https://doi.org/10.1007/s10732-016-9315-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10732-016-9315-8

Keywords

Search

Navigation