Abstract
The structural graph clustering algorithm (SCAN) is an essential graph mining tool that reveals clusters, hubs, and outliers included in a given graph. Although SCAN is used in various applications, it has two serious drawbacks when handling large graphs. First, SCAN is computationally expensive since it requires iterative computations for all nodes and edges. Second, SCAN is not designed to handle large graphs that cannot fit in the main memory. This paper presents a distributed structural graph clustering algorithm, DSCAN, to address the aforementioned problems on a cluster of computers. DSCAN employs edge pruning techniques to reduce the communication and computation overheads of the distributed algorithm. Our extensive experiments on real-world billion-edge graphs demonstrate that DSCAN outperforms state-of-the-art algorithms in terms of running time even though DSCAN outputs the same clusters as SCAN.
This work was done when T. Takahashi was a student of University of Tsukuba. He is currenly a member of Nippon Telegraph and Telphone Corporation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Blondel, V.D., Guillaume, J.L., Lambiotte, R., Mech, E.L.J.S.: Fast unfolding of communities in large networks. J. Stat. Mech.: Theory Experiment 2008(10), P10008 (2008)
Boldi, P., Vigna, S.: The WebGraph framework I: compression techniques. In: Proceedings of the 13th International Conference on World Wide Web, pp. 595–601 (2004)
Chang, L., Li, W., Qin, L., Zhang, W., Yang, S.: pSCAN: fast and exact structural graph clustering. IEEE Trans. Knowl. Data Eng. 29(2), 387–401 (2017)
Che, Y., Sun, S., Luo, Q.: Parallelizing pruning-based graph structural clustering. In: Proceedings of the 47th International Conference on Parallel Processing, pp. 77:1–77:10. ICPP (2018)
Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms. The MIT Press, Cambridge (2009)
Inoue, H., Ohara, M., Taura, K.: Faster Set Intersection with SIMD instructions by Reducing Branch Mispredictions. Proc. Very Learge Data Bases (PVLDB) 8(3), 293–304 (2015)
Karypis, G., Kumar, V.: A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM J. Sci. Comput. 20(1), 359–392 (1998)
Kim, J., et al.: CASS: a distributed network clustering algorithm based on structure similarity for large-scale network. PLOS ONE 13(10), 1–22 (2018)
Lancichinetti, A., Fortunato, S., Radicchi, F.: Benchmark graphs for testing community detection algorithms. Phys. Rev. E 78, 046110 (2008)
Manning, C.D., Raghavan, P., Schütze, H.: Introduction to Information Retrieval. Cambridge University Press, New York (2008)
Onizuka, M., Fujimori, T., Shiokawa, H.: Graph partitioning for distributed graph processing. Data Sci. Eng. 2(1), 94–105 (2017)
ParMETIS – Parallel Graph Partitioning and Fill-reducing Matrix Ordering. http://glaros.dtc.umn.edu/gkhome/metis/parmetis/overview (2006–2008)
Rosvall, M., Axelsson, D., Bergstrom, C.T.: The map equation. The European Physical Journal Special Topics 178(1), 13–23 (2009)
Sato, T., Shiokawa, H., Yamaguchi, Y., Kitagawa, H.: FORank: fast objectrank for large heterogeneous graphs. Companion Proc. Web Conf. 2018, 103–104 (2018)
Seo, J.H., Kim, M.H.: pm-SCAN: an I/O efficient structural clustering algorithm for large-scale graphs. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management (CIKM 2017), pp. 2295–2298 (2017)
Shiokawa, H., Amagasa, T., Kitagawa, H.: Scaling Fine-grained modularity clustering for massive graphs. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. pp. 4597–4604 (2019)
Shiokawa, H., Fujiwara, Y., Onizuka, M.: SCAN++: efficient algorithm for finding clusters, hubs and outliers on large-scale graphs. Proc. Very Learge Data Bases 8(11), 1178–1189 (2015)
Shiokawa, H., Onizuka, M.: Scalable graph clustering and its applications. Encyclopedia of Social Network Analysis and Mining, pp. 2290–2299 (2018)
Shiokawa, H., Takahashi, T., Kitagawa, H.: ScaleSCAN: scalable density-based graph clustering. In: Proceedings of the 29th International Conference on Database and Expert Systems Applications, pp. 18–34. DEXA (2018)
Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The Hadoop distributed file system. In: IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST 2010), pp. 1–10 (2010)
Stovall, T.R., Kockara, S., Avci, R.: GPUSCAN: GPU-based parallel structural clustering algorithm for networks. IEEE Trans. Parallel Distrib. Syst. 26(12), 3381–3393 (2015)
Takahashi, T., Shiokawa, H., Kitagawa, H.: SCAN-XP: parallel structural graph clustering algorithm on intel xeon phi coprocessors. In: Proceedings of the 2nd International Workshop on Network Data Analytics, pp. 6:1–6:7 (2017)
Xu, X., Yuruk, N., Feng, Z., Schweiger, T.A.J.: SCAN: a structural clustering algorithm for networks. In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 824–833 (2007)
Zaharia, M., Chowdhury, M., Franklin, M.J., Shenker, S., Stoica, I.: Spark: cluster computing with working sets. In: Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing, p. 10. HotCloud 2010, USENIX Association, USA (2010)
Zhao, W., Martha, V., Xu, X.: PSCAN: a parallel structural clustering algorithm for big network in MapReduce. In: Proceedings of the 2013 IEEE 27th International Conference on Advanced Information Networking and Applications (2013)
Acknowledgement
This work was supported by JSPS KAKENHI Early-Career Scientists Grant Number JP18K18057, and JST ACT-I.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Shiokawa, H., Takahashi, T. (2020). DSCAN: Distributed Structural Graph Clustering for Billion-Edge Graphs. In: Hartmann, S., Küng, J., Kotsis, G., Tjoa, A.M., Khalil, I. (eds) Database and Expert Systems Applications. DEXA 2020. Lecture Notes in Computer Science(), vol 12391. Springer, Cham. https://doi.org/10.1007/978-3-030-59003-1_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-59003-1_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59002-4
Online ISBN: 978-3-030-59003-1
eBook Packages: Computer ScienceComputer Science (R0)