Skip to main content
SpringerLink
Log in

OffStreamNG: Partial Stream Hybrid Graph Edge Partitioning Based on Neighborhood Expansion and Greedy Heuristic

  • Conference paper
  • First Online:
New Trends in Databases and Information Systems (ADBIS 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1259))

Included in the following conference series:

Abstract

Recently, graph edge partitioning has shown better partitioning quality than the vertex graph partitioning for the skewed degree distribution of real-world graph data. Graph edge partitioning can be classified as stream and offline. The stream edge partitioning approach supports a big graph partitioning; however, it has lower partitioning quality, is affected by stream order, and it has taken much time to make partitioning compared with the offline edge partitioning. Conversely, the offline edge partitioning approach has better partitioning quality than stream edge partitioning; however, it does not support big graph partitioning. In this study, we propose partial stream hybrid graph edge partitioning OffStreamNG, which leverages the advantage of both offline and stream edge partitioning approaches by interconnecting via saved partition state layer. The OffStreamNG holds vertex and load states as partition state, while the offline component is partitioning using neighborhood expansion heuristic. And it is transferring this partition state to the online component of Greedy heuristic with minor modification of both algorithms. Experimental results show that OffStreamNG achieves attractive results in terms of replication factor, load balance, and total partitioning time.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/ansrlab/edgepart.

  2. 2.

    https://github.com/fabiopetroni/VGP.

References

  1. Abbas, Z., Kalavri, V., Carbone, P., Vlassov, V.: Streaming graph partitioning: an experimental study. Proc. VLDB Endow. 11(11), 1590–1603 (2018)

    Article  Google Scholar 

  2. Andreev, K., Racke, H.: Balanced graph partitioning. Theory Comput. Syst. 39(6), 929–939 (2006). https://doi.org/10.1007/s00224-006-1350-7

    Article  MathSciNet  MATH  Google Scholar 

  3. Ayall, T., Duan, H., Liu, C.: Edge property based stream order reduce the performance of stream edge graph partition. J. Phys. Conf. Ser. 1395, 012010 (2019). IOP Publishing

    Article  Google Scholar 

  4. Chen, R., Shi, J., Chen, Y., Zang, B., Guan, H., Chen, H.: PowerLyra: differentiated graph computation and partitioning on skewed graphs. ACM Trans. Parallel Comput. (TOPC) 5(3), 13 (2019)

    Google Scholar 

  5. Gonzalez, J.E., Low, Y., Gu, H., Bickson, D., Guestrin, C.: PowerGraph: distributed graph-parallel computation on natural graphs. In: Presented as part of the 10th \(\{\)USENIX\(\}\) Symposium on Operating Systems Design and Implementation (\(\{\)OSDI\(\}\) 12), pp. 17–30 (2012)

    Google Scholar 

  6. Karypis, G.: METIS: unstructured graph partitioning and sparse matrix ordering system. Technical report (1997)

    Google Scholar 

  7. Kunegis, J.: Konect: the koblenz network collection. In: Proceedings of the 22nd International Conference on World Wide Web. pp. 1343–1350. ACM (2013)

    Google Scholar 

  8. Leskovec, J., Krevl, A.: SNAP Datasets: Stanford large network dataset collection, June 2014. http://snap.stanford.edu/data

  9. Malewicz, G., et al.: Pregel: a system for large-scale graph processing. In: Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, pp. 135–146. ACM (2010)

    Google Scholar 

  10. Mislove, A., Marcon, M., Gummadi, K.P., Druschel, P., Bhattacharjee, B.: Measurement and analysis of online social networks. In: Proceedings of the 7th ACM SIGCOMM conference on Internet Measurement, pp. 29–42. ACM (2007)

    Google Scholar 

  11. Petroni, F., Querzoni, L., Daudjee, K., Kamali, S., Iacoboni, G.: HDRF: stream-based partitioning for power-law graphs. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pp. 243–252. ACM (2015)

    Google Scholar 

  12. Stanton, I., Kliot, G.: Streaming graph partitioning for large distributed graphs. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1222–1230. ACM (2012)

    Google Scholar 

  13. Xie, C., Yan, L., Li, W.J., Zhang, Z.: Distributed power-law graph computing: theoretical and empirical analysis. In: Advances in Neural Information Processing Systems, pp. 1673–1681 (2014)

    Google Scholar 

  14. Yang, J., Leskovec, J.: Defining and evaluating network communities based on ground-truth. Knowl. Inf. Syst. 42(1), 181–213 (2013). https://doi.org/10.1007/s10115-013-0693-z

    Article  Google Scholar 

  15. Zhang, C., Wei, F., Liu, Q., Tang, Z.G., Li, Z.: Graph edge partitioning via neighborhood heuristic. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 605–614. ACM (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China

    Tewodros Ayall, Hancong Duan & Changhong Liu

  2. Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China, Chengdu, China

    Fantahun Gereme

  3. School of Information Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China

    Mesay Deleli

Authors
  1. Tewodros Ayall
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hancong Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Changhong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fantahun Gereme
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mesay Deleli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hancong Duan .

Editor information

Editors and Affiliations

  1. Université Lumière Lyon 2, Lyon, France

    Jérôme Darmont

  2. National Research University Higher School of Economics, St. Petersburg, Russia

    Boris Novikov

  3. Poznań University of Technology, Poznań, Poland

    Robert Wrembel

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ayall, T., Duan, H., Liu, C., Gereme, F., Deleli, M. (2020). OffStreamNG: Partial Stream Hybrid Graph Edge Partitioning Based on Neighborhood Expansion and Greedy Heuristic. In: Darmont, J., Novikov, B., Wrembel, R. (eds) New Trends in Databases and Information Systems. ADBIS 2020. Communications in Computer and Information Science, vol 1259. Springer, Cham. https://doi.org/10.1007/978-3-030-54623-6_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-54623-6_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-54622-9

  • Online ISBN: 978-3-030-54623-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation