Advertisement

Survey of external memory large-scale graph processing on a multi-core system

  • Jianqiang HuangEmail author
  • Wei Qin
  • Xiaoying Wang
  • Wenguang Chen
Article
  • 61 Downloads

Abstract

The fast development of big data computing contributes to the fact that large-scale graph processing has become a basic computing model in both academic and industrial communities, and it has been applied in many actual big data computing works, such as social network analysis, Web search, and product promotion. These computing works include large-scale graphs of billions of vertices and trillions of edges. Such scale has brought many challenges to large-scale graph processing. This paper mainly introduces the essential features and challenges of large-scale graph processing and how we can handle billions of edges on a multi-core machine, for which we represent out-of-core processing system and semi-external memory processing systems. This paper also summarizes the key technologies in graph processing systems and forecasts the future development of large-scale graph processing systems.

Keywords

Graph data processing Parallel computing Computing model Graph algorithms 

Notes

Acknowledgements

The authors are grateful to the reviewers for valuable comments that have greatly improved the paper. This paper is partially supported by the Open Project of State Key Laboratory of Plateau Ecology and Agriculture, Qinghai University (No. 2020-ZZ-03), the “Qinghai Province High-end Innovative Thousand Talents Program Leading Talents” and the National Natural Science Foundation of China (Nos. 61762074 and 61962051), and National Natural Science Foundation of Qinghai Province (No. 2019-ZJ-7034).

References

  1. 1.
    Knuth DE (1993) The Stanford GraphBase: a platform for combinatorial computing. ACM Press, New YorkzbMATHGoogle Scholar
  2. 2.
    Siek JG, Lee L-Q, Lumsdaine A (2002) The boost graph library: user guide and reference manual. Addison-Wesley, BostonGoogle Scholar
  3. 3.
    Gregor D (2005) Lumsdaine A The parallel BGL: a generic library for distributed graph computations. Proc Parallel Object Oriented Sci Comput 2:1–18Google Scholar
  4. 4.
    Chan A, Dehne F, Taylor R (2005) CGMGRAPH/CGMLIB: implementing and testing CGM graph algorithms on PC clusters and shared memory machines. Int J High Perform Comput Appl 19(1):81–97CrossRefGoogle Scholar
  5. 5.
    Twitter Usage Statistics. http://www.internetlivestats.com/twitter-statistics/. Accessed October (2016)
  6. 6.
  7. 7.
    Shvachko K, Kuang H, Radia S et al (2010) The hadoop distributed file system. In: Proceedings of the 26th IEEE symposium on mass storage systems and technologies, pp 1–10Google Scholar
  8. 8.
    Dean J, Ghemawat S (2008) MapReduce: simplified data processing on large clusters. Commun ACM 51(1):107–113CrossRefGoogle Scholar
  9. 9.
    Zaharia M, Chowdhury M, Franklin MJ et al (2010) Spark: cluster computing with working sets. In: Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing, p 10Google Scholar
  10. 10.
    Zaharia M, Chowdhury M, Das T et al (2012) Resilient distributed datasets: a fault tolerant abstraction for in-memory cluster computing. In: Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation, p 2Google Scholar
  11. 11.
    Malewicz G, Austern MH, Bik AJ et al (2010) Pregel: a system for large-scale graph processing. In: Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, ACM New York, NY, USA, pp 135–146Google Scholar
  12. 12.
    Low Y, Bickson D, Gonzalez J et al (2012) Distributed GraphLab: a framework for machine learning and data mining in the cloud. Proc VLDB Endow 5(8):716–727CrossRefGoogle Scholar
  13. 13.
    Gonzalez JE, Low Y, Gu H et al (2012) PowerGraph: distributed graph parallel computation on natural graphs. In: Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation, pp 17–30Google Scholar
  14. 14.
    Kyrola A, Blelloch GE, Guestrin C (2012) GraphChi: large-scale graph computation on just a PC. In: Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation, pp 31–46Google Scholar
  15. 15.
    Roy A, Mihailovic I, Zwaenepoel W (2013) X-stream: edge-centric graph processing using streaming partitions. In: Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles, ACM New York, NY, USA, pp 472–488Google Scholar
  16. 16.
    Zhu X, Han W, Chen W (2015) GridGraph: largescale graph processing on a single machine using 2-level hierarchical partitioning. In: Proceedings of the 2015 USENIX Annual Technical Conference, pp 375–386Google Scholar
  17. 17.
    Prabhakaran V, Wu M, Weng X et al (2012) Managing large graphs on multicores with graph awareness. In: Proceedings of the 2012 USENIX Annual Technical Conference, pp 41–52Google Scholar
  18. 18.
    Han WS, Lee S, Park K et al (2013) TurboGraph: a fast parallel graph engine handling billionscale graphs in a single PC. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, pp 77–85Google Scholar
  19. 19.
    Cheng J, Liu Q, Li Z et al (2015) VENUS: vertex-centric streamlined graph computation on a single PC. In Proceedings of the 2015 IEEE 31st International Conference on Data Engineering IEEE, pp 1131–1142Google Scholar
  20. 20.
    Zheng D, Mhembere D, Burns R et al (2015) FlashGraph: processing billion-node graphs on an array of commodity SSDs. In Proceedings of the 13th USENIX Conference on File and Storage Technologies, pp 45–58Google Scholar
  21. 21.
    Yuan P, Xie C, Liu L et al (2016) PathGraph: a path centric graph processing system. IEEE Trans Parallel Distrib Syst.  https://doi.org/10.1109/TPDS.2016.2518664 CrossRefGoogle Scholar
  22. 22.
    Feng Z, Heng L, Jidong Z, Jie C, Dingyi X, Jizhong L, Yunpeng C, Xiaoyong D (2018) An adaptive breadth-first search algorithm on integrated architectures. J Supercomput 74(11):6135–6155CrossRefGoogle Scholar
  23. 23.
    Zhang M, Wu Y, Zhuo Y, Qian X, Huan C, Chen K (2018) Wonderland: a novel abstraction-based out-of-core graph processing system. In: ASPLOS, pp 608–621. ACMGoogle Scholar
  24. 24.
    Vora K, Xu GH, Gupta R (2016) Load the edges you need: a generic I/O optimization for disk-based graph processing. In: USENIX ATC, pp 507–522Google Scholar
  25. 25.
    Vora K, Gupta R, Xu G (2017) KickStarter: fast and accurate computations on streaming graphs via trimmed approximations. In: ASPLOS, pp 237–251Google Scholar
  26. 26.
    Maass S, Min C, Kashyap S, Kang W, Kumar M, Kim T (2017) Mosaic: processing a trillion-edge graph on a single machine. In: EuroSys, pp 527–543. ACMGoogle Scholar
  27. 27.
    Ai Z, Zhang M, Wu Y, Qian X, Chen K, Zheng W (2017) Squeezing out all the value of loaded data: an out-of-core graph processing system with reduced disk I/O. In: USENIX ATC, pp 125–137Google Scholar
  28. 28.
    Jun S-W, Wright A, Zhang S, Xu S (2018) Using accelerated flash storage for external graph analytics. In: ISCA. IEEE, GraFBoostGoogle Scholar
  29. 29.
    Jin-zhong L, Peng-jie T, Jie-wu X et al (2015) Advances in iterative MapReduce. Comput Eng Appl 51(12):123–132Google Scholar
  30. 30.
    Bu Y, How B, Balazinska M et al (2012) The HaLoop approach to large scale iterative data analysis. VLDB J 21(2):169–190CrossRefGoogle Scholar
  31. 31.
    Bu Y, How B, Balazinska M et al (2010) HaLoop: efficient iterative data processing on large clusters. Proc VLDB Endow 3(1):285–296CrossRefGoogle Scholar
  32. 32.
    Ekanayake J, Li H, Zhang B et al (2010) Twister: a runtime for iterative Mapreduce. In: Proceedings of the 19th ACM international symposium on high performance distributed computing, pp 810–818Google Scholar
  33. 33.
    Zhang Y, Gao Q, Gao L et al (2012) iMapReduce: a distributed computing framework for iterative computation. J Grid Comput 10(1):47–68CrossRefGoogle Scholar
  34. 34.
    Zhang Y, Gao Q, Gao L et al (2013) PrIter: a distributed framework for prioritizing iterative computations. IEEE Trans Parallel Distrib Syst 24(9):1884–1893CrossRefGoogle Scholar
  35. 35.
    Kang U, Tsourakakis CE, Faloutsos C (2009) Pegasus: a petascale graph mining system implementation and observations. In: Proceedings of the Ninth IEEE International Conference on Data Mining, IEEE, pp 229–238Google Scholar
  36. 36.
    Chen R, Weng X, He B et al (2010) Large graph processing in the cloud. In: Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, ACM, pp 1123–1126Google Scholar
  37. 37.
    Ceze L, Tuck J, Montesinos P et al (2007) BulkSC: bulk enforcement of sequential consistency. In: Proceedings of the 34th annual international symposium on computer architecture, pp 278–289Google Scholar
  38. 38.
    Shun J, Blelloch GE (2013) Ligra: a lightweight graph processing framework for shared memory. In: Proceedings of the 18th ACM SIGPLAN symposium on principles and practice of parallel programming, ACM New York, NY, USA, pp 135–146Google Scholar
  39. 39.
    Han TD, Abdelrahman TS (2011) hi CUDA: high-level GPGPU programming. IEEE Trans Parallel Distrib Syst 22(1):78–90CrossRefGoogle Scholar
  40. 40.
    Harris M (2005) GPGPU: general-purpose computation on GPUs. In: SIGGRAPH 2005 GPGPU COURSE. http://www.gpgpu.org/s2005/
  41. 41.
    Lee S, Min S, Eigenmann R (2009) Open MP to GPGPU: a compiler framework for automatic translation and optimization. In: Proceedings of the 14th ACM SIGPLAN symposium on principles and practice of parallel programming, pp 101–110Google Scholar
  42. 42.
    Harish P, Narayanan PJ (2007) Accelerating large graph algorithms on the GPU using CUDA. In: Proceedings of the 14th International Conference on High Performance Computing, pp 197–208Google Scholar
  43. 43.
    Luo L, Wong M, Hwu W (2010) An effective GPU implementation of breadth-first search. In: Proceedings of the 47th Design Automation Conference, pp 52–55Google Scholar
  44. 44.
    Katz GJ, Kider Jr JT (2008) All-pairs shortest-paths for large graphs on the GPU. In: Proceedings of the 23rd ACM SIGGRAPH symposium on graphics hardware, pp 47–55Google Scholar
  45. 45.
    Hong S, Oguntebi T, Olukotun K (2011) Efficient parallel graph exploration on multi-core CPU and GPU. In: Proceedings of the 20th International Conference on Parallel Architectures and Compilation Techniques, ACM New York, NY, USA, pp 78–88Google Scholar
  46. 46.
  47. 47.
    Robinson I, Webber J, Eifrem E (2015) Graph databases: new opportunities for connected data. O’Reilly Media Inc., SebastopolGoogle Scholar
  48. 48.
    Zhong J, He B (2012) An overview of medusa: simplified graph processing on GPUs. In: Proceedings of the 17th ACM SIGPLAN symposium on principles and practice of parallel programming, ACM New York, NY, USA, pp 283–284Google Scholar
  49. 49.
    Khorasani F, Vora K, Gupta R et al (2014) CuSha: vertex-centric graph processing on GPUs. In: Proceedings of the 23rd international symposium on high-performance parallel and distributed computing, ACM New York, NY, USA, pp 239–252Google Scholar
  50. 50.
    Lingxiao M, Zhi Y, Han C, Jilong X, Yafei D (2017) Garaph: efficient GPU-accelerated graph processing on a single machine with balanced replication. In: USENIX Annual Technical Conference (ATC’), Santa Clara, CA, USA, pp 195–207Google Scholar
  51. 51.
    Zhisong F, Michael P, Bryan T (2014) MapGraph: a high level API for fast development of high performance graph analytics on GPUs. In: Proceedings of workshop on graph data management experiences and systems (GRADES’14). ACM, New York, NY, USA, Article 2Google Scholar
  52. 52.
  53. 53.
    Ben-Nun T, Sutton M, Pai S et al (2017) Groute: an asynchronous multi-GPU programming model for irregular computations. In: Proceedings of the 23rd ACM SIGPLAN symposium on principles and practice of parallel programming, Austin, pp 235–248Google Scholar
  54. 54.
    Sha M, Li Y, He B et al (2017) Accelerating dynamic graph analytics on GPUs. Proc VLDB Endow 11:107–120CrossRefGoogle Scholar
  55. 55.
    Zhang JL, Li J (2018) Degree-aware hybrid graph traversal on FPGA-HMC platform. In: Proceedings of the 26th ACM/SIGDA international symposium on field-programmable gate arrays, Monterey, pp 229–238Google Scholar
  56. 56.
    Zhou SJ, Prasanna VK (2017) Accelerating graph analytics on CPU-FPGA heterogeneous platform. In: Proceedings of the 29th international symposium on computer architecture and high performance computing, Campinas, pp 137–144Google Scholar
  57. 57.
    Zhang MX, Zhuo YW, Wang C et al (2018) GraphP: reducing communication for PIM-based graph processing with efficient data partition. In: Proceedings of the 24th IEEE international symposium on high-performance computer architecture, Vienna, pp 544–557Google Scholar
  58. 58.
    Dai G, Huang T, Chi Y et al (2017) Fore-graph: exploring large-scale graph processing on multi-FPGA architecture. In: Proceedings of the 25th ACM/SIGDA international symposium on field-programmable gate arrays, Monterey, pp 217–226Google Scholar
  59. 59.
    Shi XH, Liang JL, Di S et al (2015) Optimization of asynchronous graph processing on GPU with hybrid coloring model. In: Proceedings of the 20th ACM SIGPLAN symposium on principles and practice of parallel programming, San Francisco, pp 271–272Google Scholar
  60. 60.
    Dai GH, Huang TH, Chi YZ et al (2018) GraphH: a processing-in-memory architecture for large-scale graph processing. IEEE Trans Comput Aided Des Integr Circuits Syst.  https://doi.org/10.1109/TCAD.2018.2821565 CrossRefGoogle Scholar
  61. 61.
    Kang U, Tong H, Sun J et al (2012) GBASE: an efficient analysis platform for large graphs. VLDB J 21(5):637–650CrossRefGoogle Scholar
  62. 62.
    Valiant LG (1990) A bridging model for parallel computation. Commun ACM 33(8):103–111CrossRefGoogle Scholar
  63. 63.
    Tasci S, Demirbas M (2013) Giraphx: parallel yet serializable largescale graph processing. In: Proceedings of European Conference on Parallel Processing. Springer, Berlin, pp 458–469CrossRefGoogle Scholar
  64. 64.
    Khayyat Z, Awara K, Alonazi A et al (2013) Mizan: a system for dynamic load balancing in largescale graph processing. In: Proceedings of the 8th ACM European Conference on Computer Systems. ACM, pp 169–182Google Scholar
  65. 65.
    Yan D, Cheng J, Lu Y et al (2015) Effective techniques for message reduction and load balancing in distributed graph computation. In: Proceedings of the 24th International Conference on World Wide Web. ACM, pp 1307–1317Google Scholar
  66. 66.
    Bao NT, Suzumura T (2013) Towards highly scalable pregel based graph processing platform with x10. In: Proceedings of the 22nd International Conference on World Wide Web Companion, International World Wide Web Conferences Steering Committee, pp 501–508Google Scholar
  67. 67.
    Donald N, Andrew L, Keshav P (2013) A lightweight infrastructure for graph analytics. In: Proceedings of the twenty-fourth symposium on operating systems principles (SOSP’13), ACM, pp 456–471Google Scholar
  68. 68.
    Zhang K, Chen R, Chen H (2015) NUMA-aware graph-structured analytics. In: Proceedings of the 20th ACM SIGPLAN symposium on principles and practice of parallel programming, PPoPP, pp 183–193Google Scholar
  69. 69.
    Abdullah G, Beltrao CL, Elizeu S-N, Matei R (2012) A yoke of oxen and a thousand chickens for heavy lifting graph processing. In: Proceedings of the 21st International Conference on Parallel Architectures and Compilation Techniques (PACT’12). ACM, New York, NY, USA, pp 345–354Google Scholar
  70. 70.
    Brandes U (2001) A faster algorithm for betweenness centrality. J Math Sociol 25(2):163–177CrossRefGoogle Scholar
  71. 71.
    Broder A, Kumar R, Maghoul F, Raghavan P, Rajagopalan S, Stata R, Tomkins A, Wiener J (2000) Graph structure in the web. Comput Netw 33(1):309–320CrossRefGoogle Scholar
  72. 72.
    Su BY, Keutzer K (2012) clSpMV: a cross-platform OpenCL SpMV framework on GPUs. In: Proceedings of the 26th ACM International Conference on Supercomputing, ACM, pp 353–364Google Scholar
  73. 73.
    Backstrom L, Huttenlocher D, Kleinberg J, Lan X (2006) Group formation in large social networks: membership, growth, and evolution. In: Proceedings of KDD, pp 44–54Google Scholar
  74. 74.
    Kwak H, Lee C, Park H, Moon S (2010) What is Twitter, a social network or a news media? In: Proceedings of WWW, pp 591–600Google Scholar
  75. 75.
    Boldi P, Rosa M, Santini M, Vigna S (2011) Layered label propagation: a multiresolution coordinate-free ordering for compressing social networks. In: Proceedings of WWW, pp 587–596Google Scholar
  76. 76.
    Yahoo: Yahoo WebScope (2002) Yahoo! altavista web page hyperlink connectivity graph. https://webscope.sandbox.yahoo.com/

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Jianqiang Huang
    • 1
    • 2
    Email author
  • Wei Qin
    • 1
  • Xiaoying Wang
    • 2
  • Wenguang Chen
    • 1
    • 2
  1. 1.Department of Computer Science and TechnologyTsinghua UniversityBeijingChina
  2. 2.Department of Computer Technology and ApplicationQinghai UniversityXiningChina

Personalised recommendations