Skip to main content
SpringerLink
Log in

A low-latency computing framework for time-evolving graphs

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The demand to deliver fast responses in processing time-evolving graphs is higher than ever before in a large number of big data applications. This problem promotes extensive uses of an incremental computing model, which executes the underlying graph algorithm on the newly updated graph structure by taking the results of the computation on the outdated graph structure as initial values, in distributed time-evolving graph computing systems. In this paper, we experimentally study how the initial values of the computation on a newly updated graph structure influence the convergence of the iterative graph analysis, and we develop an optimization framework on the basis of the incremental computing model to accelerate the convergence of processing time-evolving graphs thus achieving high performance for time-evolving graph analysis. In contrast to the traditional incremental computing model, which uses the results of the computation on the outdated graph structure directly, the proposed framework predicts the optimal initial values of the computation on the new graph structure and thereby reduces the number of iterations. Two different prediction approaches are designed to optimize the initial values based on a combination of the results of the computation on the previous graph data and the newly incoming graph data. We have evaluated our optimization framework using the graph algorithms PageRank and KMeans on Amazon EC2 clusters. The experiments demonstrate that the incremental computing implementation with the initial value prediction have reduced the number of iterations by 30% for the PageRank algorithm and 13.7% for the KMeans algorithm, and reduced the response time by 12.7% and 10.6% accordingly compared to the traditional incremental computing model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Amazon (2018) Amazon ec2. https://aws.amazon.com/cn/ec2/

  2. Apache (2012) Apache giraph. http://giraph.apache.org/

  3. Broder AZ, Lempel R, Maghoul F, Pedersen JO (2004) Efficient pagerank approximation via graph aggregation. In: International World Wide Web Conferences, pp 484–485

  4. Cai Z, Logothetis D, Siganos G (2012) Facilitating real-time graph mining. In: International Workshop on Cloud Data Management, ACM, pp 1–8

  5. Cheng R, Hong J, Kyrola A, Miao Y, Weng X, Wu M, Yang F, Zhou L, Zhao F, Chen E (2012) Kineograph: taking the pulse of a fast-changing and connected world. In: European Conference on Computer Systems, ACM, pp 85–98

  6. Dean J, Ghemawat S (2008) Mapreduce: simplified data processing on large clusters. Commun ACM 51(1):107–113

    Article  Google Scholar 

  7. Gaito S, Zignani M, Rossi GP, Sala A, Zhao X, Zheng H, Zhao BY (2012) On the bursty evolution of online social networks. In: Proceedings of the First ACM International Workshop on Hot Topics on Interdisciplinary Social Networks Research, pp 1–8

  8. Gonzalez JE, Low Y, Gu H, Bickson D, Guestrin C (2012) Powergraph: distributed graph-parallel computation on natural graphs. In: Operating Systems Design and Implementation, pp 17–30

  9. Gonzalez JE, Xin RS, Dave A, Crankshaw D, Franklin MJ, Stoica I (2014) Graphx: graph processing in a distributed dataflow framework. In: Operating Systems Design and Implementation, pp 599–613

  10. Han W, Miao Y, Li K, Wu M, Yang F, Zhou L, Prabhakaran V, Chen W, Chen E (2014) Chronos: a graph engine for temporal graph analysis. In: European Conference on Computer Systems, pp 1–14

  11. Iyer AP, Li LE, Das T, Stoica I (2016) Time-evolving graph processing at scale. In: International Workshop on Graph Data Management Experiences and Systems. ACM, pp 1–6

  12. Ji S, Zhao Y (2018) A local approximation approach for processing time-evolving graphs. Symmetry 10(7):247

    Article  Google Scholar 

  13. Ju W, Li J, Yu W, Zhang R (2016) iGraph: an incremental data processing system for dynamic graph. Front Comput Sci 10(3):462–476

    Article  Google Scholar 

  14. Kan M, Thi HON (2005) Fast webpage classification using URL features. In: Conference on Information and Knowledge Management, pp 325–326

  15. Konect (2017) Konect network dataset. http://konect.uni-koblenz.de/

  16. Malewicz G, Austern MH, Bik AJ, Dehnert JC, Horn I, Leiser N, Czajkowski G (2010) Pregel: a system for large-scale graph processing. In: ACM SIGMOD International Conference on Management of Data. ACM, pp 135–146

  17. McCune RR, Weninger T, Madey G (2015) Thinking like a vertex: a survey of vertex-centric frameworks for large-scale distributed graph processing. ACM Comput Surv 48(2):25

    Article  Google Scholar 

  18. Morshed SJ, Rana J, Milrad M (2016) Real-time data analytics: an algorithmic perspective. In: International Conference on Data Mining, pp 311–320

  19. Murray DG, Mcsherry F, Isaacs R, Isard M, Barham P, Abadi M (2013) Naiad: a timely dataflow system. In: Symposium on Operating Systems Principles, pp 439–455

  20. Sha M, Li Y, He B, Tan KL (2017) Accelerating dynamic graph analytics on gpus. Very Large Data Bases 11(1):107–120

    Google Scholar 

  21. Shi X, Cui B, Shao Y, Tong Y (2016) Tornado: a system for real-time iterative analysis over evolving data. In: International Conference on Management of Data, pp 417–430

  22. TSP (2013) World TSP. http://www.math.uwaterloo.ca/tsp/world/

  23. Valiant LG (1990) A bridging model for parallel computation. Commun ACM 33(8):103–111

    Article  Google Scholar 

  24. Vaquero LM, Cuadrado F, Logothetis D, Martella C (2013) xDGP: a dynamic graph processing system with adaptive partitioning. arXiv preprint arXiv:1309.1049

  25. Vaquero LM, Cuadrado F, Ripeanu M (2014) Systems for near real-time analysis of large-scale dynamic graphs. arXiv preprint arXiv:1410.1903

  26. Vazirgiannis M, Drosos D, Senellart P, Vlachou A (2008) Web page rank prediction with Markov models. In: International World Wide Web Conferences, pp 1075–1076

Download references

Acknowledgements

We thank our colleagues for their collaboration. We also thank all the reviewers for their specific comments and suggestions. This work is supported by National Natural Science Foundation of China through Grants No. 61640219.

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Xi’an Jiaotong University, Xi’an, 710049, People’s Republic of China

    Shuo Ji & Yinliang Zhao

  2. School of Software Engineering, Xi’an Jiaotong University, Xi’an, 710049, People’s Republic of China

    Xiaomei Zhao

Authors
  1. Shuo Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yinliang Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaomei Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yinliang Zhao.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ji, S., Zhao, Y. & Zhao, X. A low-latency computing framework for time-evolving graphs. J Supercomput 75, 3673–3692 (2019). https://doi.org/10.1007/s11227-018-2725-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-018-2725-7

Keywords

Search

Navigation