The VLDB Journal

, Volume 27, Issue 2, pp 271–296 | Cite as

Accelerating reachability query processing based on \(\varvec{DAG}\) reduction

  • Junfeng Zhou
  • Jeffrey Xu Yu
  • Na Li
  • Hao Wei
  • Ziyang Chen
  • Xian Tang
Regular Paper
  • 128 Downloads

Abstract

Answering reachability queries is one of the fundamental graph operations. The existing approaches build indexes and answer reachability queries on a directed acyclic graph (DAG) \(G\), which is constructed by coalescing each strongly connected component of the given directed graph \(\mathcal {G}\) into a node of \(G\). Considering that \(G\) can still be large to be processed efficiently, there are studies to further reduce \(G\) to a smaller graph. However, these approaches suffer from either inefficiency in answering reachability queries, or cannot scale to large graphs. In this paper, we study DAG reduction to accelerate reachability query processing, which reduces the size of \(G\) by computing transitive reduction (TR) followed by computing equivalence reduction (ER). For TR, we propose a bottom-up algorithm, namely buTR, which removes from \(G\) all redundant edges to get the unique smallest DAG \(G^{t}\) satisfying that \(G^{t}\) has the same transitive closure as that of \(G\). For ER, we propose a divide-and-conquer algorithm, namely linear-ER. Given the result \(G^{t}\) of TR, linear-ER gets a smaller DAG \(G^{\varepsilon }\) in linear time based on equivalence relationship between nodes in \(G\). Our DAG reduction approaches (TR and ER) significantly improve the cost of time and space and can be scaled to large graphs. Based on the result of DAG reduction, we further propose a graph decomposition-based algorithm to efficiently answer reachability queries. We confirm the efficiency of our approaches by extensive experimental studies for TR, ER, and reachability query processing using 20 real datasets. The complete source code is available for download at https://pan.baidu.com/s/1skHBXXN.

Keywords

Reachability query processing Transitive reduction Equivalence reduction 

Notes

Acknowledgements

This work was partly supported by grants from the Natural Science Foundation of China (No. 61472339, 61303040, 61572421, 61272124), and Jeffrey Xu Yu was partly supported by the grant of the Research Grants Council of Hong Kong SAR, China, No. 14209314 and No. 14221716.

Supplementary material

778_2018_495_MOESM1_ESM.pdf (839 kb)
Supplementary material 1 (pdf 838 KB)

References

  1. 1.
    Agrawal, R., Borgida, A., Jagadish, H.V.: Efficient management of transitive relationships in large data and knowledge bases. In: SIGMOD, pp. 253–262 (1989)Google Scholar
  2. 2.
    Aho, A.V., Garey, M.R., Ullman, J.D.: The transitive reduction of a directed graph. SIAM J. Comput. 1(2), 131–137 (1972)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Boldi, P., Santini, M., Vigna, S.: A large time-aware web graph. SIGIR Forum 42(2), 33–38 (2008)CrossRefGoogle Scholar
  4. 4.
    Cha, M., Haddadi, H., Benevenuto, F., Gummadi, P.K.: Measuring user influence in twitter: the million follower fallacy. In: ICWSM (2010)Google Scholar
  5. 5.
    Cheng, J., Huang, S., Wu, H., Fu, A.W.: TF-label: a topological-folding labeling scheme for reachability querying in a large graph. In: SIGMOD, pp. 193–204 (2013)Google Scholar
  6. 6.
    Cohen, E.: Estimating the size of the transitive closure in linear time. In: 35th Annual Symposium on Foundations of Computer Science, pp. 190–200 (1994)Google Scholar
  7. 7.
    Cohen, E., Halperin, E., Kaplan, H., Zwick, U.: Reachability and distance queries via 2-hop labels. In: ACM-SIAM, pp. 937–946 (2002)Google Scholar
  8. 8.
    Fan, W., Li, J., Wang, X., Wu, Y.: Query preserving graph compression. In: SIGMOD, pp. 157–168 (2012)Google Scholar
  9. 9.
    Habib, M., Morvan, M., Rampon, J.: On the calculation of transitive reduction—closure of orders. Discrete Math. 111(1–3), 289–303 (1993)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Jiang, H., Wang, W., Lu, H., Yu, J.X.: Holistic twig joins on indexed XML documents. In: VLDB, pp. 273–284 (2003)Google Scholar
  11. 11.
    Jin, R., Ruan, N., Dey, S., Yu, J.X.: SCARAB: scaling reachability computation on large graphs. In: SIGMOD, pp. 169–180 (2012)Google Scholar
  12. 12.
    Jin, R., Ruan, N., Xiang, Y., Wang, H.: Path-tree: an efficient reachability indexing scheme for large directed graphs. ACM Trans. Database Syst. 36(1), 7 (2011)CrossRefGoogle Scholar
  13. 13.
    Jin, R., Wang, G.: Simple, fast, and scalable reachability oracle. PVLDB 6(14), 1978–1989 (2013)Google Scholar
  14. 14.
    Jin, R., Xiang, Y., Ruan, N., Fuhry, D.: 3-hop: a high-compression indexing scheme for reachability query. In: SIGMOD, pp. 813–826 (2009)Google Scholar
  15. 15.
    Jin, R., Xiang, Y., Ruan, N., Wang, H.: Efficiently answering reachability queries on very large directed graphs. In: SIGMOD, pp. 595–608 (2008)Google Scholar
  16. 16.
    Katajainen, J., Träff, J.L.: A meticulous analysis of mergesort programs. In: CIAC’97, pp. 217–228 (1997)Google Scholar
  17. 17.
    Kornaropoulos, E.M., Tollis, I.G.: Weak dominance drawings and linear extension diameter. CoRR. arXiv:1108.1439 [cs.DS] (2011)
  18. 18.
    Ma, T.-H., Spinrad, J.: Transitive closure for restricted classes of partial orders. Order 8(2), 175–183 (1991)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Seufert, S., Anand, A., Bedathur, S.J., Weikum, G.: FERRARI: flexible and efficient reachability range assignment for graph indexing. In: ICDE, pp. 1009–1020 (2013)Google Scholar
  20. 20.
    Simon, K.: An improved algorithm for transitive closure on acyclic digraphs. Theor. Comput. Sci. 58, 325–346 (1988)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Su, J., Zhu, Q., Wei, H., Yu, J.X.: Reachability querying: Can it be even faster? TKDE 29(3), 683–697 (2017)Google Scholar
  22. 22.
    Tarjan, R.E.: Depth-first search and linear graph algorithms. SIAM J. Comput. 1(2), 146–160 (1972)MathSciNetCrossRefMATHGoogle Scholar
  23. 23.
    Trißl, S., Leser, U.: Fast and practical indexing and querying of very large graphs. In: SIGMOD, pp. 845–856 (2007)Google Scholar
  24. 24.
    Valdes, J., Tarjan, R.E., Lawler, E.L.: The recognition of series parallel digraphs. SIAM J. Comput. 11(2), 298–313 (1982)MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    van Schaik, S.J., de Moor, O.: A memory efficient reachability data structure through bit vector compression. In: SIGMOD, pp. 913–924 (2011)Google Scholar
  26. 26.
    Veloso, R.R., Cerf, L., Junior, W.M., Zaki, M.J.: Reachability queries in very large graphs: a fast refined online search approach. In: EDBT, pp. 511–522 (2014)Google Scholar
  27. 27.
    Wei, H., Yu, J.X., Lu, C., Jin, R.: Reachability querying: an independent permutation labeling approach. PVLDB 7(12), 1191–1202 (2014)Google Scholar
  28. 28.
    Williams, V.V.: Multiplying matrices faster than Coppersmith–Winograd. In: STOC, pp. 887–898 (2012)Google Scholar
  29. 29.
    Yano, Y., Akiba, T., Iwata, Y., Yoshida, Y.: Fast and scalable reachability queries on graphs by pruned labeling with landmarks and paths. In: CIKM, pp. 1601–1606 (2013)Google Scholar
  30. 30.
    Yildirim, H., Chaoji, V., Zaki, M.J.: GRAIL: scalable reachability index for large graphs. PVLDB 3(1), 276–284 (2010)Google Scholar
  31. 31.
    Yildirim, H., Chaoji, V., Zaki, M.J.: GRAIL: a scalable index for reachability queries in very large graphs. VLDB J. 21(4), 509–534 (2012)CrossRefGoogle Scholar
  32. 32.
    Zhou, J., Zhou, S., Yu, J.X., Wei, H., Chen, Z., Tang, X.: DAG reduction: fast answering reachability queries. In: SIGMOD, pp. 375–390 (2017)Google Scholar
  33. 33.
    Zhu, A.D., Lin, W., Wang, S., Xiao, X.: Reachability queries on large dynamic graphs: a total order approach. In: SIGMOD, pp. 1323–1334 (2014)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Computer Science and TechnologyDonghua UniversityShanghaiChina
  2. 2.Chinese University of Hong KongSha TinHong Kong
  3. 3.Yanshan UniversityQinhuangdaoChina
  4. 4.Shanghai Lixin University of Accounting and FinanceShanghaiChina
  5. 5.Shanghai University of Engineering ScienceShanghaiChina

Personalised recommendations