Advertisement

A Parallel Method for All-Pair SimRank Similarity Computation

  • Xuan Huang
  • Xingkun Gao
  • Jie Tang
  • Gangshan Wu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11334)

Abstract

How to measure SimRank similarity of all-pair vertices in a graph is a very important research topic which has a wide range of applications in many fields. However, computation of SimRank is costly in both time and space, making traditional computing methods failing to handle graph data of ever-growing size.

This paper proposes a parallel multi-level solution for all-pair SimRank similarity computing on large graphs. We partition the objective graph first with the idea of modularity maximization and get a collapsed graph based on the blocks. Then we compute the similarities between verteices inside a block as well as the similarities between the blocks. In the end, we integrate these two types of similarities and calculate the approximate SimRank simlarities between all vertex pairs. The method is implemented on Spark platform and it makes an improvement on time efficiency while maintaining the effectiveness compared to SimRank.

Keywords

SimRank similarity Spark Parallel computing 

References

  1. 1.
    Antonellis, I., Garcia-Molina, H., Chang, C.: SimRank++: query rewriting through link analysis of the click graph. PVLDB 1(1), 408–421 (2008)Google Scholar
  2. 2.
    Baeza-Yates, R., Ribeiro-Neto, B., et al.: Modern Information Retrieval, vol. 463. ACM Press, New York (1999)Google Scholar
  3. 3.
    Bhattacharya, I., Getoor, L.: Entity resolution in graphs. In: Mining Graph Data, p. 311 (2006)CrossRefGoogle Scholar
  4. 4.
    Bui, T.N., Moon, B.R.: Genetic algorithm and graph partitioning. IEEE Trans. Comput. 45(7), 841–855 (1996)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Cao, L., Cho, B., Kim, H.D., Li, Z., Tsai, M.H., Gupta, I.: Delta-SimRank computing on MapReduce. In: Proceedings of the 1st International Workshop on Big Data, Streams and Heterogeneous Source Mining: Algorithms, Systems, Programming Models and Applications, pp. 28–35. ACM (2012)Google Scholar
  6. 6.
    Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. In: OSDI, pp. 137–150. USENIX Association (2004)Google Scholar
  7. 7.
    Dean, J., Henzinger, M.R.: Finding related pages in the world wide web. Comput. Netw. 31(11), 1467–1479 (1999)CrossRefGoogle Scholar
  8. 8.
    Dice, L.R.: Measures of the amount of ecologic association between species. Ecology 26(3), 297–302 (1945)CrossRefGoogle Scholar
  9. 9.
    Fiduccia, C.M., Mattheyses, R.M.: A linear-time heuristic for improving network partitions. Papers on Twenty-Five Years of Electronic Design Automation, pp. 241–247. ACM (1988)Google Scholar
  10. 10.
    Fouss, F., Pirotte, A., Renders, J.M., Saerens, M.: Random-walk computation of similarities between nodes of a graph with application to collaborative recommendation. IEEE Trans. Knowl. Data Eng. 19(3), 355–369 (2007)CrossRefGoogle Scholar
  11. 11.
    He, G., Feng, H., Li, C., Chen, H.: Parallel SimRank computation on large graphs with iterative aggregation. In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 543–552. ACM (2010)Google Scholar
  12. 12.
    Hendrickson, B., Leland, R.W.: An improved spectral graph partitioning algorithm for mapping parallel computations. SIAM J. Sci. Comput. 16(2), 452–469 (1995)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Jaccard, P.: Etude comparative de la distribution florale dans uneportion des Alpes et du Jura. Impr. Corbaz (1901)Google Scholar
  14. 14.
    Jeh, G., Widom, J.: SimRank: a measure of structural-context similarity. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 538–543. ACM (2002)Google Scholar
  15. 15.
    Johnson, D.S., Aragon, C.R., McGeoch, L.A., Schevon, C.: Optimization by simulated annealing: an experimental evaluation; part I, graph partitioning. Oper. Res. 37(6), 865–892 (1989)CrossRefGoogle Scholar
  16. 16.
    Kamvar, S., Haveliwala, T., Manning, C., Golub, G.: Exploiting the block structure of the web for computing PageRank. Technical report 2003-17, Stanford InfoLab (2003)Google Scholar
  17. 17.
    Karypis, G., Kumar, V.: METIS - unstructured graph partitioning and sparse matrix ordering system, version 2.0. Technical report (1995)Google Scholar
  18. 18.
    Kaufman, L., Rousseeuw, P.: Finding Groups in Data: An Introduction to Cluster Analysis. Wiley, Hoboken (1990)CrossRefGoogle Scholar
  19. 19.
    Kernighan, B.W., Lin, S.: An efficient heuristic procedure for partitioning graphs. Bell Syst. Tech. J. 49(2), 291–307 (1970)CrossRefGoogle Scholar
  20. 20.
    Kusumoto, M., Maehara, T., Kawarabayashi, K.I.: Scalable similarity search for SimRank. In: Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data, pp. 325–336. ACM (2014)Google Scholar
  21. 21.
    Li, C., et al.: Fast computation of SimRank for static and dynamic information networks. In: Proceedings of the 13th International Conference on Extending Database Technology, pp. 465–476. ACM (2010)Google Scholar
  22. 22.
    Li, L., Li, C., Chen, H., Du, X.: MapReduce-based SimRank computation and its application in social recommender system. In: BigData Congress, pp. 133–140. IEEE Computer Society (2013)Google Scholar
  23. 23.
    Li, Z., Fang, Y., Liu, Q., Cheng, J., Cheng, R., Lui, J.C.S.: Walking in the cloud: parallel SimRank at scale. PVLDB 9(1), 24–35 (2015)Google Scholar
  24. 24.
    Lizorkin, D., Velikhov, P., Grinev, M., Turdakov, D.: Accuracy estimate and optimization techniques for SimRank computation. Proc. VLDB Endow. 1(1), 422–433 (2008)CrossRefGoogle Scholar
  25. 25.
    Maehara, T., Kusumoto, M., Kawarabayashi, K.: Efficient SimRank computation via linearization. CoRR abs/1411.7228 (2014)Google Scholar
  26. 26.
    Newman, M.E.: Modularity and community structure in networks. Proc. Nat. Acad. Sci. 103(23), 8577–8582 (2006)CrossRefGoogle Scholar
  27. 27.
    Page, L., Brin, S., Motwani, R., Winograd, T.: The PageRank citation ranking: bringing order to the web (1999)Google Scholar
  28. 28.
    Rothe, S., Schütze, H.: CoSimRank: a flexible & efficient graph-theoretic similarity measure. In: ACL (1), pp. 1392–1402. The Association for Computer Linguistics (2014)Google Scholar
  29. 29.
    Yu, W., Lin, X., Zhang, W.: Towards efficient SimRank computation on large networks. In: 2013 IEEE 29th International Conference on Data Engineering (ICDE), pp. 601–612. IEEE (2013)Google Scholar
  30. 30.
    Yu, W., Zhang, W., Lin, X., Zhang, Q., Le, J.: A space and time efficient algorithm for SimRank computation. World Wide Web 15(3), 327–353 (2012)CrossRefGoogle Scholar
  31. 31.
    Zaharia, M., Chowdhury, M., Franklin, M.J., Shenker, S., Stoica, I.: Spark: cluster computing with working sets. In: HotCloud. USENIX Association (2010)Google Scholar
  32. 32.
    Zhao, P., Han, J., Sun, Y.: P-rank: a comprehensive structural similarity measure over information networks. In: CIKM, pp. 553–562. ACM (2009)Google Scholar
  33. 33.
    Zhu, R., Zou, Z., Li, J.: SimRank computation on uncertain graphs. In: ICDE, pp. 565–576. IEEE Computer Society (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Xuan Huang
    • 1
    • 2
  • Xingkun Gao
    • 1
    • 2
  • Jie Tang
    • 1
    • 2
  • Gangshan Wu
    • 1
    • 2
  1. 1.Department of Computer Science and TechnologyNanjing UniversityNanjingChina
  2. 2.State Key Laboratory for Novel Software TechnologyNanjing UniversityNanjingChina

Personalised recommendations