LinkAUC: Unsupervised Evaluation of Multiple Network Node Ranks Using Link Prediction

  • Emmanouil KrasanakisEmail author
  • Symeon Papadopoulos
  • Yiannis Kompatsiaris
Conference paper
Part of the Studies in Computational Intelligence book series (SCI, volume 881)


An emerging problem in network analysis is ranking network nodes based on their relevance to metadata groups that share attributes of interest, for example in the context of recommender systems or node discovery services. For this task, it is important to evaluate ranking algorithms and parameters and select the ones most suited to each network. Unfortunately, large real-world networks often comprise sparsely labelled nodes that hinder supervised evaluation, whereas unsupervised measures of community quality, such as density and conductance, favor structural characteristics that may not be indicative of metadata group quality. In this work, we introduce LinkAUC, a new unsupervised approach that evaluates network node ranks of multiple metadata groups by measuring how well they predict network edges. We explain that this accounts for relation knowledge encapsulated in known members of metadata groups and show that it enriches density-based evaluation. Experiments on one synthetic and two real-world networks indicate that LinkAUC agrees with AUC and NDCG for comparing ranking algorithms more than other unsupervised measures.



This work was partially funded by the European Commission under contract numbers H2020-761634 FuturePulse and H2020-825585 HELIOS.


  1. 1.
    Fortunato, S., Hric, D.: Community detection in networks: a user guide. Phys. Rep. 659, 1–44 (2016)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Leskovec, J., Lang, K.J., Mahoney, M.: Empirical comparison of algorithms for network community detection. In: Proceedings of the 19th International Conference on World Wide Web, pp. 631–640. ACM (2010)Google Scholar
  3. 3.
    Xie, J., Kelley, S., Szymanski, B.K.: Overlapping community detection in networks: the state-of-the-art and comparative study. ACM Comput. Surv. (CSUR) 45(4), 43 (2013)CrossRefGoogle Scholar
  4. 4.
    Papadopoulos, S., Kompatsiaris, Y., Vakali, A., Spyridonos, P.: Community detection in social media. Data Min. Knowl. Discov. 24(3), 515–554 (2012)CrossRefGoogle Scholar
  5. 5.
    Hric, D., Darst, R.K., Fortunato, S.: Community detection in networks: structural communities versus ground truth. Phys. Rev. E 90(6), 062805 (2014)CrossRefGoogle Scholar
  6. 6.
    Hric, D., Peixoto, T.P., Fortunato, S.: Network structure, metadata, and the prediction of missing nodes and annotations. Phys. Rev. X 6(3), 031038 (2016)Google Scholar
  7. 7.
    Peel, L., Larremore, D.B., Clauset, A.: The ground truth about metadata and community detection in networks. Sci. Adv. 3(5), e1602548 (2017)CrossRefGoogle Scholar
  8. 8.
    Perer, A., Shneiderman, B.: Balancing systematic and flexible exploration of social networks. IEEE Trans. Visual Comput. Graphics 12(5), 693–700 (2006)CrossRefGoogle Scholar
  9. 9.
    De Domenico, M., Solé-Ribalta, A., Omodei, E., Gómez, S., Arenas, A.: Ranking in interconnected multilayer networks reveals versatile nodes. Nat. Commun. 6, 6868 (2015)CrossRefGoogle Scholar
  10. 10.
    Leskovec, J., Lang, K.J., Dasgupta, A., Mahoney, M.W.: Community structure in large networks: natural cluster sizes and the absence of large well-defined clusters. Internet Math. 6(1), 29–123 (2009)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Lancichinetti, A., Fortunato, S., Kertész, J.: Detecting the overlapping and hierarchical community structure in complex networks. New J. Phys. 11(3), 033015 (2009) CrossRefGoogle Scholar
  12. 12.
    Andersen, R., Chung, F., Lang, K.: Local graph partitioning using pagerank vectors. In: 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS 2006), pp. 475–486. IEEE (2006)Google Scholar
  13. 13.
    Whang, J.J., Gleich, D.F., Dhillon, I.S.: Overlapping community detection using neighborhood-inflated seed expansion. IEEE Trans. Knowl. Data Eng. 28(5), 1272–1284 (2016)CrossRefGoogle Scholar
  14. 14.
    Hsu, C.-C., Lai, Y.-A., Chen, W.-H., Feng, M.-H., Lin, S.-D.: Unsupervised ranking using graph structures and node attributes. In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, pp. 771–779. ACM (2017)Google Scholar
  15. 15.
    Shani, G., Gunawardana, A.: Evaluating recommendation systems. In: Recommender Systems Handbook, pp. 257–297. Springer (2011)Google Scholar
  16. 16.
    Wang, Y., Wang, L., Li, Y., He, D., Chen, W., Liu, T.-Y.: A theoretical analysis of NDCG ranking measures. In: Proceedings of the 26th Annual Conference on Learning Theory (COLT 2013), vol. 8, p. 6 (2013)Google Scholar
  17. 17.
    Isinkaye, F., Folajimi, Y., Ojokoh, B.: Recommendation systems: principles, methods and evaluation. Egypt. Inform. J. 16(3), 261–273 (2015)CrossRefGoogle Scholar
  18. 18.
    Kowalik, Ł.: Approximation scheme for lowest outdegree orientation and graph density measures. In: International Symposium on Algorithms and Computation, pp. 557–566. Springer (2006)Google Scholar
  19. 19.
    Newman, M.E.: Modularity and community structure in networks. Proc. Natl. Acad. Sci. 103(23), 8577–8582 (2006)CrossRefGoogle Scholar
  20. 20.
    Chalupa, D.: A memetic algorithm for the minimum conductance graph partitioning problem, arXiv preprint arXiv:1704.02854 (2017)
  21. 21.
    Jeub, L.G., Balachandran, P., Porter, M.A., Mucha, P.J., Mahoney, M.W.: Think locally, act locally: detection of small, medium-sized, and large communities in large networks. Phys. Rev. E 91(1), 012821 (2015)CrossRefGoogle Scholar
  22. 22.
    McPherson, M., Smith-Lovin, L., Cook, J.M.: Birds of a feather: homophily in social networks. Annu. Rev. Sociol. 27(1), 415–444 (2001)CrossRefGoogle Scholar
  23. 23.
    Duan, L., Ma, S., Aggarwal, C., Ma, T., Huai, J.: An ensemble approach to link prediction. IEEE Trans. Knowl. Data Eng. 29(11), 2402–2416 (2017)CrossRefGoogle Scholar
  24. 24.
    Koren, Y., Bell, R.: Advances in collaborative filtering. In: Recommender Systems Handbook, pp. 77–118. Springer (2015)Google Scholar
  25. 25.
    Liben-Nowell, D., Kleinberg, J.: The link-prediction problem for social networks. J. Am. Soc. Inf. Sci. Technol. 58(7), 1019–1031 (2007)CrossRefGoogle Scholar
  26. 26.
    Lü, L., Zhou, T.: Link prediction in complex networks: a survey. Phys. A 390(6), 1150–1170 (2011)CrossRefGoogle Scholar
  27. 27.
    Ortega, A., Frossard, P., Kovačević, J., Moura, J.M., Vandergheynst, P.: Graph signal processing: overview, challenges, and applications. Proc. IEEE 106(5), 808–828 (2018)CrossRefGoogle Scholar
  28. 28.
    Martínez, V., Berzal, F., Cubero, J.-C.: A survey of link prediction in complex networks. ACM Comput. Surv. (CSUR) 49(4), 69 (2017)Google Scholar
  29. 29.
    Hanley, J.A., McNeil, B.J.: The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143(1), 29–36 (1982)CrossRefGoogle Scholar
  30. 30.
    Mason, S.J., Graham, N.E.: Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: statistical significance and interpretation. Q. J. R. Meteorol. Soc. 128(584), 2145–2166 (2002)CrossRefGoogle Scholar
  31. 31.
    Schaeffer, S.E.: Graph clustering. Comput. Sci. Rev. 1(1), 27–64 (2007)CrossRefGoogle Scholar
  32. 32.
    Görke, R., Kappes, A., Wagner, D.: Experiments on density-constrained graph clustering. J. Exp. Algorithmics (JEA) 19, 3–3 (2015)MathSciNetzbMATHGoogle Scholar
  33. 33.
    Holland, P.W., Laskey, K.B., Leinhardt, S.: Stochastic blockmodels: first steps. Soc. Netw. 5(2), 109–137 (1983)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Leskovec, J., Adamic, L.A., Huberman, B.A.: The dynamics of viral marketing. ACM Trans. Web (TWEB) 1(1), 5 (2007)CrossRefGoogle Scholar
  35. 35.
    Rohe, K., Chatterjee, S., Yu, B., et al.: Spectral clustering and the high-dimensional stochastic blockmodel. Ann. Stat. 39(4), 1878–1915 (2011)MathSciNetCrossRefGoogle Scholar
  36. 36.
    Abbe, E., Bandeira, A.S., Hall, G.: Exact recovery in the stochastic block model. IEEE Trans. Inf. Theory 62(1), 471–487 (2016) MathSciNetCrossRefGoogle Scholar
  37. 37.
    Tang, J., Zhang, J., Yao, L., Li, J., Zhang, L., Su, Z.: Arnetminer: extraction and mining of academic social networks. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 990–998. ACM (2008)Google Scholar
  38. 38.
    Lofgren, P., Banerjee, S., Goel, A.: Personalized pagerank estimation and search: a bidirectional approach. In: Proceedings of the Ninth ACM International Conference on Web Search and Data Mining, pp. 163–172. ACM (2016)Google Scholar
  39. 39.
    Krasanakis, E., Schinas, E., Papadopoulos, S., Kompatsiaris, Y., Symeonidis, A.: Boosted seed oversampling for local community ranking. Inf. Process. Manag. 102053 (2019, in press).
  40. 40.
    Kloster, K., Gleich, D.F.: Heat kernel based community detection. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1386–1395. ACM (2014)Google Scholar
  41. 41.
    Andersen, R., Chung, F., Lang, K.: Local partitioning for directed graphs using pagerank. Internet Math. 5(1–2), 3–22 (2008)MathSciNetCrossRefGoogle Scholar
  42. 42.
    Borgs, C., Chayes, J., Mahdian, M., Saberi, A.: Exploring the community structure of newsgroups. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 783–787. ACM (2004)Google Scholar
  43. 43.
    Gleich, D., Kloster, K.: Seeded pagerank solution paths. Eur. J. Appl. Math. 27(6), 812–845 (2016)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Emmanouil Krasanakis
    • 1
    Email author
  • Symeon Papadopoulos
    • 1
  • Yiannis Kompatsiaris
    • 1
  1. 1.CERTH-ITIThessalonikiGreece

Personalised recommendations