Advertisement

DeepRank: improving unsupervised node ranking via link discovery

  • Yi-An Lai
  • Chin-Chi HsuEmail author
  • Wen-Hao Chen
  • Mi-Yen Yeh
  • Shou-De Lin
Article
  • 71 Downloads

Abstract

This paper proposes an unsupervised node-ranking model that considers not only the attributes of nodes in a graph but also the incompleteness of the graph structure. We formulate the unsupervised ranking task into an optimization task and propose a deep neural network (DNN) structure to solve it. The rich representation capability of the DNN structure together with a novel design of the objectives allow the proposed model to significantly outperform the state-of-the-art ranking solutions.

Keywords

Unsupervised learning Node ranking PageRank Link prediction Neural networks 

Notes

Acknowledgements

This material is based upon work supported by the Air Force Office of Scientific Research, AOARD under Award Number FA2386-17-1-4038, and Taiwan Ministry of Science and Technology (MOST) under Grant Number 106-2218-E-002-042.

References

  1. Backstrom L, Leskovec J (2011) Supervised random walks: predicting and recommending links in social networks. In: Proceedings of the fourth ACM international conference on web search and data mining (WSDM’11). ACM, New York, NY, USA, pp 635–644.  https://doi.org/10.1145/1935826.1935914
  2. Bogolubsky L, Dvurechensky P, Gasnikov A, Gusev G, Nesterov Y, Raigorodskii AM, Tikhonov A, Zhukovskii M (2016) Learning supervised pagerank with gradient-based and gradient-free optimization methods. In: Lee DD, Sugiyama M, Luxburg UV, Guyon I, Garnett R (eds) Advances in neural information processing systems 29. Curran Associates, Inc., pp 4914–4922. http://papers.nips.cc/paper/6565-learning-supervised-pagerank-with-gradient-based-and-gradient-free-optimization-methods.pdf
  3. Bromley J, Guyon I, LeCun Y, Säckinger E, Shah R (1993) Signature verification using a “siamese” time delay neural network. In: Proceedings of the 6th international conference on neural information processing systems (NIPS’93). Morgan Kaufmann, San Francisco, CA, USA, pp 737–744. http://dl.acm.org/citation.cfm?id=2987189.2987282
  4. Chang H, Cohn D, McCallum A (2000) Learning to create customized authority lists. In: Langley P (ed) Proceedings of the seventeenth international conference on machine learning (ICML 2000), Stanford University, Stanford, CA, USA, June 29–July 2, 2000. Morgan Kaufmann, pp 127–134Google Scholar
  5. Clevert D, Unterthiner T, Hochreiter S (2016) Fast and accurate deep network learning by exponential linear units (elus). In: International conference on learning representations arXiv:1511.07289
  6. Freeman LC (1978) Centrality in social networks conceptual clarification. Soc Netw 1(3):215–239.  https://doi.org/10.1016/0378-8733(78)90021-7 CrossRefGoogle Scholar
  7. Gantner Z, Drumond L, Freudenthaler C, Schmidt-Thieme L (2011) Personalized ranking for non-uniformly sampled items. In: Proceedings of the 2011 international conference on KDD Cup 2011—Volume 18, JMLR.org (KDDCUP’11), pp 231–247. http://dl.acm.org/citation.cfm?id=3000375.3000390
  8. Gao B, Liu TY, Wei W, Wang T, Li H (2011) Semi-supervised ranking on very large graphs with rich metadata. In: Proceedings of the 17th ACM SIGKDD international conference on knowledge discovery and data mining (KDD’11). ACM, New York, NY, USA, pp 96–104.  https://doi.org/10.1145/2020408.2020430
  9. Gehrke J, Ginsparg P, Kleinberg J (2003) Overview of the 2003 KDD cup. SIGKDD Explor Newsl 5(2):149–151.  https://doi.org/10.1145/980972.980992 CrossRefGoogle Scholar
  10. Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: Gordon G, Dunson D, Dudk M (eds) Proceedings of the fourteenth international conference on artificial intelligence and statistics, PMLR, Fort Lauderdale, FL, USA, proceedings of machine learning research, vol 15, pp 315–323. http://proceedings.mlr.press/v15/glorot11a.html
  11. Gyöngyi Z, Garcia-Molina H, Pedersen J (2004) Combating web spam with trustrank. In: Proceedings of the thirtieth international conference on very large data bases—volume 30, VLDB Endowment (VLDB’04), pp 576–587. http://dl.acm.org/citation.cfm?id=1316689.1316740
  12. He X, Liao L, Zhang H, Nie L, Hu X, Chua TS (2017) Neural collaborative filtering. In: Proceedings of the 26th international conference on World Wide Web, International World Wide Web Conferences Steering Committee (WWW’17), Republic and Canton of Geneva, Switzerland, pp 173–182.  https://doi.org/10.1145/3038912.3052569
  13. Heidemann J, Klier M, Probst F (2010) Identifying key users in online social networks: a pagerank based approach. In: Sabherwal R, Sumner M (eds) Proceedings of the international conference on information systems (ICIS 2010), Saint Louis, Missouri, USA, December 12–15, 2010. Association for Information Systems, p 79. http://aisel.aisnet.org/icis2010_submissions/79
  14. Hsu CC, Lai YA, Chen WH, Feng MH, Lin SD (2017) Unsupervised ranking using graph structures and node attributes. In: Proceedings of the tenth ACM international conference on web search and data mining (WSDM’17). ACM, New York, NY, USA, pp 771–779.  https://doi.org/10.1145/3018661.3018668
  15. Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: International conference on learning representations arXiv:1412.6980
  16. Kleinberg JM (1999) Authoritative sources in a hyperlinked environment. J ACM 46(5):604–632.  https://doi.org/10.1145/324133.324140 MathSciNetCrossRefzbMATHGoogle Scholar
  17. Lai YA, Hsu CC, Chen WH, Yeh MY, Lin SD (2017) Prune: preserving proximity and global ranking for network embedding. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds) Advances in neural information processing systems 30. Curran Associates, Inc., pp 5257–5266. http://papers.nips.cc/paper/7110-prune-preserving-proximity-and-global-ranking-for-network-embedding.pdf
  18. Liben-Nowell D, Kleinberg J (2007) The link-prediction problem for social networks. J Am Soc Inf Sci Technol 58(7):1019–1031.  https://doi.org/10.1002/asi.v58:7 CrossRefGoogle Scholar
  19. Lichtenwalter RN, Lussier JT, Chawla NV (2010) New perspectives and methods in link prediction. In: Proceedings of the 16th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, New York, NY, USA (KDD’10), pp 243–252.  https://doi.org/10.1145/1835804.1835837
  20. Lü L, Zhou T (2011) Link prediction in complex networks: a survey. Phys A Stat Mech Its Appl 390(6):1150–1170.  https://doi.org/10.1016/j.physa.2010.11.027 CrossRefGoogle Scholar
  21. Martínez V, Berzal F, Cubero JC (2016) A survey of link prediction in complex networks. ACM Comput Surv 49(4):69:1–69:33.  https://doi.org/10.1145/3012704 CrossRefGoogle Scholar
  22. Martins AFT, Astudillo RF (2016) From softmax to sparsemax: a sparse model of attention and multi-label classification. In: Proceedings of the 33rd international conference on international conference on machine learning—volume 48, JMLR.org, ICML’16, pp 1614–1623. http://dl.acm.org/citation.cfm?id=3045390.3045561
  23. Menon AK, Elkan C (2011) Link prediction via matrix factorization. In: Proceedings of the 2011 European conference on machine learning and knowledge discovery in databases—volume part II (ECML PKDD’11). Springer, Berlin, pp 437–452. http://dl.acm.org/citation.cfm?id=2034117.2034146
  24. Newman MEJ (2001) Clustering and preferential attachment in growing networks. Phys Rev E 64:025102.  https://doi.org/10.1103/PhysRevE.64.025102 CrossRefGoogle Scholar
  25. Page L, Brin S, Motwani R, Winograd T (1999) The pagerank citation ranking: bringing order to the web. Technical Report 1999-66, Stanford InfoLab. http://ilpubs.stanford.edu:8090/422/, previous number = SIDL-WP-1999-0120
  26. Rendle S, Freudenthaler C (2014) Improving pairwise learning for item recommendation from implicit feedback. In: Proceedings of the 7th ACM international conference on web search and data mining (WSDM’14). ACM, New York, NY, USA, pp 273–282.  https://doi.org/10.1145/2556195.2556248
  27. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958. http://jmlr.org/papers/v15/srivastava14a.html
  28. Stern DH, Herbrich R, Graepel T (2009) Matchbox: Large scale online bayesian recommendations. In: Proceedings of the 18th international conference on World Wide Web (WWW’09). ACM, New York, NY, USA, pp 111–120.  https://doi.org/10.1145/1526709.1526725
  29. Tsoi AC, Morini G, Scarselli F, Hagenbuchner M, Maggini M (2003) Adaptive ranking of web pages. In: Proceedings of the 12th international conference on World Wide Web (WWW’03). ACM, New York, NY, USA, pp 356–365.  https://doi.org/10.1145/775152.775203
  30. Viswanath B, Mislove A, Cha M, Gummadi KP (2009) On the evolution of user interaction in facebook. In: Proceedings of the 2Nd ACM workshop on online social networks (WOSN’09). ACM, New York, NY, USA, pp 37–42.  https://doi.org/10.1145/1592665.1592675
  31. Wang Y, Tong Y, Zeng M (2013) Ranking scientific articles by exploiting citations, authors, journals, and time information. In: desJardins M, Littman ML (eds) Proceedings of the twenty-seventh AAAI conference on artificial intelligence, July 14–18, 2013, Bellevue, Washington, USA. AAAI Press. http://www.aaai.org/ocs/index.php/AAAI/AAAI13/paper/view/6363
  32. Wang P, Xu B, Wu Y, Zhou X (2015) Link prediction in social networks: the state-of-the-art. Sci China Inf Sci 58(1):1–38.  https://doi.org/10.1007/s11432-014-5237-y CrossRefGoogle Scholar
  33. Xing W, Ghorbani A (2004) Weighted pagerank algorithm. In: Proceedings. In: Second annual conference on communication networks and services research, 2004, pp 305–314.  https://doi.org/10.1109/DNSR.2004.1344743
  34. Zhai S, Zhang ZM (2015) Dropout training of matrix factorization and autoencoder for link prediction in sparse graphs. In: Venkatasubramanian S, Ye J (eds) Proceedings of the 2015 SIAM international conference on data mining, Vancouver, BC, Canada, April 30–May 2, 2015. SIAM, pp 451–459.  https://doi.org/10.1137/1.9781611974010.51
  35. Zhukovskiy M, Gusev G, Serdyukov P (2014) Supervised nested pagerank. In: Proceedings of the 23rd ACM international conference on information and knowledge management (CIKM’14). ACM, New York, NY, USA, pp 1059–1068.  https://doi.org/10.1145/2661829.2661969

Copyright information

© The Author(s) 2019

Authors and Affiliations

  1. 1.Department of Computer Science and Information EngineeringNational Taiwan UniversityTaipeiTaiwan
  2. 2.Institute of Information ScienceAcademia SinicaTaipeiTaiwan
  3. 3.Research Center for Information TechnologyAcademia SinicaTaipeiTaiwan

Personalised recommendations