Increasing sampling efficiency for the fixed degree sequence model with phase transitions

  • Christian Brugger
  • André Lucas Chinazzo
  • Alexandre Flores John
  • Christian De Schryver
  • Norbert Wehn
  • Wolfgang Schlauch
  • Katharina Anna Zweig
Original Article

Abstract

Real-world network data is often very noisy and contains erroneous or missing edges. These superfluous and missing edges can be identified statistically by assessing the number of common neighbors of the two incident nodes. To evaluate whether this number of common neighbors, the so-called co-occurrence, is statistically significant, a comparison with the expected co-occurrence in a suitable random graph model is required. For networks with a skewed degree distribution, including most real-world networks, it is known that the fixed degree sequence model (FDSM), which maintains the degrees of nodes, is favorable over using simplified graph models that are based on an independence assumption. However, the use of a FDSM requires sampling from the space of all graphs with the given degree sequence and measuring the co-occurrence of each pair of nodes in each of the samples, since there is no known closed formula known for this statistic. While there exist log-linear approaches such as Markov chain Monte Carlo sampling, the computational complexity still depends on the length of the Markov chain and the number of samples, which is significant in large-scale networks. In this article, we show based on ground truth data for different data sets that there are various phase transition-like tipping points that enable us to choose a comparatively low number of samples and to reduce the length of the Markov chains without reducing the quality of the significance test. As a result, the computational effort can be reduced by an order of magnitudes. Furthermore, we present and evaluate practically usable strategies for speeding up the randomization process of input graphs and heuristics for phase transition-based computation stopping.

Keywords

Graph processing Fixed degree sequence model Link assessment Online heuristics Phase transitions Data cleaning Randomization strategies Markov chain Monte Carlo 

References

  1. Berger A, Müller-Hannemann M (2010) Uniform sampling of digraphs with a fixed degree sequence. In: Thilikos DM (ed) Graph theoretic concepts in computer science, vol 6410. Springer, Heidelberg, pp 220–231Google Scholar
  2. Brualdi RA (1980) Matrices of zeros and ones with fixed row and column sum vectors. Linear Algebra Appl 33:159–231MathSciNetCrossRefMATHGoogle Scholar
  3. Brugger C, Chinazzo AL, John AF, De Schryver C, Wehn N, Spitz A, Zweig KA (2015) Exploiting phase transitions for the efficient sampling of the fixed degree sequence model. In: Proceedings of the 2015 IEEE/ACM international conference on advances in social networks analysis and mining (ASONAM), ASONAM’15. IEEE, ACM, New York, NY, pp 308–313, August 2015Google Scholar
  4. Cobb GW, Chen Y-P (2003) An application of Markov Chain Monte Carlo to community ecology. Am Math Mon 110:265–288MathSciNetCrossRefMATHGoogle Scholar
  5. Del Genio CI, Kim H, Toroczkai Z, Bassler KE (2010) Efficient and exact sampling of simple graphs with given arbitrary degree sequence. PloS one 5(4):e10012CrossRefGoogle Scholar
  6. Gionis A et al (2007) Assessing data mining results via swap randomization. ACM Trans Knowl Discov Data 1(3), Article No 14Google Scholar
  7. Gionis A, Mannila H, Mielikäinen T, Tsaparas P (2006) Assessing data mining results via swap randomization. In: Proceedings of the twelfth ACM SIGKDD international conference on knowledge discovery and data mining (KDD’06)Google Scholar
  8. Horvát E-Á, Zweig KA (2013) A fixed degree sequence model for the one-mode projection of multiplex bipartite graphs. Soc Netw Anal Min 4:164Google Scholar
  9. Jerrum M, Sinclair A, Vigoda E (2004) A polynomial-time approximation algorithm for the permanent of a matrix with nonnegative entries. JACM 51(4):671–697MathSciNetCrossRefMATHGoogle Scholar
  10. Kannan R, Tetali P, Vempala S (1999) Simple markov-chain algorithms for generating bipartite graphs and tournaments. Random Struct Algorithms 14(4):293–308MathSciNetCrossRefMATHGoogle Scholar
  11. Liben-Nowell D, Kleinberg J (2007) The link-prediction problem for social networks. J Am Soc Inform Sci Technol 58(7):1019–1031. doi:10.1002/asi.20591 CrossRefGoogle Scholar
  12. Milo R, Itzkovitz S, Kashtan N, Levitt R, Alon U (2004) Response to comment on “Network motifs: simple building blocks of complex networks” and “Superfamilies of evolved and designed networks”. Science 305:1107dCrossRefGoogle Scholar
  13. Uhlmann S, Mannsperger H, Zhang JD, Horvat E-Á, Schmidt C, Küblbeck M, Ward A, Tschulena U, Zweig K, Korf U, Wiemann S, Sahin Ö (2012) Global miRNA regulation of a local protein network: case study with the EGFR-driven cell cycle network in breast cancer. Mol Syst Biol 8:570CrossRefGoogle Scholar
  14. Zweig KA (2010) How to forget the second side of the story: a new method for the one-mode projection of bipartite graphs. In: Proceedings of the 2010 international conference on advances in social networks analysis and mining ASONAM 2010, pp 200–207Google Scholar
  15. Zweig KA (2011) Good versus optimal: why network analytic methods need more systematic evaluation. Central Eur J Comput Sci 1:137–153Google Scholar
  16. Zweig KA, Kaufmann M (2011) A systematic approach to the one-mode projection of bipartite graphs. Soc Netw Anal Min 1(3):187–218CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Wien 2016

Authors and Affiliations

  1. 1.Microelectronic System Design Research GroupUniversity of KaiserslauternKaiserslauternGermany
  2. 2.Complex Network Analysis GroupUniversity of KaiserslauternKaiserslauternGermany

Personalised recommendations