On the computation of entropy production in stationary social networks

  • Tobias Hoßfeld
  • Valentin Burger
  • Haye Hinrichsen
  • Matthias Hirth
  • Phuoc Tran-Gia
Original Article
Part of the following topical collections:
  1. Social Systems as Complex Networks
  2. Social Systems as Complex Networks


Completing their initial phase of rapid growth, social networks are expected to reach a plateau from where on they are in a statistically stationary state. Such stationary conditions may have different dynamical properties. For example, if each message in a network is followed by a reply in opposite direction, the dynamics is locally balanced. Otherwise, if messages are ignored or forwarded to a different user, one may reach a stationary state with a directed flow of information. To distinguish between the two situations, we propose a quantity called entropy production that was introduced in statistical physics as a measure for non-vanishing probability currents in nonequilibrium stationary states. The proposed quantity closes a gap for characterizing online social networks. As major contribution, we show the relation and difference between entropy production and existing metrics. The comparison shows that computational intensive metrics like centrality can be approximated by entropy production for typical online social networks. To compute the entropy production from real-world measurements, the need for Bayesian inference and the limits of naïve estimates for those probability currents are shown. As further contribution, a general scheme is presented to measure the entropy production in small-world networks using Bayesian inference. The scheme is then applied for a specific example of the R mailing list.


Social network characterization Entropy production  Bayesian inference Practical computation 

Notion of variables

Variables describing the measurement data


Measurement period over which messages between individuals are recorded


Number of messages sent from \(i\) to \(j\) during time \(T\)


Total number of individuals, i.e., nodes in the graph, communicating during \(T\)


Total number of recorded messages, i.e., directed link, \(M=\sum _{i,j=1}^N n_{ij}\)

\(\delta _{a,b}\)

Kronecker delta defined by \(\delta _{a,b}={\left\{ \begin{array}{ll} 1 &{} a=b \\ 0 &{} a\ne b \end{array}\right. }\)


Total number of directed links, \(L=\sum _{i,j=1}^N 1-\delta _{0,n_{ij}}\)


Number of outgoing messages from node \(i\), \(n_i^{\mathrm{out}}=\sum _{j=1}^N n_{ij}\)


Number of incoming messages to node \(i\), \(n_i^{\mathrm{in}}=\sum _{j=1}^N n_{ji}\)


Outgoing degree of node \(i\)


Incoming degree of node \(i\)


Degree of node \(i\)

\(\Delta n_i\)

Difference of outgoing and incoming messages of node \(i\)


Probability that \(n\) messages are sent on a link, \(P(n)= \sum _{i,j=1}^N \delta _{n,n_{ij}}/N(N-1)\)

\(\mathcal {A}\)

Adjacency matrix with matrix elements \(\mathcal {A}_{ij}=1-\delta _{0,n_{ij}}\)

Variables describing entropy production


Message rate from \(i\) to \(j\) estimated by measured \(n_{ij}\) over \(T\)

\(\mathcal {W}\)

Rate matrix with matrix elements \(\mathcal {W}_{ij}=w_{ij}\)

\(\Delta H_{ij}\)

Amount of entropy increased for each message sent from \(i\) to \(j\), \(\Delta H_{ij}=\ln \frac{w_{ij}}{w_{ji}}\)


Entropy production per link, \(H_{ij} = \left( n_{ij}-n_{ji}\right) \ln \frac{w_{ij}}{w_{ji}}\)


Entropy production per node \(i\), \(H_i=\frac{1}{2}\sum _{j=1}^N H_{ij}\)


Entropy production of total network, \(H=\sum _{i=1}^N H_i\)


Node entropy production per message, \(h_i = \frac{H_i}{n_i^{\mathrm{out}}+n_i^{\mathrm{in}}}\)

Variables for estimating message rates


Posterior distribution of message rates \(w\) conditional on observed messages \(n\)


Prior distribution of message rates; assumed to follow a power law in social networks with \(P(w) \sim w^{-1-\alpha }\); normalization with suitable lower cutoff leads to inverse gamma distribution \(P(w) \;=\; \frac{\beta ^{\alpha }}{\Gamma (\alpha )} {w^{-\alpha -1} e^{-\beta /w}}\)


Shape parameter of the inverse gamma distribution


Lower cutoff parameter for the rate \(w\) concerning inverse gamma distribution


Normalizing marginal likelihood

\(\langle \ln w \rangle _n\)

Expectation value for given \(n\), \(\langle \ln w \rangle _n \;=\; \int \limits _0^{\infty } \mathrm{d}w\, \ln w P(w|n)\)

\(K_\nu (z)\)

Modified Bessel function of the second kind and \(z=2\sqrt{\beta T}\)


  1. Andrieux D, Gaspard P (2004) Fluctuation theorem and onsager reciprocity relations. J Chem Phys 121(13)Google Scholar
  2. Andrieux D, Gaspard P, Ciliberto S, Garnier N, Joubaud S, Petrosyan A (2007) Entropy production and time asymmetry in nonequilibrium fluctuations. Phys Rev Lett 98:150601CrossRefGoogle Scholar
  3. Barnes NG, Andonian J. (2011) The 2011 fortune 500 and social media adoption: Have america’s largest companies reached a social media plateau?
  4. Bilgin C, Yener B (2010) Dynamic network evolution: models, clustering, anomaly detection. Technical report, Rensselaer University, NYGoogle Scholar
  5. Bird C, Gourley A, Devanbu P, Gertz M, Swaminathan A (2006) Mining email social networks. In: Proceedings of the 2006 international workshop on Mining software repositories. ACM, pp 137–143Google Scholar
  6. Box GEP, Tiao GC (1973) Bayesian inference in statistical analysis. Wiley, New YorkzbMATHGoogle Scholar
  7. Brandes U (2001) A faster algorithm for betweenness centrality. J Math Sociol 25(2):163–177CrossRefzbMATHGoogle Scholar
  8. Broder A, Kumar R, Maghoul F, Raghavan P, Rajagopalan S, Stata R, Tomkins A, Wiener J (2000) Graph structure in the web. Comput Netw 33(1):309–320CrossRefGoogle Scholar
  9. Chen H, Shen H, Xiong J, Tan S, Cheng X (2006) Social network structure behind the mailing lists: Ict-iiis at trec 2006 expert finding track. In: Voorhees EM, Buckland LP (eds) Proceedings of the fifteenth text retrieval conference, TREC 2006, volume Special Publication 500–272. National Institute of Standards and Technology (NIST)Google Scholar
  10. Dehmer M, Mowshowitz A (2011) A history of graph entropy measures. Inf Sci 181(1):57–78CrossRefMathSciNetzbMATHGoogle Scholar
  11. Ebel H, Mielsch L-I, Bornholdt S (2002) Scale-free topology of e-mail networks. Phys Rev E 66 (2002).  10.1103/PhysRevE.66.035103
  12. Fagiolo G (2007) Clustering in complex directed networks. Phys Rev E 76(2):026107CrossRefMathSciNetGoogle Scholar
  13. Fazeen M, Dantu R, Guturu P (2011) Identification of leaders, lurkers, associates and spammers in a social network: context-dependent and context-independent approaches. Soc Netw Anal Min 1(3):241–254CrossRefGoogle Scholar
  14. Garrido A (2011) Symmetry in complex networks. Symmetry 3(1):1–15. doi: 10.3390/sym3010001
  15. Gilbert F, Simonetto P, Zaidi F, Jourdan F, Bourqui R (2011) Communities and hierarchical structures in dynamic social networks: analysis and visualization. Soc Netw Anal Min 1(2):83–95CrossRefGoogle Scholar
  16. Gómez-Gardeñes J, Latora V (2008) Entropy rate of diffusion processes on complex networks. Phys Rev E 78(6):065102CrossRefGoogle Scholar
  17. Hirth M, Lehrieder F, Oberste-Vorth S, Hoßfeld T, Tran-Gia P (2012) Wikipedia and its network of authors from a social network perspective. In: International conference on communications and electronics (ICCE), Hue, VietnamGoogle Scholar
  18. Hoßfeld T, Hirth M, Tran-Gia P (2011a) Modeling of crowdsourcing platforms and granularity of work organization in future internet. In: International teletraffic congress (ITC), San FranciscoGoogle Scholar
  19. Hoßfeld T, Lehrieder F, Hock D, Oechsner S, Despotovic Z, Kellerer W, Michel M (2011b) Characterization of bittorrent swarms and their distribution in the internet. Comput Netw 55(5)Google Scholar
  20. Jin EM, Girvan M, Newman MEJ (2001) Structure of growing social networks. Phys Rev E 64(4):046132–046139. doi: 10.1103/PhysRevE.64.046132
  21. Kossinets G, Watts DJ (2006) Empirical analysis of an evolving social network. Science 311(5757):88–90CrossRefMathSciNetzbMATHGoogle Scholar
  22. Kourtellis N, Alahakoon T, Simha R, Iamnitchi A, Tripathi R (2013) Identifying high betweenness centrality nodes in large social networks. Soc Netw Anal Min 3(4):899–914CrossRefGoogle Scholar
  23. Leskovec J, Kleinberg J, Faloutsos C (2005) Graphs over time: densification laws, shrinking diameters and possible explanations. In: Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining, pp 177–187Google Scholar
  24. Mihaljev T, de Arcangelis L, Herrmann HJ (2011) Interarrival times of message propagation on directed networks. Phys Rev E 84:026112CrossRefGoogle Scholar
  25. Mislove A, Marcon M, Gummadi KP, Druschel P, Bhattacharjee B (2007) Measurement and analysis of online social networks. In: Proceedings of the 7th ACM SIGCOMM conference on Internet, measurement, pp. 29–42Google Scholar
  26. Moler C (2011) Experiments with matlab. The MathWorks Co, NatickGoogle Scholar
  27. Mowshowitz A, Dehmer M (2010) A symmetry index for graphs. Symmetr Cult Sci 21(4):321–327Google Scholar
  28. Mowshowitz A, Dehmer M (2012) Entropy and the complexity of graphs revisited. Entropy 14(3):559–570CrossRefMathSciNetzbMATHGoogle Scholar
  29. Page L, Brin S, Motwani R, Winograd T (1999) The pagerank citation ranking: bringing order to the web. Technical Report 1999–66, Stanford InfoLabGoogle Scholar
  30. Pincus SM, Huang W-M (1992) Approximate entropy: statistical properties and applications. Commun Stat Theory Methods 21(11):3061–3077CrossRefzbMATHGoogle Scholar
  31. R Mailing Lists. (2013)
  32. Sallaberry A, Zaidi F, Melanton G (2013) Model for generating artificial social networks having community structures with small-world and scale-free properties. Soc Netw Anal Min 3(3):597–609CrossRefGoogle Scholar
  33. Schnakenberg J (1976) Network theory of microscopic and macroscopic behavior of master equation systems. Rev Mod Phys 48(4):571–585. doi: 10.1103/RevModPhys.48.571
  34. Schreiber T (2000) Measuring information transfer. Phys Rev Lett 85(2):461–464. doi: 10.1103/PhysRevLett.85.461
  35. Seifert U (2005) Entropy production along a stochastic trajectory and an integral fluctuation theorem. Phys Rev Lett 95(4):040602–040605. doi: 10.1103/PhysRevLett.95.040602
  36. Sinatra R, Gómez-Gardeñes J, Lambiotte R, Nicosia V, Latora V (2011) Maximal-entropy random walks in complex networks with limited information. Phys Rev E 83:030103CrossRefGoogle Scholar
  37. Smilkov D, Kocarev L (2012) Influence of the network topology on epidemic spreading. Phys Rev E 85:016114CrossRefGoogle Scholar
  38. Tietz C, Schuler S, Speck T, Seifert U, Wrachtrup J (2006) Measurement of stochastic entropy production. Phys Rev Lett 97:050602CrossRefGoogle Scholar
  39. Vasudevan M, Deo N (2012) Efficient community identification in complex networks. Soc Netw Anal Min 2(4):345–359CrossRefGoogle Scholar
  40. Wang J, De Wilde P (2004) Properties of evolving e-mail networks. Phys Rev E 70:066121CrossRefGoogle Scholar
  41. West J, Lacasa L, Severini S, Teschendorff A (2012) Approximate entropy of network parameters. Phys Rev E 85:046111CrossRefGoogle Scholar
  42. Xiao Y-H, Wu W-T, Wang H, Xiong M, Wang W (2008) Symmetry-based structure entropy of complex networks. Phys A Stat Mech Appl 387(11):2611–2619Google Scholar
  43. Zeraati S, Jafarpour FH, Hinrichsen H (2012) Entropy production of nonequilibrium steady states with irreversible transitions. J Stat Mech Theory Exp 2012(12):L12001CrossRefMathSciNetGoogle Scholar
  44. Zhu C, Kuh A, Wang J, De Wilde P (2006) Analysis of an evolving email network. Phys Rev E 74:046109CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Wien 2014

Authors and Affiliations

  • Tobias Hoßfeld
    • 1
  • Valentin Burger
    • 1
  • Haye Hinrichsen
    • 2
  • Matthias Hirth
    • 1
  • Phuoc Tran-Gia
    • 1
  1. 1.Chair of Communication Networks, Institute of Computer ScienceUniversity of WürzburgWürzburgGermany
  2. 2.Department of Physics and AstronomyUniversity of WürzburgWürzburgGermany

Personalised recommendations