Application of varentropy as a measure of probabilistic uncertainty for complex networks

Varentropy is used as a general measure of probabilistic uncertainty for a complex network, inspired by the first and second laws of thermodynamics, but not limited to the equilibrium system. By exploring the relationship between the varentropy of the scale free distribution and the exponent of power laws as well as network size, we get the optimal design of a scale-free network against random failures. The behaviors of varentropy and the Shannon entropy of double Pareto law degree distribution are analyzed to compare their usefulness. Our conclusion is that varentropy is suitable and reliable.

Complex networks have recently attracted much interest from the community of physicists thanks to the possible application of various techniques inspired by statistical mechanics. Networks describe different interaction patterns in complex systems. In many cases, the interaction is changeable by time, and the topology of the network changes according to the external conditions. If a network is treated as a thermodynamic system which has an energy function depending on its topology, and entropy is introduced to describe the disorder strength, it could constitute a topological ensemble whose physical quantities are described by methods of statistical mechanics [1][2][3][4][5][6][7][8][9][10][11]. Equilibrium network ensembles are defined as stationary ensembles of graphs generated by restructuring processes, like edges of the graph are removed and/or inserted obeying detailed balance and ergodicity. During such processes, topological phase transitions are expected as the temperature is varied. However, this thermodynamic analogy has not yet been fully explored.
In particular, the concept of energy, which is very important in the study of statistical ensemble, is a big challenge. The function form of entropy in complex networks is not given a priori. Many different expressions have been suggested in recent research. They include functions of the node degree, number of links, or some global properties of the network. For instance, Farkas et al. [4] constructed several patterns of the energy E in the network based on vertex degrees or degrees of neighboring vertices, such as there is little convincing theoretical derivation for the definition of the energy in a complex network. One of our aims in the present paper is to tentatively solve this problem. If the energy function of a complex network is given, we can obtain an optimized network ensemble which minimizes the energy through zero temperature dynamics. On the other hand, we can get various topologies of the network at finite temperature. It is well known that many real world networks are scale-free networks and they are robust to random failures but vulnerable to targeted attacks. In addition, the character of heterogeneity of a network is directly related to this feature and can be measured by entropy [13,14]. Sole et al. [14] introduced the entropy of the remaining degree and the mutual information to study networks with different heterogeneity and randomness. In the present paper, we investigate the entropic behavior of the power law degree distribution to describe a network's heterogeneity, based on a generalized information measure called varentropy [15]. We find that the optimization of a scale-free network's robustness to random failures is equivalent to maximizing the entropy of the degree distribution. Through optimization, we can get a scenario of optimal design of scale-free networks against random failures.

Optimization of a scale-free network against random failures
We give a simple introduction to a universal measure of probabilistic uncertainty proposed in our previous work [15]. First, we suppose that we have a random discrete variable k i with a probability distribution p i = f (k i ) /Z, where i is the state index and Z is a normalization constant. The average value of k i is given by is the work done to the system by external forces where q j are the extensive variables such as volume, or distance. Based on the first law of thermodynamics, the quantity is the heat change in the system; that is, reversible process, where S is the thermodynamic entropy and T the absolute temperature. S has the following variational relation: As is well known, S measures the dynamical disorder of the system or the uncertainty of the probability distribution of energy in the dynamics. In contrast to another measure is just an extension of eq. (4) to arbitrary random variables k and to arbitrary systems (even out of equilibrium), and is a generic definition of maximum uncertainty measure for any random variable. Regarding a scale-free degree distribution i and  is an exponent, we derived the entropy formula in previous work [15]: I is known as the varentropy. The variation in varentropy against the exponent is shown in Figure 1. It suggests that varentropy decreases sharply with the increase of the exponent from 1.0 to 3.0. It is worth noticing that when   →1, the value of varentropy increases without bound, and when   < 1, varentropy becomes negative, which is meaningless in a physical context. Form the point of mathematics, the reason for this may be related to the divergence of the normalization constant Z at  < 1. Thus, the behavior of varentropy for scale-free degree distribution at   < 1 is still obscure. In fact, the majority of scale-free networks in nature have an exponent between 2 and 3 and they are finite networks with finite degree k = k min ,k min +1,...,k max where k min is the minimum degree and k max is the maximum degree cutoff present in the network, which are in the range of valid applications of varentropy.
We also can use varentropy to study the resilience of networks, which is very important to understand how to design networks that are optimally robust against both failures and attacks. For this subject, many different approaches have been proposed, such as the percolation theory [17,18], evolutionary algorithm [13], entropic principle [19], spectral and statistical measurement [20]. In the present paper, we focus on the entropy of the degree distribution to describe the network's heterogeneity, which measures the diversity of the degree distribution [14] within the context of a complex network. For a scale-free network, the effect of diversity (long tail) is an increase in the uncertainty. When the exponent increases or the cutoff degree decreases, the network becomes less heterogeneous and smaller entropy is observed [14]. Here, based on the relationship among k max , N and α in the network, N is the total number of vertices in network) [2]. Eq. (5) could be rewritten as Thus the varentropy I can be expressed as a function of  and N.  to the stable value. Varentropy decreases with the increase of exponent , which is consistent with the behavior of Figure 1. We conclude that the network designed with larger size N is more robust with given exponent , and the network with small exponent  will be more robust with given N.

Application to the double Pareto law distribution
In the real world, there are many truncated degree distribution in complex network such as the exponential truncated and double Pareto laws. Here, we pay more attention to the double Pareto law distribution and investigate its entropic behavior. First, we express the probability distribution as follows: where k c is the intersection of two power laws and ,  > 0.
With the normalization condition 1 According to eq. (1), we obtain the expression of the entropy function:  88. In general, the vertex in the network with bigger size often has larger degree, and thus this curve suggests that the size effect of the network becomes trivial when its size reaches some critical scale. In Figure 3(d), the entropy increases monotonically with increasing of k c at k max = 10 3 ,  = 1.2,  = 1.5. When k c is equal to 1 or 1000, the truncated distribution p evolves into single distribution p 2 or p 1 , and then the value of each entropy is equal to I p 2 = 25.5 or I p 1 = 86.1, respectively. Hence, from the above discussion, we find that for a network with a double Pareto law distribution, it will become more heterogeneous when the degree distribution has two smaller exponents and a larger cross over point with larger network size. In addition, we make a comparison with the Shannon entropy formula using the conditions of Figure 3(a). According where c 1 and c 2 are as in eq. (8). Here, we investigate the variation in Shannon entropy versus . In Figure 4, we can find that the tendency of Shannon entropy at k c = 100,  = 3, k max = 10 3 is similar to that in Figure 3(a). Both are monotonically decreasing functions. However, the difference between the two pictures is that when  increases without bound, the Shannon entropy trends to 0, but the varentropy tends to 1. We believe that the latter result is more reasonable because the value of the entropy of S p 2 is not 0, but 1 when    . Thus the Shannon entropy may be invalid for this kind of double Pareto law distribution in this condition.

Equilibrium network ensembles
Energy is often an important concept in optimization problems. It is not possible to derive an energy expression for graphs from first principles. We can find analogies with well-established equilibrium graph ensembles by assigning a statistical weight to each allowed graph. Phenomenological and heuristic arguments could lead to such various energy functions such as those described in the introduction. As is well known, the entropy is the bridge between the microscopic dynamics and macroscopic behaviors. According to the first law in equilibrium thermodynamics, Figure 4 Variation in Shannon entropy versus  under the same conditions as in Figure 3(a) with k c = 100,  = 3, k max = 10 3 . Shannon entropy decays with the increasing of , which is similar to that of varentropy. However it tends to zero at larger , which is different from that in Figure  3(a) with limiting value 1.0. In fact, when  increases without bound, S p 1 is close to zero, the Shannon entropy of function p should be equal to S p 2 which is not zero.
where E is the internal or mean energy of the system, T is the temperature of the system, S is the entropy of the system, and dW is the work done to the system by external forces. It could be rewritten as dS = ( dE  dW) /T and in another form of probability If p i = exp(E i ) /Z (Z is the partition function), after integrating of the above equation, we will get the Shannon-Gibbs entropy ln To investigate what will happen if p i is not an exponential distribution or E i is not the energy of the system of the state i, we get the extensive expression of the entropy in the formation of eq. (1), substitute the scale-free distribution p i = k i  /Z (k is the degree of the node in the network) into it, and obtain Comparing with dS = ( dE  dW) /T, the above equation is rewritten as Then, the thermodynamic equation of the scale-free network could be , k   1 1 , where parameters , ,    are analogous to the system energy, temperature, and free energy in thermodynamics respectively. k is the average degree of a scale-free network. Thus, based on the foundation of the first and second laws of thermodynamics, thermodynamical quantities of scale-free networks such as "energy", "temperature", and "free energy" are given. Although the assumption of an "equilibrium network ensemble" is used here, we believe that it is still a reasonable approach to apply statistical mechanisms and thermodynamics to complex networks.

Discussion and conclusions
In the present paper, we introduced an alternative method for the optimization of scale-free networks in terms of network entropy. This network entropy is not the traditional Shannon entropy. It is another definition of uncertainty measure of the system called varentropy, resulting from the first and second laws of thermodynamics, proposed in our previous work. Our findings indicate that small exponent and large network size can facilitate the optimal design of the scale-free network against random failures.
In addition, we studied the entropic behavior of the double Pareto law distribution with varentropy and Shannon entropy formula. The theoretical expression of each entropy function is obtained and their variance versus different parameters of distribution is analyzed. Through the comparison, we find that Shannon entropy may be less reasonable for calculation with this kind of distribution in some conditions. Lastly, we have shown that scale-free network ensembles can be described naturally by statistical mechanics and the thermodynamics methods, and provided the theoretical estimation of the quantities like "energy", "temperature" and "free energy" of the network ensemble, which is different from many previous works with various arbitrary definitions without any derivation. The results presented in this paper are only a tiny fraction of what can be done with the thermodynamics equation of scale-free network ensembles. They can facilitate the application of statistical physics and thermodynamics tools in the field of complex networks, and can also help to expand the analysis towards problems, such as optimization, extracting information and topological phase transitions.