Abstract
Identifying network structures from dynamical observations is a fundamental problem currently pervading scientific research on complex systems, as understanding and modeling the structure of a complex network will lead to greater knowledge of its evolutionary mechanisms and to a better understanding of its functional behaviors. Usually, one needs to identify a network’s structure through a limited number of observations. Particularly, couplings of many real-world networks are sparse and continuously varying with time. In this study, a new framework is developed via optimization for identifying structures of continuously-varying weighted networks formed by sparsely-connected dynamical systems. Furthermore, a regularization technique is employed to increase the numerical stability of the parameter estimation algorithm. Three numerical examples are provided to illustrate the feasibility and effectiveness of the proposed identification method. In comparison with other existing techniques, the main advantages of our method include its ability to identify structures of continuously-varying weighted networks in addition to static ones, as well as its requirement of a relatively small number of observations. The proposed method has a potential applicability to a variety of evolving complex dynamical networks.
Similar content being viewed by others
Introduction
Time-varying networks describe a large number of systems whose constituents and interactions evolve over time1,2,3. This subject has attracted extensive attention from researchers in various fields. For instance, Starnini et al. proposed several immunization strategies for epidemic processes in time-varying networks2, Amritkar et al. studied the synchronization properties of coupled dynamics on time-varying networks and the corresponding time-average network4 and Nicola et al. explored the random walk process in a fairly general class of time-varying networks5. Indeed, complex systems taking the form of time-varying networks are ubiquitous. Some plausible representations of the relational information among entities in dynamical systems are time-varying networks that are topologically rewiring and semantically evolving over time6,7,8, such as a social community, living cells, e-mail messages, mobile telephone calls, functional brain networks, food webs and other networks of species. It is noteworthy that topological structures of complex networks play an important role in determining their evolutionary dynamics and functional behaviors and may have significant consequences for many real-world applications9. However, in many practical situations, the precise structure of a complex dynamical network is unknown or uncertain. Therefore, to find a general solution to the structure identification problem of complex networks is of primary importance.
Many achievements have been made on structure identification of complex networks10,11,12,13,14,15,16. For example, Zhang et al. constructed an auxiliary complex network in a very general form and designed some adaptive controllers to identify network structures based upon generalized outer synchronization10, Wu et al. used Granger causality test to infer network structures11,12, Wang et al. estimated network parameters based on compressive sensing13,14, Kolar et al. presented two new machine learning methods for estimating time-varying networks based on a temporally smoothed l1-regularized logistic regression formalism15 and Rao et al. used a state-space model to infer time-varying network topologies from gene expression data16. However, most if not all of these studies focus on the case that network structures are static and those works regarding estimating time-varying structures of networks only care about topological changes in the structures, not about continuous changes of coupling weights. Yet, networks are not only specified by their structures but also by the dynamics of information taking place on the structure17. Examples include a social network where there exists stronger or weaker social ties between individuals, a metabolic network where there are more or less flux along particular reaction pathways, a food web where there are varying energy or carbon flow between predator-prey pairs18, among many others. Thus it is practical to assign varying weights for each edge of a complex network without changing the structure17.
In this paper, we propose a method to identify structures of networks whose edge weights continuously vary with time. In many situations, the interactions among elements of a large system are rapidly changing and are usually characterized by processes whose timing and duration are defined on a very short temporal scale3. Since interactive weights among elements in a real-world network sometimes are varying continuously with time, the assumption of static structures or activity-driven varying structures is sometimes inappropriate. Continuously-varying structures should thus be taken into account for analysis of this kind of realistic complex networks. Furthermore, due to the fact that the structure of a real-world network normally is sparse and there is only a limited number of observations, we propose an optimization framework to identify the continuously-varying weights of a complex network. In designing the algorithm, a regularization technique is incorporated, which led to an efficient and robust scheme as finally verified by accurate or noisy data sets observed from a 6-node directed chaotic network, a 50-node undirected network, as well as a 50000-node small-world network.
Network analysis
Mathematical notations
Some necessary notations that will be used throughout the paper are introduced. x⊤ (or M⊤) denotes the transpose of a vector x (or a matrix M), is the Euclidean-norm of x, is the l0 norm of x, ⊗ represents the Kronecker product, is the N-dimensional real space, denotes the identity matrix of order N.
Consider the following complex dynamical network consisting of N identical nodes with continuously-varying couplings:
where is the state vector of the i-th node and is a smooth nonlinear vector-valued function governing the dynamics of the i-th node. Matrix is the inner coupling matrix determining the interaction of component variables. The configuration matrix describes the continuously-varying topology of the weighted network. Specifically, at time point t, if there is a link from node j to node i (j ≠ i), then the coupling strength cij(t) > 0, otherwise cij(t) = 0. The diagonal entries of matrix C(t) are defined as
Denote and . Then, network (1) can be rewritten as
Furthermore, by denoting and , one has
In general, can be approximated by its first-order difference quotient, that is,
Here, t* is a time point that is sufficiently close to t so that can be more accurately approximated.
Let represent the observed node dynamics X(t) at time point t for the k-th time, where m − 1(m ∈ ) is the total number of observations. Then one has
where Mt = M(t) and represent the k-th observations of X(t) and Y(t), respectively.
From Eq. (2), one has , where is a row vector whose elements are ones and is a row vector whose elements are zeros. Let . Then, one has
If every row of C(t) ⊗ Γ is a sparse vector, the above problem can be translated into solving the following minimization problem:
where index i represents the i-th row of matrix Mt. To ensure the restricted isometry property, one will normalize At with dividing elements in each column with the Euclidean norm of that column13.
Results
In this section, three numerical examples are provided to illustrate the effectiveness of the identification technique proposed in the preceding section. In the first two examples, each node in the network represents a well-known three-dimensional chaotic Lorenz system, as described by
where a = 10, b = 8/3, c = 28.
In the following numerical simulations, the fourth-order Runge–Kutta method is employed to solve the ordinary differential equations of the dynamical network. The m − 1 observations are collected by solving the ordinary differential equations in Eq. (3) with m − 1 sets of different initial values.
Numerical validation on a 6-node weighted directed network
Consider a weighted directed network (1) consisting of 6 Lorenz oscillators, with the continuously-varying structure matrix being
and the inner coupling matrix being Γ = I3×3. In the following simulations, it is always assumed that the observed X(t) in Eq. (4) is accurate. The time step used in the Runge-Kutta method for generating data is h = 10−6 and 11(m = 12) different trajectories are generated from (1) starting from 11 sets of random initial values. Therefore, at time point t, 11 observations can be collected to identify instantaneous network structures.
Figures 1, 2, 3 display successful identification of C(t) from noise-free dynamical observations, where only nonzero elements of matrix C(t) are displayed. In all the figures, the dots (cij) represent the estimated values at different time points and the lines are the original curves of cij(t). From these figures, it is obvious that the estimated values match perfectly with the original structures.
In real-world applications, observation noise commonly exists. Thus, in order to test the robustness of the proposed optimization-based structure identification algorithm in practical scenarios, two more numerical simulations are carried out. Figure 4 presents identification results from noisy observations, where observed node dynamics are assumed to be contaminated by white noise. The left panel of Fig. 4 shows the result when observation noise satisfying a uniform distribution on the interval [0, 0.01] is inserted. It is seen that the varying structures are still correctly identified. In the right panel, the noise is stronger and satisfies a uniform distribution over the interval [0, 0.1]. The inferred values still match those of the original ones, though there is some slight disagreement caused by noise. This example illustrates the strong robustness of the proposed approach.
As is known, synchronization is an obstacle to topology identification19,20. Figure 5 displays the identification results with a longer time span without noise perturbation. From the left panel, it is observed that structures can be correctly identified during the time period [0, 3.6). At about t = 3.6, the structures cannot be identified anymore. It is because that the network gets into partial synchronization, which results in identification failure. The right panel of Fig. 5 shows the time evolutions of the first component variables of all the nodes starting from one set of initial values, which arrive at synchronization after a short transient period. Therefore, it is important to ensure that nodes in a dynamical network are not in any kind of synchronization status, otherwise structure identification is likely to fail.
Numerical validation on a 50-node small-world network
In this subsection, the proposed method is tested on a network consisting of 50 nodes. The Watts–Strogatz (W–S) algorithm21 is employed here to generate a small-world network. Specifically, start with a ring of N = 50 nodes, each connecting to its k = 4 nearest neighbours by undirected edges. Then, rewire each edge in such a way that the beginning end of the edge is kept but the other end is disconnected with probability p and then reconnected to another node randomly selected from the network. A W–S small-world network generated with rewiring probability p = 0.5, as shown in Fig. 6, is employed here for illustration. For simplicity, assume c13(t) = c31(t) = |sin t|, c24(t) = c42(t) = ln(1 + t). For all the other couplings, if there is a link from node j to node i (j ≠ i) at time point t, then cij(t) = 1, otherwise cij(t) = 0. The diagonal entries of matrix C(t) satisfy The inner coupling matrix is Γ = I3×3.
In the following simulations, it is always assumed that node dynamics can be exactly observed. The time step used in the Runge-Kutta method for generating data is still h = 10−6 and m − 1(m = 101) different trajectories are generated from (1) starting from m − 1 sets of different initial values.
For brevity, Fig. 7 only shows the identification results for continuously-varying couplings c13(t) and c24(t). It is obvious that the couplings are correctly identified at different time points, which again illustrates the effectiveness of the proposed identification method from limited observations.
Numerical validation on a 50000-node small-world network
In this example, the one-dimensional multi-agent system proposed by Saber et al.22 is taken as node dynamics. Denote xi as the state of agent i and suppose that the dynamics of agent i is described by
where N is the number of agents. The collective dynamics of the group of interacting agents can be rewritten as
where .
An undirected small-world network consisting of 50000 agents is used to illustrate the effectiveness of the proposed method. The network starts with a ring of N = 50000 nodes, each connects to its k = 8 nearest neighbours by undirected edges that are rewired with probability p = 0.4. Specifically, assume , . For all the other couplings, if there is a link between node i and node j (j ≠ i) at time point t, then cij(t) = cji(t) = 1, otherwise cij(t) = 0. The diagonal entries of matrix C(t) satisfy for
The time step used in the Runge-Kutta method for generating data is h = 10−6 and m − 1(m = 10000) different trajectories are generated from (1) starting from m − 1 sets of different initial values.
For brevity, Fig. 8 only shows the identification results for continuously-varying couplings c29(t) and c68(t). It is obvious that the couplings are correctly identified at different time points, which again illustrates the effectiveness of the proposed identification method from limited observations.
Discussion
Identifying network structures is not only a very important research topic itself but also has applications in a variety of domains. Various methods had been presented to solve the problem of inferring structures of static networks from observations and perhaps through synchronization or control of dynamics. Within the growing body of work concerning time-varying networks, few studies have focused on the issue of identifying structures of continuously-varying weighted networks. In this paper, we have proposed a new method based on optimization to identify structures of complex dynamical networks with continuously-varying weights from limited observations. Two problems, which can be used to solve optimal problems and increase the numerical stability, have been established by rigorous mathematical analysis. In comparison with other existing methods, the main advantages of this approach are that it can handle the challenging identification problem of structures of continuously-varying weighted networks other than static ones. In addition, it only requires a relatively small number data, so it can dramatically reduce the difficulty in obtaining large numbers of observations. Furthermore, by taking into consideration observation noise, the robustness of the method has been demonstrated. In the proposed algorithm, a regularization method has been incorporated, which can avoid large deviations caused by small singular values thereby further increase the numerical stability. Therefore, the method provides a sensible way to identify structures of continuously-varying weighted networks from observations, which has a clear potential applicability to a wide variety of complex dynamical networks.
Methods
Theoretical analysis and algorithm design (more details can be found in the SI)
Here, we design an algorithm to solve optimization problem (8). Topology identification for a complex network with limited observations can be casted into reconstructing a vector from an underdetermined system of linear equations which has more unknowns than equations. The problem can be described by , where A is an m × n matrix, and m is the number of measurements, with m < n. Thus, the underdetermined system Ax = y may have infinitely many solutions. In order to find a solution to such a system, preferably optimal in some sense, one must impose extra constraints as appropriate. In this paper, it is assumed that matrix A has a full row-rank and x is sparse. In this case, one can minimize the number of nonzero components of x to obtain the sparsest solution to Ax = y, that is, to solve the following optimization problem23:
where . The minimization problem (12) can be transformed to the following problem24,25:
where is the l1 norm of the sparse vector x. However, when the matrix A is ill-conditioned and y cannot be accurately observed, numerical instability will arise (more details can be found in the SI). In order to increase the numerical stability, a classical regularization is introduced here. The regularized problem is
where α = α(δ) > 0 is the regularization parameter used to avoid large deviation from the optimal solution. Here, is an erroneous observation, δ is the noise level and , η represents the observation error.
Therefore, the following problem is formulated for the case with accurate observation data.
Problem 0.1 By introducing multipliers, we can reformulate the augmented Lagrangian of problem (13) as
where and ρ > 0 are the Lagrange multipliers and penalty parameter, respectively; z is an auxiliary vector that has the same dimension as x and satisfies the constraint x − z = 0; g(x) is the indicator function of to ensure x satisfying the constraint Ax = y.
For noisy observations, one has the following reformulation.
Problem 0.2 When noise exists in observations, the optimal solution to problem (13) can be approximated by the solution to the optimization problem (14). By introducing multipliers, one has
where and ρ > 0 are the Lagrange multipliers and penalty parameter, respectively; z is an auxiliary vector that has the same dimension as x and satisfies the constraint x − z = 0; .
Remark 0.1 Solving Problems 0.1 and 0.2, one can obtain solutions to the convex optimization problems (13) and (14) using the well-known ADMM algorithm26.
Additional Information
How to cite this article: Mei, G. et al. Identifying structures of continuously-varying weighted networks. Sci. Rep. 6, 26649; doi: 10.1038/srep26649 (2016).
References
Ribeiro, B., Perra, N. & Baronchelli, A. Quantifying the effect of temporal resolution on time-varying networks. Scientific Reports 3, 3006, 10.1038/srep03006 (2013).
Starnini, M., Machens, A., Cattuto, C., Barrat, A. & Pastor-Satorras, R. Immunization strategies for epidemic processes in time-varying contact networks. Journal of Theoretical Biology 337, 89–100 (2013).
Perra, N., Gonçalves, B., Pastor-Satorras, R. & Vespignani, A. Activity driven modeling of time varying networks. Scientific Reports 2, 469, 10.1038/srep00469 (2012).
Amritkar, R. E. & Hu, C. K. Synchronized state of coupled dynamics on time-varying networks. Chaos 16, 015117, 10.1063/1.2168395 (2006).
Nicola, P. et al. Random walks and search in time-varying networks. Physical Review Letters 109, 824–830 (2012).
Holme, P. & Saramäki, J. Temporal networks. Physics Reports 519, 97–125 (2012).
Ahmed, A. & Xing, E. P. Recovering time-varying networks of dependencies in social and biological studies. Proceedings of the National Academy of Sciences 106, 11878–11883 (2009).
Kársai, M., Perra, N. & Vespignani, A. Time varying networks and the weakness of strong ties. Scientific Reports 4, 4001, 10.1038/srep04001 (2014).
Newman, M. E. The structure and function of complex networks. SIAM Review 45, 167–256 (2003).
Zhang, S., Wu, X., Lu, J., Feng, H. & Lu, J. Recovering structures of complex dynamical networks based on generalized outer synchronization. IEEE Transactions on Circuits and Systems I: Regular Papers 61, 3216–3224 (2014).
Wu, X., Zhou, C., Chen, G. & Lu, J. Detecting the topologies of complex networks with stochastic perturbations. Chaos 21, 043129, 10.1063/1.3664396 (2011).
Wu, X., Wang, W. & Zheng, W. Inferring topologies of complex networks with hidden variables. Physical Review E 86, 046106, 10.1103/PhysRevE.86.046106 (2012).
Wang, W., Yang, R., Lai, Y., Kovanis, V. & Grebogi, C. Predicting catastrophes in nonlinear dynamical systems by compressive sensing. Physical Review Letters 106, 154101, 10.1103/PhysRevLett.106.154101 (2011).
Li, G., Wu, X., Liu, J., Lu, J. A. & Guo, C. Recovering network topologies via taylor expansion and compressive sensing. Chaos 25 043102, 10.1063/1.4916788 (2015).
Kolar, M., Song, L., Ahmed, A. & Xing, E. P. Estimating time-varying networks. Annals of Applied Statistics 4, 94–123 (2008).
Rao, A., Rd, H. A., States, D. J. & Engel, J. D. Inferring time-varying network topologies from gene expression data. Eurasip Journal on Bioinformatics and Systems Biology 2007, 12, 10.1155/2007/51947 (2007).
Barrat, A., Barthelemy, M., Pastor-Satorras, R. & Vespignani, A. The architecture of complex weighted networks. Proceedings of the National Academy of Sciences of the United States of America 101, 3747–3752 (2004).
Newman, M. E. Analysis of weighted networks. Physical Review E 70, 056131, 10.1103/PhysRevE.70.056131 (2004).
He, T., Lu, X., Wu, X., Lu, J. & Zheng, W. Optimization-based structure identification of dynamical networks. Physica A 392, 1038–1049 (2013).
Chen, L., Lu, J. & Tse, C. K. Synchronization: An obstacle to identification of network topology. IEEE Transactions on Circuits and Systems II: Express Briefs 56, 310- 314 (2009).
Watts, D. J. & Strogatz, S. H. Collective dynamics of ‘small-world’ networks. Nature 393, 440–442 (1998).
Olfati-Saber, R., Fax, A. & Murray, R. M. Consensus and cooperation in networked multi-agent systems. Proceedings of the IEEE 95, 215–233 (2007).
Donoho, D. L. Compressed sensing. IEEE Transactions on Information Theory 52, 1289–1306 (2006).
Candes, E. J. & Tao, T. Decoding by linear programming. IEEE Transactions on Information Theory 51, 4203–4215 (2006).
Donoho, D. L. For most large underdetermined systems of linear equations the minimal l1-norm solution is also the sparsest solution. Communications on Pure and Applied Mathematics 59, 797–829 (2006).
Boyd, S., Parikh, N., Chu, E., Peleato, B. & Eckstein, J. Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Foundations and Trends in Machine Learning 3, 1–122 (2011).
Acknowledgements
The authors would also like to express their sincere gratitude to Dr. Pierre-André Noël and Dr. Márton Pósfai for their thorough review, helpful suggestions and grammatical corrections. This work is supported by the National Natural Science Foundation of China under Grant Nos 61573262, 61174028 and 11172215 and the Hong Kong Research Grants Council under the GRF Grant CityU-11208515.
Author information
Authors and Affiliations
Contributions
G.M. and X.W. conceived and designed the research. G.M., X.W. and J.L. developed the theoretical analysis. G.M. carried out the numerical simulations. X.W., G.C. and J.L. analyzed the results. G.M., X.W. and G.C. wrote the manuscript. All authors reviewed the manuscript.
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Electronic supplementary material
Rights and permissions
This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/
About this article
Cite this article
Mei, G., Wu, X., Chen, G. et al. Identifying structures of continuously-varying weighted networks. Sci Rep 6, 26649 (2016). https://doi.org/10.1038/srep26649
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/srep26649
- Springer Nature Limited
This article is cited by
-
Critical analysis of (Quasi-)Surprise for community detection in complex networks
Scientific Reports (2018)
-
Reconstruction of Complex Directional Networks with Group Lasso Nonlinear Conditional Granger Causality
Scientific Reports (2017)
-
Transcriptomic basis for drought-resistance in Brassica napus L.
Scientific Reports (2017)