Abstract
In the past decade, resting-state functional magnetic resonance imaging (rs-fMRI) and graph-based measures have been widely used to quantitatively characterize the architectures of brain functional networks in healthy individuals and in patients with abnormalities related to psychopathic and neurological disorders. To accurately evaluate the topological organization of brain functional networks, the definition of the nodes and edges for the construction of functional networks is critical. Furthermore, both types of brain functional networks (binarized networks and weighted networks) are widely used to analyze topological organization. However, how to best select the network type is still debated. Consequently, we investigated the test-retest reliability of brain functional networks with binarized and weighted edges using two independent datasets and four strategies for defining nodes. We revealed fair to good reliability for a majority of network topological attributes and overall higher reliabilities for weighted networks than for binarized networks. For regional nodal efficiency, weighted networks also showed higher reliability across nodes. Thus, our findings imply that weighted networks contain more information, leading to more stable results. In addition, we found that the reliability of brain functional networks was influenced by the node definition strategy and that more precise of nodal definition were associated with higher reliability. The time effect of reliability was restricted, as no differences between long-term and short-term reliability were observed. In conclusion, our results suggest that weighted networks have better reliability because they reflect more topological information, implying broader applications of weighted networks related to normal and disordered function of the human brain.
Similar content being viewed by others
References
Achard, S., & Bullmore, E. (2007). Efficiency and cost of economical brain functional networks. PLoS Computational Biology, 3(2), e17.
Achard, S., Salvador, R., Whitcher, B., Suckling, J., & Bullmore, E. (2006). A resilient, low-frequency, small-world human brain functional network with highly connected association cortical hubs. Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 26(1), 63–72.
Aso, T., Okamura, S., Matsuguchi, T., Sakamoto, N., Sata, T., & Niho, Y. (2011). Rich-Club Organization of the Human Connectome. Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 31(44), 15775–15786.
Bassett, D. S., & Bullmore, E. T. (2016). Small-world brain networks revisited. Neuroscientist A Review Journal Bringing Neurobiology Neurology & Psychiatry, 23(5).
Bassett, D. S., Brown, J. A., Deshpande, V., Carlson, J. M., & Grafton, S. T. (2011a). Conserved and variable architecture of human white matter connectivity. Neuroimage, 54(2), 1262–1279.
Bassett, D. S., Wymbs, N. F., Porter, M. A., Mucha, P. J., Carlson, J. M., & Grafton, S. T. (2011b). Dynamic reconfiguration of human brain networks during learning. Proceedings of the National Academy of Sciences of the United States of America, 108(18), 7641–7646.
Bassett, D. S., Nelson, B. G., Mueller, B. A., Camchong, J., & Lim, K. O. (2012). Altered resting state complexity in schizophrenia. Neuroimage, 59(3), 2196–2207.
Biswal, B. B., Mennes, M., Zuo, X. N., Gohel, S., Kelly, C., Smith, S. M., Beckmann, C. F., Adelstein, J. S., Buckner, R. L., & Colcombe, S. (2009). Toward discovery science of human brain function. Proceedings of the National Academy of Sciences of the United States of America, 71(10), 4734–4739.
Braun, U., Plichta, M. M., Esslinger, C., Sauer, C., Haddad, L., Grimm, O., Mier, D., Mohnke, S., Heinz, A., Erk, S., Walter, H., Seiferth, N., Kirsch, P., & Meyer-Lindenberg, A. (2012). Test–retest reliability of resting-state connectivity network characteristics using fMRI and graph theoretical measures. NeuroImage, 59(2), 1404–1412.
Buckner, R. L., Andrewshanna, J. R., & Schacter, D. L. (2008). The brain's default network: Anatomy, function, and relevance to disease. Annals of the New York Academy of Sciences, 1124(1), 1–38.
Butts, C. T. (2009). Revisiting the foundations of network analysis. Science, 325(5939), 414–416.
Cao, H., Plichta, M. M., Schäfer, A., Haddad, L., Grimm, O., Schneider, M., Esslinger, C., Kirsch, P., Meyer Lindenberg, A., & Tost, H. (2014). Test-retest reliability of fMRI-based graph theoretical properties during working memory, emotion processing, and resting state. Neuroimage, 84(1), 888–900.
Charles, L. (2014). Resolving structure in human brain organization: Identifying mesoscale Organization in Weighted Network Representations. PLoS Computational Biology, 10(10), e1003712.
Cocito, C., Vanlinden, F., & Branlant, C. (2012). Exposure to subliminal arousing stimuli induces robust activation in the amygdala, hippocampus, anterior cingulate, insular cortex and primary visual cortex: A systematic meta-analysis of fMRI studies. Neuroimage, 59(3), 2962–2973.
Cole, M. W., Pathak, S., & Schneider, W. (2010). Identifying the brain's most globally connected regions. Neuroimage, 49(4), 3132–3148.
Doria, V., Beckmann, C. F., Arichi, T., Merchant, N., Groppo, M., Turkheimer, F. E., Counsell, S. J., Murgasova, M., Aljabar, P., & Nunes, R. G. (2010). Emergence of resting state networks in the preterm human brain. Proceedings of the National Academy of Sciences of the United States of America, 107(46), 20015–20020.
Dosenbach, N. U. F., Nardos, B., Cohen, A. L., Fair, D. A., Power, J. D., Church, J. A., Nelson, S. M., Wig, G. S., Vogel, A. C., & Lessovschlaggar, C. N. (2010). Prediction of individual brain maturity using fMRI. Science, 329(5997), 1358–1361.
Fan, L., Li, H., Zhuo, J., Zhang, Y., Wang, J., Chen, L., Yang, Z., Chu, C., Xie, S., & Laird, A. R. (2016). The human Brainnetome atlas: A new brain atlas based on connectional architecture. Cerebral Cortex, 26(8), 3508–3526.
Farine, D. R. (2014). Measuring phenotypic assortment in animal social networks: Weighted associations are more robust than binary edges. Animal Behaviour, 89(3), 141–153.
Finn, E. S., Shen, X., Scheinost, D., Rosenberg, M. D., Huang, J., Chun, M. M., Papademetris, X., & Constable, R. T. (2015). Functional connectome fingerprinting: Identifying individuals based on patterns of brain connectivity. Nature Neuroscience, 18(11), 1664–1671.
Gilman, S. R., Iossifov, I., Levy, D., Ronemus, M., Wigler, M., & Vitkup, D. (2011). Rare de novo variants associated with autism implicate a large functional network of genes involved in formation and function of synapses. Neuron, 70(5), 898–907.
Glasser, M. F., Smith, S. M., Marcus, D. S., Andersson, J. L. R., Auerbach, E. J., Behrens, T. E. J., Coalson, T. S., Harms, M. P., Jenkinson, M., & Moeller, S. (2016). The human connectome Project's neuroimaging approach. Nature Neuroscience, 19(9), 1175–1187.
Gong, G., He, Y., Concha, L., Lebel, C., Gross, D. W., Evans, A. C., & Beaulieu, C. (2009). Mapping anatomical connectivity patterns of human cerebral cortex using in vivo diffusion tensor imaging Tractography. Cerebral Cortex, 19(3), 524–536.
Guo, C. C., Kurth, F., Zhou, J., Mayer, E. A., Eickhoff, S. B., Kramer, J. H., & Seeley, W. W. (2012). One-year test-retest reliability of intrinsic connectivity network fMRI in older adults. Neuroimage, 61(4), 1471–1483.
Hayasaka, S., & Laurienti, P. J. (2010). Comparison of characteristics between region-and voxel-based network analyses in resting-state fMRI data. Neuroimage, 50(2), 499–508.
He, Y., Chen, Z., & Evans, A. (2008). Structural insights into aberrant topological patterns of large-scale cortical networks in Alzheimer's disease. Journal of Neuroscience, 4(4), T284–T285.
He, Y., Dagher, A., Chen, Z., Charil, A., Zijdenbos, A., Worsley, K., & Evans, A. (2009). Impaired small-world efficiency in structural cortical networks in multiple sclerosis associated with white matter lesion load. Brain: A Journal of Neurology, 132(Pt 12), 3366–3379.
Heuvel, M. P. V. D., & Sporns, O. (2013). Network hubs in the human brain. Trends in Cognitive Sciences, 17(12), 683–696.
Horn, A., Ostwald, D., Reisert, M., & Blankenburg, F. (2014). The structural-functional connectome and the default mode network of the human brain. Neuroimage, 102142–102151.
Kim, J., Chey, J., Kim, S. E., & Kim, H. (2015). The effect of education on regional brain metabolism and its functional connectivity in an aged population utilizing positron emission tomography. Neuroscience Research, 94(3), 50–61.
Klimm, F., Bassett, D.S., Carlson, J.M., Mucha, P.J., (2014). Resolving Structural Variability in Network Models and the Brain. PLoS Computational Biology, 10,3(2014-3-27) 10(3), e1003491.
Koch, W., Teipel, S., Mueller, S., Buerger, K., Bokde, A. L. W., Hampel, H., Coates, U., Reiser, M., & Meindl, T. (2010). Effects of aging on default mode network activity in resting state fMRI: Does the method of analysis matter? Neuroimage, 51(1), 280–287.
Latora, V., & Marchiori, M. (2001). Efficient behavior of small-world networks. Physical Review Letters, 87(19), 198701.
Mazoyer, B., Zago, L., Mellet, E., Bricogne, S., Etard, O., Houdé, O., Crivello, F., Joliot, M., Petit, L., & Tzourio-Mazoyer, N. (2001). Cortical networks for working memory and executive functions sustain the conscious resting state in man. Brain Research Bulletin, 54(3), 287–298.
Meunier, D., Achard, S., Morcom, A., & Bullmore, E. (2009). Age-related changes in modular organization of human brain functional networks. Neuroimage, 44(3), 715–723.
Motter, A. E., Changsong, Z., & Jürgen, K. (2005). Network synchronization, diffusion, and the paradox of heterogeneity. Physical Review. E, Statistical, Nonlinear, and Soft Matter Physics, 71(2), 016116.
Newman, M. E. (2002). Assortative mixing in networks. Physical Review Letters, 89(20), 208701.
Newman, M. E. (2006). Modularity and community structure in networks. APS March Meeting, pp., 8577–8582.
Plichta, M. M., Schwarz, A. J., Grimm, O., Morgen, K., Mier, D., Haddad, L., Gerdes, A. B., Sauer, C., Tost, H., & Esslinger, C. (2012). Test-retest reliability of evoked BOLD signals from a cognitive-emotive fMRI test battery. Neuroimage, 60(3), 1746–1758.
Power, J. D., Cohen, A. L., Nelson, S. M., Wig, G. S., Barnes, K. A., Church, J. A., Vogel, A. C., Laumann, T. O., Miezin, F. M., & Schlaggar, B. L. (2011). Functional network Organization of the Human Brain. Neuron, 72(4), 665–678.
Raichle, M. E., Macleod, A. M., Snyder, A. Z., Powers, W. J., Gusnard, D. A., & Shulman, G. L. (2001). A default mode of brain function. Proceedings of the National Academy of Sciences of the United States of America, 98(2), 676–682.
Ravasz, E., & Barabási, A. L. (2003). Hierarchical organization in complex networks. Physical Review. E, Statistical, Nonlinear, and Soft Matter Physics, 67(2), 026112.
Rubinov, M., & Sporns, O. (2010). Complex network measures of brain connectivity: Uses and interpretations. Neuroimage, 52(3), 1059–1069.
Rubinov, M., & Sporns, O. (2011). Weight-conserving characterization of complex functional brain networks. Neuroimage, 56(4), 2068–2079.
Sampat, M. P., Whitman, G. J., Stephens, T. W., Broemeling, L. D., Heger, N. A., Bovik, A. C., & Markey, M. K. (2006). The reliability of measuring physical characteristics of spiculated masses on mammography. British Journal of Radiology 79 Spec No, 2(special_issue_2), S134.
Sanabriadiaz, G., Meliegarcía, L., Iturriamedina, Y., Alemángómez, Y., Hernándezgonzález, G., Valdésurrutia, L., Galán, L., & Valdéssosa, P. (2010). Surface area and cortical thickness descriptors reveal different attributes of the structural human brain networks. Neuroimage, 50(4), 1497–1510.
Shrout, P. E., & Fleiss, J. L. (2015). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420.
Shulman, G. L., Fiez, J. A., Corbetta, M., Buckner, R. L., Miezin, F. M., Raichle, M. E., & Petersen, S. E. (1997). Common blood flow changes across visual tasks: II. Decreases in cerebral cortex. Journal of Cognitive Neuroscience, 9(5), 648–663.
Smith, S. M., Miller, K. L., Salimikhorshidi, G., Webster, M., Beckmann, C. F., Nichols, T. E., Ramsey, J. D., & Woolrich, M. W. (2011). Network modelling methods for FMRI. Neuroimage, 54(2), 875–891.
Smith, S. M., Vidaurre, D., Beckmann, C. F., Glasser, M. F., Jenkinson, M., Miller, K. L., Nichols, T. E., Robinson, E., Salimikhorshidi, G., & Woolrich, M. W. (2013). Functional connectomics from resting-state fMRI. Trends in Cognitive Sciences, 17(12), 666–682.
Sporns, O. (2002). Network analysis, complexity, and brain function. Complexity, 8(1), 56–60.
Sporns, O. (2013). Making sense of brain network data. Nature Methods, 10(6), 491–493.
Sporns, O., Tononi, G., & Kötter, R. (2005). The human connectome: A structural description of the human brain. PLoS Computational Biology, 1(4), e42.
Tian, L., Wang, J., Yan, C., & He, Y. (2011). Hemisphere- and gender-related differences in small-world brain networks: A resting-state functional MRI study. Neuroimage, 54(1), 191–202.
Tijms, B. M., Wink, A. M., De, H. W., Wm, V. D. F., Stam, C. J., Scheltens, P., & Barkhof, F. (2013). Alzheimer's disease: Connecting findings from graph theoretical studies of brain networks. Neurobiology of Aging, 34(8), 2023–2036.
Tzouriomazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N., Mazoyer, B., & Joliot, M. (2002). Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage, 15(1), 273–289.
Van Den Heuvel, M., Stam, C. J., Kahn, R. S., & Hulshoff Pol, H. E. (2009). Efficiency of functional brain networks and intellectual performance. Journal of Neuroscience, 29(23), 7619–7624.
Van Wijk, B. C., Stam, C. J., & Daffertshofer, A. (2010). Comparing brain networks of different size and connectivity density using graph theory. PLoS One, 5(10), e13701.
Wandell, B. A., Dumoulin, S. O., & Brewer, A. A. (2007). Visual field maps in human cortex. Neuron, 56(2), 366–383.
Wang, J., Zuo, X., & He, Y. (2010a). Graph-based network analysis of resting-state functional MRI. Frontiers in Systems Neuroscience, 4(16), 16.
Wang, L., Li, Y., Metzak, P., He, Y., & Woodward, T. S. (2010b). Age-related changes in topological patterns of large-scale brain functional networks during memory encoding and recognition. Neuroimage, 50(3), 862–872.
Wang, L., Zhu, C., He, Y., Zang, Y., Cao, Q., Zhang, H., Zhong, Q., & Wang, Y. (2010c). Altered small-world brain functional networks in children with attention-deficit/hyperactivity disorder. Human Brain Mapping, 30(2), 638–649.
Wang, J. H., Zuo, X. N., Gohel, S., Milham, M. P., Biswal, B. B., & He, Y. (2011). Graph theoretical analysis of functional brain networks: Test-retest evaluation on short- and long-term resting-state functional MRI data. PLoS One, 6(7), e21976.
Watts, D.J., Strogatz, S.H., (1998). Collective dynamics of 'small-world' networks, Collective dynamics of ‘small-world’ networks.
Weber, M. J., Detre, J. A., Thompsonschill, S. L., & Avants, B. B. (2013). Reproducibility of functional network metrics and network structure: A comparison of task-related BOLD, resting ASL with BOLD contrast, and resting cerebral blood flow. Cognitive, Affective, & Behavioral Neuroscience, 13(3), 627–640.
Wig, G. S., Schlaggar, B. L., & Petersen, S. E. (2011). Concepts and principles in the analysis of brain networks. Annals of the New York Academy of Sciences, 1224(1), 126–146.
Wijk, B. C. M. V., Stam, C. J., & Daffertshofer, A. (2010). Comparing brain networks of different size and connectivity density using graph theory. PLoS One, 5(10), e13701.
Winer, B. J. (1962). Statistical principles in experimental design. International Student Edition, 29, 7304–7309.
Zalesky, A., Fornito, A., Harding, I. H., Cocchi, L., Yücel, M., Pantelis, C., & Bullmore, E. T. (2010). Whole-brain anatomical networks: Does the choice of nodes matter? Neuroimage, 50(3), 970–983.
Zhao, X., Liu, Y., Wang, X., Liu, B., Xi, Q., Guo, Q., Jiang, H., Jiang, T., & Wang, P. (2012). Disrupted small-world brain networks in moderate Alzheimer's disease: A resting-state FMRI study. PLoS One, 7(3), e33540.
Zhao, K., Yan, W. J., Chen, Y. H., Zuo, X. N., & Fu, X. (2013). Amygdala volume predicts inter-individual differences in fearful face recognition. PLoS One, 8(8), e74096.
Zhou, J., Gennatas, E. D., Kramer, J. H., Miller, B. L., & Seeley, W. W. (2012). Predicting regional neurodegeneration from the healthy brain functional connectome. Neuron, 73(6), 1216–1227.
Zuo, X. N., & Xing, X. X. (2014). Test-retest reliabilities of resting-state FMRI measurements in human brain functional connectomics: A systems neuroscience perspective. Neuroscience and Biobehavioral Reviews, 45, 100–118.
Zuo, X. N., Ehmke, R., Mennes, M., Imperati, D., Castellanos, F. X., Sporns, O., & Milham, M. P. (2012). Network centrality in the human functional connectome. Cerebral Cortex, 22(8), 1862–1875.
Acknowledgements
This project is supported by the National Natural Science Foundation of China (61503272, 61873178 and 61876124), the Natural Science Foundation of Shanxi (201801D121135), the International Science and Technology Cooperation Project of Shanxi (201803D421047), and the Youth Science and Technology Research Fund (201701D221119). Also, we would like to thank PhD Lynne Hyman for the professional language editing services.
Funding
This project is supported by the National Natural Science Foundation of China (61503272, 61873178 and 61876124), the Natural Science Foundation of Shanxi (201801D121135), the International Science and Technology Cooperation Project of Shanxi (201803D421047), and the Youth Science and Technology Research Fund (201701D221119).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interest
Jie Xiang, Jiayue Xue, Hao Guo, Dandan Li, Xiaohong Cui, Yan Niu, Ting Yan, Rui Cao, Yao Ma, Yanli Yang and Bin Wang declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethics approval
The dataset of IPCAS_1 and NYU CSC are used in the study. The IPCAS_1 dataset was approved by the Institute of Psychology, Chinese Academy of Sciences, the NYU CSC dataset was approved by the New York University, Child Study Center.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
ESM 1
(PDF 1583 kb)
Rights and permissions
About this article
Cite this article
Xiang, J., Xue, J., Guo, H. et al. Graph-based network analysis of resting-state fMRI: test-retest reliability of binarized and weighted networks. Brain Imaging and Behavior 14, 1361–1372 (2020). https://doi.org/10.1007/s11682-019-00042-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11682-019-00042-6