Skip to main content
Log in

Graph-based network analysis of resting-state fMRI: test-retest reliability of binarized and weighted networks

  • ORIGINAL RESEARCH
  • Published:
Brain Imaging and Behavior Aims and scope Submit manuscript

Abstract

In the past decade, resting-state functional magnetic resonance imaging (rs-fMRI) and graph-based measures have been widely used to quantitatively characterize the architectures of brain functional networks in healthy individuals and in patients with abnormalities related to psychopathic and neurological disorders. To accurately evaluate the topological organization of brain functional networks, the definition of the nodes and edges for the construction of functional networks is critical. Furthermore, both types of brain functional networks (binarized networks and weighted networks) are widely used to analyze topological organization. However, how to best select the network type is still debated. Consequently, we investigated the test-retest reliability of brain functional networks with binarized and weighted edges using two independent datasets and four strategies for defining nodes. We revealed fair to good reliability for a majority of network topological attributes and overall higher reliabilities for weighted networks than for binarized networks. For regional nodal efficiency, weighted networks also showed higher reliability across nodes. Thus, our findings imply that weighted networks contain more information, leading to more stable results. In addition, we found that the reliability of brain functional networks was influenced by the node definition strategy and that more precise of nodal definition were associated with higher reliability. The time effect of reliability was restricted, as no differences between long-term and short-term reliability were observed. In conclusion, our results suggest that weighted networks have better reliability because they reflect more topological information, implying broader applications of weighted networks related to normal and disordered function of the human brain.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Achard, S., & Bullmore, E. (2007). Efficiency and cost of economical brain functional networks. PLoS Computational Biology, 3(2), e17.

    PubMed  PubMed Central  Google Scholar 

  • Achard, S., Salvador, R., Whitcher, B., Suckling, J., & Bullmore, E. (2006). A resilient, low-frequency, small-world human brain functional network with highly connected association cortical hubs. Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 26(1), 63–72.

    CAS  Google Scholar 

  • Aso, T., Okamura, S., Matsuguchi, T., Sakamoto, N., Sata, T., & Niho, Y. (2011). Rich-Club Organization of the Human Connectome. Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 31(44), 15775–15786.

    Google Scholar 

  • Bassett, D. S., & Bullmore, E. T. (2016). Small-world brain networks revisited. Neuroscientist A Review Journal Bringing Neurobiology Neurology & Psychiatry, 23(5).

  • Bassett, D. S., Brown, J. A., Deshpande, V., Carlson, J. M., & Grafton, S. T. (2011a). Conserved and variable architecture of human white matter connectivity. Neuroimage, 54(2), 1262–1279.

    PubMed  Google Scholar 

  • Bassett, D. S., Wymbs, N. F., Porter, M. A., Mucha, P. J., Carlson, J. M., & Grafton, S. T. (2011b). Dynamic reconfiguration of human brain networks during learning. Proceedings of the National Academy of Sciences of the United States of America, 108(18), 7641–7646.

    CAS  PubMed  PubMed Central  Google Scholar 

  • Bassett, D. S., Nelson, B. G., Mueller, B. A., Camchong, J., & Lim, K. O. (2012). Altered resting state complexity in schizophrenia. Neuroimage, 59(3), 2196–2207.

    PubMed  Google Scholar 

  • Biswal, B. B., Mennes, M., Zuo, X. N., Gohel, S., Kelly, C., Smith, S. M., Beckmann, C. F., Adelstein, J. S., Buckner, R. L., & Colcombe, S. (2009). Toward discovery science of human brain function. Proceedings of the National Academy of Sciences of the United States of America, 71(10), 4734–4739.

    Google Scholar 

  • Braun, U., Plichta, M. M., Esslinger, C., Sauer, C., Haddad, L., Grimm, O., Mier, D., Mohnke, S., Heinz, A., Erk, S., Walter, H., Seiferth, N., Kirsch, P., & Meyer-Lindenberg, A. (2012). Test–retest reliability of resting-state connectivity network characteristics using fMRI and graph theoretical measures. NeuroImage, 59(2), 1404–1412.

    PubMed  Google Scholar 

  • Buckner, R. L., Andrewshanna, J. R., & Schacter, D. L. (2008). The brain's default network: Anatomy, function, and relevance to disease. Annals of the New York Academy of Sciences, 1124(1), 1–38.

    PubMed  Google Scholar 

  • Butts, C. T. (2009). Revisiting the foundations of network analysis. Science, 325(5939), 414–416.

    CAS  PubMed  Google Scholar 

  • Cao, H., Plichta, M. M., Schäfer, A., Haddad, L., Grimm, O., Schneider, M., Esslinger, C., Kirsch, P., Meyer Lindenberg, A., & Tost, H. (2014). Test-retest reliability of fMRI-based graph theoretical properties during working memory, emotion processing, and resting state. Neuroimage, 84(1), 888–900.

    PubMed  Google Scholar 

  • Charles, L. (2014). Resolving structure in human brain organization: Identifying mesoscale Organization in Weighted Network Representations. PLoS Computational Biology, 10(10), e1003712.

    Google Scholar 

  • Cocito, C., Vanlinden, F., & Branlant, C. (2012). Exposure to subliminal arousing stimuli induces robust activation in the amygdala, hippocampus, anterior cingulate, insular cortex and primary visual cortex: A systematic meta-analysis of fMRI studies. Neuroimage, 59(3), 2962–2973.

    Google Scholar 

  • Cole, M. W., Pathak, S., & Schneider, W. (2010). Identifying the brain's most globally connected regions. Neuroimage, 49(4), 3132–3148.

    PubMed  Google Scholar 

  • Doria, V., Beckmann, C. F., Arichi, T., Merchant, N., Groppo, M., Turkheimer, F. E., Counsell, S. J., Murgasova, M., Aljabar, P., & Nunes, R. G. (2010). Emergence of resting state networks in the preterm human brain. Proceedings of the National Academy of Sciences of the United States of America, 107(46), 20015–20020.

    CAS  PubMed  PubMed Central  Google Scholar 

  • Dosenbach, N. U. F., Nardos, B., Cohen, A. L., Fair, D. A., Power, J. D., Church, J. A., Nelson, S. M., Wig, G. S., Vogel, A. C., & Lessovschlaggar, C. N. (2010). Prediction of individual brain maturity using fMRI. Science, 329(5997), 1358–1361.

    CAS  PubMed  PubMed Central  Google Scholar 

  • Fan, L., Li, H., Zhuo, J., Zhang, Y., Wang, J., Chen, L., Yang, Z., Chu, C., Xie, S., & Laird, A. R. (2016). The human Brainnetome atlas: A new brain atlas based on connectional architecture. Cerebral Cortex, 26(8), 3508–3526.

    PubMed  Google Scholar 

  • Farine, D. R. (2014). Measuring phenotypic assortment in animal social networks: Weighted associations are more robust than binary edges. Animal Behaviour, 89(3), 141–153.

    Google Scholar 

  • Finn, E. S., Shen, X., Scheinost, D., Rosenberg, M. D., Huang, J., Chun, M. M., Papademetris, X., & Constable, R. T. (2015). Functional connectome fingerprinting: Identifying individuals based on patterns of brain connectivity. Nature Neuroscience, 18(11), 1664–1671.

    CAS  PubMed  PubMed Central  Google Scholar 

  • Gilman, S. R., Iossifov, I., Levy, D., Ronemus, M., Wigler, M., & Vitkup, D. (2011). Rare de novo variants associated with autism implicate a large functional network of genes involved in formation and function of synapses. Neuron, 70(5), 898–907.

    CAS  PubMed  PubMed Central  Google Scholar 

  • Glasser, M. F., Smith, S. M., Marcus, D. S., Andersson, J. L. R., Auerbach, E. J., Behrens, T. E. J., Coalson, T. S., Harms, M. P., Jenkinson, M., & Moeller, S. (2016). The human connectome Project's neuroimaging approach. Nature Neuroscience, 19(9), 1175–1187.

    PubMed  PubMed Central  Google Scholar 

  • Gong, G., He, Y., Concha, L., Lebel, C., Gross, D. W., Evans, A. C., & Beaulieu, C. (2009). Mapping anatomical connectivity patterns of human cerebral cortex using in vivo diffusion tensor imaging Tractography. Cerebral Cortex, 19(3), 524–536.

    PubMed  Google Scholar 

  • Guo, C. C., Kurth, F., Zhou, J., Mayer, E. A., Eickhoff, S. B., Kramer, J. H., & Seeley, W. W. (2012). One-year test-retest reliability of intrinsic connectivity network fMRI in older adults. Neuroimage, 61(4), 1471–1483.

    PubMed  PubMed Central  Google Scholar 

  • Hayasaka, S., & Laurienti, P. J. (2010). Comparison of characteristics between region-and voxel-based network analyses in resting-state fMRI data. Neuroimage, 50(2), 499–508.

    PubMed  Google Scholar 

  • He, Y., Chen, Z., & Evans, A. (2008). Structural insights into aberrant topological patterns of large-scale cortical networks in Alzheimer's disease. Journal of Neuroscience, 4(4), T284–T285.

    Google Scholar 

  • He, Y., Dagher, A., Chen, Z., Charil, A., Zijdenbos, A., Worsley, K., & Evans, A. (2009). Impaired small-world efficiency in structural cortical networks in multiple sclerosis associated with white matter lesion load. Brain: A Journal of Neurology, 132(Pt 12), 3366–3379.

    Google Scholar 

  • Heuvel, M. P. V. D., & Sporns, O. (2013). Network hubs in the human brain. Trends in Cognitive Sciences, 17(12), 683–696.

    PubMed  Google Scholar 

  • Horn, A., Ostwald, D., Reisert, M., & Blankenburg, F. (2014). The structural-functional connectome and the default mode network of the human brain. Neuroimage, 102142–102151.

  • Kim, J., Chey, J., Kim, S. E., & Kim, H. (2015). The effect of education on regional brain metabolism and its functional connectivity in an aged population utilizing positron emission tomography. Neuroscience Research, 94(3), 50–61.

    PubMed  Google Scholar 

  • Klimm, F., Bassett, D.S., Carlson, J.M., Mucha, P.J., (2014). Resolving Structural Variability in Network Models and the Brain. PLoS Computational Biology, 10,3(2014-3-27) 10(3), e1003491.

  • Koch, W., Teipel, S., Mueller, S., Buerger, K., Bokde, A. L. W., Hampel, H., Coates, U., Reiser, M., & Meindl, T. (2010). Effects of aging on default mode network activity in resting state fMRI: Does the method of analysis matter? Neuroimage, 51(1), 280–287.

    CAS  PubMed  Google Scholar 

  • Latora, V., & Marchiori, M. (2001). Efficient behavior of small-world networks. Physical Review Letters, 87(19), 198701.

    CAS  PubMed  Google Scholar 

  • Mazoyer, B., Zago, L., Mellet, E., Bricogne, S., Etard, O., Houdé, O., Crivello, F., Joliot, M., Petit, L., & Tzourio-Mazoyer, N. (2001). Cortical networks for working memory and executive functions sustain the conscious resting state in man. Brain Research Bulletin, 54(3), 287–298.

    CAS  PubMed  Google Scholar 

  • Meunier, D., Achard, S., Morcom, A., & Bullmore, E. (2009). Age-related changes in modular organization of human brain functional networks. Neuroimage, 44(3), 715–723.

    PubMed  Google Scholar 

  • Motter, A. E., Changsong, Z., & Jürgen, K. (2005). Network synchronization, diffusion, and the paradox of heterogeneity. Physical Review. E, Statistical, Nonlinear, and Soft Matter Physics, 71(2), 016116.

    PubMed  Google Scholar 

  • Newman, M. E. (2002). Assortative mixing in networks. Physical Review Letters, 89(20), 208701.

    CAS  PubMed  Google Scholar 

  • Newman, M. E. (2006). Modularity and community structure in networks. APS March Meeting, pp., 8577–8582.

  • Plichta, M. M., Schwarz, A. J., Grimm, O., Morgen, K., Mier, D., Haddad, L., Gerdes, A. B., Sauer, C., Tost, H., & Esslinger, C. (2012). Test-retest reliability of evoked BOLD signals from a cognitive-emotive fMRI test battery. Neuroimage, 60(3), 1746–1758.

    PubMed  Google Scholar 

  • Power, J. D., Cohen, A. L., Nelson, S. M., Wig, G. S., Barnes, K. A., Church, J. A., Vogel, A. C., Laumann, T. O., Miezin, F. M., & Schlaggar, B. L. (2011). Functional network Organization of the Human Brain. Neuron, 72(4), 665–678.

    CAS  PubMed  PubMed Central  Google Scholar 

  • Raichle, M. E., Macleod, A. M., Snyder, A. Z., Powers, W. J., Gusnard, D. A., & Shulman, G. L. (2001). A default mode of brain function. Proceedings of the National Academy of Sciences of the United States of America, 98(2), 676–682.

    CAS  PubMed  PubMed Central  Google Scholar 

  • Ravasz, E., & Barabási, A. L. (2003). Hierarchical organization in complex networks. Physical Review. E, Statistical, Nonlinear, and Soft Matter Physics, 67(2), 026112.

    PubMed  Google Scholar 

  • Rubinov, M., & Sporns, O. (2010). Complex network measures of brain connectivity: Uses and interpretations. Neuroimage, 52(3), 1059–1069.

    PubMed  Google Scholar 

  • Rubinov, M., & Sporns, O. (2011). Weight-conserving characterization of complex functional brain networks. Neuroimage, 56(4), 2068–2079.

    PubMed  Google Scholar 

  • Sampat, M. P., Whitman, G. J., Stephens, T. W., Broemeling, L. D., Heger, N. A., Bovik, A. C., & Markey, M. K. (2006). The reliability of measuring physical characteristics of spiculated masses on mammography. British Journal of Radiology 79 Spec No, 2(special_issue_2), S134.

    Google Scholar 

  • Sanabriadiaz, G., Meliegarcía, L., Iturriamedina, Y., Alemángómez, Y., Hernándezgonzález, G., Valdésurrutia, L., Galán, L., & Valdéssosa, P. (2010). Surface area and cortical thickness descriptors reveal different attributes of the structural human brain networks. Neuroimage, 50(4), 1497–1510.

    Google Scholar 

  • Shrout, P. E., & Fleiss, J. L. (2015). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420.

    Google Scholar 

  • Shulman, G. L., Fiez, J. A., Corbetta, M., Buckner, R. L., Miezin, F. M., Raichle, M. E., & Petersen, S. E. (1997). Common blood flow changes across visual tasks: II. Decreases in cerebral cortex. Journal of Cognitive Neuroscience, 9(5), 648–663.

    CAS  PubMed  Google Scholar 

  • Smith, S. M., Miller, K. L., Salimikhorshidi, G., Webster, M., Beckmann, C. F., Nichols, T. E., Ramsey, J. D., & Woolrich, M. W. (2011). Network modelling methods for FMRI. Neuroimage, 54(2), 875–891.

    PubMed  Google Scholar 

  • Smith, S. M., Vidaurre, D., Beckmann, C. F., Glasser, M. F., Jenkinson, M., Miller, K. L., Nichols, T. E., Robinson, E., Salimikhorshidi, G., & Woolrich, M. W. (2013). Functional connectomics from resting-state fMRI. Trends in Cognitive Sciences, 17(12), 666–682.

    PubMed  PubMed Central  Google Scholar 

  • Sporns, O. (2002). Network analysis, complexity, and brain function. Complexity, 8(1), 56–60.

    Google Scholar 

  • Sporns, O. (2013). Making sense of brain network data. Nature Methods, 10(6), 491–493.

    CAS  PubMed  Google Scholar 

  • Sporns, O., Tononi, G., & Kötter, R. (2005). The human connectome: A structural description of the human brain. PLoS Computational Biology, 1(4), e42.

    PubMed  PubMed Central  Google Scholar 

  • Tian, L., Wang, J., Yan, C., & He, Y. (2011). Hemisphere- and gender-related differences in small-world brain networks: A resting-state functional MRI study. Neuroimage, 54(1), 191–202.

    PubMed  Google Scholar 

  • Tijms, B. M., Wink, A. M., De, H. W., Wm, V. D. F., Stam, C. J., Scheltens, P., & Barkhof, F. (2013). Alzheimer's disease: Connecting findings from graph theoretical studies of brain networks. Neurobiology of Aging, 34(8), 2023–2036.

    PubMed  Google Scholar 

  • Tzouriomazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N., Mazoyer, B., & Joliot, M. (2002). Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage, 15(1), 273–289.

    CAS  Google Scholar 

  • Van Den Heuvel, M., Stam, C. J., Kahn, R. S., & Hulshoff Pol, H. E. (2009). Efficiency of functional brain networks and intellectual performance. Journal of Neuroscience, 29(23), 7619–7624.

    PubMed  Google Scholar 

  • Van Wijk, B. C., Stam, C. J., & Daffertshofer, A. (2010). Comparing brain networks of different size and connectivity density using graph theory. PLoS One, 5(10), e13701.

    PubMed  PubMed Central  Google Scholar 

  • Wandell, B. A., Dumoulin, S. O., & Brewer, A. A. (2007). Visual field maps in human cortex. Neuron, 56(2), 366–383.

    CAS  PubMed  Google Scholar 

  • Wang, J., Zuo, X., & He, Y. (2010a). Graph-based network analysis of resting-state functional MRI. Frontiers in Systems Neuroscience, 4(16), 16.

    PubMed  PubMed Central  Google Scholar 

  • Wang, L., Li, Y., Metzak, P., He, Y., & Woodward, T. S. (2010b). Age-related changes in topological patterns of large-scale brain functional networks during memory encoding and recognition. Neuroimage, 50(3), 862–872.

    PubMed  Google Scholar 

  • Wang, L., Zhu, C., He, Y., Zang, Y., Cao, Q., Zhang, H., Zhong, Q., & Wang, Y. (2010c). Altered small-world brain functional networks in children with attention-deficit/hyperactivity disorder. Human Brain Mapping, 30(2), 638–649.

    Google Scholar 

  • Wang, J. H., Zuo, X. N., Gohel, S., Milham, M. P., Biswal, B. B., & He, Y. (2011). Graph theoretical analysis of functional brain networks: Test-retest evaluation on short- and long-term resting-state functional MRI data. PLoS One, 6(7), e21976.

    CAS  PubMed  PubMed Central  Google Scholar 

  • Watts, D.J., Strogatz, S.H., (1998). Collective dynamics of 'small-world' networks, Collective dynamics of ‘small-world’ networks.

  • Weber, M. J., Detre, J. A., Thompsonschill, S. L., & Avants, B. B. (2013). Reproducibility of functional network metrics and network structure: A comparison of task-related BOLD, resting ASL with BOLD contrast, and resting cerebral blood flow. Cognitive, Affective, & Behavioral Neuroscience, 13(3), 627–640.

    Google Scholar 

  • Wig, G. S., Schlaggar, B. L., & Petersen, S. E. (2011). Concepts and principles in the analysis of brain networks. Annals of the New York Academy of Sciences, 1224(1), 126–146.

    PubMed  Google Scholar 

  • Wijk, B. C. M. V., Stam, C. J., & Daffertshofer, A. (2010). Comparing brain networks of different size and connectivity density using graph theory. PLoS One, 5(10), e13701.

    PubMed  PubMed Central  Google Scholar 

  • Winer, B. J. (1962). Statistical principles in experimental design. International Student Edition, 29, 7304–7309.

    Google Scholar 

  • Zalesky, A., Fornito, A., Harding, I. H., Cocchi, L., Yücel, M., Pantelis, C., & Bullmore, E. T. (2010). Whole-brain anatomical networks: Does the choice of nodes matter? Neuroimage, 50(3), 970–983.

    PubMed  Google Scholar 

  • Zhao, X., Liu, Y., Wang, X., Liu, B., Xi, Q., Guo, Q., Jiang, H., Jiang, T., & Wang, P. (2012). Disrupted small-world brain networks in moderate Alzheimer's disease: A resting-state FMRI study. PLoS One, 7(3), e33540.

    CAS  PubMed  PubMed Central  Google Scholar 

  • Zhao, K., Yan, W. J., Chen, Y. H., Zuo, X. N., & Fu, X. (2013). Amygdala volume predicts inter-individual differences in fearful face recognition. PLoS One, 8(8), e74096.

    CAS  PubMed  PubMed Central  Google Scholar 

  • Zhou, J., Gennatas, E. D., Kramer, J. H., Miller, B. L., & Seeley, W. W. (2012). Predicting regional neurodegeneration from the healthy brain functional connectome. Neuron, 73(6), 1216–1227.

    CAS  PubMed  PubMed Central  Google Scholar 

  • Zuo, X. N., & Xing, X. X. (2014). Test-retest reliabilities of resting-state FMRI measurements in human brain functional connectomics: A systems neuroscience perspective. Neuroscience and Biobehavioral Reviews, 45, 100–118.

    PubMed  Google Scholar 

  • Zuo, X. N., Ehmke, R., Mennes, M., Imperati, D., Castellanos, F. X., Sporns, O., & Milham, M. P. (2012). Network centrality in the human functional connectome. Cerebral Cortex, 22(8), 1862–1875.

    PubMed  Google Scholar 

Download references

Acknowledgements

This project is supported by the National Natural Science Foundation of China (61503272, 61873178 and 61876124), the Natural Science Foundation of Shanxi (201801D121135), the International Science and Technology Cooperation Project of Shanxi (201803D421047), and the Youth Science and Technology Research Fund (201701D221119). Also, we would like to thank PhD Lynne Hyman for the professional language editing services.

Funding

This project is supported by the National Natural Science Foundation of China (61503272, 61873178 and 61876124), the Natural Science Foundation of Shanxi (201801D121135), the International Science and Technology Cooperation Project of Shanxi (201803D421047), and the Youth Science and Technology Research Fund (201701D221119).

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Dandan Li or Bin Wang.

Ethics declarations

Conflict of interest

Jie Xiang, Jiayue Xue, Hao Guo, Dandan Li, Xiaohong Cui, Yan Niu, Ting Yan, Rui Cao, Yao Ma, Yanli Yang and Bin Wang declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethics approval

The dataset of IPCAS_1 and NYU CSC are used in the study. The IPCAS_1 dataset was approved by the Institute of Psychology, Chinese Academy of Sciences, the NYU CSC dataset was approved by the New York University, Child Study Center.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(PDF 1583 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiang, J., Xue, J., Guo, H. et al. Graph-based network analysis of resting-state fMRI: test-retest reliability of binarized and weighted networks. Brain Imaging and Behavior 14, 1361–1372 (2020). https://doi.org/10.1007/s11682-019-00042-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11682-019-00042-6

Keywords

Navigation