, Volume 11, Issue 4, pp 469–476 | Cite as

Eyes-Open/Eyes-Closed Dataset Sharing for Reproducibility Evaluation of Resting State fMRI Data Analysis Methods

  • Dongqiang LiuEmail author
  • Zhangye Dong
  • Xinian Zuo
  • Jue Wang
  • Yufeng Zang
Data Original Article


The multi-scan resting state fMRI (rs-fMRI) dataset was recently released; thus the test-retest (TRT) reliability of rs-fMRI measures can be assessed. However, because this dataset was acquired only from a single group under a single condition, we cannot directly evaluate whether the rs-fMRI measures can generate reproducible between-condition or between-group results. Because the modulation of resting state activity has gained increasing attention, it is important to know whether one rs-fMRI metric can reliably detect the alteration of the resting activity. Here, we shared a public Eyes-Open (EO)/Eyes-Closed (EC) dataset for evaluating the split-half reproducibility of the rs-fMRI measures in detecting changes of the resting state activity between EO and EC. As examples, we assessed the split-half reproducibility of three widely applied rs-fMRI metrics: amplitude of low frequency fluctuation, regional homogeneity, and seed-based correlation analysis. Our results demonstrated that reproducible patterns of EO-EC differences can be detected by all three measures, suggesting the feasibility of the EO/EC dataset for performing reproducibility assessment for other rs-fMRI measures.


Resting state fMRI measures Reproducibility Data sharing Eyes open Eyes closed 



This work was supported by the National Natural Science Foundation of China (Grant No. 30770594) and the National High Technology Program of China (863) (Grant No. 2008AA02Z405).

Conflict of Interest

The authors declare that they have no conflict of interest.

Supplementary material

12021_2013_9187_MOESM1_ESM.pdf (9 kb)
Supplemental Table 1 (PDF 9 kb)


  1. Albert, N. B., Robertson, E. M., & Miall, R. C. (2009). The resting human brain and motor learning. Current Biology, 19(12), 1023–1027.PubMedCrossRefGoogle Scholar
  2. Barnes, A., Bullmore, E. T., & Suckling, J. (2009). Endogenous human brain dynamics recover slowly following cognitive effort. PLoS One, 4(8), e6626.PubMedCrossRefGoogle Scholar
  3. Beckmann, C. F., DeLuca, M., Devlin, J. T., & Smith, S. M. (2005). Investigations into resting-state connectivity using independent component analysis. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 360(1457), 1001–1013.PubMedCrossRefGoogle Scholar
  4. Biswal, B., Yetkin, F. Z., Haughton, V. M., & Hyde, J. S. (1995). Functional connectivity in the motor cortex of resting human brain using echo-planar MRI. Magnetic Resonance in Medicine, 34(4), 537–541.PubMedCrossRefGoogle Scholar
  5. Castellanos, F. X., Margulies, D. S., Kelly, C., Uddin, L. Q., Ghaffari, M., et al. (2008). Cingulate-precuneus interactions: a new locus of dysfunction in adult attention-deficit/hyperactivity disorder. Biological Psychiatry, 63(3), 332–337.PubMedCrossRefGoogle Scholar
  6. Cole, D. M., Smith, S. M., & Beckmann, C. F. (2010). Advances and pitfalls in the analysis and interpretation of resting-state fMRI data. Frontiers in Systems Neuroscience, 4, 8.Google Scholar
  7. Cordes, D., Haughton, V. M., Arfanakis, K., Wendt, G. J., Turski, P. A., Moritz, C. H., Quigley, M. A., & Meyerand, M. E. (2000). Mapping functionally related regions of brain with functional connectivity MR imaging. American Journal of Neuroradiology, 21(9), 1636–1644.PubMedGoogle Scholar
  8. Cox, R. W. (1996). AFNI: Software for analysis and visualization of functional magnetic resonance neuroimages. Computers and Biomedical Research, 29(3), 162–173.PubMedCrossRefGoogle Scholar
  9. Di Martino, A., Scheres, A., Margulies, D. S., Kelly, A. M., Uddin, L. Q., Shehzad, Z., Biswal, B., Walters, J. R., Castellanos, F. X., & Milham, M. P. (2008). Functional connectivity of human striatum: a resting state FMRI study. Cerebral Cortex, 18(12), 2735–2747.PubMedCrossRefGoogle Scholar
  10. Fox, M. D., Snyder, A. Z., Vincent, J. L., Corbetta, M., Van Essen, D. C., & Raichle M. E. (2005). The human brain is intrinsically organized into dynamic, anticorrelated functional networks. Proceedings of the National Academy of Sciences in United States of America, 105(27), 9673–9678.Google Scholar
  11. Fox, M. D., Corbetta, M., Snyder, A. Z., Vincent, J. L., & Raichle, M. E. (2006). Spontaneous neuronal activity distinguishes human dorsal and ventral attention systems. Proceedings of the National Academy of Sciences in United States of America, 103(26), 10046–10051.CrossRefGoogle Scholar
  12. Friston, K. J., Holmes, A. P., Price, C. J., Buchel, C., & Worsley, K. J. (1999). Multisubject fMRI studies and conjunction analyses. NeuroImage, 10(4), 385–396.PubMedCrossRefGoogle Scholar
  13. Greicius, M. D., Krasnow, B., Reiss, A. L., & Menon, V. (2003). Functional connectivity in the resting brain: a network analysis of the default mode hypothesis. Proceedings of the National Academy of Sciences in United States of America, 100(1), 253–258.CrossRefGoogle Scholar
  14. Grigg, O., & Grady, C. L. (2010). Task-related effects on the temporal and spatial dynamics of resting-state functional connectivity in the default network. PLoS One, 5(10), e13311.PubMedCrossRefGoogle Scholar
  15. Hoptman, M. J., Zuo, X. N., Butler, P. D., Javitt, D. C., D’Angelo, D., Mauro, C. J., & Milham, M. P. (2010). Amplitude of low-frequency oscillations in schizophrenia: a resting state fMRI study. Schizophrenia Research, 117(1), 13–20.PubMedCrossRefGoogle Scholar
  16. Kendall, M., & Gibbons, J. D. (1990). Rank correlation methods. Oxford: Oxford University Press.Google Scholar
  17. Lai, M. C., Lombardo, M. V., Chakrabarti, B., Sadek, S. A., Pasco, G., Wheelwright, S. J., Bullmore, E. T., Baron-Cohen, S., & Suckling, J. (2010). A shift to randomness of brain oscillations in people with autism. Biological Psychiatry, 68(12), 1092–1099.PubMedCrossRefGoogle Scholar
  18. Ledberg, A., Akerman, S., & Roland, P. E. (1998). Estimation of the probabilities of 3D clusters in functional brain images. NeuroImage, 8(2), 113–128.PubMedCrossRefGoogle Scholar
  19. Lowe, M. J., Mock, B. J., & Sorenson, J. A. (1998). Functional connectivity in single and multislice echoplanar imaging using resting-state fluctuations. NeuroImage, 7(2), 119–132.PubMedCrossRefGoogle Scholar
  20. Machielsen, W. C., Rombouts, S. A., Barkhof, F., Scheltens, P., & Witter, M. P. (2000). FMRI of visual encoding: reproducibility of activation. Human Brain Mapping, 9(3), 156–164.PubMedCrossRefGoogle Scholar
  21. Margulies, D. S., Bottger, J., Long, X., Lv, Y., Kelly, C., Schafer, A., Goldhahn, D., Abbushi, A., Milham, M. P., Lohmann, G., & Villringer, A. (2010). Resting developments: a review of fMRI post-processing methodologies for spontaneous brain activity. Magma, 23(5–6), 289–307.PubMedCrossRefGoogle Scholar
  22. Nichols, T., Brett, M., Andersson, J., Wager, T., & Poline, J. B. (2005). Valid conjunction inference with the minimum statistic. NeuroImage, 25(3), 653–660.PubMedCrossRefGoogle Scholar
  23. Paakki, J. J., Rahko, J., Long, X., Moilanen, I., Tervonen, O., Nikkinen, J., Starck, T., Remes, J., Hurtig, T., Haapsamo, H., Jussila, K., Kuusikko-Gauffin, S., Mattila, M. L., Zang, Y., & Kiviniemi, V. (2010). Alterations in regional homogeneity of resting-state brain activity in autism spectrum disorders. Brain Research, 1321, 169–179.PubMedCrossRefGoogle Scholar
  24. Press, W. H. T. S., Vetterling, W. T., & Flannery, B. P. (1992). Numerical Recipes in C (2nd ed.). Cambridge: Cambridge University Press.Google Scholar
  25. Raichle, M. E., MacLeod, A. M., Snyder, A. Z., Powers, W. J., Gusnard, D. A., & Shulman, G. L. (2001). A default mode of brain function. Proceedings of the National Academy of Sciences in United States of America, 98(2), 676–682.CrossRefGoogle Scholar
  26. Salvador, R., Martinez, A., Pomarol-Clotet, E., Sarro, S., Suckling, J., & Bullmore, E. (2007). Frequency based mutual information measures between clusters of brain regions in functional magnetic resonance imaging. NeuroImage, 35(1), 83–88.PubMedCrossRefGoogle Scholar
  27. Shehzad, Z., Kelly, A. M., Reiss, P. T., Gee, D. G., Gotimer, K., Uddin, L. Q., Lee, S. H., Margulies, D. S., Roy, A. K., Biswal, B. B., Petkova, E., Castellanos, F. X., & Milham, M. P. (2009). The resting brain: unconstrained yet reliable. Cerebral Cortex, 19(10), 2209–2229.PubMedCrossRefGoogle Scholar
  28. Sheline, Y. I., Raichle, M. E., Snyder, A. Z., Morris, J. C., Head, D., et al. (2010). Amyloid plaques disrupt resting state default mode network connectivity in cognitively normal elderly. Biological Psychiatry, 67(6), 584–587.PubMedCrossRefGoogle Scholar
  29. Song, X. W., Dong, Z. Y., Long, X. Y., Li, S. F., Zuo, X. N., Zhu, C. Z., He, Y., Yan, C. G., & Zang, Y. F. (2011). REST: a toolkit for resting-state functional magnetic resonance imaging data processing. PLoS One, 6(9), e25031.PubMedCrossRefGoogle Scholar
  30. Stiers, P., Mennes, M., & Sunaert, S. (2010). Distributed task coding throughout the multiple demand network of the human frontal-insular cortex. NeuroImage, 52(1), 252–262.PubMedCrossRefGoogle Scholar
  31. Strother, S. C., Anderson, J., Hansen, L. K., Kjems, U., Kustra, R., et al. (2002). The quantitative evaluation of functional neuroimaging experiments: the NPAIRS data analysis framework. NeuroImage, 15(4), 747–771.PubMedCrossRefGoogle Scholar
  32. Tambini, A., Ketz, N., & Davachi, L. (2010). Enhanced brain correlations during rest are related to memory for recent experiences. Neuron, 65(2), 280–290.PubMedCrossRefGoogle Scholar
  33. Van Dijk, K. R., Hedden, T., Venkataraman, A., Evans, K. C., Lazar, S. W., & Buckner, R. L. (2010). Intrinsic functional connectivity as a tool for human connectomics: theory, properties, and optimization. Journal of Neurophysiology, 103(1), 297–321.PubMedCrossRefGoogle Scholar
  34. Waites, A. B., Stanislavsky, A., Abbott, D. F., & Jackson, G. D. (2005). Effect of prior cognitive state on resting state networks measured with functional connectivity. Human Brain Mapping, 24(1), 59–68.PubMedCrossRefGoogle Scholar
  35. Wang, K., Yu, C., Xu, L., Qin, W., Li, K., Xu, L., & Jiang, T. (2009). Offline memory reprocessing: involvement of the brain’s default network in spontaneous thought processes. PLoS One, 4(3), e4867.PubMedCrossRefGoogle Scholar
  36. Wang, L., Yu, C., Chen, H., Qin, W., He, Y., Fan, F., Zhang, Y., Wang, M., Li, K., Zang, Y., Woodward, T. S., & Zhu, C. (2010). Dynamic functional reorganization of the motor execution network after stroke. Brain, 133(Pt 4), 1224–1238.PubMedCrossRefGoogle Scholar
  37. Wu, T., Long, X., Zang, Y., Wang, L., Hallett, M., Li, K., & Chan, P. (2009). Regional homogeneity changes in patients with Parkinson’s disease. Human Brain Mapping, 30(5), 1502–1510.PubMedCrossRefGoogle Scholar
  38. Yan, C., & Zang, Y. (2010). DPARSF: A MATLAB toolbox for “Pipeline” data analysis of resting-state fMRI. Frontiers in Systems Neuroscience, 4, 13.Google Scholar
  39. Yan, C., Liu, D., He, Y., Zou, Q., Zhu, C., et al. (2009). Spontaneous brain activity in the default mode network is sensitive to different resting-state conditions with limited cognitive load. PLoS One, 4(5), e5743.PubMedCrossRefGoogle Scholar
  40. Yang, H., Long, X. Y., Yang, Y., Yan, H., Zhu, C. Z., Zhou, X. P., Zang, Y. F., & Gong, Q. Y. (2007). Amplitude of low frequency fluctuation within visual areas revealed by resting-state functional MRI. NeuroImage, 36(1), 144–152.PubMedCrossRefGoogle Scholar
  41. Zang, Y., Jiang, T., Lu, Y., He, Y., & Tian, L. (2004). Regional homogeneity approach to fMRI data analysis. NeuroImage, 22(1), 394–400.PubMedCrossRefGoogle Scholar
  42. Zang, Y. F., He, Y., Zhu, C. Z., Cao, Q. J., Sui, M. Q., Liang, M., Tian, L. X., Jiang, T. Z., & Wang, Y. F. (2007). Altered baseline brain activity in children with ADHD revealed by resting-state functional MRI. Brain & Development, 29(2), 83–91.CrossRefGoogle Scholar
  43. Zhang, D., & Raichle, M. E. (2010). Disease and the brain’s dark energy. Nature Reviews. Neurology, 6(1), 15–28.PubMedCrossRefGoogle Scholar
  44. Zuo, X. N., Di Martino, A., Kelly, C., Shehzad, Z. E., Gee, D. G., Klein, D. F., Castellanos, F. X., Biswal, B. B., & Milham, M. P. (2010a). The oscillating brain: complex and reliable. NeuroImage, 49(2), 1432–1445.PubMedCrossRefGoogle Scholar
  45. Zuo, X. N., Kelly, C., Adelstein, J. S., Klein, D. F., Castellanos, F. X., & Milham, M. P. (2010b). Reliable intrinsic connectivity networks: test-retest evaluation using ICA and dual regression approach. NeuroImage, 49(3), 2163–2177.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Dongqiang Liu
    • 1
    Email author
  • Zhangye Dong
    • 2
  • Xinian Zuo
    • 3
  • Jue Wang
    • 1
    • 2
  • Yufeng Zang
    • 1
    • 2
  1. 1.Center for Cognition and Brain DisordersHangzhou Normal UniversityHangzhouChina
  2. 2.National Key Laboratory of Cognitive Neuroscience and LearningBeijing Normal UniversityBeijingChina
  3. 3.Laboratory for Functional Connectome and Development, Key Laboratory of Behavioral Science, Institute of PsychologyChinese Academy of SciencesBeijingChina

Personalised recommendations