Skip to main content

Advertisement

SpringerLink
Log in

Harnessing reliability for neuroscience research

  • Comment
  • Published:

From Nature Human Behaviour

View current issue Submit your manuscript

Neuroscientists are amassing the large-scale datasets needed to study individual differences and identify biomarkers. However, measurement reliability within individual samples is often suboptimal, thereby requiring unnecessarily large samples. We focus our comment on reliability in neuroimaging and provide examples of how the reliability can be increased.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1: Interactions between reliability, sample and effect size for a given statistical power.
Fig. 2: Reliability and validity of measurements of individual differences.

Code availability

All code employed in this effort can be found on GitHub at https://github.com/TingsterX/power__reliability_sample_size

References

  1. Laumann, T. O. et al. Neuron 87, 657–670 (2015).

    Article  CAS  Google Scholar 

  2. O’Connor, D. et al. Gigascience 6, 1–14 (2017).

    Article  Google Scholar 

  3. Xu, T., Opitz, A., Craddock, C., Zuo, X. N. & Milham, M. P. Cereb. Cortex 26, 4192–4211 (2016).

    Article  Google Scholar 

  4. Zuo, X. N. & Xing, X. X. Neurosci. Biobehav. Rev. 45, 100–118 (2014).

    Article  Google Scholar 

  5. Kanyongo, G. Y., Brook, G. P., Kyei-Blankson, L. & Gocmen, G. J. Mod. Appl. Stat. Methods 6, Article 9 (2007).

    Article  Google Scholar 

  6. Poldrack, R. A. et al. Nat. Rev. Neurosci. 18, 115–126 (2017).

    Article  CAS  Google Scholar 

  7. Bennett, C. M. & Miller, M. B. Ann. NY Acad. Sci. 1191, 133–155 (2010).

    Article  Google Scholar 

  8. Hedge, C., Powell, G. & Sumner, P. Behav. Res. Methods 50, 1166–1186 (2018).

    Article  Google Scholar 

  9. Button, K. S. et al. Nat. Rev. Neurosci. 14, 365–376 (2013).

    Article  CAS  Google Scholar 

  10. Zuo, X.N., Biswal, B.B. & Poldrack, R.A. Front. Neurosci. 13, 117 (2019).

    Article  Google Scholar 

  11. Tomasi, D. G., Shokri-Kojori, E. & Volkow, N. D. Cereb. Cortex 27, 4153–4165 (2017).

    PubMed  Google Scholar 

  12. Elliott, M. L. et al. Neuroimage 189, 516–532 (2019).

    Article  Google Scholar 

  13. Nichols, T. E. et al. Nat. Neurosci. 20, 299–303 (2017).

    Article  CAS  Google Scholar 

  14. Koo, T. K. & Li, M. Y. J. Chiropr. Med. 15, 155–163 (2016).

    Article  Google Scholar 

  15. Kraemer, H. C. Annu. Rev. Clin. Psychol. 10, 111–130 (2014).

    Article  Google Scholar 

  16. Yan, C. G. et al. Neuroimage 76, 183–201 (2013).

    Article  Google Scholar 

Download references

Acknowledgements

We thank X. Castellanos, A. Franco, H. Kimball, A. Nikolaidis and X.-X. Xing for their helpful comments in the preparation of this commentary, as well as D. Klein for his guidance and encouragement of our focus on issues of reliability over the years, X. Castellanos for his support along the way, and all the contributors from CoRR and R3BRAIN for their enthusiasm on open neuroscience and data sharing. The two consortia are supported in part by the National Basic Research (973) Program (2015CB351702).

Author information

Authors and Affiliations

  1. Department of Psychology, University of Chinese Academy of Sciences (UCAS), Beijing, Beijing, China

    Xi-Nian Zuo

  2. Key Laboratory of Brain and Education, Nanning Normal University, Nanning, Guangxi, China

    Xi-Nian Zuo

  3. Institute for Brain Research and Rehabilitation, South China Normal University, Guangzhou, China

    Xi-Nian Zuo

  4. CAS Key Laboratory of Behavioral Sciences, Research Center for Lifespan Development of Mind and Brain (CLIMB) and Magnetic Resonance Imaging Research Center, CAS Institute of Psychology, Beijing, Beijing, China

    Xi-Nian Zuo & Ting Xu

  5. Center for Developing Brain, Child Mind Institute, New York, NY, USA

    Ting Xu & Michael Peter Milham

  6. Center for Biomedical Imaging and Neuromodulation, Nathan S. Kline Institute for Psychiatric Research, Orangeburg, NY, USA

    Michael Peter Milham

Authors
  1. Xi-Nian Zuo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ting Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Peter Milham
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xi-Nian Zuo.

Ethics declarations

Competing interests

The authors declare no competing interests.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zuo, XN., Xu, T. & Milham, M.P. Harnessing reliability for neuroscience research. Nat Hum Behav 3, 768–771 (2019). https://doi.org/10.1038/s41562-019-0655-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-019-0655-x

  • Springer Nature Limited

This article is cited by

Search

Navigation