Abstract
Accelerating insight into the relation between brain and behavior entails conducting small and large-scale research endeavors that lead to reproducible results. Consensus is emerging between funding agencies, publishers, and the research community that data sharing is a fundamental requirement to ensure all such endeavors foster data reuse and fuel reproducible discoveries. Funding agency and publisher mandates to share data are bolstered by a growing number of data sharing efforts that demonstrate how information technologies can enable meaningful data reuse. Neuroinformatics evaluates scientific needs and develops solutions to facilitate the use of data across the cognitive and neurosciences. For example, electronic data capture and management tools designed to facilitate human neurocognitive research can decrease the setup time of studies, improve quality control, and streamline the process of harmonizing, curating, and sharing data across data repositories. In this article we outline the advantages and disadvantages of adopting software applications that support these features by reviewing the tools available and then presenting two contrasting neuroimaging study scenarios in the context of conducting a cross-sectional and a multisite longitudinal study.
Similar content being viewed by others
Notes
References
Beekly, D., Ramos, E., Lee, W., Deitrich, W., Jacka, M., Wu, J., et al. (2007). The national Alzheimer’s coordinating center (NACC) database: the uniform data set. Alzheimer Disease & Associated Disorders, 21(3), 249–258.
Belleau, F., Nolin, M.-A., Tourigny, N., Rigault, P., & Morissette, J. (2008). Bio2RDF: towards a mashup to build bioinformatics knowledge systems. Journal of Biomedical Informatics, 41(5). doi:10.1016/j.jbi.2008.03.004.
Bernstam, E. V., Hersh, W. R., Johnson, S. B., Chute, C. G., Nguyen, H., Sim, I., et al. (2009). Synergies and distinctions between computational disciplines in biomedical research: perspective from the clinical and translational science award programs (Vol. 84, pp. 964–970). Presented at the Academic medicine: Journal of the Association of American Medical Colleges. doi:10.1097/ACM.0b013e3181a8144d.
Biswal, B. B., Mennes, M., Zuo, X.-N., Gohel, S., Kelly, C., Smith, S. M., et al. (2010). Toward discovery science of human brain function. Proceedings of the National Academy of Sciences, 107(10), 4734–4739. doi:10.1073/pnas.0911855107.
Bjaalie, J. G., & Grillner, S. (2007). Global neuroinformatics: the international neuroinformatics coordinating facility. Journal of Neuroscience, 27(14), 3613–3615. doi:10.1523/JNEUROSCI.0558-07.2007.
Bloom, T., Ganley, E., & Winker, M. (2014). Data access for the open access literature: PLOS’s data policy. PLoS Biology, doi:10.1371/journal.pmed.1001607.
Book, G. A., Anderson, B. M., Stevens, M. C., Glahn, D. C., Assaf, M., & Pearlson, G. D. (2013). Neuroinformatics database (NiDB) - a modular, portable database for the storage, analysis, and sharing of neuroimaging data. Neuroinformatics, 11(4), 495–505. doi:10.1007/s12021-013-9194-1.
Breeze, J. L., Poline, J. B., & Kennedy, D. N. (2012). Data sharing and publishing in the field of neuroimaging. GigaScience. doi:10.1186/2047-217X-1-9.
Brinkley, J. E., & Rosse, C. (2002). Imaging and the human brain project: a review. Methods of Information in Medicine, 41(4), 245–260.
Brown, S. A., Brumback, T., Tomlinson, K., Cummins, K., Thompson, W. K., Nagel, B. J., et al. (2015). The national consortium on alcohol and neurodevelopment in adolescence (NCANDA): a multi-site study of adolescent development and substance use. Journal of Studies on Alcohol and Drugs. In Press.
Buckow, K., Quade, M., Rienhoff, O., & Nussbeck, S. Y. (2014). Changing requirements and resulting needs for IT-infrastructure for longitudinal research in the neurosciences. Neuroscience Research. doi:10.1016/j.neures.2014.08.005.
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14(5), 365–376. doi:10.1038/nrn3475.
Caspers, S., Zilles, K., Laird, A. R., & Eickhoff, S. B. (2010). ALE meta-analysis of action observation and imitation in the human brain. NeuroImage, 50(3), 1148–1167. doi:10.1016/j.neuroimage.2009.12.112.
Collins, F. S., & Tabak, L. A. (2014). Policy: NIH plans to enhance reproducibility. Nature, 505(7485), 612–613.
Cox, R. W. (1996). AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Computers and Biomedical Research, an International Journal, 29(3), 162–173.
Cozzarelli, N. R. (2004). UPSIDE: uniform principle for sharing integral data and materials expeditiously. Proceedings of the National Academy of Sciences of the United States of America, 101(11), 3721–3722. doi:10.1073/pnas.0400437101.
D’Esposito, M. (2000). Letter from the special issue editor. Journal of Cognitive Neuroscience, 12(supplement 2), 1. doi:10.1162/089892900563966.
Das, S., Zijdenbos, A. P., Harlap, J., Vins, D., & Evans, A. C. (2011). LORIS: a web-based data management system for multi-center studies. Frontiers in Neuroinformatics, 5, 37. doi:10.3389/fninf.2011.00037.
David, S. P., Ware, J. J., Chu, I. M., Loftus, P. D., Fusar-Poli, P., Radua, J., et al. (2013). Potential reporting bias in fMRI studies of the brain. PLoS ONE, 8(7), e70104. doi:10.1371/journal.pone.0070104.
Day, S., Fayers, P., & Harvey, D. (1998). Double data entry: what value, what price? Controlled Clinical Trials, 19(1), 15–24.
Di Martino, A., Yan, C.-G., Li, Q., Denio, E., Castellanos, F. X., Alaerts, K., et al. (2014). The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism. Molecular Psychiatry, 19(6), 659–667. doi:10.1038/mp.2013.78.
Dinov, I. (2009). Efficient, distributed and interactive neuroimaging data analysis using the LONI pipeline. Frontiers in Neuroinformatics, 3, 1–10. doi:10.3389/neuro.11.022.2009.
Ferguson, A. R., Nielson, J. L., Cragin, M. H., Bandrowski, A. E., & Martone, M. E. (2014). Big data from small data: data-sharing in the “long tail” of neuroscience. Nature Neuroscience, 17(11), 1442–1447. doi:10.1038/nn.3838.
Fischl, B., Salat, D. H., Busa, E., Albert, M., Dieterich, M., Haselgrove, C., et al. (2002). Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain. Neuron, 33(3), 341–355.
Fischl, B., van der Kouwe, A., Destrieux, C., Halgren, E., Ségonne, F., Salat, D. H., et al. (2004). Automatically parcellating the human cerebral cortex. Cerebral Cortex (New York, N.Y. : 1991), 14(1), 11–22.
Fjell, A. M., Walhovd, K. B., Brown, T. T., Kuperman, J. M., Chung, Y., Hagler, D. J., et al. (2012). Multimodal imaging of the self-regulating developing brain. Proceedings of the National Academy of Sciences, 109(48), 19620–19625. doi:10.1073/pnas.1208243109.
Fox, P. T., & Lancaster, J. L. (2002). Opinion: mapping context and content: the brainmap model. Nature Reviews Neuroscience, 3(4), 319–321. doi:10.1038/nrn789.
Fox, P., Fox, P. T., Laird, A. R., Laird, A., Fox, S. P., Fox, S., et al. (2005). BrainMap taxonomy of experimental design: description and evaluation. Human Brain Mapping, 25(1), 185–198. doi:10.1002/hbm.20141.
Franklin, J. D., Guidry, A., & Brinkley, J. F. (2011). A partnership approach for electronic data capture in small-scale clinical trials. Journal of Biomedical Informatics, 44(Suppl 1), S103–S108. doi:10.1016/j.jbi.2011.05.008.
Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J. P., Frith, C. D., & Frackowiak, R. S. (1994). Statistical parametric maps in functional imaging: a general linear approach. Human Brain Mapping, 2(4), 189–210.
Gadde, S., Aucoin, N., Grethe, J. S., Keator, D. B., Marcus, D. S., Pieper, S., FBIRN, MBIRN, & BIRN-CC. (2012). XCEDE: an extensible schema for biomedical data. Neuroinformatics, 10(1), 19–32. doi:10.1007/s12021-011-9119-9.
Gardner, D., Akil, H., Ascoli, G., Bowden, D., Bug, W., Donohue, D., et al. (2008). The neuroscience information framework: a data and knowledge environment for neuroscience. Neuroinformatics, 6(3), 149–160.
Gershon, R. C., Gershon, R. C., Cella, D., Cella, D., Fox, N. A., Fox, N. A., et al. (2010). Assessment of neurological and behavioural function: the NIH Toolbox. Lancet Neurology, 9(2), 138–139. doi:10.1016/S1474-4422(09)70335-7.
Gomez-Marin, A., Paton, J. J., Kampff, A. R., Costa, R. M., & Mainen, Z. F. (2014). Big behavioral data: psychology, ethology and the foundations of neuroscience. Nature Neuroscience, 17(11), 1455–1462. doi:10.1038/nn.3812.
Gorgolewski, K. J., Varoquaux, G., Rivera, G., Schwartz, Y., Sochat, V. V., Ghosh, S. S., et al. (2015). NeuroVault.org: a repository for sharing unthresholded statistical maps, parcellations, and atlases of the human brain. NeuroImage. doi:10.1016/j.neuroimage.2015.04.016.
Greenes, R., & Brinkley, J. (2006). Imaging systems in radiology. In E. H. Shortliffe & J. J. Cimino (Eds.), Biomedical and Health Informatics: Computer Applications in Healthcare (pp. 626–659). New York: Springer.
Gur, R. C., Richard, J., Hughett, P., Calkins, M. E., Macy, L., Bilker, W. B., et al. (2010). A cognitive neuroscience-based computerized battery for efficient measurement of individual differences: standardization and initial construct validation. Journal of Neuroscience Methods, 187(2), 254–262. doi:10.1016/j.jneumeth.2009.11.017.
Haak, D., Page, C.-E., Reinartz, S., Krüger, T., & Deserno, T. M. (2015). DICOM for clinical research: PACS-integrated electronic data capture in multi-center trials. Journal of Digital Imaging. doi:10.1007/s10278-015-9802-8.
Hall, D., Huerta, M. F., McAuliffe, M. J., & Farber, G. K. (2012). Sharing heterogeneous data: the national database for autism research. Neuroinformatics, 10(4), 331–339. doi:10.1007/s12021-012-9151-4.
Harris, P. A., Taylor, R., Thielke, R., Payne, J., Gonzalez, N., & Conde, J. G. (2009). Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics, 42(2), 377–381. doi:10.1016/j.jbi.2008.08.010.
Helmer, K. G., Ambite, J. L., Ambite, J. L., Ames, J., Ames, J., Ananthakrishnan, R., et al. (2011). Enabling collaborative research using the biomedical informatics research network (BIRN). Journal of the American Medical Informatics Association, 18(4), 416–422. doi:10.1136/amiajnl-2010-000032.
Howe, D., Costanzo, M., Fey, P., Gojobori, T., Hannick, L., Hide, W., et al. (2008). Big data: the future of biocuration. Nature, 455(7209), 47–50. doi:10.1038/455047a.
Hudson, K. L., & Collins, F. S. (2015). Sharing and reporting the results of clinical trials. JAMA, 313(4), 355–356. doi:10.1001/jama.2014.10716.
Huerta, M. F., & Koslow, S. H. (1996). Neuroinformatics: opportunities across disciplinary and national borders. NeuroImage, 4(3), S4–S6. doi:10.1006/nimg.1996.0040.
Hussein, R., Engelmann, U., Schroeter, A., & Meinzer, H. (2004). DICOM structured reporting. Radiographics: a Review Publication of the Radiological Society of North America, Inc, 24(3), 897–909.
Insel, T., Cuthbert, B., Garvey, M., Heinssen, R., Pine, D. S., Quinn, K., et al. (2010). Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. The American Journal of Psychiatry, 167(7), 748–751. doi:10.1176/appi.ajp.2010.09091379.
Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124. doi:10.1371/journal.pmed.0020124.
Ioannidis, J. P. A. (2011). Excess significance bias in the literature on brain volume abnormalities. Archives of General Psychiatry, 68(8), 773–780. doi:10.1001/archgenpsychiatry.2011.28.
Jack, C. R., Bernstein, M. A., Fox, N. C., Thompson, P., Alexander, G., Harvey, D., et al. (2008). The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods. Journal of Magnetic Resonance Imaging, 27(4), 685–691. doi:10.1002/jmri.21049.
Johnston, L. D., OMalley, P. M., Miech, R. A., Bachman, J. G., & Schulenberg, J. E. (2015). Monitoring the Future national survey results on drug use: 1975–2014: Overview, key findings on adolescent drug use. Resource document. Monitoring the Future Project. http://www.monitoringthefuture.org/pubs/monographs/mtf-overview2014.pdf . Accessed 25 March 2015.
Kane, R. L., & Kay, G. G. (1992). Computerized assessment in neuropsychology: a review of tests and test batteries. Neuropsychology Review, 3(1), 1–117. doi:10.1007/BF01108787.
Keator, D. B., Wei, D., Gadde, S., Bockholt, J., Grethe, J. S., Marcus, D., et al. (2009). Derived data storage and exchange workflow for large-scale neuroimaging analyses on the BIRN grid. Frontiers in Neuroinformatics. doi:10.3389/neuro.11.030.2009.
Keator, D. B., Helmer, K., Steffener, J., Turner, J. A., Van Erp, T. G., Gadde, S., et al. (2013). Towards structured sharing of raw and derived neuroimaging data across existing resources. NeuroImage, 82, 647–661. doi:10.1016/j.neuroimage.2013.05.094.
Kennedy, D. N., Haselgrove, C., Hodge, S. M., Rane, P. S., Rane, P. S., Makris, N., & Frazier, J. A. (2012). CANDIShare: a resource for pediatric neuroimaging data. Neuroinformatics, 10(3), 319–322. doi:10.1007/s12021-011-9133-y.
Kennedy, D. N., Haselgrove, C., Riehl, J., Preuss, N., & Buccigrossi, R. (2015). The Three NITRCs: a guide to neuroimaging neuroinformatics resources. Neuroinformatics. doi:10.1007/s12021-015-9263-8.
Kulikowski, C. A., Shortliffe, E. H., Currie, L. M., Elkin, P. L., Hunter, L. E., Johnson, T. R., et al. (2012). AMIA board white paper: definition of biomedical informatics and specification of core competencies for graduate education in the discipline. Journal of the American Medical Informatics Association, 19(6), 931–938. doi:10.1136/amiajnl-2012-001053.
Laird, A. R., Lancaster, J. L., & Fox, P. T. (2005). BrainMap: the social evolution of a human brain mapping database. Neuroinformatics, 3(1), 65–78.
Lancaster, J. L., Woldorff, M. G., Parsons, L. M., Liotti, M., Freitas, C. S., Rainey, L., et al. (2000). Automated Talairach atlas labels for functional brain mapping. Human Brain Mapping, 10(3), 120–131. doi:10.1002/1097-0193(200007)10:3<120::AID-HBM30>3.0.CO;2-8.
Larson, S. D., & Martone, M. E. (2013). NeuroLex.org: an online framework for neuroscience knowledge. Frontiers in Neuroinformatics. doi:10.3389/fninf.2013.00018.
Lord, C., Rutter, M., Goode, S., Heemsbergen, J., Jordan, H., Mawhood, L., & Schopler, E. (1989). Autism diagnostic observation schedule: a standardized observation of communicative and social behavior. Journal of Autism and Developmental Disorders, 19(2), 185–212.
Marcus, D., Olsen, T., Ramaratnam, M., & Buckner, R. (2007). The extensible neuroimaging archive toolkit. Neuroinformatics, 5(1), 11–33.
Marcus, D. S., Harwell, J., Olsen, T., Hodge, M., Glasser, M. F., Prior, F., et al. (2011). Informatics and data mining tools and strategies for the human connectome project. Frontiers in Neuroinformatics. doi:10.3389/fninf.2011.00004.
Meier, M. H., Caspi, A., Ambler, A., Harrington, H., Houts, R., Keefe, R. S. E., et al. (2012). Persistent cannabis users show neuropsychological decline from childhood to midlife. Proceedings of the National Academy of Sciences, 109(40), E2657–E2664. doi:10.1073/pnas.1206820109.
Mennes, M., Biswal, B. B., Castellanos, F. X., & Milham, M. P. (2013). Making data sharing work: the FCP/INDI experience. NeuroImage, 82, 683–691. doi:10.1016/j.neuroimage.2012.10.064.
Milchenko, M., & Marcus, D. (2012). Obscuring surface anatomy in volumetric imaging data. Neuroinformatics. doi:10.1007/s12021-012-9160-3.
Morris, J., Weintraub, S., Chui, H., Cummings, J., DeCarli, C., Ferris, S., et al. (2006). The uniform data Set (UDS): clinical and cognitive variables and descriptive data from Alzheimer disease centers. Alzheimer Disease & Associated Disorders, 20(4), 210–216.
Nichols, B. N., Mejino, J. L., Jr., Detwiler, L., Nilsen, T. T., Martone, M. E., Turner, J. A., et al. (2014). Neuroanatomical domain of the foundational model of anatomy ontology. Journal of Biomedical Semantics. doi:10.1186/2041-1480-5-1.
Pernet, C., & Poline, J. B. (2015). Improving functional magnetic resonance imaging reproducibility. GigaScience. doi:10.1186/s13742-015-0055-8.
Poldrack, R. A., & Gorgolewski, K. J. (2014). Making big data open: data sharing in neuroimaging. Nature Neuroscience, 17(11), 1510–1517. doi:10.1038/nn.3818.
Poldrack, R. A., Kittur, A., Kalar, D., Miller, E., Seppa, C., Gil, Y., et al. (2011). The cognitive atlas: toward a knowledge foundation for cognitive neuroscience. Frontiers in Neuroinformatics. doi:10.3389/fninf.2011.00017.
Poldrack, R. A., Barch, D. M., Mitchell, J. P., Wager, T. D., Wagner, A. D., Devlin, J. T., et al. (2013). Toward open sharing of task-based fMRI data: the OpenfMRI project. Frontiers in Neuroinformatics. doi:10.3389/fninf.2013.00012.
Poline, J. B., Breeze, J. L., Ghosh, S., Gorgolewski, K., Halchenko, Y. O., Hanke, M., et al. (2012). Data sharing in neuroimaging research. Frontiers in Neuroinformatics. doi:10.3389/fninf.2012.00009.
Rohlfing, T., Cummins, K., Henthorn, T., Chu, W., & Nichols, B. N. (2013). N-CANDA data integration: anatomy of an asynchronous infrastructure for multi-site, multi-instrument longitudinal data capture. Journal of the American Medical Informatics Association. doi:10.1136/amiajnl-2013-002367.
Salimi-Khorshidi, G., Smith, S. M., Keltner, J. R., Wager, T. D., & Nichols, T. E. (2009). Meta-analysis of neuroimaging data: a comparison of image-based and coordinate-based pooling of studies. NeuroImage, 45(3), 810–823. doi:10.1016/j.neuroimage.2008.12.039.
Scott, A., Courtney, W., Wood, D., De la Garza, R., Lane, S., King, M., et al. (2011). COINS: an innovative informatics and neuroimaging tool suite built for large heterogeneous datasets. Frontiers in Neuroinformatics. doi:10.3389/fninf.2011.00033.
Shepherd, G. M. (2002). Supporting databases for neuroscience research. Journal of Neuroscience, 22(5), 1497.
Shepherd, G. M., Mirsky, J. S., Healy, M. D., Singer, M. S., Skoufos, E., Hines, M. S., et al. (1998). The human brain project: neuroinformatics tools for integrating, searching and modeling multidisciplinary neuroscience data. Trends in Neurosciences, 21(11), 460–468.
Smith, S. M., Jenkinson, M., Woolrich, M. W., Beckmann, C. F., Behrens, T. E. J., Johansen-Berg, H., et al. (2004). Advances in functional and structural MR image analysis and implementation as FSL. NeuroImage, 23(Suppl 1), S208–S219. doi:10.1016/j.neuroimage.2004.07.051.
Stover, P. J., Harlan, W. R., Hammond, J. A., Hendershot, T., & Hamilton, C. M. (2010). PhenX: a toolkit for interdisciplinary genetics research. Current Opinion in Lipidology, 21(2), 136–140. doi:10.1097/MOL.0b013e3283377395.
Thompson, P. M., Stein, J. L., Medland, S. E., Hibar, D. P., Vasquez, A. A., Rentería, M. E., et al. (2014). The ENIGMA consortium: large-scale collaborative analyses of neuroimaging and genetic data. Brain Imaging and Behavior, 8(2), 153–182. doi:10.1007/s11682-013-9269-5.
Toga, A. W., Crawford, K. L., & Alzheimer’s Disease Neuroimaging Initiative. (2010). The informatics core of the Alzheimer’s disease neuroimaging initiative. Alzheimer’s & Dementia: the Journal of the Alzheimer’s Association, 6(3), 247–256. doi:10.1016/j.jalz.2010.03.001.
Tzourio-Mazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N., et al. (2002). Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. NeuroImage, 15(1), 273–289.
Van Essen, D. C., Ugurbil, K., Auerbach, E., Barch, D., Behrens, T. E. J., Bucholz, R., et al. (2012). The human connectome project: a data acquisition perspective. NeuroImage, 62(4), 2222–2231. doi:10.1016/j.neuroimage.2012.02.018.
Van Horn, J. D., & Toga, A. W. (2009). Is it time to re-prioritize neuroimaging databases and digital repositories? NeuroImage, 47(4), 1720–1734. doi:10.1016/j.neuroimage.2009.03.086.
Van Horn, J. D., & Toga, A. W. (2014). Human neuroimaging as a “big data” science. Brain Imaging and Behavior, 8(2), 323–331. doi:10.1007/s11682-013-9255-y.
Yarkoni, T., Poldrack, R. A., Nichols, T. E., Van Essen, D. C., & Wager, T. D. (2011). Large-scale automated synthesis of human functional neuroimaging data. Nature Methods, 8(8), 665–670. doi:10.1038/nmeth.1635.
Young, M. P., & Scannell, J. W. (2000). Brain structure-function relationships: advances from neuroinformatics. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 355(1393), 3–6. doi:10.1098/rstb.2000.0545.
Acknowledgements
This work was supported by the U.S. National Institute on Alcohol Abuse and Alcoholism (NIAAA) (U01 AA021697, R01 AA005965, R01 AA012388, U01 AA013521, U01 AA017347, U01 AA017923). It was also supported by the Creative and Novel Ideas in HIV Research Program (CNIHR) through a supplement to the University of California at San Francisco (UCSF) Center For AIDS Research funding (P30 AI027763). This funding was made possible by collaborative efforts of the Office of AIDS Research, the National Institutes of Allergies and Infectious Diseases, and the International AIDS Society.
Funding Sources
U.S. National Institute on Alcohol Abuse and Alcoholism (NIAAA).
(U01 AA021697, R01 AA005965, R01 AA012388, U01 AA013521, U01 AA017347, U01 AA017923).
Creative and Novel Ideas in HIV Research Program (CNIHR) (P30 AI027763).
Conflict of interest
Neither author has conflicts of interest with the information presented herein.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Nichols, B.N., Pohl, K.M. Neuroinformatics Software Applications Supporting Electronic Data Capture, Management, and Sharing for the Neuroimaging Community. Neuropsychol Rev 25, 356–368 (2015). https://doi.org/10.1007/s11065-015-9293-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11065-015-9293-x