Skip to main content
SpringerLink
Log in

FreeSurfer 5.3 versus 6.0: are volumes comparable? A Chronic Effects of Neurotrauma Consortium study

  • ORIGINAL RESEARCH
  • Published:
Brain Imaging and Behavior Aims and scope Submit manuscript

Abstract

Automated neuroimaging methods like FreeSurfer (https://surfer.nmr.mgh.harvard.edu/) have revolutionized quantitative neuroimaging analyses. Such analyses provide a variety of metrics used for image quantification, including magnetic resonance imaging (MRI) volumetrics. With the release of FreeSurfer version 6.0, it is important to assess its comparability to the widely-used previous version 5.3. The current study used data from the initial 249 participants in the ongoing Chronic Effects of Neurotrauma Consortium (CENC) multicenter observational study to compare the volumetric output of versions 5.3 and 6.0 across various regions of interest (ROI). In the current investigation, the following ROIs were examined: total intracranial volume, total white matter volume, total ventricular volume, total gray matter volume, and right and left volumes for the thalamus, pallidum, putamen, caudate, amygdala and hippocampus. Absolute ROI volumes derived from FreeSurfer 6.0 differed significantly from those obtained using version 5.3. We also employed a clinically-based evaluation strategy to compare both versions in their prediction of age-mediated volume reductions (or ventricular increase) in the aforementioned structures. Statistical comparison involved both general linear modeling (GLM) and random forest (RF) methods, where cross-validation error was significantly higher using segmentations from FreeSurfer version 5.3 versus version 6.0 (GLM: t = 4.97, df = 99, p value = 2.706e-06; RF: t = 4.85, df = 99, p value = 4.424e-06). Additionally, the relative importance of ROIs used to predict age using RFs differed between FreeSurfer versions, indicating substantial differences in the two versions. However, from the perspective of correlational analyses, fitted regression lines and their slopes were similar between the two versions, regardless of version used. While absolute volumes are not interchangeable between version 5.3 and 6.0, ROI correlational analyses appear to yield similar results, suggesting the interchangeability of ROI volume for correlational studies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Bigler, E. D. (2015). Structural image analysis of the brain in neuropsychology using magnetic resonance imaging (MRI) techniques. Neuropsychology Review, 25(3), 224–249. https://doi.org/10.1007/s11065-015-9290-0.

    Article  PubMed  Google Scholar 

  • Bigler, E. D. (2016). Systems biology, neuroimaging, neuropsychology, neuroconnectivity and traumatic brain injury. Frontiers in Systems Neuroscience, 10, 55. https://doi.org/10.3389/fnsys.2016.00055.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Bigler, E. D., Abildskov, T. J., Wilde, E. A., McCauley, S. R., Li, X., Merkley, T. L., . . . Levin, H. S. (2010). Diffuse damage in pediatric traumatic brain injury: a comparison of automated versus operator-controlled quantification methods. Neuroimage, 50(3), 1017–1026. https://doi.org/10.1016/j.neuroimage.2010.01.003.

  • Blatter, D. D., Bigler, E. D., Gale, S. D., Johnson, S. C., Anderson, C. V., Burnett, B. M., . . . Horn, S. D. (1995). Quantitative volumetric analysis of brain MR: normative database spanning 5 decades of life. AJNR. American Journal of Neuroradiology, 16(2), 241–251. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/7726068.

  • Breiman, L. (2001). Random forests. Machine Learning, 45, 5–32.

    Article  Google Scholar 

  • Brewer, J. B., Magda, S., Airriess, C., & Smith, M. E. (2009). Fully-automated quantification of regional brain volumes for improved detection of focal atrophy in Alzheimer disease. AJNR. American Journal of Neuroradiology, 30(3), 578–580. https://doi.org/10.3174/ajnr.A1402.

  • Chepkoech J. L., Walhovd K. B., & Grydeland H. (2016). Fjell AM Alzheimer's Disease Neuroimaging Initiative. Effects of change in FreeSurfer version on classification accuracy of patients with Alzheimer's disease and mild cognitive impairment. Human Brain Mapping, 37(5), 1831–41. https://doi.org/10.1002/hbm.23139

  • Cifu, D. X., & Dixon, K. J. (2016). Chronic effects of neurotrauma consortium. Brain Injury, 30(12), 1397–1398. https://doi.org/10.1080/02699052.2016.1219065.

    Article  PubMed  Google Scholar 

  • Dewey, J., Hana, G., Russell, T., Price, J., McCaffrey, D., Harezlak, J., . . . Consortium, H. I. V. N. (2010). Reliability and validity of MRI-based automated volumetry software relative to auto-assisted manual measurement of subcortical structures in HIV-infected patients from a multisite study. Neuroimage, 51(4), 1334–1344. https://doi.org/10.1016/j.neuroimage.2010.03.033.

  • Fischl, B. (2012). FreeSurfer. Neuroimage, 62(2), 774–781. https://doi.org/10.1016/j.neuroimage.2012.01.021.

    Article  PubMed  PubMed Central  Google Scholar 

  • Gronenschild E. H., Habets P., Jacobs H. I., Mengelers R., Rozendaal N., van Os J., & Marcelis M. (2012). The effects of FreeSurfer version, workstation type, and Macintosh operating system version on anatomical volume and cortical thickness measurements. PLoS One. 7(6):e38234. https://doi.org/10.1371/journal.pone.0038234

  • Guzman Perez-Carrillo, G. J., Owen, C., Schwetye, K. E., McFarlane, S., Vellimana, A. K., Mar, S., . . . Benzinger, T. L. S. (2017). The use of hippocampal volumetric measurements to improve diagnostic accuracy in pediatric patients with mesial temporal sclerosis. Journal of Neurosurgery. Pediatrics, 19(6), 720–728. https://doi.org/10.3171/2016.12.PEDS16335.

  • Jack, C. R., Jr., Barnes, J., Bernstein, M. A., Borowski, B. J., Brewer, J., Clegg, S., . . . Weiner, M. (2015). Magnetic resonance imaging in Alzheimer's disease neuroimaging initiative 2. Alzheimers Dement, 11(7), 740–756. https://doi.org/10.1016/j.jalz.2015.05.002.

  • Jahanshad, N., & Thompson, P. M. (2017). Multimodal neuroimaging of male and female brain structure in health and disease across the life span. Journal of Neuroscience Research, 95(1–2), 371–379. https://doi.org/10.1002/jnr.23919.

    Article  CAS  PubMed  Google Scholar 

  • Kuhn, M. (2008). Building predictive models in R using the caret package. Journal of Statistical Software, 28(5) Retrieved from http://www.jstatsoft.org.

  • Makowski, C., Beland, S., Kostopoulos, P., Bhagwat, N., Devenyi, G. A., Malla, A. K., et al. (2017). Evaluating accuracy of striatal, pallidal, and thalamic segmentation methods: comparing automated approaches to manual delineation. NeuroImage, 170, 182–198. https://doi.org/10.1016/j.neuroimage.2017.02.069.

    Article  PubMed  Google Scholar 

  • Makris, N., Hodge, S. M., Haselgrove, C., Kennedy, D. N., Dale, A., Fischl, B., . . . Schmahmann, J. D. (2003). Human cerebellum: surface-assisted cortical parcellation and volumetry with magnetic resonance imaging. Journal of Cognitive Neuroscience, 15(4), 584–599. https://doi.org/10.1162/089892903321662967.

  • Marizzoni, M., Antelmi, L., Bosch, B., Bartres-Faz, D., Muller, B. W., Wiltfang, J., . . . PharmaCog, C. (2015). Longitudinal reproducibility of automatically segmented hippocampal subfields: a multisite European 3T study on healthy elderly. Human Brain Mapping, 36(9), 3516–3527. https://doi.org/10.1002/hbm.22859.

  • Matsumae, M., Kikinis, R., Morocz, I. A., Lorenzo, A. V., Sandor, T., Albert, M. S., . . . Jolesz, F. A. (1996). Age-related changes in intracranial compartment volumes in normal adults assessed by magnetic resonance imaging. Journal of Neurosurgery, 84(6), 982–991. https://doi.org/10.3171/jns.1996.84.6.0982.

  • Michael, A. P., Stout, J., Roskos, P. T., Bolzenius, J., Gfeller, J., Mogul, D., & Bucholz, R. (2015). Evaluation of cortical thickness after traumatic brain injury in military veterans. Journal of Neurotrauma, 32(22), 1751–1758. https://doi.org/10.1089/neu.2015.3918.

    Article  PubMed  Google Scholar 

  • Ochs, A. L., Ross, D. E., Zannoni, M. D., Abildskov, T. J., Bigler, E. D., & Alzheimer's Disease Neuroimaging, I. (2015). Comparison of automated brain volume measures obtained with NeuroQuant and FreeSurfer. Journal of Neuroimaging, 25(5), 721–727. https://doi.org/10.1111/jon.12229.

    Article  PubMed  Google Scholar 

  • Pfefferbaum, A., Mathalon, D. H., Sullivan, E. V., Rawles, J. M., Zipursky, R. B., & Lim, K. O. (1994). A quantitative magnetic resonance imaging study of changes in brain morphology from infancy to late adulthood. Archives of Neurology, 51(9), 874–887 Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/8080387.

    Article  CAS  Google Scholar 

  • Seal, K. H., Bertenthal, D., Barnes, D. E., Byers, A. L., Strigo, I., Yaffe, K., & Chronic Effects of Neurotrauma Consortium Study, G. (2017). Association of Traumatic Brain Injury with chronic pain in Iraq and Afghanistan veterans: effect of comorbid mental health conditions. Archives of Physical Medicine and Rehabilitation, 98(8), 1636–1645. https://doi.org/10.1016/j.apmr.2017.03.026.

    Article  PubMed  Google Scholar 

  • Stelmokas, J., Yassay, L., Giordani, B., Dodge, H. H., Dinov, I. D., Bhaumik, A., . . . Hampstead, B. M. (2017). Translational MRI Volumetry with NeuroQuant: effects of version and normative data on relationships with memory performance in healthy older adults and patients with mild cognitive impairment. Journal of Alzheimer's Disease, 60(4), 1499–1510. https://doi.org/10.3233/JAD-170306.

  • Tate, D. F., Wade, B. S., Velez, C. S., Drennon, A. M., Bolzenius, J., Gutman, B. A., . . . York, G. E. (2016). Volumetric and shape analyses of subcortical structures in United States service members with mild traumatic brain injury. Journal of Neurology, 263(10), 2065–2079. https://doi.org/10.1007/s00415-016-8236-7.

  • Tustison, N. J., Cook, P. A., Klein, A., Song, G., Das, S. R., Duda, J. T., . . . Avants, B. B. (2014). Large-scale evaluation of ANTs and FreeSurfer cortical thickness measurements. Neuroimage, 99, 166–179. https://doi.org/10.1016/j.neuroimage.2014.05.044.

  • Vanderploeg, R. D., Silva, M. A., Soble, J. R., Curtiss, G., Belanger, H. G., Donnell, A. J., & Scott, S. G. (2015). The structure of postconcussion symptoms on the neurobehavioral symptom inventory: a comparison of alternative models. The Journal of Head Trauma Rehabilitation, 30(1), 1–11. https://doi.org/10.1097/HTR.0000000000000009.

    Article  PubMed  Google Scholar 

  • Visser, E., Keuken, M. C., Douaud, G., Gaura, V., Bachoud-Levi, A. C., Remy, P., . . . Jenkinson, M. (2016). Automatic segmentation of the striatum and globus pallidus using MIST: Multimodal image segmentation tool. Neuroimage, 125, 479–497. https://doi.org/10.1016/j.neuroimage.2015.10.013.

  • Walker, W. C., Carne, W., Franke, L. M., Nolen, T., Dikmen, S. D., Cifu, D. X., . . . Williams, R. (2016). The Chronic Effects of Neurotrauma Consortium (CENC) multi-centre observational study: Description of study and characteristics of early participants. Brain Injury, 30(12), 1469–1480. doi:https://doi.org/10.1080/02699052.2016.1219061.

  • Weathers, F. W., Litz, B. T., Herman, D. S., Huska, J. A., & Keane, T. M. (1993). The PTSD checklist (PCL): Reliability, validity, and diagnostic utility. San Antonio: Annual Convention of the International Society for Traumatic Stress Studies.

    Google Scholar 

  • Wilde, E. A., Bouix, S., Tate, D. F., Lin, A. P., Newsome, M. R., Taylor, B. A., . . . York, G. (2015). Advanced neuroimaging applied to veterans and service personnel with traumatic brain injury: state of the art and potential benefits. Brain Imaging and Behavior, 9(3), 367–402. https://doi.org/10.1007/s11682-015-9444-y.

  • Wilde, E. A., Bigler, E. D., Huff, T., Wang, H., Black, G. M., Christensen, Z. P., . . . Tate, D. F. (2016). Quantitative structural neuroimaging of mild traumatic brain injury in the Chronic Effects of Neurotrauma Consortium (CENC): comparison of volumetric data within and across scanners. Brain Injury, 30(12), 1442–1451. https://doi.org/10.1080/02699052.2016.1219063.

  • Wilkins, K. C., Lang, A. J., & Norman, S. B. (2011). Synthesis of the psychometric properties of the PTSD checklist (PCL) military, civilian, and specific versions. Depression and Anxiety, 28(7), 596–606. https://doi.org/10.1002/da.20837.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The CENC Observational Study Site PIs or co-PIs also include: Heather Belanger PhD (Tampa), Carlos Jaramillo MD (San Antonio), Ajit Pai MD (Richmond), Heechin Chae MD (Fort Belvoir), Terri Pogoda PhD (Boston), Scott Sponheim PhD (Minneapolis), and Kathleen Carlson PhD (Portland). We also acknowledge the efforts of the entire CENC Observational Study Leadership Working Group and Core Team members who, besides the authors, also include: Justin Alicea, Jessica Berumen, Cody Blankenship, Jennifer Boyce, Linda Brunson, Katrina Burson, Julia Christensen, Margaret Clarke, Doreen Collins, Sureyya Dikmen, Esra Doud, Connie Duncan, Stephanie Edmunds, Robyn Endsley, Elizabeth Fogleman, Laura M. Franke, Katelyn Gormley, Brenda Hair, Jim Henry, Nancy Hsu, Cheryl Ford-Smith, George Gitchel, Col. Sidney Hinds (Consortium Co-PI), Caitlin Jones, Kimbra Kenney, Sunchai Khemalaap, Valerie Larson, Tiffany Lewis, Scott McDonald, Tamara McKenzie-Hartman, Frank Mierzwa, Alison Molitor, Joe Montanari, Johnnie Mortenson, Nicholas Pastorek, Judy Pulliam, Risa Richardson, Callie Riggs, Rachel Rosenfield, Sara Salkind, James K. Sickinger, Taylor Swankie, Nancy Temkin, Doug Theriaque, Maya Troyanskaya, Rodney Vanderploeg, and Carmen Vasquez.

Additionally, we wish to acknowledge additional members of the CENC Neuroimaging Core including: Marlene Diaz, Carlo Pierpaoli, Amritha Nayak, Carmen Velez, Gerald E. York, Jennifer Nathan, Rajan Agarwal, Timothy Duncan, Michael Lennon, Aaron M. Betts, Jorge De Villasante, Robert Cadrain, Garrett Black, Naomi J. Goodrich-Hunsaker, Zili D. Chu, and Rhonda O’Donovan.

Finally, we extend our gratitude to the participants and family members of the Chronic Effects of Neurotrauma Consortium.

Funding

This material is based in part upon work supported by the U.S. Army Medical Research and Material Command and from the U.S. Department of Veterans Affairs Chronic Effects of Neurotrauma Consortium under Award No. W81XWH-13-2-0095 and the CENC Neuroimaging Core under 1I01RX001062. The U.S. Army Medical Research Acquisition Activity, 820 Chandler Street, Fort Detrick MD 21702–5014 is the awarding and administering acquisition office. Any opinions, findings, conclusions or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the U.S. Government, or the U.S. Department of Veterans Affairs, and no official endorsement should be inferred. Dr. Bigler received royalties from Oxford University Press, and honorarium from the American Psychological Association for editorial work and, during the time of this investigation, directed the Neuropsychological Assessment and Research Lab at Brigham Young University, which did provide forensic consultation.

Author information

Authors and Affiliations

  1. Psychology Department and Neuroscience Center, Brigham Young University, Provo, UT, 84602, USA

    Erin D. Bigler, Marc Skiles & Tracy J. Abildskov

  2. Missouri Institute of Mental Health, University of Missouri-St. Louis, St. Louis, MO, USA

    Benjamin S. C. Wade & David F. Tate

  3. Imaging Genetics Center, University of Southern California, Marina del Rey, CA, USA

    Benjamin S. C. Wade

  4. Ahmanson-Lovelace Brain Mapping Center, Department of Neurology, UCLA, Los Angeles, CA, USA

    Benjamin S. C. Wade

  5. Department of Radiology and Medical Imaging, University of Virginia, Charlottesville, VA, USA

    Nick J. Tustison & James R. Stone

  6. Michael DeBakey VA Medical Center and Baylor College of Medicine, Houston, TX, USA

    Randall S. Scheibel, Mary R. Newsome, Harvey S. Levin & Elisabeth A. Wilde

  7. University of New Mexico, Albuquerque, NM, USA

    Andrew R. Mayer

  8. Virginia Commonwealth University, Richmond, VA, USA

    Brian A. Taylor & William C. Walker

  9. University of Utah, Salt Lake City, UT, USA

    Elisabeth A. Wilde

Authors
  1. Erin D. Bigler
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marc Skiles
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Benjamin S. C. Wade
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tracy J. Abildskov
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nick J. Tustison
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Randall S. Scheibel
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mary R. Newsome
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Andrew R. Mayer
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. James R. Stone
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Brian A. Taylor
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. David F. Tate
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. William C. Walker
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Harvey S. Levin
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Elisabeth A. Wilde
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Erin D. Bigler.

Ethics declarations

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study. Institutional review board (IRB) approval was obtained across all participating intuitions and for all phases of the study.

Conflict of interest

No other investigator reported any financial conflict of interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(DOCX 670 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bigler, E.D., Skiles, M., Wade, B.S.C. et al. FreeSurfer 5.3 versus 6.0: are volumes comparable? A Chronic Effects of Neurotrauma Consortium study. Brain Imaging and Behavior 14, 1318–1327 (2020). https://doi.org/10.1007/s11682-018-9994-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11682-018-9994-x

Keywords

Search

Navigation