Imputation Strategy for Reliable Regional MRI Morphological Measurements

  • Shaina Sta. Cruz
  • Ivo D. Dinov
  • Megan M. Herting
  • Clio González-Zacarías
  • Hosung Kim
  • Arthur W. Toga
  • Farshid SepehrbandEmail author
Original Article


Regional morphological analysis represents a crucial step in most neuroimaging studies. Results from brain segmentation techniques are intrinsically prone to certain degrees of variability, mainly as results of suboptimal segmentation. To reduce this inherent variability, the errors are often identified through visual inspection and then corrected (semi)manually. Identification and correction of incorrect segmentation could be very expensive for large-scale studies. While identification of the incorrect results can be done relatively fast even with manual inspection, the correction step is extremely time-consuming, as it requires training staff to perform laborious manual corrections. Here we frame the correction phase of this problem as a missing data problem. Instead of manually adjusting the segmentation outputs, our computational approach aims to derive accurate morphological measures by machine learning imputation. Data imputation techniques may be used to replace missing or incorrect region average values with carefully chosen imputed values, all of which are computed based on other available multivariate information. We examined our approach of correcting segmentation outputs on a cohort of 970 subjects, which were undergone an extensive, time-consuming, manual post-segmentation correction. A random forest imputation technique recovered the gold standard results with a significant accuracy (r = 0.93, p < 0.0001; when 30% of the segmentations were considered incorrect in a non-random fashion). The random forest technique proved to be most effective for big data studies (N > 250).


Brain segmentation FreeSurfer Post-segmentation correction Imputation Random forest Big data 



This work was supported by the National Institute of Biomedical Imaging and Bioengineering (P41EB015922 and U54 EB020406), the Eunice Kennedy Shriver National Institute of Child Health and Human Development (R00HD065832), the National Institute of Mental Health (R01MH094343; K01MH1087610), National Institute of Diabetes and Digestive and Kidney Diseases (P30DK089503), National Institute of Neurological Disorders and Stroke (P30DK089503), National Institute of Nursing Research (P20 NR015331). This work was partially supported by NSF grants 1734853, 1636840, 1416953, 0716055 and 1023115. Many colleagues, who are part of the Big Data Discovery Science (BDDS) community, contributed indirectly to this research. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIBIB, NICHD, NIMH, NIDDK, NINDS, NINR or NIH.

This study was conducted as part of the “Big Data Discovery and Diversity through Research Education Advancement and Partnerships (BD3-REAP)” Project funded by National Institutes of Health (NIH)-R25; Grant number is IR25MD010397-01. Data collection and sharing for this project was funded by the Philadelphia Neurodevelopmental Cohort (PNC) and the Pediatric Imaging, Neurocognition and Genetics Study (PING) (National Institutes of Health Grant RC2DA029475).


  1. Azur, M. J., Stuart, E. A., Frangakis, C., & Leaf, P. J. (2011). Multiple imputation by chained equations: What is it and how does it work? International Journal of Methods in Psychiatric Research, 20, 40–49.
  2. Coupe, P., Yger, P., Prima, S., Hellier, P., Kervrann, C., & Barillot, C. (2008). An optimized blockwise nonlocal means denoising filter for 3-D magnetic resonance images. IEEE Transactions on Medical Imaging, 27, 425–441. Scholar
  3. Dale, A., Fischl, B., & Sereno, M. I. (1999). Cortical surface-based analysis: I. segmentation and surface reconstruction. Neuroimage, 9, 179–194. Scholar
  4. Desikan, R. S., Ségonne, F., Fischl, B., Quinn, B. T., Dickerson, B. C., Blacker, D., Buckner, R. L., Dale, A. M., Maguire, R. P., Hyman, B. T., Albert, M. S., & Killiany, R. J. (2006). An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage, 31, 968–980. Scholar
  5. Dinov, I. D. (2018). Data science and predictive analytics: Biomedical and health applications using R. Berlin: Springer.CrossRefGoogle Scholar
  6. Dinov, I. D., Van Horn, J. D., Lozev, K. M., Magsipoc, R., Petrosyan, P., Liu, Z., MacKenzie-Graham, A., Eggert, P., Parker, D. S., & Toga, A. W. (2009). Efficient, distributed and interactive neuroimaging data analysis using the LONI pipeline. Frontiers in Neuroinformatics, 3, 22.
  7. Dinov, I., Lozev, K., Petrosyan, P., Liu, Z., & Eggert, P. (2010). Neuroimaging study designs, computational analyses and data provenance using the LONI pipeline. PLoS One, 5, e13070. Scholar
  8. Eckert, M. (2004). Neuroanatomical markers for dyslexia: A review of dyslexia structural imaging studies. Neuroscientist, 10, 362–371. Scholar
  9. Eggert, L. D., Sommer, J., Jansen, A., Kircher, T., & Konrad, C. (2012). Accuracy and reliability of automated gray matter segmentation pathways on real and simulated structural magnetic resonance images of the human brain. PLoS One, 7, e45081. Scholar
  10. Eskildsen, S., Coupé, P., Fonov, V., Ostergaard, L.R., Collins, L., 2011. Effect of non-local means denoising on cortical segmentation accuracy with FACE, in: Organization for Human Brain Mapping 2011 Annual Meeting.Google Scholar
  11. Fischl, B. (2012). FreeSurfer. Neuroimage, 62, 774–781.
  12. Fischl, B., & Dale, A. M. (2000). Measuring the thickness of the human cerebral cortex from magnetic resonance images. Proceedings of the National Academy of Sciences, 97, 11050–11055.
  13. Fischl, B., Sereno, M. I., & Dale, A. (1999). Cortical surface-based analysis: II: Inflation, flattening, and a surface-based coordinate system. Neuroimage, 9, 195–207. Scholar
  14. Fischl, B., Salat, D. H., Busa, E., Albert, M., Dieterich, M., Haselgrove, C., Van Der Kouwe, A., Killiany, R., Kennedy, D., & Klaveness, S. (2002). Whole brain segmentation: Automated labeling of neuroanatomical structures in the human brain. Neuron, 33, 341–355.
  15. Fischl, B., Salat, D. H., van der Kouwe, A. J. W., Makris, N., Ségonne, F., Quinn, B. T., & Dale, A. M. (2004a). Sequence-independent segmentation of magnetic resonance images. Neuroimage, 23, S69–S84. Scholar
  16. Fischl, B., van der Kouwe, A., Destrieux, C., Halgren, E., Ségonne, F., Salat, D. H., Busa, E., Seidman, L. J., Goldstein, J., Kennedy, D., Caviness, V., Makris, N., Rosen, B., & Dale, A. M. (2004b). Automatically Parcellating the human cerebral cortex. Cerebral Cortex, 14, 11–22. Scholar
  17. Gedamu, E. L., Collins, D. L., & Arnold, D. L. (2008). Automated quality control of brain MR images. Journal of Magnetic Resonance Imaging, 28, 308–319. Scholar
  18. Gómez-Carracedo, M. P., Andrade, J. M., López-Mahía, P., Muniategui, S., & Prada, D. (2014). A practical comparison of single and multiple imputation methods to handle complex missing data in air quality datasets. Chemometrics and Intelligent Laboratory Systems, 134, 23–33.
  19. Gondara, L., & Wang, K. (2017). Multiple imputation using deep denoising. arXiv preprint arXiv:1705.02737.Google Scholar
  20. Graham, J. W. (2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60, 549–576. Scholar
  21. Gronenschild, E. H. B. M., Habets, P., Jacobs, H. I. L., Mengelers, R., Rozendaal, N., van Os, J., & Marcelis, M. (2012). The effects of FreeSurfer version, workstation type, and Macintosh operating system version on anatomical volume and cortical thickness measurements. PLoS One, 7, e38234. Scholar
  22. Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning. Springer Series in Statistics.
  23. Hastie, T., Tibshirani, R., Balasubramanian, N., Chu, G., 2016. Impute: Imputation for microarray data. R package version 1.48. 0.Google Scholar
  24. Hudak, A. T., Crookston, N. L., Evans, J. S., Hall, D. E., & Falkowski, M. J. (2008). Nearest neighbor imputation of species-level, plot-scale forest structure attributes from LiDAR data. Remote Sensing of Environment, 112, 2232–2245.
  25. Klapwijk, E. T., Van De Kamp, F., Van Der Meulen, M., Peters, S., & Wierenga, L. M. (2019). Qoala-T: A supervised-learning tool for quality control of FreeSurfer segmented MRI data. Neuroimage., 189, 116–129.
  26. Lee, M. R., Bartholow, B. D., McCarthy, D. M., Pedersen, S. L., & Sher, K. J. (2015). Two alternative approaches to conventional person-mean imputation scoring of the self-rating of the effects of alcohol scale (SRE). Psychology of Addictive Behaviors, 29, 231–236. Scholar
  27. Long, X., Liao, W., Jiang, C., Liang, D., Qiu, B., & Zhang, L. (2012). Healthy aging: an automatic analysis of global and regional morphological alterations of human brain. Academic Radiology, 19, 785–793. Scholar
  28. Luders, E., Narr, K. L., Thompson, P. M., Rex, D. E., Woods, R. P., DeLuca, H., Jancke, L., & Toga, A. W. (2006). Gender effects on cortical thickness and the influence of scaling. Human Brain Mapping, 27, 314–324. Scholar
  29. Makowski, C., Beland, S., Kostopoulos, P., Bhagwat, N., Devenyi, G. A., Malla, A. K., Joober, R., Lepage, M., & Chakravarty, M. M. (2017). Evaluating accuracy of striatal, pallidal, and thalamic segmentation methods: Comparing automated approaches to manual delineation. Neuroimage, 170, 182–198. Scholar
  30. Manjón, J. V., Coupé, P., Martí-Bonmatí, L., Collins, D. L., & Robles, M. (2010). Adaptive non-local means denoising of MR images with spatially varying noise levels. Journal of Magnetic Resonance Imaging, 31, 192–203.
  31. Markovsky, I., & Usevich, K. (2012). Low Rank Approximation: Algorithms, Implementation, Applications. London: Springer.CrossRefGoogle Scholar
  32. Mazumder, R., Hastie, T., & Tibshirani, R. (2010). Spectral regularization algorithms for learning large incomplete matrices. Journal of Machine Learning Research, 11, 2287–2322.Google Scholar
  33. Moon, S. W., Dinov, I. D., Kim, J., Zamanyan, A., Hobel, S., Thompson, P. M., & Toga, A. W. (2015). Structural neuroimaging genetics interactions in Alzheimer’s disease. Journal of Alzheimer's Disease, 48, 1051–1063. Scholar
  34. Morey, R. A., Petty, C. M., Xu, Y., Hayes, J. P., Wagner, H. R., 2nd, Lewis, D. V., LaBar, K. S., Styner, M., & McCarthy, G. (2009). A comparison of automated segmentation and manual tracing for quantifying hippocampal and amygdala volumes. Neuroimage, 45, 855–866. Scholar
  35. Mortamet, B., Bernstein, M. A., Jack, C. R. J., Gunter, J. L., Ward, C., Britson, P. J., Meuli, R., Thiran, J.-P., & Krueger, G. (2009). Automatic quality assessment in structural brain magnetic resonance imaging. Magnetic Resonance in Medicine, 62, 365–372. Scholar
  36. Perez, D. L., Matin, N., Williams, B., Tanev, K., Makris, N., LaFrance, W. C. J., & Dickerson, B. C. (2018). Cortical thickness alterations linked to somatoform and psychological dissociation in functional neurological disorders. Human Brain Mapping, 39, 428–439. Scholar
  37. Perlaki, G., Horvath, R., Nagy, S. A., Bogner, P., Doczi, T., Janszky, J., & Orsi, G. (2017). Comparison of accuracy between FSL’s FIRST and Freesurfer for caudate nucleus and putamen segmentation. Scientific Reports, 7, 2418. Scholar
  38. Reuter, M., & Fischl, B. (2011). Avoiding asymmetry-induced Bias in longitudinal image processing. Neuroimage, 57, 19–21. Scholar
  39. Reuter, M., Rosas, H. D., & Fischl, B. (2010). Highly accurate inverse consistent registration: A robust approach. Neuroimage, 53, 1181–1196. Scholar
  40. Reuter, M., Schmansky, N. J., Rosas, H. D., & Fischl, B. (2012). Within-subject template estimation for unbiased longitudinal image analysis. Neuroimage, 61, 1402–1418. Scholar
  41. Rubin, D. B. (2004). Multiple imputation for nonresponse in surveys. Hoboken: John Wiley & Sons.Google Scholar
  42. Satterthwaite, T. D., Elliott, M. A., Ruparel, K., Loughead, J., Prabhakaran, K., Calkins, M. E., Hopson, R., Jackson, C., Keefe, J., Riley, M., Mentch, F. D., Sleiman, P., Verma, R., Davatzikos, C., Hakonarson, H., Gur, R. C., & Gur, R. E. (2014). Neuroimaging of the Philadelphia neurodevelopmental cohort. Neuroimage, 86, 544–553. Scholar
  43. Satterthwaite, T. D., Connolly, J. J., Ruparel, K., Calkins, M. E., Jackson, C., Elliott, M. A., Roalf, D. R., Hopsona, R., Prabhakaran, K., Behr, M., Qiu, H., Mentch, F. D., Chiavacci, R., Sleiman, P. M. A., Gur, R. C., Hakonarson, H., & Gur, R. E. (2016). The Philadelphia neurodevelopmental cohort: A publicly available resource for the study of normal and abnormal brain development in youth. Neuroimage, 124, 1115–1119. Scholar
  44. Schafer, J. L. (1999). Multiple imputation: a primer. Statistical Methods in Medical Research, 8, 3–15. Scholar
  45. Segonne, F., Dale, A. M., Busa, E., Glessner, M., Salat, D., Hahn, H. K., & Fischl, B. (2004). A hybrid approach to the skull stripping problem in MRI. Neuroimage, 22, 1060–1075. Scholar
  46. Segonne, F., Pacheco, J., & Fischl, B. (2007). Geometrically accurate topology-correction of cortical surfaces using nonseparating loops. IEEE Transactions on Medical Imaging, 26, 518–529.
  47. Sepehrband, F., Lynch, K. M., Cabeen, R. P., Gonzalez-Zacarias, C., Zhao, L., D’Arcy, M., Kesselman, C., Herting, M. M., Dinov, I. D., & Toga, A. W. (2018). Neuroanatomical morphometric characterization of sex differences in youth using statistical learning. Neuroimage, 172, 217–227.
  48. Shah, A. D., Bartlett, J. W., Carpenter, J., Nicholas, O., & Hemingway, H. (2014). Comparison of random forest and parametric imputation models for imputing missing data using MICE: A CALIBER study. American Journal of Epidemiology, 179, 764–774. Scholar
  49. Sled, J. G., Zijdenbos, A. P., & Evans, A. C. (1998). A nonparametric method for automatic correction of intensity nonuniformity in MRI data. IEEE Transactions on Medical Imaging, 17, 87–97. Scholar
  50. Stekhoven, D. J., & Bühlmann, P. (2011). MissForest—Non-parametric missing value imputation for mixed-type data. Bioinformatics, 28, 112–118.
  51. Toga, A. W., Foster, I., Kesselman, C., Madduri, R., Chard, K., Deutsch, E. W., Price, N. D., Glusman, G., Heavner, B. D., Dinov, I. D., Ames, J., Van Horn, J., Kramer, R., & Hood, L. (2015). Big biomedical data as the key resource for discovery science. Journal of the American Medical Informatics Association, 22, 1126–1131. Scholar
  52. Torri, F., Dinov, I. D., Zamanyan, A., Hobel, S., Genco, A., Petrosyan, P., Clark, A. P., Liu, Z., Eggert, P., Pierce, J., Knowles, J. A., Ames, J., Kesselman, C., Toga, A. W., Potkin, S. G., Vawter, M. P., & Macciardi, F. (2012). Next generation sequence analysis and computational genomics using graphical pipeline workflows. Genes (Basel), 3, 545–575. Scholar
  53. Tustison, N. J., Cook, P. A., Klein, A., Song, G., Das, S. R., Duda, J. T., Kandel, B. M., van Strien, N., Stone, J. R., Gee, J. C., & Avants, B. B. (2014). Large-scale evaluation of ANTs and FreeSurfer cortical thickness measurements. Neuroimage, 99, 166–179. Scholar
  54. van Buuren, S., & Groothuis-Oudshoorn, K. (2010). Mice: Multivariate imputation by chained equations in R. Journal of Statistical Software, 1–68.Google Scholar
  55. Vijayakumar, N., Allen, N. B., Youssef, G., Dennison, M., Yucel, M., Simmons, J. G., & Whittle, S. (2016). Brain development during adolescence: A mixed-longitudinal investigation of cortical thickness, surface area, and volume. Human Brain Mapping, 37, 2027–2038. Scholar
  56. Waljee, A. K., Mukherjee, A., Singal, A. G., Zhang, Y., Warren, J., Balis, U., Marrero, J., Zhu, J., & Higgins, P. D. (2013). Comparison of imputation methods for missing laboratory data in medicine. BMJ Open, 3, e002847. Scholar
  57. Waters, A.B., Mace, R.A., Sawyer, K.S., & Gansler, D. A. (2018). Identifying errors in Freesurfer automated skull stripping and the incremental utility of manual intervention. Brain imaging and behavior, 1-11.
  58. Wiest-Daesslé, N., Prima, S., Coupé, P., Morrissey, S.P., Barillot, C., 2008. Rician noise removal by non-local means filtering for low signal-to-noise ratio MRI: Applications to DT-MRI, in: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics).

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Communication Sciences and DisordersCalifornia State UniversityFullertonUSA
  2. 2.Public Health Graduate ProgramUniversity of California MercedMercedUSA
  3. 3.Laboratory of Neuro Imaging, USC Mark and Mary Stevens Neuroimaging and Informatics Institute, Keck School of Medicine of USCUniversity of Southern CaliforniaLos AngelesUSA
  4. 4.Statistics Online Computational Resource, Department of Health Behavior and Biological, Michigan Institute for Data ScienceUniversity of MichiganAnn ArborUSA
  5. 5.Department of Preventive Medicine, Keck School of Medicine of USCUniversity of Southern CaliforniaLos AngelesUSA
  6. 6.Department of Pediatrics, Keck School of Medicine of USCUniversity of Southern CaliforniaLos AngelesUSA
  7. 7.Neuroscience Graduate ProgramUniversity of Southern CaliforniaLos AngelesUSA

Personalised recommendations