Abstract
High dimensional neuroimaging datasets and machine learning have been used to estimate and predict domain-specific cognition, but comparisons with simpler models composed of easy-to-measure variables are limited. Regularization methods in particular may help identify regions-of-interest related to domain-specific cognition. Using data from the Northern Manhattan Study, a cohort study of mostly Hispanic older adults, we compared three models estimating domain-specific cognitive performance: sociodemographics and APOE ε4 allele status (basic model), the basic model and MRI markers, and a model with only MRI markers. We used several machine learning methods to fit our regression models: elastic net, support vector regression, random forest, and principal components regression. Model performance was assessed with the RMSE, MAE, and R2 statistics using 5-fold cross-validation. To assess whether prediction models with imaging biomarkers were more predictive than prediction models built with randomly generated biomarkers, we refit the elastic net models using 1000 datasets with random biomarkers and compared the distribution of the RMSE and R2 in models using these random biomarkers to the RMSE and R2 from observed models. Basic models explained ~ 31–38% of the variance in domain-specific cognition. Addition of MRI markers did not improve estimation. However, elastic net models with only MRI markers performed significantly better than random MRI markers (one-sided P < .05) and yielded regions-of-interest consistent with previous literature and others not previously explored. Therefore, structural brain MRI markers may be more useful for etiological than predictive modeling.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Andersson, J., Smith, S., & Jenkinson, M. (2008). Fnirt—fmrib’s non-linear image registration tool. Annual Meeting of the Organization Hum Brain Mapp..
Brickman, A. M., Tosto, G., Gutierrez, J., Andrews, H., Gu, Y., Narkhede, A., … Mayeux, R. (2018). An MRI measure of degenerative and cerebrovascular pathology in Alzheimer disease. Neurology, 91(15), e1402–e1412. https://doi.org/10.1212/WNL.0000000000006310.
Bunea, F., She, Y., Ombao, H., Gongvatana, A., Devlin, K., & Cohen, R. (2011). Penalized least squares regression methods and applications to neuroimaging. Neuroimage, 55(4), 1519–1527. https://doi.org/10.1016/j.neuroimage.2010.12.028.
Casanova, R., Barnard, R. T., Gaussoin, S. A., Saldana, S., Hayden, K. M., Manson, J. E., … Chen, J.-C. (2018). Using high-dimensional machine learning methods to estimate an anatomical risk factor for Alzheimer’s disease across imaging databases. NeuroImage, 183, 401–411. https://doi.org/10.1016/j.neuroimage.2018.08.040.
D’Elia, L. F., Satz, P., Uchiyama, C. L., & White, T. (1996). Color trails test professional manual.
Dale, A. M., Fischl, B., & Sereno, M. I. (1999). Cortical surface-based analysis: I. segmentation and surface reconstruction. Neuroimage, 9, 179–194.
Debette, S., Schilling, S., Duperron, M.-G., Larsson, S. C., & Markus, H. S. (2018). Clinical significance of magnetic resonance imaging markers of vascular brain injury: a systematic review and meta-analysis. JAMA Neurology. https://doi.org/10.1001/jamaneurol.2018.3122.
DeCarli, C., Fletcher, E., Ramey, V., Harvey, D., & Jagust, W. J. (2005). Anatomical mapping of white matter hyperintensities (WMH): exploring the relationships between periventricular WMH, deep WMH, and total WMH burden. Stroke, 36(1), 50–55. https://doi.org/10.1161/01.STR.0000150668.58689.f2.
Desikan, R. S., Segonne, F., Fischl, B., Quinn, B. T., Dickerson, B. C., Blacker, D., … Killiany, R. J. (2006). An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage, 31(3), 968–980. https://doi.org/10.1016/j.neuroimage.2006.01.021.
Dhamoon, M. S., Cheung, Y.-K., Bagci, A., Alperin, N., Sacco, R. L., Elkind, M. S. V., & Wright, C. B. (2018). Periventricular white matter hyperintensities and functional decline. Journal of the American Geriatrics Society, 66(1), 113–119. https://doi.org/10.1111/jgs.15149.
Dong, C., Nabizadeh, N., Caunca, M., Cheung, Y. K., Rundek, T., Elkind, M. S., … Wright, C. B. (2015). Cognitive correlates of white matter lesion load and brain atrophy: the Northern Manhattan Study. Neurology, 85(5), 441–449. https://doi.org/10.1212/wnl.0000000000001716.
Duchesne, S., Caroli, A., Geroldi, C., Collins, D. L., & Frisoni, G. B. (2009). Relating one-year cognitive change in mild cognitive impairment to baseline MRI features. NeuroImage, 47(4), 1363–1370. https://doi.org/10.1016/j.neuroimage.2009.04.023.
Fischl, B., Salat, D. H., Busa, E., Albert, M., Dieterich, M., Haselgrove, C., … Dale, A. M. (2002). Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain. Neuron, 33, 341–355.
Flanagan, J. L., & Jackson, S. T. (1997). Test-retest reliability of three aphasia tests: performance of non-brain-damaged older adults. Journal of Communication Disorders, 30(1), 33–42.
Flowers, K. A., & Robertson, C. (1985). The effect of Parkinson’s disease on the ability to maintain a mental set. Journal of Neurology, Neurosurgery, and Psychiatry, 48(6), 517–529.
Frisoni, G. B., Fox, N. C., Jack, C. R. Jr., Scheltens, P., & Thompson, P. M. (2010). The clinical use of structural MRI in Alzheimer disease. Nature Reviews. Neurology, 6(2), 67–77. https://doi.org/10.1038/nrneurol.2009.215.
Harrison, J. E., Buxton, P., Husain, M., & Wise, R. (2000). Short test of semantic and phonological fluency: normal performance, validity and test-retest reliability. The British Journal of Clinical Psychology, 39(Pt 2), 181–191.
Hixson, J. E., & Vernier, D. T. (1990). Restriction isotyping of human apolipprotein E by gene amplification and cleavage with HhaI. Journal of Lipid Research, 31, 545–548.
Kuhn, M. (2008). Building predictive models in R using the caret package. Journal of Statistical Software, 28(5), 1–26.
Liu, X., Cao, P., Wang, J., Kong, J., & Zhao, D. (2018). Fused group lasso regularized multi-task feature learning and its application to the cognitive performance prediction of Alzheimer’s Disease. Neuroinformatics. https://doi.org/10.1007/s12021-018-9398-5.
Lockhart, S. N., & DeCarli, C. (2014). Structural imaging measures of brain aging. Neuropsychology Review, 24(3), 271–289. https://doi.org/10.1007/s11065-014-9268-3.
Mateos-Perez, J. M., Dadar, M., Lacalle-Aurioles, M., Iturria-Medina, Y., Zeighami, Y., & Evans, A. C. (2018). Structural neuroimaging as clinical predictor: A review of machine learning applications. NeuroImage. Clinical, 20, 506–522. https://doi.org/10.1016/j.nicl.2018.08.019.
Matthews, C. G., & Klove, H. (1964). Instruction manual for the adult neuropsychology test battery.
Mayeda, E. R., Glymour, M. M., Quesenberry, C. P., & Whitmer, R. A. (2016). Inequalities in dementia incidence between six racial and ethnic groups over 14 years. Alzheimers Dement, 12(3), 216–224. https://doi.org/10.1016/j.jalz.2015.12.007.
Oliveros, J. C. (n.d.), Venny. An interactive tool for comparing lists with Venn’s diagrams. Retrieved December 14, 2018, from http://bioinfogp.cnb.csic.es/tools/venny/index.html.
Sacco, R. L., Anand, K., Lee, H. S., Boden-Albala, B., Stabler, S., Allen, R., & Paik, M. C. (2004). Homocysteine and the risk of ischemic stroke in a triethnic cohort: the Northern Manhattan Study. Stroke, 35(10), 2263–2269. https://doi.org/10.1161/01.str.0000142374.33919.92.
Siedlecki, K. L., Stern, Y., Reuben, A., Sacco, R. L., Elkind, M. S., & Wright, C. B. (2009). Construct validity of cognitive reserve in a multiethnic cohort: The Northern Manhattan Study. Journal of the International Neuropsychological Society, 15(4), 558–569. https://doi.org/10.1017/S1355617709090857.
Siedlecki, K. L., Rundek, T., Elkind, M. S., Sacco, R. L., Stern, Y., & Wright, C. B. (2012). Using contextual analyses to examine the meaning of neuropsychological variables across samples of english-speaking and spanish-speaking older adults. Journal of the International Neuropsychological Society, 18(2), 223–233. https://doi.org/10.1017/S135561771100155X.
Smith, S. M. (2002). Fast robust automated brain extraction. Human Brain Mapping, 17(3), 143–155. https://doi.org/10.1002/hbm.10062.
Smith, S. M., & Nichols, T. E. (2018). Statistical Challenges in “Big Data” Human Neuroimaging. Neuron, 97(2), 263–268. https://doi.org/10.1016/j.neuron.2017.12.018.
Stonnington, C. M., Chu, C., Kloppel, S., Jack, C. R. J., Ashburner, J., & Frackowiak, R. S. J. (2010). Predicting clinical scores from magnetic resonance scans in Alzheimer’s disease. NeuroImage, 51(4), 1405–1413. https://doi.org/10.1016/j.neuroimage.2010.03.051.
Taylor, J., & Tibshirani, R. J. (2015). Statistical learning and selective inference. Proceedings of the National Academy of Sciences of the United States of America, 112(25), 7629–7634. https://doi.org/10.1073/pnas.1507583112.
Ten Kate, M., Ingala, S., Schwarz, A. J., Fox, N. C., Chetelat, G., van Berckel, B. N. M., … Barkhof, F. (2018). Secondary prevention of Alzheimer’s dementia: neuroimaging contributions. Alzheimer’s Research & Therapy, 10(1), 112. https://doi.org/10.1186/s13195-018-0438-z.
Wakana, S., Caprihan, A., Panzenboeck, M. M., Fallon, J. H., Perry, M., Gollub, R. L., … Mori, S. (2007). Reproducibility of quantitative tractography methods applied to cerebral white matter. NeuroImage, 36(3), 630–644. https://doi.org/10.1016/j.neuroimage.2007.02.049.
Wang, Y., Fan, Y., Bhatt, P., & Davatzikos, C. (2010). High-dimensional pattern regression using machine learning: from medical images to continuous clinical variables. NeuroImage, 50(4), 1519–1535. https://doi.org/10.1016/j.neuroimage.2009.12.092.
Wickham, H. (2009). ggplot2: elegant graphics for data analysis.
Zhang, D., & Shen, D. (2012). Multi-modal multi-task learning for joint prediction of multiple regression and classification variables in Alzheimer’s disease. NeuroImage, 59(2), 895–907. https://doi.org/10.1016/j.neuroimage.2011.09.069.
Zhang, Y., Brady, M., & Smith, S. (2001). Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm. IEEE Transactions on Medical Imaging, 20(1), 45–57. https://doi.org/10.1109/42.906424.
Acknowledgements
We would like to thank the NOMAS participants for their time and effort and our project manager, Janet De Rosa, MPH.
Funding
This work was supported by the National Institutes of Neurological Disease and Stroke (R01 NS29993, F30 NS103462) and the Evelyn F. McKnight Brain Institute.
Author information
Authors and Affiliations
Contributions
Author contributions included conception and study design (MRC, LW), data collection or acquisition (YKC, NA, SHL, MSVE, RLS, CBW, TR), statistical analysis (MRC, LW), interpretation of results (all authors), drafting of manuscript (MRC), revising manuscript for important intellectual content (all authors), and approval of final version (all authors).
Corresponding author
Ethics declarations
Participants provided written informed consent, and the institutional review boards at the University of Miami and Columbia University approved this study.
Conflicts of interest
Dr. Mitchell S.V. Elkind receives compensation for providing consultative services for the BMS-Pfizer Alliance for Eliquis, and Roche for a clinical trial of stroke prevention; has given expert legal opinions on behalf of Organon (NuvaRing and stroke litigation), Auxilium (testosterone and stroke), and LivaNova (cardiac surgery and stroke); and serves on the National, Founders Affiliate, and New York City Chapter Boards of the American Heart Association/American Stroke Association. He receives royalties from UpToDate for chapters related to stroke.
Dr. Ralph L. Sacco receives federal grant support (R01 NS 29993, CTSA UL1 TR002736), private foundation support (American Heart Association Bugher Center), and pharma research support (Boehringer Ingelheim).
Dr. Clinton B. Wright receives royalties for 2 chapters on Vascular Dementia from UpToDate.
Drs. Michelle Caunca, Lily Wang, Ying Kuen Cheung, Noam Alperin, and Tatjana Rundek and Mr. Sang Lee report no disclosures.
Additional information
Publisher’s note
Springer Nature remains neutral with regard tojurisdictionalclaims in published maps and institutional affiliations.
Electronic supplementary material
ESM 1
(DOCX 297 kb)
Rights and permissions
About this article
Cite this article
Caunca, M.R., Wang, L., Cheung, Y.K. et al. Machine learning-based estimation of cognitive performance using regional brain MRI markers: the Northern Manhattan Study. Brain Imaging and Behavior 15, 1270–1278 (2021). https://doi.org/10.1007/s11682-020-00325-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11682-020-00325-3