Abstract
In genetics, aggregation of many loci with small effect sizes into a single score improved prediction. Nevertheless, studies applying easily replicable weighted scores to neuroimaging data are lacking. Our aim was to assess the reliability and validity of the Neuroimaging Association Score (NAS), which combines information from structural brain features previously linked to mental disorders. Participants were 726 youth (aged 6–14) from two cities in Brazil who underwent MRI and psychopathology assessment at baseline and 387 at 3-year follow-up. Results were replicated in two samples: IMAGEN (n = 1627) and the Healthy Brain Network (n = 843). NAS were derived by summing the product of each standardized brain feature by the effect size of the association of that brain feature with seven psychiatric disorders documented by previous meta-analyses. NAS were calculated for surface area, cortical thickness and subcortical volumes using T1-weighted scans. NAS reliability, temporal stability and psychopathology and cognition prediction were analyzed. NAS for surface area showed high internal consistency and 3-year stability and predicted general psychopathology and cognition with higher replicability than specific symptomatic domains for all samples. They also predicted general psychopathology with higher replicability than single structures alone, accounting for 1–3% of the variance, but without directionality. The NAS for cortical thickness and subcortical volumes showed lower internal consistency and less replicable associations with behavioural phenotypes. These findings indicate the NAS based on surface area might be replicable markers of general psychopathology, but these links are unlikely to be causal or clinically useful yet.
Similar content being viewed by others
References
Torkamani A, Wineinger NE, Topol EJ (2018) The personal and clinical utility of polygenic risk scores. Nat Rev Genet 19:581–590. https://doi.org/10.1038/s41576-018-0018-x
Kapur S, Phillips AG, Insel TR (2012) Why has it taken so long for biological psychiatry to develop clinical tests and what to do about it? Mol Psychiatry 17:1174–1179. https://doi.org/10.1038/mp.2012.105
Milham MP, Craddock RC, Klein A (2017) Clinically useful brain imaging for neuropsychiatry: How can we get there? Depress Anxiety 34:578–587. https://doi.org/10.1002/da.22627
Woo C-W, Chang LJ, Lindquist MA, Wager TD (2017) Building better biomarkers: brain models in translational neuroimaging. Nat Neurosci 20:365–377. https://doi.org/10.1038/nn.4478
Boekel W, Wagenmakers E-J, Belay L et al (2015) A purely confirmatory replication study of structural brain-behavior correlations. Cortex 66:115–133. https://doi.org/10.1016/j.cortex.2014.11.019
Thompson PM, Andreassen OA, Arias-Vasquez A et al (2017) ENIGMA and the individual: predicting factors that affect the brain in 35 countries worldwide. Neuroimage 145:389–408. https://doi.org/10.1016/j.neuroimage.2015.11.057
van Erp TGM, Walton E, Hibar DP et al (2018) Cortical brain abnormalities in 4474 individuals with schizophrenia and 5098 control subjects via the enhancing neuro imaging genetics through Meta Analysis (ENIGMA) Consortium. Biol Psychiatry 84:644–654. https://doi.org/10.1016/j.biopsych.2018.04.023
van Erp TGM, Hibar DP, Rasmussen JM et al (2016) Subcortical brain volume abnormalities in 2028 individuals with schizophrenia and 2540 healthy controls via the ENIGMA consortium. Mol Psychiatry 21:547–553. https://doi.org/10.1038/mp.2015.63
Schmaal L, Hibar DP, Samann PG et al (2017) Cortical abnormalities in adults and adolescents with major depression based on brain scans from 20 cohorts worldwide in the ENIGMA Major Depressive Disorder Working Group. Mol Psychiatry 22:900–909. https://doi.org/10.1038/mp.2016.60
Schmaal L, Veltman DJ, van Erp TGM et al (2016) Subcortical brain alterations in major depressive disorder: findings from the ENIGMA Major Depressive Disorder working group. Mol Psychiatry 21:806–812. https://doi.org/10.1038/mp.2015.69
Renteria ME, Schmaal L, Hibar DP et al (2017) Subcortical brain structure and suicidal behaviour in major depressive disorder: a meta-analysis from the ENIGMA-MDD working group. Transl Psychiatry 7:e1116. https://doi.org/10.1038/tp.2017.84
Hibar DP, Westlye LT, Doan NT et al (2018) Cortical abnormalities in bipolar disorder: an MRI analysis of 6503 individuals from the ENIGMA Bipolar Disorder Working Group. Mol Psychiatry 23:932–942. https://doi.org/10.1038/mp.2017.73
Boedhoe PSW, Schmaal L, Abe Y et al (2018) Cortical abnormalities associated with pediatric and adult obsessive-compulsive disorder: findings from the ENIGMA Obsessive-Compulsive Disorder Working Group. Am J Psychiatry 175:453–462. https://doi.org/10.1176/appi.ajp.2017.17050485
Boedhoe PSW, Schmaal L, Abe Y et al (2017) Distinct subcortical volume alterations in pediatric and adult OCD: a worldwide meta- and mega-analysis. Am J Psychiatry 174:60–69. https://doi.org/10.1176/appi.ajp.2016.16020201
Mackey S, Allgaier N, Chaarani B et al (2019) Mega-analysis of gray matter volume in substance dependence: general and substance-specific regional effects. Am J Psychiatry 176:119–128. https://doi.org/10.1176/appi.ajp.2018.17040415
van Rooij D, Anagnostou E, Arango C et al (2018) Cortical and subcortical brain morphometry differences between patients with autism spectrum disorder and healthy individuals across the lifespan: results from the ENIGMA ASD Working Group. Am J Psychiatry 175:359–369. https://doi.org/10.1176/appi.ajp.2017.17010100
Hoogman M, Bralten J, Hibar DP et al (2017) Subcortical brain volume differences in participants with attention deficit hyperactivity disorder in children and adults: a cross-sectional mega-analysis. Lancet Psychiatry 4:310–319. https://doi.org/10.1016/S2215-0366(17)30049-4
Hoogman M, Muetzel R, Guimaraes JP et al (2019) Brain imaging of the cortex in ADHD: a coordinated analysis of large-scale clinical and population-based samples. Am J Psychiatry 176:531–542. https://doi.org/10.1176/appi.ajp.2019.18091033
Chatterjee N, Shi J, Garcia-Closas M (2016) Developing and evaluating polygenic risk prediction models for stratified disease prevention. Nat Rev Genet 17:392–406. https://doi.org/10.1038/nrg.2016.27
Martin AR, Daly MJ, Robinson EB et al (2019) Predicting polygenic risk of psychiatric disorders. Biol Psychiatry 86:97–109. https://doi.org/10.1016/j.biopsych.2018.12.015
Ripke S, Neale BM, Corvin A et al (2014) Biological insights from 108 schizophrenia-associated genetic loci. Nature 511:421–427. https://doi.org/10.1038/nature13595
Craig JE, Han X, Qassim A et al (2020) Multitrait analysis of glaucoma identifies new risk loci and enables polygenic prediction of disease susceptibility and progression. Nat Genet. https://doi.org/10.1038/s41588-019-0556-y
Lambert SA, Abraham G, Inouye M (2019) Towards clinical utility of polygenic risk scores. Hum Mol Genet 28:R133–R142. https://doi.org/10.1093/hmg/ddz187
Koutsouleris N, Meisenzahl EM, Davatzikos C et al (2009) Use of neuroanatomical pattern classification to identify subjects in at-risk mental states of psychosis and predict disease transition. Arch Gen Psychiatry 66:700–712. https://doi.org/10.1001/archgenpsychiatry.2009.62
Clementz BA, Sweeney JA, Hamm JP et al (2016) Identification of distinct psychosis biotypes using brain-based biomarkers. Am J Psychiatry 173:373–384. https://doi.org/10.1176/appi.ajp.2015.14091200
Schmaal L, Marquand AF, Rhebergen D et al (2015) Predicting the naturalistic course of major depressive disorder using clinical and multimodal neuroimaging information: a multivariate pattern recognition study. Biol Psychiatry 78:278–286. https://doi.org/10.1016/j.biopsych.2014.11.018
Squarcina L, Castellani U, Bellani M et al (2017) Classification of first-episode psychosis in a large cohort of patients using support vector machine and multiple kernel learning techniques. Neuroimage 145:238–245. https://doi.org/10.1016/j.neuroimage.2015.12.007
Steele VR, Rao V, Calhoun VD, Kiehl KA (2017) Machine learning of structural magnetic resonance imaging predicts psychopathic traits in adolescent offenders. Neuroimage 145:265–273. https://doi.org/10.1016/j.neuroimage.2015.12.013
Salum GA, Gadelha A, Pan PM et al (2015) High risk cohort study for psychiatric disorders in childhood: rationale, design, methods and preliminary results. Int J Methods Psychiatr Res 24:58–73. https://doi.org/10.1002/mpr.1459
Desikan RS, Cabral HJ, Hess CP et al (2009) Automated MRI measures identify individuals with mild cognitive impairment and Alzheimer’s disease. Brain 132:2048–2057. https://doi.org/10.1093/brain/awp123
Fischl B, Salat DH, Busa E et al (2002) Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain. Neuron 33:341–355. https://doi.org/10.1016/s0896-6273(02)00569-x
Zugman A, Harrewijn A, Cardinale EM et al (2020) Mega-analysis methods in ENIGMA: The experience of the generalized anxiety disorder working group. Hum Brain Map. https://doi.org/10.1002/hbm.25096
Goodman R, Ford T, Richards H et al (2000) The Development and Well-Being Assessment: description and initial validation of an integrated assessment of child and adolescent psychopathology. J Child Psychol Psychiatry 41:645–655
Achenbach TM (1991) Manual for the child behavior checklist/4-18 and 1991 profile. Department of Psychiatry, University of Vermont, Burlington, VT, USA
Martel MM, Pan PM, Hoffmann MS et al (2017) A general psychopathology factor (P factor) in children: structural model analysis and external validation through familial risk and child global executive function. J Abnorm Psychol 126:137–148. https://doi.org/10.1037/abn0000205
Vleeschouwer M, Schubart CD, Henquet C et al (2014) Does assessment type matter? A measurement invariance analysis of online and paper and pencil assessment of the Community Assessment of Psychic Experiences (CAPE). PLoS ONE 9:e84011. https://doi.org/10.1371/journal.pone.0084011
Liddle EB, Batty MJ, Goodman R (2009) The Social Aptitudes Scale: an initial validation. Soc Psychiatry Psychiatr Epidemiol 44:508–513. https://doi.org/10.1007/s00127-008-0456-4
Wechsler D, Simões M, Ferreira C (2002) WISC-III: escala de inteligência de Wechsler para crianças III
Vandierendonck A, Kemps E, Fastame MC, Szmalec A (2004) Working memory components of the Corsi blocks task. Br J Psychol 95:57–79. https://doi.org/10.1348/000712604322779460
Hogan AM, Vargha-Khadem F, Kirkham FJ, Baldeweg T (2005) Maturation of action monitoring from adolescence to adulthood: an ERP study. Dev Sci 8:525–534. https://doi.org/10.1111/j.1467-7687.2005.00444.x
Bitsakou P, Psychogiou L, Thompson M, Sonuga-Barke EJS (2008) Inhibitory deficits in attention-deficit/hyperactivity disorder are independent of basic processing efficiency and IQ. J Neural Transm 115:261–268
Alexander LM, Escalera J, Ai L et al (2017) An open resource for transdiagnostic research in pediatric mental health and learning disorders. Sci Data 4:170181. https://doi.org/10.1038/sdata.2017.181
Schumann G, Loth E, Banaschewski T et al (2010) The IMAGEN study: reinforcement-related behaviour in normal brain function and psychopathology. Mol Psychiatry 15:1128–1139. https://doi.org/10.1038/mp.2010.4
Goodman R (1997) The strengths and difficulties questionnaire: a research note. J Child Psychol Psychiatry 38:581–586. https://doi.org/10.1111/j.1469-7610.1997.tb01545.x
George D, Mallery P (2003) SPSS for Windows step by step: a simple guide and reference. 11.0 update. wps. ablongman. com/wps/media/objects/385. George 4answers pdf
Koo TK, Li MY (2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 15:155–163. https://doi.org/10.1016/j.jcm.2016.02.012
Tamnes CK, Herting MM, Goddings A-L et al (2017) Development of the cerebral cortex across adolescence: a multisample study of inter-related longitudinal changes in cortical volume, surface area, and thickness. J Neurosci 37:3402–3412. https://doi.org/10.1523/JNEUROSCI.3302-16.2017
Caspi A, Houts RM, Belsky DW et al (2014) The p factor: one general psychopathology factor in the structure of psychiatric disorders? Clin Psychol Sci 2:119–137. https://doi.org/10.1177/2167702613497473
Muetzel RL, Blanken LME, van der Ende J et al (2018) Tracking brain development and dimensional psychiatric symptoms in children: a longitudinal population-based neuroimaging study. Am J Psychiatry 175:54–62. https://doi.org/10.1176/appi.ajp.2017.16070813
Insel T, Cuthbert B, Garvey M et al (2010) Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. Am J Psychiatry 167:748–751. https://doi.org/10.1176/appi.ajp.2010.09091379
Frisch S (2016) Are mental disorders brain diseases, and what does this mean? A clinical-neuropsychological perspective. Psychopathology 49:135–142. https://doi.org/10.1159/000447359
Acknowledgements
We thank the children and families from the Brazilian High-Risk Study for their participation, which made this research possible.
Funding
This work was funded through research grants by the Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq, Brazil; grant number 573974/2008–0), the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES, Brazil), the Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP, Brazil; grant number 2008/57896–8) and the Fundação de Amparo à Pesquisa do Estado do Rio Grande do Sul (FAPERGS, Brazil). All of them are public institutions of the Brazilian government developed for scientific research support. It was also funded by the National Institute of Mental Health (grant number 1-R01-MH-120482–01). Funding sources have no involvement in this study, including no role in data collection, analysis, and interpretation of the data. IMAGEN consortium is composed of Tobias Banaschewski, M.D., Ph.D.; Gareth J. Barker, Ph.D.; Arun L.W. Bokde, Ph.D.; Erin Burke Quinlan, PhD; Sylvane Desrivières, Ph.D.; Herta Flor, Ph.D.; Antoine Grigis, Ph.D.; Hugh Garavan, Ph.D.; Penny Gowland, Ph.D.; Andreas Heinz, M.D., Ph.D.; Bernd Ittermann, Ph.D.; Jean-Luc Martinot, M.D., Ph.D., Marie-Laure Paillère Martinot, M.D., Ph.D., Eric Artiges, M.D., Ph.D.; Frauke Nees, Ph.D.; Dimitri Papadopoulos Orfanos, Ph.D., Herve Lemaitre, Ph.D; Tomáš Paus, M.D., Ph.D.; Luise Poustka, M.D., Sarah Hohmann, M.D, Sabina Millenet, PhD; Juliane H. Fröhner, MSc; Michael N. Smolka, M.D; Henrik Walter, M.D., Ph.D.; Robert Whelan, Ph.D. and Gunter Schumann, M.D. IMAGEN work received support from the following sources: the European Union-funded FP6 Integrated Project IMAGEN (Reinforcement-related behaviour in normal brain function and psychopathology) (LSHM-CT- 2007–037286), the Horizon 2020 funded ERC Advanced Grant ‘STRATIFY’ (Brain network based stratification of reinforcement-related disorders) (695313), ERANID (Understanding the Interplay between Cultural, Biological and Subjective Factors in Drug Use Pathways) (PR-ST-0416-10004), BRIDGET (JPND: Brain Imaging, cognition Dementia and next generation Genomics) (MR/N027558/1), Human Brain Project (HBP SGA 2, 785907), the FP7 project MATRICS (603016), the Medical Research Council Grant ‘c-VEDA’ (Consortium on Vulnerability to Externalizing Disorders and Addictions) (MR/N000390/1), the National Institute for Health Research (NIHR) Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King’s College London, the Bundesministeriumfür Bildung und Forschung (BMBF grants 01GS08152; 01EV0711; Forschungsnetz AERIAL 01EE1406A, 01EE1406B), the Deutsche Forschungsgemeinschaft (DFG grants SM 80/7-2, SFB 940, TRR 265, NE 1383/14-1), the Medical Research Foundation and Medical Research Council (grants MR/R00465X/1 and MR/S020306/1), the National Institutes of Health (NIH) funded ENIGMA (grants 5U54EB020403-05 and 1R56AG058854-01). Further support was provided by grants from: the ANR (ANR-12-SAMA-0004, AAPG2019—GeBra), the Eranet Neuron (AF12-NEUR0008-01—WM2NA; and ANR-18-NEUR00002-01—ADORe), the Fondation de France (00081242), the Fondation pour la Recherche Médicale (DPA20140629802), the Mission Interministérielle de Lutte-contre-les-Drogues-et-les-Conduites-Addictives (MILDECA), the Assistance-Publique-Hôpitaux-de-Paris and INSERM (interface grant), Paris Sud University IDEX 2012, the Fondation de l’Avenir (Grant AP-RM-17-013), the Fédération pour la Recherche sur le Cerveau; the National Institutes of Health, Science Foundation Ireland (16/ERCD/3797), U.S.A. (Axon, Testosterone and Mental Health during Adolescence; RO1 MH085772-01A1), and by NIH Consortium grant U54 EB020403, supported by a cross-NIH alliance that funds Big Data to Knowledge Centres of Excellence. Further support was provided by grants from: –the ANR (ANR-12-SAMA-0004, AAPG2019—GeBra), the Eranet Neuron (AF12-NEUR0008-01—WM2NA; and ANR-18-NEUR00002-01—ADORe), the Fondation de France (00,081,242), the Fondation pour la Recherche Médicale (DPA20140629802), the Mission Interministérielle de Lutte-contre-les-Drogues-et-les-Conduites-Addictives (MILDECA), the Assistance-Publique-Hôpitaux-de-Paris and INSERM (interface grant), Paris Sud University IDEX 2012, the Fondation de l’Avenir (grant AP-RM-17–013), the Fédération pour la Recherche sur le Cerveau. The Healthy Brain Network (https://www.healthybrainnetwork.org) work was supported by philanthropic contributions from the following individuals, foundations, and organizations: Lee Alexander, Robert Allard, Lisa Bilotti Foundation, Inc., Margaret Billoti, Christopher Boles, Brooklyn Nets, Agapi and Bruce Burkhard, Randolph Cowen and Phyllis Green, Elizabeth and David DePaolo, Charlotte Ford, Valesca Guerrand-Hermes, Sarah and Geoffrey Gund, George Hall, Joseph Healey and Elaine Thomas, Hearst Foundations, Eve and Ross Joffe, Anton and Robin Katz, Rachael and Marshall Levine, Ke Li, Jessica Lupovici, Javier Macaya, Christine and Richard Mack, Susan Miller and Byron Grote, John and Amy Phelan, Linnea and George Roberts, Jim and Linda Robinson Foundation, Inc, Caren and Barry Roseman, Zibby Schwarzman, David Shapiro and Abby Pogrebin, Stavros Niarchos Foundation, Nicholas Van Dusen, David Wolkoff and Stephanie Winston Wolkoff, and the Donors to the Brant Art Auction of 2012. MPM is a Randolph Cowen and Phyllis Green Scholar.
Author information
Authors and Affiliations
Consortia
Corresponding author
Ethics declarations
Conflict of interest
Ms. Axelrud, Dr. Simioni, Dr. Pine, Dr. Winkler, Dr. Barker, Dr. Sato, Dr. Zugman, Dr. Parker, Dr. Picon, Dr. Jackowski, Dr. Hoexter, Dr Satterthwaite, Dr. Barker, Dr. JL Martinot, Dr. MLP Martinot, Dr Milham and Dr. Salum report no biomedical financial interests or potential conflicts of interest. Dr. Pan has received payment for the development of educational material for Janssen-Cilag and Astra-Zeneca. Dr. Rohde has received Honoraria, has been on the speakers' bureau/advisory board and/or has acted as a consultant for Eli-Lilly, Janssen-Cilag, Novartis and Shire in the last 2 years. He receives authorship royalties from Oxford Press and ArtMed. He also received travel awards for taking part of 2014 APA and 2015 WFADHD meetings from Shire. The ADHD and Juvenile Bipolar Disorder Outpatient Programs chaired by him received unrestricted educational and research support from the following pharmaceutical companies in the last three years: Eli-Lilly, Janssen-Cilag, Novartis, and Shire. Regarding the IMAGEN Consortium, Dr. Banaschewski served in an advisory or consultancy role for Lundbeck, Medice, Neurim Pharmaceuticals, Oberberg GmbH, Shire. He received conference support or speaker’s fee by Lilly, Medice, Novartis and Shire. He has been involved in clinical trials conducted by Shire & Viforpharma. He received royalties from Hogrefe, Kohlhammer, CIP Medien, Oxford University Press. The other authors report no biomedical financial interests or potential conflicts of interest.
Ethical approval
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. The Ethics Committee of the University of São Paulo approved the study. Parents of the participants and participants provided written or verbal consent.
Additional information
The members of the IMAGEN Consortium group are mentioned in "Acknowledgements" section.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Axelrud, L.K., Simioni, A.R., Pine, D.S. et al. Neuroimaging Association Scores: reliability and validity of aggregate measures of brain structural features linked to mental disorders in youth. Eur Child Adolesc Psychiatry 30, 1895–1906 (2021). https://doi.org/10.1007/s00787-020-01653-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00787-020-01653-x