Automatically computed rating scales from MRI for patients with cognitive disorders

  • Juha R. Koikkalainen
  • Hanneke F. M. Rhodius-Meester
  • Kristian S. Frederiksen
  • Marie Bruun
  • Steen G. Hasselbalch
  • Marta Baroni
  • Patrizia Mecocci
  • Ritva Vanninen
  • Anne Remes
  • Hilkka Soininen
  • Mark van Gils
  • Wiesje M. van der Flier
  • Philip Scheltens
  • Frederik Barkhof
  • Timo Erkinjuntti
  • Jyrki M. P. LötjönenEmail author
  • for the Alzheimer’s Disease Neuroimaging Initiative



The aims of this study were to examine whether visual MRI rating scales used in diagnostics of cognitive disorders can be estimated computationally and to compare the visual rating scales with their computed counterparts in differential diagnostics.


A set of volumetry and voxel-based morphometry imaging biomarkers was extracted from T1-weighted and FLAIR images. A regression model was developed for estimating visual rating scale values from a combination of imaging biomarkers. We studied three visual rating scales: medial temporal lobe atrophy (MTA), global cortical atrophy (GCA), and white matter hyperintensities (WMHs) measured by the Fazekas scale. Images and visual ratings from the Amsterdam Dementia Cohort (ADC) (N = 513) were used to develop the models and cross-validate them. The PredictND (N = 672) and ADNI (N = 752) cohorts were used for independent validation to test generalizability.


The correlation coefficients between visual and computed rating scale values were 0.83/0.78 (MTA-left), 0.83/0.79 (MTA-right), 0.64/0.64 (GCA), and 0.76/0.75 (Fazekas) in ADC/PredictND cohorts. When performance in differential diagnostics was studied for the main types of dementia, the highest balanced accuracy, 0.75–0.86, was observed for separating different dementias from cognitively normal subjects using computed GCA. The lowest accuracy of about 0.5 for all the visual and computed scales was observed for the differentiation between Alzheimer’s disease and frontotemporal lobar degeneration. Computed scales produced higher balanced accuracies than visual scales for MTA and GCA (statistically significant).


MTA, GCA, and WMHs can be reliably estimated automatically helping to provide consistent imaging biomarkers for diagnosing cognitive disorders, even among less experienced readers.

Key Points

Visual rating scales used in diagnostics of cognitive disorders can be estimated computationally from MRI images with intraclass correlations ranging from 0.64 (GCA) to 0.84 (MTA).

Computed scales provided high diagnostic accuracy with single-subject data (area under the receiver operating curve range, 0.84–0.94).


Magnetic resonance imaging Cognition disorders Atrophy 



Alzheimer’s disease


Amsterdam Dementia Cohort


Alzheimer’s Disease Neuroimaging Initiative


Balanced accuracy


Cognitively normal


Dementia with Lewy bodies


Fluid-attenuated inversion recovery


Frontotemporal lobar degeneration


Global cortical atrophy


Intraclass correlation coefficient


Medial temporal lobe atrophy on the left (MTA-L) and right (MTA-R)


Other dementias but AD, VaD, FTLD, and DLB


Vascular dementia


Voxel-based morphometry


White matter hyperintensities



This work has received funding from the European Union’s Seventh Framework Programme for research, technological development, and demonstration under grant agreement no. 611005 (PredictND) and no. 601055 (VPH-DARE@IT). Research of the VUmc Alzheimer Center is part of the neurodegeneration research program of the Neuroscience Campus Amsterdam. The VUmc Alzheimer Center is supported by Stichting Alzheimer Nederland and Stichting VUmc funds. The clinical database structure of the VUmc Alzheimer Center was developed with funding from Stichting Dioraphte. FB was supported by the NIHR-UCLH Biomedical Research Centre.

Data collection and sharing for this project was funded by the Alzheimer’s Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012). ADNI is funded by the National Institute on Aging and the National Institute of Biomedical Imaging and Bioengineering and through generous contributions from the following: AbbVie; Alzheimer’s Association; Alzheimer’s Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc.; Biogen; Bristol-Myers Squibb Company; CereSpir, Inc.; Cogstate; Eisai Inc.; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; EuroImmun; F. Hoffmann-La Roche Ltd. and its affiliated company Genentech, Inc.; Fujirebio; GE Healthcare; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research & Development LLC.; Lumosity; Lundbeck; Merck & Co., Inc.; Meso Scale Diagnostics, LLC.; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health ( The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer’s Therapeutic Research Institute at the University of Southern California. ADNI data are disseminated by the Laboratory for NeuroImaging at the University of Southern California.

Compliance with ethical standards


The scientific guarantor of this publication is Prof. Jyrki Lötjönen.

Conflict of interest

The authors of this manuscript declare relationships with the following companies: JK and JL are employees, co-founders, and shareholders of Combinostics Ltd. JL has given an educational presentation for Merck and Sanofi that paid to his institution.

Research programs of WF have been funded by ZonMW, NWO, EU-FP7, Alzheimer Nederland, CardioVascular Onderzoek Nederland, stichting Dioraphte, Gieskes-Strijbis fonds, Boehringer Ingelheim, Piramal Neuroimaging, Roche BV, Janssen Stellar, and Combinostics. WF has performed contract research for Boehringer Ingelheim. WF has been an invited speaker at Boehringer Ingelheim. All funding is paid to her institution.

PS has acquired grant support (for the institution) from GE Healthcare, Danone Research, Piramal, and MERCK. In the past 2 years, he has received consultancy/speaker fees (paid to the institution) from Lilly, GE Healthcare, Novartis, Probiodrug, Biogen, Roche, and EIP Pharma.

FB has consultancy payments from Biogen-Idec, TEVA, Merck-Serono, Novartis, Roche, Jansen Research, Genzyme-Sanofi, IXICO Ltd., GeNeuro, and Apitope Ltd. and payments for development of educational presentations from Biogen-IDEC and IXICO Ltd.

Statistics and biometry

One of the authors has significant statistical expertise (MvG).

Informed consent

Written informed consent was obtained from all subjects (patients) in this study.

Ethical approval

Institutional Review Board approval was obtained.

Study subjects or cohorts overlap

Some study subjects or cohorts have been previously reported in the following:

Amsterdam Dementia Cohort (ADC): van der Flier WM, Pijnenburg YAL, Prins N, et al (2014) Optimizing patient care and research: the Amsterdam Dementia Cohort. J Alzheimers Dis. 41:313–327

Cohort from the PredictND EU FP7 project (PredictND): Bruun M, Gjerum L, Frederiksen K, et al (2017) Data-driven diagnosis of dementia disorders: the PredictND validation study. Alzheimer’s & Dementia 13(7):405–407 (Supplement)

Alzheimer’s Disease Neuroimaging Initiative (ADNI): Petersen RC, Aisen P, Beckett L, et al (2010) Alzheimer’s Disease Neuroimaging Initiative (ADNI). Neurology 74(3):201–209


• retrospective

• cross-sectional study

• multicenter study

Supplementary material

330_2019_6067_MOESM1_ESM.docx (745 kb)
ESM 1 (DOCX 744 kb)


  1. 1.
    McKhann GM, Knopman DS, Chertkow H et al (2011) The diagnosis of dementia due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement (Amst) 7(3):263–269CrossRefGoogle Scholar
  2. 2.
    McKeith IG, Galasko D, Kosaka K et al (1996) Consensus guidelines for the clinical and pathologic diagnosis of dementia with Lewy bodies (DLB): report of the consortium on DLB international workshop. Neurology 47(5):1113–1124CrossRefPubMedGoogle Scholar
  3. 3.
    Neary D, Snowden JS, Gustafson L et al (1998) Frontotemporal lobar degeneration: a consensus on clinical diagnostic criteria. Neurology 51(6):1546–1554CrossRefPubMedGoogle Scholar
  4. 4.
    Rascovsky K, Hodges JR, Knopman D et al (2011) Sensitivity of revised diagnostic criteria for the behavioural variant of frontotemporal dementia. Brain 134(9):2456–2477CrossRefPubMedPubMedCentralGoogle Scholar
  5. 5.
    Román GC, Tatemichi TK, Erkinjuntti T et al (1993) Vascular dementia: diagnostic criteria for research studies. Report of the NINDS-AIREN International Workshop. Neurology 43(2):250–260CrossRefPubMedGoogle Scholar
  6. 6.
    Dubois B, Feldman HH, Jacova C (2007) Research criteria for the diagnosis of Alzheimer’s disease: revising the NINCDS-ADRDA criteria. Lancet Neurol 6(8):734–746CrossRefPubMedGoogle Scholar
  7. 7.
    Harper L, Fumagalli GG, Barkhof F et al (2016) MRI visual rating scales in the diagnosis of dementia: evaluation in 184 post-mortem confirmed cases. Brain 139:1211–1225CrossRefPubMedPubMedCentralGoogle Scholar
  8. 8.
    Koedam EL, Lehmann M, van der Flier WM et al (2011) Visual assessment of posterior atrophy development of a MRI rating scale. Eur Radiol 21(12):2618–2625CrossRefPubMedPubMedCentralGoogle Scholar
  9. 9.
    Fazekas F, Chawluk J, Alavi A, Hurtig H, Zimmerman R (1987) MR signal abnormalities at 1.5 T in Alzheimer’s dementia and normal aging. AJR Am J Roentgenol 149(2):351–356CrossRefPubMedGoogle Scholar
  10. 10.
    O’Brien JT, Erkinjuntti T, Reisberg B et al (2003) Vascular cognitive impairment. Lancet Neurol 2(2):89–98CrossRefPubMedGoogle Scholar
  11. 11.
    Pantoni L, Basile AM, Pracucci G et al (2005) Impact of age-related cerebral white matter changes on the transition to disability. The LADIS study: rationale, design, and methodology. Neuroepidemiology 24(1–2):51–62CrossRefPubMedGoogle Scholar
  12. 12.
    Rhodius-Meester HFM, Benedictus M, Wattjes M et al (2017) MRI visual ratings of brain atrophy and white matter hyperintensities across the spectrum of cognitive decline are differently affected by age and diagnosis. Front Aging Neurosci 9:117CrossRefPubMedPubMedCentralGoogle Scholar
  13. 13.
    Vernooij MW, Haller S, Frisoni G et al (2018) Dementia imaging in Europe: results from the European Society for Neuroradiology (ESNR) Diagnostic Subcommittee Survey. Available via Accessed 17 Aug 2018
  14. 14.
    Scheltens P, Launer LJ, Barkhof F, Weinstein H, van Gool W (1995) Visual assessment of medial temporal lobe atrophy on magnetic resonance imaging: interobserver reliability. J Neurol 242(9):557–560CrossRefPubMedGoogle Scholar
  15. 15.
    Pasquier F, Leys D, Weerts J, Mounier-Vehier F, Barkhof F, Scheltens P (1996) Inter and intraobserver reproducibility of cerebral atrophy assessment on MRI scans with hemispheric infarcts. Eur Neurol 36(5):268–272CrossRefPubMedGoogle Scholar
  16. 16.
    Torisson G, van Westen D, Stavenow L, Minthon L, Londos E (2015) Medial temporal lobe atrophy is underreported and may have important clinical correlates in medical inpatients. BMC Geriatr 15:65CrossRefPubMedPubMedCentralGoogle Scholar
  17. 17.
    van der Flier WM, Pijnenburg YAL, Prins N et al (2014) Optimizing patient care and research: the Amsterdam Dementia Cohort. J Alzheimers Dis 41:313–327CrossRefPubMedGoogle Scholar
  18. 18.
    McKhann G, Drachman D, Folstein M, Katzman R, Price D, Stadlan EM (1984) Clinical diagnosis of Alzheimer’s disease: report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer’s Disease. Neurology 34(7):939–944CrossRefPubMedGoogle Scholar
  19. 19.
    Petersen RC (2004) Mild cognitive impairment as a diagnostic entitity. J Intern Med 256(3):183–194CrossRefPubMedGoogle Scholar
  20. 20.
    Albert MS, DeKosky ST, Dickson D et al (2011) The diagnosis of mild cognitive impairment due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement (Amst) 7:270–279CrossRefGoogle Scholar
  21. 21.
    Lötjönen JM, Wolz R, Koikkalainen JR, Thurfjell L, Waldemar G, Soininen H, Rueckert D (2010) Fast and robust multi-atlas segmentation of brain magnetic resonance images. Neuroimage 49(3):2352–2365CrossRefPubMedGoogle Scholar
  22. 22.
    Koikkalainen JR, Rhodius-Meester HFM, Tolonen A et al (2016) Differential diagnosis of neurodegenerative diseases using structural MRI data. Neuroimage Clin 11:435–449CrossRefPubMedPubMedCentralGoogle Scholar
  23. 23.
    Wang Y, Catindig JA, Hilal S et al (2012) Multi-stage segmentation of white matter hyperintensity, cortical and lacunar infarcts. Neuroimage 60(4):2379–2388CrossRefPubMedGoogle Scholar
  24. 24.
    Buckner RL, Head D, Parker J et al (2004) A unified approach for morphometric and functional data analysis in young, old, and demented adults using automated atlas-based head size normalization: reliability and validation against manual measurement of total intracranial volume. Neuroimage 23:724–738Google Scholar
  25. 25.
    Cole TJ, Green PJ (1992) Smoothing reference centile curves: the LMS method and penalized likelihood. Stat Med 11(10):1305–1319CrossRefPubMedGoogle Scholar
  26. 26.
    Ashburner J, Friston KJ (2000) Voxel-based morphometry—the methods. Neuroimage 11(6):805–821CrossRefPubMedGoogle Scholar
  27. 27.
    Wattjes M, Henneman W, van der Flier WM et al (2009) Diagnostics imaging of patients in a memory clinic: comparison of MR imaging and 64-detector row CT. Radiology 253(1):174–183CrossRefPubMedGoogle Scholar
  28. 28.
    Kate M, Barkhof F, Boccardi M et al (2017) Clinical validity of medial temporal atrophy as a biomarker for Alzheimer’s disease in the context of a structured 5-phase development framework. Neurobiol Aging 52:167–182CrossRefPubMedGoogle Scholar
  29. 29.
    Warlaw JJ, Smith EE, Biessels GJ et al (2013) Neuroimaging standards for research into small vessel disease and its contribution to ageing and neurodegeneration. Lancet Neurol 12:822–838CrossRefGoogle Scholar
  30. 30.
    Verhagen MV, Guit GL, Hafkamp GJ, Kalisvaart K (2016) The impact of MRI combined with visual rating scales on the clinical diagnosis of dementia: a prospective study. Eur Radiol 26(6):1716–1722CrossRefPubMedGoogle Scholar
  31. 31.
    Barkhof F, Polvikoski TM, van Straaten EC et al (2007) The significance of medial temporal lobe atrophy: a postmortem MRI study in the very old. Neurology 69(15):1521–1527CrossRefPubMedGoogle Scholar
  32. 32.
    van de Pol LA, Hensel A, van der Flier WM et al (2006) Hippocampal atrophy on MRI in frontotemporal lobar degeneration and Alzheimer’s disease. J Neurol Neurosurg Psychiatry 77(4):439–442CrossRefPubMedGoogle Scholar
  33. 33.
    Canu E, Agosta F, Mandic-Stojmenovic G et al (2017) Multiparametric MRI to distinguish early onset Alzheimer’s disease and behavioural variant of frontotemporal dementia. Neuroimage Clin 15:428–438CrossRefPubMedPubMedCentralGoogle Scholar
  34. 34.
    Bruun M, Rhodius-Meester HFM, Koikkalainen JR et al (2018) Evaluating combinations of diagnostic tests to discriminate different dementia types. Alzheimers Dement (Amst) 10:509–518Google Scholar
  35. 35.
    Burton EJ, Barber R, Mukaetova-Ladinska EB et al (2009) Medial temporal lobe atrophy on MRI differentiates Alzheimer’s disease from dementia with Lewy bodies and vascular cognitive impairment: a prospective study with pathological verification of diagnosis. Brain 132:195–203CrossRefPubMedGoogle Scholar
  36. 36.
    Bresciani L, Rossi R, Testa C et al (2005) Visual assessment of medial temporal atrophy on MR films in Alzheimer’s disease: comparison with volumetry. Aging Clin Exp Res 17(1):8–13Google Scholar
  37. 37.
    Cavallin L, Bronge L, Zhang Y et al (2012) Comparison between visual assessment of MTA and hippocampal volumes in an elderly, non-demented population. Acta Radiol 53(5):573–579CrossRefPubMedGoogle Scholar
  38. 38.
    Persson K, Barca M, Cavallin L et al (2017) Comparison of automated volumetry of the hippocampus using NeuroQuant® and visual assessment of the medial temporal lobe in Alzheimer’s disease. Acta Radiol 59(8):997–1001CrossRefPubMedGoogle Scholar
  39. 39.
    Clerx L, van Rossum IA, Burns L et al (2013) Measurements of medial temporal lobe atrophy for prediction of Alzheimer’s disease in subjects with mild cognitive impairment. Neurobiol Aging 34:2003–2013CrossRefPubMedGoogle Scholar
  40. 40.
    van Straaten E, Fazekas F, Rostrup E et al (2006) Impact of white matter hyperintensities scoring method on correlations with clinical data. Stroke 37:836–840CrossRefPubMedGoogle Scholar
  41. 41.
    Clerx L, Jacobs HI, Burgmans S et al (2013) Sensitivity of different MRI-techniques to assess gray matter atrophy patterns in Alzheimer’s disease is region-specific. Curr Alzheimer Res 10:940–951CrossRefPubMedGoogle Scholar

Copyright information

© European Society of Radiology 2019

Authors and Affiliations

  • Juha R. Koikkalainen
    • 1
  • Hanneke F. M. Rhodius-Meester
    • 2
  • Kristian S. Frederiksen
    • 3
  • Marie Bruun
    • 3
  • Steen G. Hasselbalch
    • 3
  • Marta Baroni
    • 4
  • Patrizia Mecocci
    • 4
  • Ritva Vanninen
    • 5
    • 6
  • Anne Remes
    • 6
  • Hilkka Soininen
    • 6
  • Mark van Gils
    • 7
  • Wiesje M. van der Flier
    • 2
    • 8
  • Philip Scheltens
    • 2
  • Frederik Barkhof
    • 2
    • 9
    • 10
  • Timo Erkinjuntti
    • 11
  • Jyrki M. P. Lötjönen
    • 1
    Email author
  • for the Alzheimer’s Disease Neuroimaging Initiative
  1. 1.Combinostics Ltd.TampereFinland
  2. 2.Alzheimer Center, Department of NeurologyVU University Medical Centre, Amsterdam NeuroscienceAmsterdamthe Netherlands
  3. 3.Danish Dementia Research Centre, Department of Neurology, RigshospitaletUniversity of CopenhagenCopenhagenDenmark
  4. 4.Institute of Gerontology and GeriatricsUniversity of PerugiaPerugiaItaly
  5. 5.Institute of Clinical Medicine, RadiologyUniversity of Eastern FinlandKuopioFinland
  6. 6.Department of Clinical RadiologyKuopio University HospitalKuopioFinland
  7. 7.VTT Technical Research Center of Finland LtdTampereFinland
  8. 8.Department of Epidemiology and BiostatisticsVU University Medical CentreAmsterdamthe Netherlands
  9. 9.Institute of NeurologyUniversity College LondonLondonUK
  10. 10.Institute of Healthcare EngineeringUniversity College LondonLondonUK
  11. 11.Clinical Neurosciences, NeurologyUniversity of Helsinki and Helsinki University HospitalHelsinkiFinland

Personalised recommendations