Advertisement

TADPOLE Challenge: Accurate Alzheimer’s Disease Prediction Through Crowdsourced Forecasting of Future Data

  • Răzvan V. MarinescuEmail author
  • Neil P. Oxtoby
  • Alexandra L. Young
  • Esther E. Bron
  • Arthur W. Toga
  • Michael W. Weiner
  • Frederik Barkhof
  • Nick C. Fox
  • Polina Golland
  • Stefan Klein
  • Daniel C. Alexander
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11843)

Abstract

The Alzheimer’s Disease Prediction Of Longitudinal Evolution (TADPOLE) Challenge compares the performance of algorithms at predicting the future evolution of individuals at risk of Alzheimer’s disease. TADPOLE Challenge participants train their models and algorithms on historical data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) study. Participants are then required to make forecasts of three key outcomes for ADNI-3 rollover participants: clinical diagnosis, Alzheimer’s Disease Assessment Scale Cognitive Subdomain (ADAS-Cog 13), and total volume of the ventricles – which are then compared with future measurements. Strong points of the challenge are that the test data did not exist at the time of forecasting (it was acquired afterwards), and that it focuses on the challenging problem of cohort selection for clinical trials by identifying fast progressors. The submission phase of TADPOLE was open until 15 November 2017; since then data has been acquired until April 2019 from 219 subjects with 223 clinical visits and 150 Magnetic Resonance Imaging (MRI) scans, which was used for the evaluation of the participants’ predictions. Thirty-three teams participated with a total of 92 submissions. No single submission was best at predicting all three outcomes. For diagnosis prediction, the best forecast (team Frog), which was based on gradient boosting, obtained a multiclass area under the receiver-operating curve (MAUC) of 0.931, while for ventricle prediction the best forecast (team EMC1), which was based on disease progression modelling and spline regression, obtained mean absolute error of 0.41% of total intracranial volume (ICV). For ADAS-Cog 13, no forecast was considerably better than the benchmark mixed effects model (BenchmarkME), provided to participants before the submission deadline. Further analysis can help understand which input features and algorithms are most suitable for Alzheimer’s disease prediction and for aiding patient stratification in clinical trials. The submission system remains open via the website: https://tadpole.grand-challenge.org/.

Keywords

Alzheimer’s disease Future prediction Community challenge 

Notes

Acknowledgement

TADPOLE Challenge has been organised by the European Progression Of Neurological Disease (EuroPOND) consortium, in collaboration with the ADNI. We thank all the participants and advisors, in particular Clifford R. Jack Jr. from Mayo Clinic, Rochester, United States and Bruno M. Jedynak from Portland State University, Portland, United States for useful input and feedback.

The organisers are extremely grateful to The Alzheimer’s Association, The Alzheimer’s Society and Alzheimer’s Research UK for sponsoring the challenge by providing the prize fund and providing invaluable advice into its construction and organisation. Similarly, we thank the ADNI leadership and members of our advisory board and other members of the EuroPOND consortium for their valuable advice and support.

RVM was supported by the EPSRC Centre For Doctoral Training in Medical Imaging with grant EP/L016478/1 and by the Neuroimaging Analysis Center with grant NIH NIBIB NAC P41EB015902. NPO, FB, SK, and DCA are supported by EuroPOND, which is an EU Horizon 2020 project. ALY was supported by an EPSRC Doctoral Prize fellowship and by EPSRC grant EP/J020990/01. PG was supported by NIH grant NIBIB NAC P41EB015902 and by grant NINDS R01NS086905. DCA was supported by EPSRC grants J020990, M006093 and M020533. The UCL-affiliated researchers received support from the NIHR UCLH Biomedical Research Centre. Data collection and sharing for this project was funded by the Alzheimer’s Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012). FB was supported by the NIHR UCLH Biomedical Research Centre and the AMYPAD project, which has received support from the EU-EFPIA Innovative Medicines Initiatives 2 Joint Undertaking (AMYPAD project, grant 115952). This project has received funding from the European Union Horizon 2020 research and innovation programme under grant agreement No. 666992.

References

  1. 1.
    Mehta, D., Jackson, R., Paul, G., Shi, J., Sabbagh, M.: Why do trials for Alzheimer’s disease drugs keep failing? A discontinued drug perspective for 2010–2015. Expert Opin. Investig. Drugs 26(6), 735 (2017)CrossRefGoogle Scholar
  2. 2.
    Scahill, R.I., Schott, J.M., Stevens, J.M., Rossor, M.N., Fox, N.C.: Mapping the evolution of regional atrophy in Alzheimer’s disease: unbiased analysis of fluid-registered serial MRI. Proc. Natl. Acad. Sci. 99(7), 4703–4707 (2002)CrossRefGoogle Scholar
  3. 3.
    Yang, E., et al.: Quantifying the pathophysiological timeline of Alzheimer’s disease. J. Alzheimer’s Dis. 26(4), 745–753 (2011)CrossRefGoogle Scholar
  4. 4.
    Guerrero, R., et al.: Instantiated mixed effects modeling of Alzheimer’s disease markers. NeuroImage 142, 113–125 (2016)CrossRefGoogle Scholar
  5. 5.
    Klöppel, S., et al.: Automatic classification of MR scans in Alzheimer’s disease. Brain 131(3), 681–689 (2008)CrossRefGoogle Scholar
  6. 6.
    Young, J., et al.: Accurate multimodal probabilistic prediction of conversion to Alzheimer’s disease in patients with mild cognitive impairment. NeuroImage Clin. 2, 735–745 (2013)CrossRefGoogle Scholar
  7. 7.
    Young, A.L., et al.: A data-driven model of biomarker changes in sporadic Alzheimer’s disease. Brain 137(9), 2564–2577 (2014)CrossRefGoogle Scholar
  8. 8.
    Lorenzi, M., Filippone, M., Frisoni, G.B., Alexander, D.C., Ourselin, S., Alzheimer’s Disease Neuroimaging Initiative, et al.: Probabilistic disease progression modeling to characterize diagnostic uncertainty: application to staging and prediction in Alzheimer’s disease. NeuroImage 190, 56–68 (2017)Google Scholar
  9. 9.
    Bron, E.E., et al.: Standardized evaluation of algorithms for computer-aided diagnosis of dementia based on structural MRI: the CADDementia challenge. NeuroImage 111, 562–579 (2015)CrossRefGoogle Scholar
  10. 10.
    Sarica, A., Cerasa, A., Quattrone, A., Calhoun, V.: Editorial on special issue: machine learning on MCI. J. Neurosci. Methods 302, 1 (2018)CrossRefGoogle Scholar
  11. 11.
    Weiner, M.W., et al.: Recent publications from the Alzheimer’s disease neuroimaging initiative: reviewing progress toward improved AD clinical trials. Alzheimer’s Dement. 13(4), e1–e85 (2017)CrossRefGoogle Scholar
  12. 12.
    Allen, G.I., et al.: Crowdsourced estimation of cognitive decline and resilience in Alzheimer’s disease. Alzheimer’s Dement. J. Alzheimer’s Assoc. 12(6), 645–653 (2016)CrossRefGoogle Scholar
  13. 13.
    Marinescu, R.V., et al.: Tadpole challenge: prediction of longitudinal evolution in Alzheimer’s disease. arXiv preprint arXiv:1805.03909 (2018)
  14. 14.
  15. 15.
    Beach, T.G., Monsell, S.E., Phillips, L.E., Kukull, W.: Accuracy of the clinical diagnosis of alzheimer disease at national institute on aging alzheimer disease centers, 2005–2010. J. Neuropathol. Exp. Neurol. 71(4), 266–273 (2012)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Răzvan V. Marinescu
    • 1
    • 2
    Email author
  • Neil P. Oxtoby
    • 2
  • Alexandra L. Young
    • 2
  • Esther E. Bron
    • 3
  • Arthur W. Toga
    • 4
  • Michael W. Weiner
    • 5
  • Frederik Barkhof
    • 3
    • 6
  • Nick C. Fox
    • 7
  • Polina Golland
    • 1
  • Stefan Klein
    • 3
  • Daniel C. Alexander
    • 2
  1. 1.Computer Science and Artificial Intelligence LaboratoryMITCambridgeUSA
  2. 2.Centre for Medical Image ComputingUniversity College LondonLondonUK
  3. 3.Biomedical Imaging Group RotterdamErasmus MCRotterdamThe Netherlands
  4. 4.Laboratory of NeuroImagingUniversity of Southern CaliforniaLos AngelesUSA
  5. 5.Center for Imaging of Neurodegenerative DiseasesUCSFSan FranciscoUSA
  6. 6.Department of Radiology and Nuclear MedicineVU Medical CentreAmsterdamThe Netherlands
  7. 7.Dementia Research CentreUCL Institute of NeurologyLondonUK

Personalised recommendations