Abstract
Background
Increasing prognostic models for amyotrophic lateral sclerosis (ALS) have been developed. However, no comprehensive evaluation of these models has been done. The purpose of this study was to map the prognostic models for ALS to assess their potential contribution and suggest future improvements on modeling strategy.
Methods
Databases including Medline, Embase, Web of Science, and Cochrane library were searched from inception to 20 February 2021. All studies developing and/or validating prognostic models for ALS were selected. Information regarding modelling method and methodological quality was extracted.
Results
A total of 28 studies describing the development of 34 models and the external validation of 19 models were included. The outcomes concerned were ALS progression (n = 12; 35%), change in weight (n = 1; 3%), respiratory insufficiency (n = 2; 6%), and survival (n = 19; 56%). Among the models predicting ALS progression or survival, the most frequently used predictors were age, ALS Functional Rating Scale/ALS Functional Rating Scale-Revised, site of onset, and disease duration. The modelling method adopted most was machine learning (n = 16; 47%). Most of the models (n = 25; 74%) were not presented. Discrimination and calibration were assessed in 12 (35%) and 2 (6%) models, respectively. Only one model by Westeneng et al. (Lancet Neurol 17:423–433, 2018) was assessed with overall low risk of bias and it performed well in both discrimination and calibration, suggesting a relatively reliable model for practice.
Conclusions
This study systematically reviewed the prognostic models for ALS. Their usefulness is questionable due to several methodological pitfalls and the lack of external validation done by fully independent researchers. Future research should pay more attention to the addition of novel promising predictors, external validation, and head-to-head comparisons of existing models.
Similar content being viewed by others
Data availability
All data generated or analyzed during this study are included in this article and its supplementary information files.
References
Brown RH, Al-Chalabi A (2017) Amyotrophic lateral sclerosis. N Engl J Med 377:162–172. https://doi.org/10.1056/NEJMra1603471
Hardiman O, Al-Chalabi A, Chio A, Corr EM, Logroscino G, Robberecht W, Shaw PJ, Simmons Z, van den Berg LH (2017) Amyotrophic lateral sclerosis. Nat Rev Dis Primers 3:17071. https://doi.org/10.1038/nrdp.2017.85
Logroscino G, Piccininni M, Marin B, Nichols E, Abd-Allah F, Abdelalim A, Alahdab F, Asgedom SW, Awasthi A, Chaiah Y, Daryani A, Do HP, Dubey M, Elbaz A, Eskandarieh S, Farhadi F, Farzadfar F, Fereshtehnejad S-M, Fernandes E, Filip I, Foreman KJ, Gebre AK, Gnedovskaya EV, Hamidi S, Hay SI, Irvani SSN, Ji JS, Kasaeian A, Kim YJ, Mantovani LG, Mashamba-Thompson TP, Mehndiratta MM, Mokdad AH, Nagel G, Nguyen TH, Nixon MR, Olagunju AT, Owolabi MO, Piradov MA, Qorbani M, Radfar A, Reiner RC, Sahraian MA, Sarvi S, Sharif M, Temsah O, Tran BX, Truong NT, Venketasubramanian N, Winkler AS, Yimer EM, Feigin VL, Vos T, Murray CJL (2018) Global, regional, and national burden of motor neuron diseases 1990–2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet Neurol 17:1083–1097. https://doi.org/10.1016/s1474-4422(18)30404-6
Elamin M, Bede P, Montuschi A, Pender N, Chio A, Hardiman O (2015) Predicting prognosis in amyotrophic lateral sclerosis: a simple algorithm. J Neurol 262:1447–1454. https://doi.org/10.1007/s00415-015-7731-6
Knibb JA, Keren N, Kulka A, Leigh PN, Martin S, Shaw CE, Tsuda M, Al-Chalabi A (2016) A clinical tool for predicting survival in ALS. J Neurol Neurosurg Psychiatry 87:1361–1367. https://doi.org/10.1136/jnnp-2015-312908
Moons KG, de Groot JA, Bouwmeester W, Vergouwe Y, Mallett S, Altman DG, Reitsma JB, Collins GS (2014) Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Med 11:e1001744. https://doi.org/10.1371/journal.pmed.1001744
Zandonà A, Vasta R, Chiò A, Di Camillo B (2019) A dynamic bayesian network model for the simulation of amyotrophic lateral sclerosis progression. BMC Bioinformatics 20:118. https://doi.org/10.1186/s12859-019-2692-x
Carreiro AV, Amaral PMT, Pinto S, Tomás P, de Carvalho M, Madeira SC (2015) Prognostic models based on patient snapshots and time windows: predicting disease progression to assisted ventilation in Amyotrophic Lateral Sclerosis. J Biomed Inform 58:133–144. https://doi.org/10.1016/j.jbi.2015.09.021
Bellou V, Belbasis L, Konstantinidis AK, Tzoulaki I, Evangelou E (2019) Prognostic models for outcome prediction in patients with chronic obstructive pulmonary disease: systematic review and critical appraisal. BMJ 367:l5358. https://doi.org/10.1136/bmj.l5358
Nagendran M, Chen Y, Lovejoy CA, Gordon AC, Komorowski M, Harvey H, Topol EJ, Ioannidis JPA, Collins GS, Maruthappu M (2020) Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies. BMJ 368:m689. https://doi.org/10.1136/bmj.m689
Collins GS, Reitsma JB, Altman DG, Moons KG (2015) Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med 162:55–63. https://doi.org/10.7326/M14-0697
Debray TP, Damen JA, Snell KI, Ensor J, Hooft L, Reitsma JB, Riley RD, Moons KG (2017) A guide to systematic review and meta-analysis of prediction model performance. BMJ 356:i6460. https://doi.org/10.1136/bmj.i6460
Moons KGM, Wolff RF, Riley RD, Whiting PF, Westwood M, Collins GS, Reitsma JB, Kleijnen J, Mallett S (2019) PROBAST: A tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration. Ann Intern Med 170:W1–W33. https://doi.org/10.7326/M18-1377
Damen JA, Hooft L, Schuit E, Debray TP, Collins GS, Tzoulaki I, Lassale CM, Siontis GC, Chiocchia V, Roberts C, Schlussel MM, Gerry S, Black JA, Heus P, van der Schouw YT, Peelen LM, Moons KG (2016) Prediction models for cardiovascular disease risk in the general population: systematic review. BMJ 353:i2416. https://doi.org/10.1136/bmj.i2416
Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW (2010) Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 21:128–138. https://doi.org/10.1097/EDE.0b013e3181c30fb2
Wolff RF, Moons KGM, Riley RD, Whiting PF, Westwood M, Collins GS, Reitsma JB, Kleijnen J, Mallett S, Groupdagger P (2019) PROBAST: a tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med 170:51–58. https://doi.org/10.7326/M18-1376
Jablecki CK, Berry C, Leach J (1989) Survival prediction in amyotrophic lateral sclerosis. Muscle Nerve 12:833–841
Westeneng HJ, Debray TPA, Visser AE, van Eijk RPA, Rooney JPK, Calvo A, Martin S, McDermott CJ, Thompson AG, Pinto S, Kobeleva X, Rosenbohm A, Stubendorff B, Sommer H, Middelkoop BM, Dekker AM, van Vugt JJFA, van Rheenen W, Vajda A, Heverin M, Kazoka M, Hollinger H, Gromicho M, Körner S, Ringer TM, Rödiger A, Gunkel A, Shaw CE, Bredenoord AL, van Es MA, Corcia P, Couratier P, Weber M, Grosskreutz J, Ludolph AC, Petri S, de Carvalho M, Van Damme P, Talbot K, Turner MR, Shaw PJ, Al-Chalabi A, Chiò A, Hardiman O, Moons KGM, Veldink JH, van den Berg LH (2018) Prognosis for patients with amyotrophic lateral sclerosis: development and validation of a personalised prediction model. Lancet Neurol 17:423–433. https://doi.org/10.1016/S1474-4422(18)30089-9
Yip CH, Evans DG, Agarwal G, Buccimazza I, Kwong A, Morant R, Prakash I, Song CY, Taib NA, Tausch C, Ung O, Meterissian S (2019) Global disparities in breast cancer genetics testing, counselling and management. World J Surg 43:1264–1270. https://doi.org/10.1007/s00268-018-04897-6
Ngoya PS, Muhogora WE, Pitcher RD (2016) Defining the diagnostic divide: an analysis of registered radiological equipment resources in a low-income African country. Pan Afr Med J 25:99. https://doi.org/10.11604/pamj.2016.25.99.9736
Wang M-D, Little J, Gomes J, Cashman NR, Krewski D (2017) Identification of risk factors associated with onset and progression of amyotrophic lateral sclerosis using systematic review and meta-analysis. Neurotoxicology 61:101–130. https://doi.org/10.1016/j.neuro.2016.06.015
García-Redondo A, Dols-Icardo O, Rojas-García R, Esteban-Pérez J, Cordero-Vázquez P, Muñoz-Blanco JL, Catalina I, González-Muñoz M, Varona L, Sarasola E, Povedano M, Sevilla T, Guerrero A, Pardo J, López de Munain A, Márquez-Infante C, de Rivera FJR, Pastor P, Jericó I, de Arcaya AÁ, Mora JS, Clarimón J, Gonzalo-Martínez JF, Juárez-Rufián A, Atencia G, Jiménez-Bautista R, Morán Y, Mascías J, Hernández-Barral M, Kapetanovic S, García-Barcina M, Alcalá C, Vela A, Ramírez-Ramos C, Galán L, Pérez-Tur J, Quintáns B, Sobrido MJ, Fernández-Torrón R, Poza JJ, Gorostidi A, Paradas C, Villoslada P, Larrodé P, Capablo JL, Pascual-Calvet J, Goñi M, Morgado Y, Guitart M, Moreno-Laguna S, Rueda A, Martín-Estefanía C, Cemillán C, Blesa R, Lleó A (2013) Analysis of the C9orf72 gene in patients with amyotrophic lateral sclerosis in Spain and different populations worldwide. Hum Mutat 34:79–82. https://doi.org/10.1002/humu.22211
Atassi N, Berry J, Shui A, Zach N, Sherman A, Sinani E, Walker J, Katsovskiy I, Schoenfeld D, Cudkowicz M, Leitner M (2014) The PRO-ACT database: design, initial analyses, and predictive features. Neurology 83:1719–1725. https://doi.org/10.1212/WNL.0000000000000951
Aggarwal SP, Zinman L, Simpson E, McKinley J, Jackson KE, Pinto H, Kaufman P, Conwit RA, Schoenfeld D, Shefner J, Cudkowicz M (2010) Safety and efficacy of lithium in combination with riluzole for treatment of amyotrophic lateral sclerosis: a randomised, double-blind, placebo-controlled trial. Lancet Neurol 9:481–488. https://doi.org/10.1016/S1474-4422(10)70068-5
Groenwold RH, Moons KG, Pajouheshnia R, Altman DG, Collins GS, Debray TP, Reitsma JB, Riley RD, Peelen LM (2016) Explicit inclusion of treatment in prognostic modeling was recommended in observational and randomized settings. J Clin Epidemiol 78:90–100. https://doi.org/10.1016/j.jclinepi.2016.03.017
Küffner R, Zach N, Norel R, Hawe J, Schoenfeld D, Wang L, Li G, Fang L, MacKey L, Hardiman O, Cudkowicz M, Sherman A, Ertaylan G, Grosse-Wentrup M, Hothorn T, Van Ligtenberg J, MacKe JH, Meyer T, Schölkopf B, Tran L, Vaughan R, Stolovitzky G, Leitner ML (2015) Crowdsourced analysis of clinical trial data to predict amyotrophic lateral sclerosis progression. Nat Biotechnol 33:51–57. https://doi.org/10.1038/nbt.3051
Kueffner R, Zach N, Bronfeld M, Norel R, Atassi N, Balagurusamy V, Di Camillo B, Chio A, Cudkowicz M, Dillenberger D, Garcia-Garcia J, Hardiman O, Hoff B, Knight J, Leitner ML, Li G, Mangravite L, Norman T, Wang L, Alkallas R, Anghel C, Avril J, Bacardit J, Balser B, Balser J, Bar-Sinai Y, Ben-David N, Ben-Zion E, Bliss R, Cai J, Chernyshev A, Chiang J-H, Chicco D, Corriveau BAN, Dai J, Deshpande Y, Desplats E, Durgin JS, Espiritu SMG, Fan F, Fevrier P, Fridley BL, Godzik A, Golinska A, Gordon J, Graw S, Guo Y, Herpelinck T, Hopkins J, Huang B, Jacobsen J, Jahandideh S, Jeon J, Ji W, Jung K, Karanevich A, Koestler DC, Kozak M, Kurz C, Lalansingh C, Larrieu T, Lazzarini N, Lerner B, Lesinski W, Liang X, Lin X, Lowe J, Mackey L, Meier R, Min W, Mnich K, Nahmias V, Noel-MacDonnell J, O’Donnell A, Paadre S, Park J, Polewko-Klim A, Raghavan R, Rudnicki W, Saghapour E, Salomond J-B, Sankaran K, Sendorek D, Sharan V, Shiah Y-J, Sirois J-K, Sumanaweera DN, Usset J, Vang YS, Vens C, Wadden D, Wang D, Wong WC, Xie X, Xu Z, Yang H-T, Yu X, Zhang H, Zhang L, Zhang S, Zhu S, Xiao J, Fang W-C, Peng J, Yang C, Chang H-J, Stolovitzky G, Consortium ALSS (2019) Stratification of amyotrophic lateral sclerosis patients: a crowdsourcing approach. Sci Rep 9:690. https://doi.org/10.1038/s41598-018-36873-4
Andersen PM, Abrahams S, Borasio GD, de Carvalho M, Chio A, Van Damme P, Hardiman O, Kollewe K, Morrison KE, Petri S, Pradat PF, Silani V, Tomik B, Wasner M, Weber M (2012) EFNS guidelines on the clinical management of amyotrophic lateral sclerosis (MALS)–revised report of an EFNS task force. Eur J Neurol 19:360–375. https://doi.org/10.1111/j.1468-1331.2011.03501.x
Miller RG, Rosenberg JA, Gelinas DF, Mitsumoto H, Newman D, Sufit R, Borasio GD, Bradley WG, Bromberg MB, Brooks BR, Kasarskis EJ, Munsat TL, Oppenheimer EA (1999) Practice parameter: the care of the patient with amyotrophic lateral sclerosis (an evidence-based review): report of the Quality Standards Subcommittee of the American Academy of Neurology: ALS Practice Parameters Task Force. Neurology 52:1311–1323. https://doi.org/10.1212/wnl.52.7.1311
Neurology JSo (2013) ALS clinical practice guidelines 2013. Nankodo Co Ltd, Tokyo
Tramacere I, Dalla Bella E, Chiò A, Mora G, Filippini G, Lauria G (2015) The MITOS system predicts long-term survival in amyotrophic lateral sclerosis. J Neurol Neurosurg Psychiatry 86:1180–1185. https://doi.org/10.1136/jnnp-2014-310176
Roche JC, Rojas-Garcia R, Scott KM, Scotton W, Ellis CE, Burman R, Wijesekera L, Turner MR, Leigh PN, Shaw CE, Al-Chalabi A (2012) A proposed staging system for amyotrophic lateral sclerosis. Brain 135:847–852. https://doi.org/10.1093/brain/awr351
Chiò A, Hammond ER, Mora G, Bonito V, Filippini G (2015) Development and evaluation of a clinical staging system for amyotrophic lateral sclerosis. J Neurol Neurosurg Psychiatry 86:38–44. https://doi.org/10.1136/jnnp-2013-306589
Thakore NJ, Lapin BR, Kinzy TG, Pioro EP (2018) Deconstructing progression of amyotrophic lateral sclerosis in stages: a Markov modeling approach. Amyotroph Lateral Scler Frontotemporal Degener 19:483–494. https://doi.org/10.1080/21678421.2018.1484925
Grollemund V, Le Chat G, Secchi-Buhour MS, Delbot F, Pradat-Peyre JF, Bede P, Pradat PF (2021) Manifold learning for amyotrophic lateral sclerosis functional loss assessment: Development and validation of a prognosis model. J Neurol 268:825–850. https://doi.org/10.1007/s00415-020-10181-2
van der Burgh HK, Schmidt R, Westeneng HJ, de Reus MA, van den Berg LH, van den Heuvel MP (2017) Deep learning predictions of survival based on MRI in amyotrophic lateral sclerosis. Neuroimage Clin 13:361–369. https://doi.org/10.1016/j.nicl.2016.10.008
Agosta F, Spinelli EG, Riva N, Fontana A, Basaia S, Canu E, Castelnovo V, Falzone Y, Carrera P, Comi G, Filippi M (2019) Survival prediction models in motor neuron disease. Eur J Neurol 26:1143–1152. https://doi.org/10.1111/ene.13957
Bock M, Duong YN, Kim A, Allen I, Murphy J, Lomen-Hoerth C (2017) Progression and effect of cognitive-behavioral changes in patients with amyotrophic lateral sclerosis. Neurol Clin Pract 7:488–498. https://doi.org/10.1212/CPJ.0000000000000397
Agosta F, Pagani E, Petrolini M, Sormani MP, Caputo D, Perini M, Prelle A, Salvi F, Filippi M (2010) MRI predictors of long-term evolution in amyotrophic lateral sclerosis. Eur J Neurosci 32:1490–1496. https://doi.org/10.1111/j.1460-9568.2010.07445.x
Chiò A, Calvo A, Bovio G, Canosa A, Bertuzzo D, Galmozzi F, Cugnasco P, Clerico M, De Mercanti S, Bersano E, Cammarosano S, Ilardi A, Manera U, Moglia C, Sideri R, Marinou K, Bottacchi E, Pisano F, Cantello R, Mazzini L, Mora G (2014) Amyotrophic lateral sclerosis outcome measures and the role of albumin and creatinine: a population-based study. JAMA Neurol 71:1134–1142. https://doi.org/10.1001/jamaneurol.2014.1129
Paganoni S, Nicholson K, Chan J, Shui A, Schoenfeld D, Sherman A, Berry J, Cudkowicz M, Atassi N (2018) Urate levels predict survival in amyotrophic lateral sclerosis: analysis of the expanded Pooled Resource Open-Access ALS clinical trials database. Muscle Nerve 57:430–434. https://doi.org/10.1002/mus.25950
Benatar M, Zhang L, Wang L, Granit V, Statland J, Barohn R, Swenson A, Ravits J, Jackson C, Burns TM, Trivedi J, Pioro EP, Caress J, Katz J, McCauley JL, Rademakers R, Malaspina A, Ostrow LW, Wuu J (2020) Validation of serum neurofilaments as prognostic and potential pharmacodynamic biomarkers for ALS. Neurology 95:e59–e69. https://doi.org/10.1212/WNL.0000000000009559
Poesen K, Van Damme P (2018) Diagnostic and prognostic performance of neurofilaments in ALS. Front Neurol 9:1167. https://doi.org/10.3389/fneur.2018.01167
Lee CT, Chiu YW, Wang KC, Hwang CS, Lin KH, Lee IT, Tsai CP (2013) Riluzole and prognostic factors in amyotrophic lateral sclerosis long-term and short-term survival: a population-based study of 1149 cases in Taiwan. J Epidemiol 23:35–40. https://doi.org/10.2188/jea.je20120119
Mandrioli J, Malerba SA, Beghi E, Fini N, Fasano A, Zucchi E, De Pasqua S, Guidi C, Terlizzi E, Sette E, Ravasio A, Casmiro M, Salvi F, Liguori R, Zinno L, Handouk Y, Rizzi R, Borghi A, Rinaldi R, Medici D, Santangelo M, Granieri E, Mussuto V, Aiello M, Ferro S, Vinceti M (2018) Riluzole and other prognostic factors in ALS: a population-based registry study in Italy. J Neurol 265:817–827. https://doi.org/10.1007/s00415-018-8778-y
Brown FS, Glasmacher SA, Kearns PKA, MacDougall N, Hunt D, Connick P, Chandran S (2020) Systematic review of prediction models in relapsing remitting multiple sclerosis. PLoS ONE 15:e0233575. https://doi.org/10.1371/journal.pone.0233575
Jampathong N, Laopaiboon M, Rattanakanokchai S, Pattanittum P (2018) Prognostic models for complete recovery in ischemic stroke: a systematic review and meta-analysis. BMC Neurol 18:26. https://doi.org/10.1186/s12883-018-1032-5
Steyerberg EW (2019) Clinical prediction models: a practical approach to development, validation, and updating, 2nd edn. In: Springer Nature Switzerland
Grollemund V, Chat GL, Secchi-Buhour MS, Delbot F, Pradat-Peyre JF, Bede P, Pradat PF (2020) Development and validation of a 1-year survival prognosis estimation model for amyotrophic lateral sclerosis using manifold learning algorithm UMAP. Sci Rep 10:13378. https://doi.org/10.1038/s41598-020-70125-8
Collins GS, Moons KGM (2019) Reporting of artificial intelligence prediction models. Lancet 393:1577–1579. https://doi.org/10.1016/S0140-6736(19)30037-6
Funding
National Natural Science Foundation of China (91646107, 81973146); National Key Research and Development Program of China (2018YFC1311704); National research program for key issues in air pollution control (DQGG0404).
Author information
Authors and Affiliations
Contributions
Conception: SYZ, SFW, LX; Design: SFW, LX; Administrative support: SFW, SYZ; Provision of study materials: LX; Collection and assembly of data: LX, BJH, YJZ; Data analysis and interpretation: LX, BJH, LC, DSF; Manuscript writing: LX, BJH; Final approval of manuscript: All authors.
Corresponding authors
Ethics declarations
Conflicts of interests
The authors declare that they have no competing interests.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Xu, L., He, B., Zhang, Y. et al. Prognostic models for amyotrophic lateral sclerosis: a systematic review. J Neurol 268, 3361–3370 (2021). https://doi.org/10.1007/s00415-021-10508-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00415-021-10508-7