Abstract
We present a novel deep learning framework that uses dynamic functional connectivity to simultaneously localize the language and motor areas of the eloquent cortex in brain tumor patients. Our method leverages convolutional layers to extract graph-based features from the dynamic connectivity matrices and a long-short term memory (LSTM) attention network to weight the relevant time points during classification. The final stage of our model employs multi-task learning to identify different eloquent subsystems. Our unique training strategy finds a shared representation between the cognitive networks of interest, which enables us to handle missing patient data. We evaluate our method on resting-state fMRI data from 56 brain tumor patients while using task fMRI activations as surrogate ground-truth labels for training and testing. Our model achieves higher localization accuracies than conventional deep learning approaches and can identify bilateral language areas even when trained on left-hemisphere lateralized cases. Hence, our method may ultimately be useful for preoperative mapping in tumor patients.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ojemann, G.A., Whitaker, H.A.: Language localization and variability. Brain Lang. 6(2), 239–260 (1978)
Tomasi, D., Volkow, N.: Language network: segregation, laterality and connectivity. Mol. Psychiatry 17(8), 759 (2012)
Tzourio-Mazoyer, N., Josse, G., Crivello, F., Mazoyer, B.: Interindividual variability in the hemispheric organization for speech. Neuroimage 21(1), 422–435 (2004)
Gupta, D.K., Chandra, P., Ojha, B., Sharma, B., Mahapatra, A., Mehta, V.: Awake craniotomy versus surgery under general anesthesia for resection of intrinsic lesions of eloquent cortex-a prospective randomised study. Clin. Neurol. Neurosurg. 109(4), 335–343 (2007)
Berger, M.S., Kincaid, J., Ojemann, G.A., Lettich, E.: Brain mapping techniques to maximize resection, safety, and seizure control in children with brain tumors. Neurosurgery 25(5), 786–792 (1989)
Kokkonen, S.-M., et al.: Preoperative localization of the sensorimotor area using independent component analysis of resting-state fMRI. Magn. Reson. Imaging 27(6), 733–740 (2009)
Lee, M.H., et al.: Clinical resting-state fMRI in the preoperative setting: are we ready for prime time? Top. Magn. Reson. Imaging TMRI 25(1), 11 (2016)
Van Den Heuvel, M.P., Pol, H.E.H.: Exploring the brain network: a review on resting-state fMRI functional connectivity. Eur. Neuropsychopharmacol. 20(8), 519–534 (2010)
Nandakumar, N., Manzoor, K., Pillai, J.J., Gujar, S.K., Sair, H.I., Venkataraman, A.: A novel graph neural network to localize eloquent cortex in brain tumor patients from resting-state fMRI connectivity. In: Schirmer, M.D., Venkataraman, A., Rekik, I., Kim, M., Chung, A.W. (eds.) CNI 2019. LNCS, vol. 11848, pp. 10–20. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32391-2_2
Sair, H.I., et al.: Presurgical brain mapping of the language network in patients with brain tumors using resting-state fMRI: comparison with task fMRI. Hum. Brain Mapp. 37(3), 913–923 (2016)
Tie, Y., et al.: Defining language networks from resting-state fMRI for surgical planning-a feasibility study. Hum. Brain Mapp. 35(3), 1018–1030 (2014)
Hacker, C.D., et al.: Resting state network estimation in individual subjects. Neuroimage 82, 616–633 (2013)
Leuthardt, E.C., et al.: Integration of resting state functional MRI into clinical practice-a large single institution experience. PloS One 13(6), e0198349 (2018)
Dvornek, N.C., Li, X., Zhuang, J., Duncan, J.S.: Jointly discriminative and generative recurrent neural networks for learning from fMRI. In: Suk, H.-I., Liu, M., Yan, P., Lian, C. (eds.) MLMI 2019. LNCS, vol. 11861, pp. 382–390. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32692-0_44
Yan, W., Zhang, H., Sui, J., Shen, D.: deep chronnectome learning via full bidirectional long short-term memory networks for MCI diagnosis. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11072, pp. 249–257. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00931-1_29
Dvornek, N.C., Ventola, P., Pelphrey, K.A., Duncan, J.S.: Identifying autism from resting-state fMRI using long short-term memory networks. In: Wang, Q., Shi, Y., Suk, H.-I., Suzuki, K. (eds.) MLMI 2017. LNCS, vol. 10541, pp. 362–370. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67389-9_42
Rashid, B., et al.: Classification of schizophrenia and bipolar patients using static and dynamic resting-state fMRI brain connectivity. Neuroimage 134, 645–657 (2016)
El-Gazzar, A., Quaak, M., Cerliani, L., Bloem, P., van Wingen, G., Mani Thomas, R.: A hybrid 3DCNN and 3DC-LSTM based model for 4D spatio-temporal fMRI data: an ABIDE autism classification study. In: Zhou, L., et al. (eds.) OR 2.0/MLCN -2019. LNCS, vol. 11796, pp. 95–102. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32695-1_11
Kunert-Graf, J.M., Eschenburg, K., Galas, D., Kutz, J.N., Rane, S., Brunton, B.W.: Extracting reproducible time-resolved resting state networks using dynamic mode decomposition. Front. Comput. Neurosci. 13, 75 (2019)
Hutchison, R.M., et al.: Dynamic functional connectivity: promise, issues, and interpretations. Neuroimage 80, 360–378 (2013)
Kawahara, J., et al.: BrainNetCNN: convolutional neural networks for brain networks; towards predicting neurodevelopment. NeuroImage 146, 1038–1049 (2017)
Xingjian, S., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-K., Woo, W.-C.: Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In: Advances in Neural Information Processing Systems, pp. 802–810 (2015)
Li, H., Fan, Y.: Brain decoding from functional MRI using long short-term memory recurrent neural networks. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11072, pp. 320–328. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00931-1_37
Suresh, S., et al.: Risk-sensitive loss functions for sparse multi-category classification problems. Inf. Sci. 178(12), 2621–2638 (2008)
Jack Jr., C.R., et al.: Sensory motor cortex: correlation of presurgical mapping with functional MR imaging and invasive cortical mapping. Radiology 190(1), 85–92 (1994)
Mazaika, P.K., Hoeft, F., Glover, G.H., Reiss, A.L., et al.: Methods and software for fMRI analysis of clinical subjects. Neuroimage 47(Suppl 1), S58 (2009)
Behzadi, Y., Restom, K., Liau, J., Liu, T.T.: A component based noise correction method (CompCor) for BOLD and perfusion based fMRI. Neuroimage 37(1), 90–101 (2007)
Craddock, R.C., et al.: A whole brain fMRI atlas generated via spatially constrained spectral clustering. Hum. Brain Mapp. 33(8), 1914–1928 (2012)
McAuliffe, M.J., Lalonde, F.M., McGarry, D., Gandler, W., Csaky, K., Trus, B.L.: Medical image processing, analysis and visualization in clinical research. In: Proceedings 14th IEEE Symposium on Computer-Based Medical Systems. CBMS 2001, pp. 381–386. IEEE (2001)
Acknowledgement
This work was supported by the National Science Foundation CAREER award 1845430 (PI: Venkataraman) and the Research & Education Foundation Carestream Health RSNA Research Scholar Grant RSCH1420.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Nandakumar, N. et al. (2020). A Multi-task Deep Learning Framework to Localize the Eloquent Cortex in Brain Tumor Patients Using Dynamic Functional Connectivity. In: Kia, S.M., et al. Machine Learning in Clinical Neuroimaging and Radiogenomics in Neuro-oncology. MLCN RNO-AI 2020 2020. Lecture Notes in Computer Science(), vol 12449. Springer, Cham. https://doi.org/10.1007/978-3-030-66843-3_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-66843-3_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-66842-6
Online ISBN: 978-3-030-66843-3
eBook Packages: Computer ScienceComputer Science (R0)
