Abstract
Single-subject mapping of resting-state brain functional activity to non-imaging phenotypes is a major goal of neuroimaging. The large majority of learning approaches applied today rely either on static representations or on short-term temporal correlations. This is at odds with the nature of brain activity which is dynamic and exhibit both short- and long-range dependencies. Further, new sophisticated deep learning approaches have been developed and validated on single tasks/datasets. The application of these models for the study of a different targets typically require exhaustive hyperparameter search, model engineering and trial and error to obtain competitive results with simpler linear models. This in turn limit their adoption and hinder fair benchmarking in a rapidly developing area of research. To this end, we propose fMRI-S4; a versatile deep learning model for the classification of phenotypes and psychiatric disorders from the timecourses of resting-state functional magnetic resonance imaging scans. fMRI-S4 capture short- and long-range temporal dependencies in the signal using 1D convolutions and the recently introduced state-space models S4. The proposed architecture is lightweight, sample-efficient and robust across tasks/datasets. We validate fMRI-S4 on the tasks of diagnosing major depressive disorder (MDD), autism spectrum disorder (ASD) and sex classification on three multi-site rs-fMRI datasets. We show that fMRI-S4 can outperform existing methods on all three tasks and can be trained as a plug &play model without special hyperpararameter tuning for each setting (Code available at https://github.com/elgazzarr/fMRI-S4.)
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arbabshirani, M.R., Plis, S., Sui, J., Calhoun, V.D.: Single subject prediction of brain disorders in neuroimaging: promises and pitfalls. Neuroimage 145, 137–165 (2017)
Biewald, L.: Experiment tracking with weights and biases (2020). https://www.wandb.com/. Software available from wandb.com
Desikan, R.S., et al.: An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage 31(3), 968–980 (2006)
Di Martino, A., et al.: The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism. Mol. Psychiatry 19(6), 659–667 (2014)
Dvornek, N.C., Ventola, P., Pelphrey, K.A., Duncan, J.S.: Identifying autism from resting-state fMRI using long short-term memory networks. In: Wang, Q., Shi, Y., Suk, H.-I., Suzuki, K. (eds.) MLMI 2017. LNCS, vol. 10541, pp. 362–370. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67389-9_42
Eavani, H., Satterthwaite, T.D., Gur, R.E., Gur, R.C., Davatzikos, C.: Unsupervised learning of functional network dynamics in resting state fMRI. In: Gee, J.C., Joshi, S., Pohl, K.M., Wells, W.M., Zöllei, L. (eds.) IPMI 2013. LNCS, vol. 7917, pp. 426–437. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38868-2_36
El Gazzar, A., Cerliani, L., van Wingen, G., Thomas, R.M.: Simple 1-d convolutional networks for resting-state fMRI based classification in autism. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–6. IEEE (2019)
El-Gazzar, A., Quaak, M., Cerliani, L., Bloem, P., van Wingen, G., Mani Thomas, R.: A hybrid 3DCNN and 3DC-LSTM based model for 4d spatio-temporal fMRI data: an ABIDE autism classification study. In: Zhou, L., et al. (eds.) OR 2.0/MLCN -2019. LNCS, vol. 11796, pp. 95–102. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32695-1_11
El-Gazzar, A., Thomas, R.M., van Wingen, G.: Dynamic adaptive spatio-temporal graph convolution for fMRI modelling. In: Abdulkadir, A., et al. (eds.) MLCN 2021. LNCS, vol. 13001, pp. 125–134. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87586-2_13
Friston, K.J.: Functional and effective connectivity in neuroimaging: a synthesis. Hum. Brain Mapp. 2(1–2), 56–78 (1994)
Friston, K.J.: Functional and effective connectivity: a review. Brain Connect. 1(1), 13–36 (2011)
Gadgil, S., Zhao, Q., Pfefferbaum, A., Sullivan, E.V., Adeli, E., Pohl, K.M.: Spatio-temporal graph convolution for resting-state fMRI analysis. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12267, pp. 528–538. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59728-3_52
Gu, A., Dao, T., Ermon, S., Rudra, A., Ré, C.: HiPPO: recurrent memory with optimal polynomial projections. Adv. Neural. Inf. Process. Syst. 33, 1474–1487 (2020)
Gu, A., Goel, K., Ré, C.: Efficiently modeling long sequences with structured state spaces. In: International Conference on Learning Representations (2022)
He, B.J.: Scale-free properties of the functional magnetic resonance imaging signal during rest and task. J. Neurosci. 31(39), 13786–13795 (2011)
He, T., et al.: Deep neural networks and kernel regression achieve comparable accuracies for functional connectivity prediction of behavior and demographics. Neuroimage 206, 116276 (2020)
Hutchinson, R.A., Niculescu, R.S., Keller, T.A., Rustandi, I., Mitchell, T.M.: Modeling fMRI data generated by overlapping cognitive processes with unknown onsets using hidden process models. Neuroimage 46(1), 87–104 (2009)
Hutchison, R.M., et al.: Dynamic functional connectivity: promise, issues, and interpretations. Neuroimage 80, 360–378 (2013)
Janoos, F., Machiraju, R., Singh, S., Morocz, I.A.: Spatio-temporal models of mental processes from fMRI. Neuroimage 57(2), 362–377 (2011)
Kawahara, J., et al.: BrainNetCNN: convolutional neural networks for brain networks; towards predicting neurodevelopment. Neuroimage 146, 1038–1049 (2017)
Li, S., et al.: Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Advances in Neural Information Processing Systems 32 (2019)
Malkiel, I., Rosenman, G., Wolf, L., Hendler, T.: Pre-training and fine-tuning transformers for fMRI prediction tasks. arXiv preprint arXiv:2112.05761 (2021)
Ogawa, S., Lee, T.M., Kay, A.R., Tank, D.W.: Brain magnetic resonance imaging with contrast dependent on blood oxygenation. Proc. Natl. Acad. Sci. 87(24), 9868–9872 (1990)
Oord, A.v.d., et al.: WaveNet: a generative model for raw audio. arXiv preprint arXiv:1609.03499 (2016)
Pan, V.: Fast approximate computations with Cauchy matrices and polynomials. Math. Comput. 86(308), 2799–2826 (2017)
Preti, M.G., Bolton, T.A., Van De Ville, D.: The dynamic functional connectome: state-of-the-art and perspectives. Neuroimage 160, 41–54 (2017)
Sasha, R., Sidd, K.: The annotated s4. In: Blog Track at ICLR 2022 (2022). https://srush.github.io/annotated-s4/
Schulz, M.A., et al.: Different scaling of linear models and deep learning in UKBiobank brain images versus machine-learning datasets. Nat. Commun. 11(1), 1–15 (2020)
Shi, X., Chen, Z., Wang, H., Yeung, D.Y., Wong, W.K., Woo, W.C.: Convolutional LSTM network: a machine learning approach for precipitation nowcasting. Advances in Neural Information Processing Systems 28 (2015)
Sudlow, C., et al.: UK biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age. PLoS Med. 12(3), e1001779 (2015)
Suk, H.I., Wee, C.Y., Lee, S.W., Shen, D.: State-space model with deep learning for functional dynamics estimation in resting-state fMRI. Neuroimage 129, 292–307 (2016)
Sundermann, B., Herr, D., Schwindt, W., Pfleiderer, B.: Multivariate classification of blood oxygen level-dependent fMRI data with diagnostic intention: a clinical perspective. Am. J. Neuroradiol. 35(5), 848–855 (2014)
Tu, T., Paisley, J., Haufe, S., Sajda, P.: A state-space model for inferring effective connectivity of latent neural dynamics from simultaneous EEG/fMRI. Advances in Neural Information Processing Systems 32 (2019)
Tustin, A.: A method of analysing the behaviour of linear systems in terms of time series. J. Inst. Electr. Eng. Part IIA Autom. Regul. Servo Mech. 94(1), 130–142 (1947)
Tzourio-Mazoyer, N., et al.: Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage 15(1), 273–289 (2002)
Wang, Z., Yan, W., Oates, T.: Time series classification from scratch with deep neural networks: a strong baseline. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 1578–1585. IEEE (2017)
Williams, R.L., Lawrence, D.A., et al.: Linear State-Space Control Systems. Wiley, Hoboken (2007)
Yan, C.G., et al.: Reduced default mode network functional connectivity in patients with recurrent major depressive disorder. Proc. Natl. Acad. Sci. 116(18), 9078–9083 (2019)
Yan, W., et al.: Discriminating schizophrenia using recurrent neural network applied on time courses of multi-site fMRI data. EBioMedicine 47, 543–552 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
El-Gazzar, A., Thomas, R.M., van Wingen, G. (2022). fMRI-S4: Learning Short- and Long-Range Dynamic fMRI Dependencies Using 1D Convolutions and State Space Models. In: Abdulkadir, A., et al. Machine Learning in Clinical Neuroimaging. MLCN 2022. Lecture Notes in Computer Science, vol 13596. Springer, Cham. https://doi.org/10.1007/978-3-031-17899-3_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-17899-3_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-17898-6
Online ISBN: 978-3-031-17899-3
eBook Packages: Computer ScienceComputer Science (R0)
