Abstract
Complex spatiotemporal correlation and dependency embedded in functional magnetic resonance imaging (fMRI) data introduce critical challenges in related analytical methodologies. Despite remarkable successes, most of existing approaches only model spatial or temporal dependency alone and the development of a unified spatiotemporal model is still a challenge. Meanwhile, the recent emergence of deep neural networks has provided powerful models for interpreting complex spatiotemporal data. Here, we proposed a novel convolutional long-short term memory network (3DCLN) for spatiotemporal modeling of fMRI data. The proposed model is designed to decode fMRI volumes belonging to different task events by joint training a 3D convolutional neural network (CNN) for spatial dependency modeling and a long short-term memory (LSTM) network for temporal dependency modeling. We also designed a 3D deconvolution scheme for fMRI sequence reconstruction to inspect the feature learning process in the 3DCLN. The experimental results on the motor task-fMRI data from Human Connectome Project (HCP) showed that fMRI volumes can be decoded with a relatively high accuracy (76.38%). More importantly, the proposed 3DCLN can dramatically remove noises and highlights signals of interest in the reconstructed fMRI sequence and hence improve the performance of activation detection, validating the spatiotemporal feature learning in the proposed 3DCLN model.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Logothetis, N.K.: What we can do and what we cannot do with fMRI. Nature 453(7197), 869–878 (2008)
Friston, K.J.: Transients, metastability, and neuronal dynamics. NeuroImage 5, 164–171 (1997)
Hjelm, R.D., Calhoun, V.D., Salakhutdinov, R., Allen, E.A., Adali, T., Plis, S.M.: Restricted Boltzmann machines for neuroimaging: an application in identifying intrinsic networks. NeuroImage 96(8), 245–260 (2014)
Hu, X., et al.: Latent source mining in FMRI via restricted Boltzmann machine. Hum. Brain Mapp. 39(6), 2368–2380 (2018)
Xia, W., Wen, X., Li, J., Li, Y.: A new dynamic Bayesian network approach for determining effective connectivity from fMRI data. Neural Comput. Appl. 24(1), 91–97 (2014)
Graves, A.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Wang, H., et al.: Recognizing brain states using deep sparse recurrent neural network. IEEE Trans. Med. Imag. 34, 1058–1068 (2018)
Huang, S., Li, X., Zhang, Z., Wu, F., Han, J.: User-ranking video summarization with multi-stage spatio-temporal representation. IEEE Trans. Image Process. PP(99), 1 (2018)
Barch, D.M., et al.: Function in the human connectome: task-fMRI and individual differences in behavior. NeuroImage 80(8), 169–189 (2013)
Glasser, M.F., Sotiropoulos, S.N., Wilson, J.A., et al.: The minimal preprocessing pipelines for the Human Connectome Project. NeuroImage 80, 105–124 (2013)
Springenberg, J.T., Dosovitskiy, A., Brox, T., Riedmiller, M.: Striving for simplicity: the all convolutional net. arXiv preprint arXiv:1412.6806 (2014)
Zeiler, Matthew D., Fergus, R.: Visualizing and Understanding Convolutional Networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53
Huang, H., Hu, X., Zhao, Y., et al.: Modeling task fMRI data via deep convolutional autoencoder. IEEE Trans. Med. Imag. 37(7), 1551–1561 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Suo, W. et al. (2019). 3D Convolutional Long-Short Term Memory Network for Spatiotemporal Modeling of fMRI Data. In: Zhu, D., et al. Multimodal Brain Image Analysis and Mathematical Foundations of Computational Anatomy. MBIA MFCA 2019 2019. Lecture Notes in Computer Science(), vol 11846. Springer, Cham. https://doi.org/10.1007/978-3-030-33226-6_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-33226-6_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-33225-9
Online ISBN: 978-3-030-33226-6
eBook Packages: Computer ScienceComputer Science (R0)