Abstract
We propose to formulate MRI image reconstruction as an optimization problem and model the optimization trajectory as a dynamic process using ordinary differential equations (ODEs). We model the dynamics in ODE with a neural network and solve the desired ODE with the off-the-shelf (fixed) solver to obtain reconstructed images. We extend this model and incorporate the knowledge of off-the-shelf ODE solvers into the network design (learned solvers). We investigate several models based on three ODE solvers and compare models with fixed solvers and learned solvers. Our models achieve better reconstruction results and are more parameter efficient than other popular methods such as UNet and cascaded CNN. We introduce a new way of tackling the MRI reconstruction problem by modeling the continuous optimization dynamics using neural ODEs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
References
Aggarwal, H.K., Mani, M.P., Jacob, M.: Modl: model-based deep learning architecture for inverse problems. IEEE TMI 38(2), 394–405 (2018)
Akçakaya, M., Moeller, S., Weingärtner, S., Uğurbil, K.: Scan-specific robust artificial-neural-networks for k-space interpolation (raki) reconstruction: database-free deep learning for fast imaging. MRM 81(1), 439–453 (2019)
Bao, L., et al.: Undersampled MR image reconstruction using an enhanced recursive residual network. J. Magn. Reson. 305, 232–246 (2019)
Chen, T.Q., Rubanova, Y., Bettencourt, J., Duvenaud, D.K.: Neural ordinary differential equations. In: NIPS, pp. 6571–6583 (2018)
Chen, T., Xu, B., Zhang, C., Guestrin, C.: Training deep nets with sublinear memory cost. arXiv:1604.06174 (2016)
Chen, Y., Xiao, T., Li, C., Liu, Q., Wang, S.: Model-based convolutional de-aliasing network learning for parallel MR imaging. In: MICCAI, pp. 30–38. Springer (2019)
Duan, J., et al.: VS-Net: variable splitting network for accelerated parallel MRI reconstruction. In: MICCAI, pp. 713–722. Springer (2019)
Eo, T., Jun, Y., Kim, T., Jang, J., Lee, H.J., Hwang, D.: KIKI-net: cross-domain convolutional neural networks for reconstructing undersampled magnetic resonance images. MRM 80(5), 2188–2201 (2018)
Gholami, A., Keutzer, K., Biros, G.: ANODE: unconditionally accurate memory-efficient gradients for neural odes. arXiv:1902.10298 (2019)
Gilton, D., Ongie, G., Willett, R.: Neumann networks for inverse problems in imaging. arXiv:1901.03707 (2019)
Hammernik, K., et al.: Learning a variational network for reconstruction of accelerated MRI data. MRM 79(6), 3055–3071 (2018)
Hammernik, K., Schlemper, J., Qin, C., Duan, J., Summers, R.M., Rueckert, D.: \(\sum \)-net: systematic evaluation of iterative deep neural networks for fast parallel MR image reconstruction. arXiv:1912.09278 (2019)
Han, Y., Sunwoo, L., Ye, J.C.: k-space deep learning for accelerated MRI. IEEE TMI 39(2), 377–386 (2019)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)
He, X., Mo, Z., Wang, P., Liu, Y., Yang, M., Cheng, J.: ODE-inspired network design for single image super-resolution. In: CVPR, pp. 1732–1741 (2019)
Huang, Q., Yang, D., Wu, P., Qu, H., Yi, J., Metaxas, D.: MRI reconstruction via cascaded channel-wise attention network. In: ISBI, pp. 1622–1626. IEEE (2019)
Knoll, F., et al.: Deep learning methods for parallel magnetic resonance image reconstruction. arXiv:1904.01112 (2019)
Lee, D., Yoo, J., Tak, S., Ye, J.C.: Deep residual learning for accelerated MRI using magnitude and phase networks. IEEE T. Biomed. Eng. 65(9), 1985–1995 (2018)
Liang, D., Cheng, J., Ke, Z., Ying, L.: Deep MRI reconstruction: unrolled optimization algorithms meet neural networks. arXiv:1907.11711 (2019)
Liu, L., et al.: On the variance of the adaptive learning rate and beyond. arXiv:1908.03265 (2019)
Mardani, M., et al.: Deep generative adversarial networks for compressed sensing automates MRI. arXiv:1706.00051 (2017)
Paoletti, M.E., Haut, J.M., Plaza, J., Plaza, A.: Neural ordinary differential equations for hyperspectral image classification. IEEE TGRS 58(3), 1718–1734 (2019)
Pezzotti, N., et al.: Adaptive-CS-net: eastMRI with adaptive intelligence. arXiv:1912.12259 (2019)
Poddar, S., Jacob, M.: Dynamic MRI using smoothness regularization on manifolds (storm). IEEE TMI 35(4), 1106–1115 (2015)
Putzky, P., Welling, M.: Invert to learn to invert. In: NIPS, pp. 444–454 (2019)
Qin, C., Schlemper, J., Caballero, J., Price, A.N., Hajnal, J.V., Rueckert, D.: Convolutional recurrent neural networks for dynamic MR image reconstruction. IEEE TMI 38(1), 280–290 (2018)
Ravishankar, S., Ye, J.C., Fessler, J.A.: Image reconstruction: from sparsity to data-adaptive methods and machine learning. arXiv:1904.02816 (2019)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: MICCAI, pp. 234–241. Springer (2015)
Schlemper, J., Caballero, J., Hajnal, J.V., Price, A.N., Rueckert, D.: A deep cascade of convolutional neural networks for dynamic MR image reconstruction. IEEE TMI 37(2), 491–503 (2017)
Souza, R., Bento, M., Nogovitsyn, N., Chung, K.J., Lebel, R.M., Frayne, R.: Dual-domain cascade of u-nets for multi-channel magnetic resonance image reconstruction. arXiv:1911.01458 (2019)
Souza, R., Frayne, R.: A hybrid frequency-domain/image-domain deep network for magnetic resonance image reconstruction. In: SIBGRAPI, pp. 257–264. IEEE (2019)
Sriram, A., et al.: End-to-end variational networks for accelerated MRI reconstruction. arXiv:2004.06688 (2020)
Sun, J., Li, H., Xu, Z., et al.: Deep ADMM-Net for compressive sensing MRI. In: NIPS, pp. 10–18 (2016)
Wang, P., Chen, E.Z., Chen, T., Patel, V.M., Sun, S.: Pyramid convolutional RNN for MRI reconstruction. arXiv:1912.00543 (2019)
Wang, S., et al.: Dimension: dynamic MR imaging with both k-space and spatial prior knowledge obtained via multi-supervised network training. arXiv:1810.00302 (2018)
Yang, G., et al.: DAGAN: deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction. IEEE TMI 37(6), 1310–1321 (2017)
Yazdanpanah, A.P., Afacan, O., Warfield, S.K.: ODE-based deep network for MRI reconstruction. arXiv:1912.12325 (2019)
Zbontar, J., et al.: FastMRI: an open dataset and benchmarks for accelerated MRI. arXiv:1811.08839 (2018)
Zhang, J., Ghanem, B.: ISTA-net: interpretable optimization-inspired deep network for image compressive sensing. In: CVPR, pp. 1828–1837 (2018)
Zhang, M., Lucas, J., Ba, J., Hinton, G.E.: Lookahead optimizer: k steps forward, 1 step back. In: NIPS, pp. 9593–9604 (2019)
Zhang, T., et al.: ANODEV2: a coupled neural ODE evolution framework. arXiv:1906.04596 (2019)
Zhu, B., Liu, J.Z., Cauley, S.F., Rosen, B.R., Rosen, M.S.: Image reconstruction by domain-transform manifold learning. Nature 555(7697), 487 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, E.Z., Chen, T., Sun, S. (2020). MRI Image Reconstruction via Learning Optimization Using Neural ODEs. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12262. Springer, Cham. https://doi.org/10.1007/978-3-030-59713-9_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-59713-9_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59712-2
Online ISBN: 978-3-030-59713-9
eBook Packages: Computer ScienceComputer Science (R0)
