Abstract
Traditional approaches to image reconstruction uses physics-based loss with data-efficient inference, although the difficulty to properly model the inverse solution precludes learning the reconstruction across a distribution of data. Modern deep learning approaches enable expressive modeling but rely on a large number of reconstructed images (labeled data) that are often not available in practice. To combine the best of the above two lines of works, we present a novel label-free image reconstruction network that is supervised by physics-based forward operators rather than labeled data. We further present an expressive yet disentangled spatial-temporal modeling of the inverse solution, where its latent dynamics is modeled by neural ordinary differential equations and its emission over non-Euclidean geometrical domains by graph convolutional neural networks. We applied the presented method to reconstruct electrical activity on the heart surface from body-surface potential. In simulation and real-data experiments in comparison to both traditional physics-based and modern data-driven reconstruction methods, we demonstrated the ability of the presented method to learn how to reconstruct using observational data without any corresponding labels.
X. Jiang and R. Missel contributed equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aliev, R.R., Panfilov, A.V.: A simple two-variable model of cardiac excitation. Chaos Solit. Fract. 7(3), 293–301 (1996)
Aras, K., et al.: Experimental data and geometric analysis repository-EDGAR. J. Electrocardiol. 48(6), 975–981 (2015)
Cacciola, F.: Triangulated surface mesh simplification. In: CGAL User and Reference Manual. CGAL Editorial Board, 5.0.2 edn. (2020). https://doc.cgal.org/5.0.2/Manual/packages.html#PkgSurfaceMeshSimplification
Chen, E.Z., Chen, T., Sun, S.: MRI image reconstruction via learning optimization using neural odes. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI) (2020)
Chen, R.T.Q., Rubanova, Y., Bettencourt, J., Duvenaud, D.: Neural ordinary differential equations (2019)
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation (2014)
Chung, J., Kastner, K., Dinh, L., Goel, K., Courville, A., Bengio, Y.: A recurrent latent variable model for sequential data. Neural Inf. Proces. Syst. (2015)
Cohen, O., Zhu, B., Rosen, M.S.: MR fingerprinting deep reconstruction network (drone). Magn. Resonan. Med. 80(3), 885–894 (2018)
Durbin, J., Koopman, S.J.: Time Series Analysis By State Space Methods. Oxford University Press, Oxford (2012)
Erem, B., Coll-Font, J., Orellana, R., Stovicek, P., Brooks, D.: Using transmural regularization and dynamic modeling for noninvasive cardiac potential imaging of endocardial pacing with imprecise thoracic geometry. IEEE Trans. Med. Imag. 33, 726–738 (2014). https://doi.org/10.1109/TMI.2013.2295220
Fang, Q., Boas, D.A.: Tetrahedral mesh generation from volumetric binary and grayscale images. In: 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 1142–1145. Ieee (2009)
Fey, M., Eric Lenssen, J., Weichert, F., Müller, H.: Splinecnn: fast geometric deep learning with continuous b-spline kernels. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 869–877 (2018)
Ghimire, S., Dhamala, J., Gyawali, P.K., Sapp, J.L., Horacek, M., Wang, L.: Generative modeling and inverse imaging of cardiac transmembrane potential. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 508–516. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_57
Ghimire, S., Gyawali, P.K., Dhamala, J., Sapp, J.L., Horacek, M., Wang, L.: Improving generalization of deep networks for inverse reconstruction of image sequences. In: Chung, A.C.S., Gee, J.C., Yushkevich, P.A., Bao, S. (eds.) IPMI 2019. LNCS, vol. 11492, pp. 153–166. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20351-1_12
Giffard-Roisin, S., et al.: Transfer learning from simulations on a reference anatomy for ECCI personalized cardiac resynchronization therapy. IEEE Trans. Biomed. Eng. 66(2), 343–353 (2018)
Giffard-Roisin, S., et al.: Noninvasive personalization of a cardiac electrophysiology model from body surface potential mapping. IEEE Trans. Biomed. Eng. 64(9), 2206–2218 (2016)
Jiang, X., Ghimire, S., Dhamala, J., Li, Z., Gyawali, P.K., Wang, L.: Learning geometry-dependent and physics-based inverse image reconstruction. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 487–496. Springer, Berlin (2020). https://doi.org/10.1007/10704282
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Lai, K.W., Aggarwal, M., van Zijl, P., Li, X., Sulam, J.: Learned proximal networks for quantitative susceptibility mapping. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI) (2020)
Lindstrom, P., Turk, G.: Fast and memory efficient polygonal simplification. In: Proceedings Visualization’98 (Cat. No. 98CB36276), pp. 279–286. IEEE (1998)
Plonsey, R.: Bioelectr. Phenomena. McGraw Hill, New York (1969)
Rubanova, Y., Chen, R.T., Duvenaud, D.: Latent odes for irregularly-sampled time series. arXiv preprint arXiv:1907.03907 (2019)
Sapp, J.L., Dawoud, F., Clements, J.C., Horáček, B.M.: Inverse solution mapping of epicardial potentials: quantitative comparison with epicardial contact mapping. Circul. Arrhyth. Electrophysiol. 5(5), 1001–1009 (2012)
Stewart, R., Ermon, S.: Label-free supervision of neural networks with physics and domain knowledge. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31 (2017)
Ulas, C., et al.: Direct estimation of pharmacokinetic parameters from DCE-MRI using deep cnn with forward physical model loss. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 39–47. Springer (2018)
Wang, L., Zhang, H., Wong, K.C., Liu, H., Shi, P.: Physiological-model-constrained noninvasive reconstruction of volumetric myocardial transmembrane potentials. IEEE Trans. Biomed. Eng. 57(2), 296–315 (2009)
Yang, Y., Sun, J., Li, H., Xu, Z.: Deep admm-net for compressive sensing MRI. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, pp. 10–18 (2016)
Zhu, B., Liu, J.Z., Cauley, S.F., Rosen, B.R., Rosen, M.S.: Image reconstruction by domain-transform manifold learning. Nature 555(7697), 487–492 (2018)
Acknowledgement
This work is supported by the National Institutes of Health (NIH) under Award Number R01HL145590 and R01NR018301.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Jiang, X. et al. (2021). Label-Free Physics-Informed Image Sequence Reconstruction with Disentangled Spatial-Temporal Modeling. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12906. Springer, Cham. https://doi.org/10.1007/978-3-030-87231-1_35
Download citation
DOI: https://doi.org/10.1007/978-3-030-87231-1_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87230-4
Online ISBN: 978-3-030-87231-1
eBook Packages: Computer ScienceComputer Science (R0)
