Abstract
Accurately segmenting temporal frames of cine magnetic resonance imaging (MRI) is a crucial step in various real-time MRI guided cardiac interventions. To achieve fast and accurate visual assistance, there are strict requirements on the maximum latency and minimum throughput of the segmentation framework. State-of-the-art neural networks on this task are mostly hand-crafted to satisfy these constraints while achieving high accuracy. On the other hand, existing literature has demonstrated the power of neural architecture search (NAS) in automatically identifying the best neural architectures for various medical applications, within which differentiable NAS is a prevailing and efficient approach. However, they are mostly guided by accuracy, sometimes with computation complexity, but the importance of real-time constraints are overlooked. A major challenge is that such constraints are non-differentiable and thus are not compatible with the widely used differentiable NAS frameworks. In this paper, we present a strategy that can directly handle real-time constraints in differentiable NAS frameworks, named RT-DNAS. Experiments on extended 2017 MICCAI ACDC dataset show that compared with state-of-the-art manually and automatically designed architectures, RT-DNAS is able to identify neural architectures that can achieve better accuracy while satisfying the real-time constraints.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Annett, M., Ng, A., Dietz, P., Bischof, W.F., Gupta, A.: How low should we go? Understanding the perception of latency while inking. In: Graphics Interface 2014, pp. 167–174. AK Peters/CRC Press (2020)
Bosma, M., Dushatskiy, A., Grewal, M., Alderliesten, T., Bosman, P.A.: Mixed-block neural architecture search for medical image segmentation. In: SPIE Medical Imaging (2022)
Chetlur, S., et al.: cuDNN: Efficient primitives for deep learning. arXiv preprint arXiv:1410.0759 (2014)
Dong, S., et al.: DeU-Net 2.0: Enhanced deformable U-Net for 3D cardiac cine MRI segmentation. Medical Image Analysis 78, 102389 (2022)
Fernandes, F.E., Yen, G.G.: Automatic searching and pruning of deep neural networks for medical imaging diagnostic. IEEE Trans. Neural Netw. Learn. Syst. 32(12), 5664–5674 (2021)
Gaspar, T., Piorkowski, C., Gutberlet, M., Hindricks, G.: Three-dimensional real-time MRI-guided intracardiac catheter navigation. Eur. Heart J. 35(9), 589–589 (2014)
He, Y., Yang, D., Roth, H., Zhao, C., Xu, D.: Dints: differentiable neural network topology search for 3d medical image segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5841–5850 (2021)
Huang, Z., Wang, Z., Gu, L., et al.: Adwu-Net: adaptive depth and width U-Net for medical image segmentation by differentiable neural architecture search. In: Medical Imaging with Deep Learning (2021)
Iltis, P.W., Frahm, J., Voit, D., Joseph, A.A., Schoonderwaldt, E., Altenmüller, E.: High-speed real-time magnetic resonance imaging of fast tongue movements in elite horn players. Quant. Imaging Med. Surg. 5(3), 374 (2015)
Jiang, W., et al.: Hardware/software co-exploration of neural architectures. IEEE Trans. Comput.-Aided Des. Integr. Circ. Syst. 39(12), 4805–4815 (2020)
Li, Y., et al.: EDD: efficient differentiable DNN architecture and implementation co-search for embedded AI solutions. In: 2020 57th ACM/IEEE Design Automation Conference, DAC 2020. Proceedings - Design Automation Conference, Institute of Electrical and Electronics Engineers Inc., USA, July 2020
Lu, Z., et al.: Multi-scale and multi-level neural architecture search for low-dose ct denoising. arXiv preprint arXiv:2103.12995 (2021)
McVeigh, E.R., Guttman, M.A., Lederman, R.J., Li, M., Kocaturk, O., Hunt, T., Kozlov, S., Horvath, K.A.: Real-time interactive Mri-guided cardiac surgery: Aortic valve replacement using a direct apical approach. Mag. Resona. Med. 56(5), 958–964 (2006)
Peng, C., et al.: Hypersegnas: bridging one-shot neural architecture search with 3d medical image segmentation using hypernet. arXiv preprint arXiv:2112.10652 (2021)
Radau, P.E., et al.: VURTIGO: visualization platform for real-time, MRI-guided cardiac electroanatomic mapping. In: Camara, O., Konukoglu, E., Pop, M., Rhode, K., Sermesant, M., Young, A. (eds.) STACOM 2011. LNCS, vol. 7085, pp. 244–253. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-28326-0_25
Radau, Z., et al.: VURTIGO: visualization platform for real-time, MRI-guided cardiac electroanatomic mapping. In: Camara, O., Konukoglu, E., Pop, M., Rhode, K., Sermesant, M., Young, A. (eds.) STACOM 2011. LNCS, vol. 7085, pp. 244–253. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-28326-0_25
Rogers, T., et al.: Transcatheter myocardial needle chemoablation during real-time magnetic resonance imaging: a new approach to ablation therapy for rhythm disorders. Circulat. Arrhyth. Electrophysiol. 9(4), e003926 (2016)
Schaetz, S., Voit, D., Frahm, J., Uecker, M.: Accelerated computing in magnetic resonance imaging: real-time imaging using nonlinear inverse reconstruction. In: Computational and Mathematical Methods in Medicine 2017 (2017)
Vergara, C.R., et al.: Real-time magnetic resonance imaging-guided radiofrequency atrial ablation and visualization of lesion formation at 3 tesla. Heart Rhythm 8(2), 295–303 (2011)
Wang, T., et al.: MSU-Net: multiscale statistical u-net for real-time 3d cardiac MRI video segmentation. In: Shen, S., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 614–622. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_68
Wang, T., et al.: ICA-UNet: ICA inspired statistical UNet for real-time 3D cardiac cine MRI segmentation. In: Martel, M.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12266, pp. 447–457. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59725-2_43
Wang, X., et al.: BiX-NAS: searching efficient bi-directional architecture for medical image segmentation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 229–238. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_22
Xu, S., Quan, H.: ECT-NAS: Searching efficient CNN-transformers architecture for medical image segmentation. In: 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 1601–1604. IEEE (2021)
Xu, Y., et al.: PC-darts: partial channel connections for memory-efficient architecture search. arXiv preprint arXiv:1907.05737 (2019)
Yan, X., Jiang, W., Shi, Y., Zhuo, C.: MS-NAS: multi-scale neural architecture search for medical image segmentation. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 388–397. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_38
Zeng, D., et al.: Towards cardiac intervention assistance: hardware-aware neural architecture exploration for real-time 3D cardiac cine MRI segmentation. In: Proceedings of the 39th International Conference on Computer-Aided Design, pp. 1–8 (2020)
Zhu, Y., Meijering, E.: Automatic improvement of deep learning-based cell segmentation in time-lapse microscopy by neural architecture search. Bioinformatics 37(24), 4844–4850 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lu, Q. et al. (2022). RT-DNAS: Real-Time Constrained Differentiable Neural Architecture Search for 3D Cardiac Cine MRI Segmentation. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13435. Springer, Cham. https://doi.org/10.1007/978-3-031-16443-9_58
Download citation
DOI: https://doi.org/10.1007/978-3-031-16443-9_58
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16442-2
Online ISBN: 978-3-031-16443-9
eBook Packages: Computer ScienceComputer Science (R0)
