Abstract
Current deep learning based segmentation models generalize poorly to different domains due to the lack of sufficient labelled image data. An important example in radiology is generalizing from contrast enhanced CT to non-contrast CT. In real-world clinical applications, cross-domain image analysis tools are in high demand since medical images from different domains are generally used to achieve precise diagnoses. For example, contrast enhanced CT at different phases are used to enhance certain pathologies or internal organs. Many existing cross-domain image-to-image translation models show impressive results on large organ segmentation by successfully preserving large structures across domains. However, such models lack the ability to preserve fine structures during the translation process, which is significant for many clinical applications, such as segmenting small calcified plaques in the aorta and pelvic arteries. In order to preserve fine structures during medical image translation, we propose a patch-based model using shared latent variables from a Gaussian mixture. We compare our image translation framework to several state-of-the-art methods on cross-domain image translation and show our model does a better job preserving fine structures. The superior performance of our model is verified by performing two tasks with the translated images - detection and segmentation of aortic plaques and pancreas segmentation. We expect the utility of our framework will extend to other problems beyond segmentation due to the improved quality of the generated images and enhanced ability to preserve small structures.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Clark, K., et al.: The cancer imaging archive (TCIA): maintaining and operating a public information repository. J. Digit. Imaging 26(6), 1045–1057 (2013)
Dilokthanakul, N., et al.: Deep unsupervised clustering with gaussian mixture variational autoencoders. arXiv e-prints arXiv:1611.02648 (2016)
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: International Conference on Computer Vision (ICCV), pp. 2980–2988 (2017)
Huang, X., Liu, M.Y., Belongie, S., Kautz, J.: Multimodal unsupervised image-to-image translation. In: European Conference on Computer Vision (ECCV), pp. 172–189 (2018)
Jin, D., Xu, Z., Tang, Y., Harrison, A.P., Mollura, D.J.: CT-realistic lung nodule simulation from 3D conditional generative adversarial networks for robust lung segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 732–740 (2018)
Liu, J., Yao, J., Bagheri, M., Sandfort, V., Summers, R.M.: A semi-supervised CNN learning method with pseudo-class labels for atherosclerotic vascular calcification detection. In: International Symposium on Biomedical Imaging (ISBI), pp. 780–783 (2019)
Liu, L., Nie, F., Wiliem, A., Li, Z., Zhang, T., Lovell, B.C.: Multi-modal joint clustering with application for unsupervised attribute discovery. IEEE Trans. Image Process. 27(9), 4345–4356 (2018)
Liu, M.Y., Breuel, T., Kautz, J.: Unsupervised image-to-image translation networks. In: Advances in Neural Information Processing Systems (NeurIPS), pp. 700–708 (2017)
Parab, S.Y., Patil, V.P., Shetmahajan, M., Kanaparthi, A.: Coronary artery calcification on chest computed tomography scan-anaesthetic implications. Indian J. Anaesth. 63(8), 663 (2019)
Roth, H., et al.: DeepOrgan: multi-level deep convolutional networks for automated pancreas segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 556–564. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24553-9_68
Simpson, A.L., et al.: A large annotated medical image dataset for the development and evaluation of segmentation algorithms. arXiv e-prints arXiv:1902.09063 (2019)
Sriram, S.A., Paul, A., Zhu, Y., Sandfort, V., Pickhardt, P.J., Summers, R.: Multilevel U-Net for pancreas segmentation from non-contrast CT scans through domain adaptation. In: Hahn, H.K., Mazurowski, M.A. (eds.) Medical Imaging 2020: Computer-Aided Diagnosis. SPIE, March 2020
Tang, Y.B., Oh, S., Tang, Y.X., Xiao, J., Summers, R.M.: CT-realistic data augmentation using generative adversarial network for robust lymph node segmentation. In: Medical Imaging: Computer-Aided Diagnosis, vol. 10950, p. 109503V (2019)
Tang, Y., et al.: CT image enhancement using stacked generative adversarial networks and transfer learning for lesion segmentation improvement. In: International Workshop on Machine Learning in Medical Imaging, pp. 46–54 (2018)
Tang, Y., Tang, Y., Xiao, J., Summers, R.M.: XLSor: a robust and accurate lung segmentor on chest x-rays using criss-cross attention and customized radiorealistic abnormalities generation. In: International Conference on Medical Imaging with Deep Learning, pp. 457–467 (2019)
Tang, Y.X., Tang, Y.B., Han, M., Xiao, J., Summers, R.M.: Abnormal chest X-ray identification with generative adversarial one-class classifier. In: International Symposium on Biomedical Imaging, pp. 1358–1361 (2019)
Tang, Y., Tang, Y., Sandfort, V., Xiao, J., Summers, R.M.: TUNA-Net: task-oriented unsupervised adversarial network for disease recognition in cross-domain chest X-rays. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 431–440 (2019)
Tran, D., Kucukelbir, A., Dieng, A.B., Rudolph, M., Liang, D., Blei, D.M.: Edward: a library for probabilistic modeling, inference, and criticism. arXiv e-prints arXiv:1610.09787 (2016)
Yan, K., Wang, X., Lu, L., Summers, R.M.: DeepLesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning. J. Med. Imaging 5(3), 1–11 (2018)
Zhou, T., Fu, H., Chen, G., Shen, J., Shen, J., Shao, L.: Hi-net: Hybrid-fusion network for multi-modal MR image synthesis. IEEE Trans. Med. Imaging PP(99), 1 (2020)
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: International Conference on Computer Yision (ICCV), pp. 2223–2232 (2017)
Zhu, Y., Elton, D.C., Lee, S., Pickhardt, P., Summers, R.: Image translation by latent union of subspaces for cross-domain plaque detection. arXiv e-prints arXiv:2005.11384 (2020)
Zhu, Y., Huang, D., la Torre Frade, F.D., Lucey, S.: Complex non-rigid motion 3D reconstruction by union of subspaces. In: Proceedings of CVPR, June 2014
Acknowledgments
This research was supported in part by the Intramural Research Program of the National Institutes of Health Clinical Center. We thank NVIDIA for GPU card donations.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhu, Y. et al. (2020). Cross-domain Medical Image Translation by Shared Latent Gaussian Mixture Model. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12262. Springer, Cham. https://doi.org/10.1007/978-3-030-59713-9_37
Download citation
DOI: https://doi.org/10.1007/978-3-030-59713-9_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59712-2
Online ISBN: 978-3-030-59713-9
eBook Packages: Computer ScienceComputer Science (R0)
