Abstract
Medical image registration and segmentation are complementary functions and combining them can improve each other’s performance. Conventional deep learning (DL) based approaches tackle the two problems separately without leveraging their mutually beneficial information. We propose a DL based approach for joint registration and segmentation (JRS) of chest Xray images. Generative adversarial networks (GANs) are trained to register a floating image to a reference image by combining their segmentation map similarity with conventional feature maps. Intermediate segmentation maps from the GAN’s convolution layers are used in the training stage to generate the final segmentation mask at test time. Experiments on chest Xray images show that JRS gives better registration and segmentation performance than when solving them separately.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Dosovitskiy, A., Fischer, P., et. al.: Flownet: Learning optical flow with convolutional networks. In: Proceedings of IEEE ICCV, pp. 2758–2766 (2015)
van Ginneken, B., Stegmann, M., Loog, M.: Segmentation of anatomical structures in chest radiographs using supervised methods: a comparative study on a public database. Med. Imag. Anal. 10(1), 19–40 (2006)
Goodfellow, I., et al.: Generative adversarial nets. In: Proceedings of NIPS, pp. 2672–2680 (2014)
Harrison, A., Xu, Z., George, K., Lu, L.: Progressive and multi-path holistically nested neural networks for pathological lung segmentation from ct images. In: Proceedings of MICCAI, pp. 621–629 (2017)
Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: arXiv preprint arXiv:1412.6980 (2014)
Klein, S., Staring, M., Murphy, K., Viergever, M., Pluim, J.: Elastix: a toolbox for intensity based medical image registration. IEEE Trans. Med. Imag. 29(1), 196–205 (2010)
Liao, R., et al.: An artificial agent for robust image registration. In: AAAI, pp. 4168–4175 (2017)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of CVPR, pp. 3431–3440 (2015)
Mahapatra, D., Sun, Y.: Joint registration and segmentation of dynamic cardiac perfusion images using mrfs. In: Proceedings of MICCAI, pp. 493–501 (2010)
Miao, S., Z.J. Wang, Y.Z., Liao, R.: Real-time 2d/3d registration via cnn regression. In: IEEE ISBI, pp. 1430–1434 (2016)
Rohe, M., Datar, M., Heimann, T., Sermesant, M., Pennec, X.: SVF-Net: Learning deformable image registration using shape matching. In: Proceedings of MICCAI, pp. 266–274 (2017)
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Proceedings of MICCAI, pp. 234–241 (2015)
Shen, H., Wang, R., Zhang, J., McKenna, S.: Boundary-aware fully convolutional network for brain tumor segmentation. In: Proceedings of MICCAI, pp. 433–441 (2017)
Simonyan, K., Zisserman., A.: Very deep convolutional networks for large-scale image recognition. CoRR absarXiv:1409.1556 (2014)
Sokooti, H., de Vos, B., Berendsen, F., Lelieveldt, B., Isgum, I., Staring, M., et al.: Nonrigid image registration using multiscale 3d convolutional neural networks. In: MICCAI, pp. 232–239 (2017)
de Vos, B., Berendsen, F., Viergever, M., Staring, M., Isgum, I.: End-to-end unsupervised deformable image registration with a convolutional neural network. In: arXiv preprint arXiv:1704.06065 (2017)
Wang, F., Vemuri, B., Eisenschenk, S.: Joint registration and segmentation of neuroanatomic structures from brain mri. J Acad. Radiol. 12(9), 1104–1111 (2006)
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R., et al.: Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: Proceedings of CVPR (2017)
Wang, Z., et al.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Imag. Proc. 13(4), 600–612 (2004)
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Proceedings of CVPR, pp. 2921–2929 (2016)
Zhu, J., park, T., Isola, P., Efros, A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: arXiv preprint arXiv:1703.10593 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Mahapatra, D., Ge, Z., Sedai, S., Chakravorty, R. (2018). Joint Registration And Segmentation Of Xray Images Using Generative Adversarial Networks. In: Shi, Y., Suk, HI., Liu, M. (eds) Machine Learning in Medical Imaging. MLMI 2018. Lecture Notes in Computer Science(), vol 11046. Springer, Cham. https://doi.org/10.1007/978-3-030-00919-9_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-00919-9_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00918-2
Online ISBN: 978-3-030-00919-9
eBook Packages: Computer ScienceComputer Science (R0)