Abstract
Clinicians usually examine and diagnose patients with multimodality images such as CT and MRI because different modality data of the same anatomical structure are often complementary. This can provide doctors with a variety of information and help doctors to make accurate diagnoses. Inspired by this, the paper proposes a novel method of collaborative spinal segmentation based on spinal CT and MRI images. We use Siam network as architecture and ResNet50 as backbone network to extract high-level semantic features and low-level detail features of two modal images at the same time. Firstly, the high-level feature is enhanced by expanding the receptive field, and then it is input into the channel and spatial attention structure to achieve the optimal combination of high-level semantic information with the help of average pooling and maximum pooling, and learn the mutual information between different modal images. The learned high-level semantic correlation of different modalities will be combined with the up-sampled low-level features for maintaining the uniqueness of their respective modality, and finally the spinal segmentation results of the two modal images will be obtained at the same time. The experimental results show that the performance of multi-modal co-segmentation is better than that of single-modal co-segmentation and ResNet50 segmentation. All codes and data described are available at: https://github.com/1shero/CoSeg_CSA_MLMI.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Windsor, R., Jamaludin, A., Kadir, T., Zisserman, A.: A convolutional approach to vertebrae detection and labelling in whole spine MRI. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12266, pp. 712–722. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59725-2_69
Zhao, S., Chen, B., Chang, H., Wu, X., Li, S.: Discriminative dictionary-embedded network for comprehensive vertebrae tumor diagnosis. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12266, pp. 691–701. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59725-2_67
Chang, H., Zhao, S., Zheng, H., Chen, Y., Li, S.: Multi-vertebrae segmentation from arbitrary spine MR images under global view. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12266, pp. 702–711. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59725-2_68
Masuzawa, N., Kitamura, Y., Nakamura, K., Iizuka, S., Simo-Serra, E.: Automatic segmentation, localization, and identification of vertebrae in 3D CT images using cascaded convolutional neural networks. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12266, pp. 681–690. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59725-2_66
Rother, C., et al.: Cosegmentation of image pairs by histogram matching-incorporating a global constraint into MRFS. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), vol. 1, pp. 993–1000 (2006)
Cai, Y., et al.: Multi-modality vertebra recognition in arbitrary views using 3D deformable hierarchical model. IEEE Trans. Med. Imaging 34(8), 1676–1693 (2015)
Li, B., et al.: Group-wise deep object co-segmentation with co-attention recurrent neural network. In: Proceedings of the IEEE/CVF International Conference on Computer Vision 2019, pp. 8519–852 (2019)
He, K., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2016, pp. 770–778 (2016)
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 833–851. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_49
Chen, L.-C., et al.: Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2017)
Chen, J., et al.: Channel and spatial attention based deep object co-segmentation. Knowl. Based Syst. 211, 106550 (2021)
The xVertSeg Challenge. http://lit.fe.uni-lj.si/xVertSeg/. Accessed 03 Mar 2020
Acknowledgement
This study is supported by the Medical-Industrial Integration Project of Fudan University.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Zou, Y., Shi, Y. (2021). Co-segmentation of Multi-modality Spinal Image Using Channel and Spatial Attention. In: Lian, C., Cao, X., Rekik, I., Xu, X., Yan, P. (eds) Machine Learning in Medical Imaging. MLMI 2021. Lecture Notes in Computer Science(), vol 12966. Springer, Cham. https://doi.org/10.1007/978-3-030-87589-3_30
Download citation
DOI: https://doi.org/10.1007/978-3-030-87589-3_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87588-6
Online ISBN: 978-3-030-87589-3
eBook Packages: Computer ScienceComputer Science (R0)
