Abstract
Breast lesion segmentation in dynamic contrast enhanced magnetic resonance imaging (DCE-MRI) is a challenging task since DCE-MRI includes not only spatial information within slices but also temporal correlation between different sequences. Previous works only focused on one-side of the spatio-temporal information, therefore could not obtain accurate segmentation results. In this paper, we propose an automatic breast lesion segmentation framework with much less parameters utilizing both spatial and temporal features. Specifically, a breast region extraction is applied to eliminate unconcerned tissues. After that, the segmentation result can be obtained by a spatio-temporal segmentation network, where we exploit a multi-pathway structure and a fusion operation to unearth more spatio-temporal features. The proposed framework is validated on our real dataset and obtains a more accurate segmentation result compared to other approaches.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M et al (2016) Tensorflow: a system for large-scale machine learning. USENIX Symp Oper Syst Design Implement 16:265–283
Amit G, Hadad O, Alpert S, Tlusty T, Gur Y, BenAri R, Hashoul S (2017) Hybrid mass detection in breast mri combining unsupervised saliency analysis and deep learning. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 594–602
Arslan G, Çelik L, Çubuk R, Çelik L, Atasoy MM (2017) Background parenchymal enhancement: is it just an innocent effect of estrogen on the breast? Diagn Interventional Radiol 23(6):414
Badrinarayanan V, Kendall A, Cipolla R (2015) Segnet: A deep convolutional encoder-decoder architecture for image segmentation. arXiv:151100561
Bhooshan N, Giger ML, Jansen SA, Li H, Lan L, Newstead GM (2010) Cancerous breast lesions on dynamic contrast-enhanced mr images: computerized characterization for image-based prognostic markers. Radiology 254(3):680–690
Chen M, Zheng H, Lu C, Tu E, Yang J, Kasabov N (2018) A spatio-temporal fully convolutional network for breast lesion segmentation in dce-mri. In: International conference on neural information processing. Springer, pp 358–368
Chen W, Giger ML, Bick U (2006) A fuzzy c-means (fcm)-based approach for computerized segmentation of breast lesions in dynamic contrast-enhanced mr images1. Acad Radiol 13(1):63–72
Chollet F et al. (2015) Keras. https://github.com/fchollet/keras
Courbariaux M, Hubara I, Soudry D, El-Yaniv R, Bengio Y (2016) Binarized neural networks: training deep neural networks with weights and activations constrained to + 1 or -1. arXiv:1602.02830
Don S, Choi E, Min D (2011) Breast mass segmentation in digital mammography using graph cuts. In: International conference on hybrid information technology. Springer, pp 88–96
Graves A (2012) Long short-term memory. In: Supervised sequence labelling with recurrent neural networks. Springer, pp 37–45
Huang G, Liu Z, van der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: IEEE conference on computer vision and pattern recognition, pp 4700–4708
Jayender J, Chikarmane S, Jolesz FA, Gombos E (2013) Automatic segmentation of invasive breast carcinomas from dce-mri using time series analysis. J Magn Reson Imaging 40(2):467–75
Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv:1412.6980
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105
Lehman CD, Gatsonis C, Kuhl CK, Hendrick RE, Pisano ED, Hanna L, Peacock S, Smazal SF, Maki DD, Julian TB et al (2007) Mri evaluation of the contralateral breast in women with recently diagnosed breast cancer. N Engl J Med 356(13):1295–1303
Li Z, Peng C, Yu G, Zhang X, Deng Y, Sun J (2018) Detnet: a backbone network for object detection. arXiv:180406215
Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: IEEE conference on computer vision and pattern recognition, pp 3431–3440
Maicas G, Carneiro G, Bradley AP (2017a) Globally optimal breast mass segmentation from DCE-MRI using deep semantic segmentation as shape prior. In: IEEE international symposium on biomedical imaging, pp 305–309
Maicas G, Carneiro G, Bradley AP, Nascimento JC, Reid I (2017b) Deep reinforcement learning for active breast lesion detection from DCE-MRI. In: International conference on medical image computing and computer-assisted intervention, pp 665–673
Marrone S, Piantadosi G, Fusco R, Petrillo A, Sansone M, Sansone C (2013) Automatic lesion detection in breast DCE-MRI. In: International conference on image analysis and processing, pp 359–368
Milletari F, Navab N, Ahmadi SA (2016) V-net: Fully convolutional neural networks for volumetric medical image segmentation. In: International conference on 3D vision (3DV). IEEE, pp 565–571
Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9(1):62–66
Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards realtime object detection with region proposal networks. In: Advances in neural information processing systems, pp 91–99
Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, pp 234–241
Siegel RL, Miller KD, Jemal A (2015) Cancer statistics, 2016. CA Cancer J Clin 54(1):8–29
Siu AL (2016) Screening for breast cancer: Us preventive services task force recommendation statement. Ann Intern Med 164(4):279–296
Sun W, Tseng TLB, Zhang J, Qian W (2017) Enhancing deep convolutional neural network scheme for breast cancer diagnosis with unlabeled data. Comput Med Imaging Graph 57:4–9
Wu H, Gallego-Ortiz C, Martel A (2015) Deep artificial neural network approach to automated lesion segmentation in breast. In: the 3rd MICCAI workshop on breast image analysis, pp 73–80
Xingjian S, Chen Z, Wang H, Yeung DY, Wong WK, Woo WC (2015) Convolutional lstm network: a machine learning approach for precipitation nowcasting. In: Advances in neural information processing systems, pp 802–810
Yuan Y, Giger ML, Hui L, Suzuki K, Sennett C (2007) A dual-stage method for lesion segmentation on digital mammograms. Med Phys 34(11):4180–4193
Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. In: IEEE conference on computer vision and pattern recognition, pp 2881–2890
Acknowledgements
This research is partly supported by NSFC, China (No: 61572315, 61603248, 61806125, 61802247), Committee of Science and Technology, Shanghai, China (No. 19510711200) and 1000-Talent Plan (Young Program).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Chen, M., Zheng, H., Lu, C. et al. Accurate breast lesion segmentation by exploiting spatio-temporal information with deep recurrent and convolutional network. J Ambient Intell Human Comput 14, 15609–15617 (2023). https://doi.org/10.1007/s12652-019-01551-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-019-01551-4