The accuracy of the segmentation of the target lesion and at-risk surrounding organs is important for cervical cancer patients treated with three-dimensional (3D) brachytherapy. However, the nature of brachytherapy, organ deformities, metal induced artifacts caused by applicators, and limited operating time make this process a challenge. Deep learning segmentation has recently emerged as an approach to this problem. The basic concept proposed in this paper is accurate segmentation of a pelvic computed tomography image dataset such that patients can obtain more accurate radiotherapy.
In this work, we propose a solution based on 3D U-Net and Long Short-Term Memory (LSTM). The model was trained and tested on a computed tomography pelvic image dataset comprising 51 patients who underwent 3D brachytherapy. The organs that required segmentation included the bladder, bowels, sigmoid colon, rectum, and uterus. We also used self-ensemble to improve segmentation accuracy. The Dice coefficient was used as the evaluation metric to determine the segmentation results for all of the organs under consideration.
The proposed model was evaluated using organ segmentation obtained from 10 patients with newly diagnosed stage I-IVA cervical cancer. The test results showed that segmentation conducted using the proposed model resulted in the following Dice coefficient values (mean, ± standard deviation): 87 (± 6.3) %, 72 (± 9.1) %, 72 (± 7.8) %, 95 (± 3.5) %, 86 (± 4.9) %, 77 (± 8.4) %, 73 (± 10.2) %, and 93 (± 3.5) % for the HRCTV, GTV, bowels, foley, bladder, rectum, sigmoid colon, and uterus, respectively.
This method shows that the combination of 3D U-Net and LSTM and self-ensemble post-processing has high potential for segmentation of a pelvic computed tomography dataset.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
International Agency for Research on Cancer. (2012). Cervical cancer estimated incidence, mortality and prevalence worldwide in 2012. World Health Organization.
Eifel, P. J., Thoms, W. W., Jr., Smith, T. L., Morris, M., & Oswald, M. J. (1994). The relationship between brachytherapy dose and outcome in patients with bulky endocervical tumors treated with radiation alone. International Journal of Radiation Oncology Biology Physics, 28(1), 113–118. https://doi.org/10.1016/0360-3016(94)90148-1
Aghili, M., Andalib, B., Moghaddam, Z. K., Safaie, A. M., Hashemi, F. A., & Darzikolaie, N. M. (2018). Concurrent chemo-radiobrachytherapy with cisplatin and medium dose rate intra-cavitary brachytherapy for locally advanced uterine cervical cancer. Asian Pacific journal of cancer prevention: APJCP, 19(10), 2745. https://doi.org/10.22034/APJCP.2018.19.10.2745
Sorbe, B. G., Horvath, G., Andersson, H., Boman, K., Lundgren, C., & Pettersson, B. (2012). External pelvic and vaginal irradiation versus vaginal irradiation alone as postoperative therapy in medium-risk endometrial carcinoma: a prospective, randomized study—quality-of-life analysis. International Journal of Gynecologic Cancer, 22(7), 1281–1288. https://doi.org/10.1097/IGC.0b013e3182643ba0
Chargari, C., Deutsch, E., Blanchard, P., Gouy, S., Martelli, H., Guérin, F., & Haie-Meder, C. (2019). Brachytherapy: An overview for clinicians. CA: A Cancer Journal for Clinicians, 69(5), 386–401. https://doi.org/10.3322/caac.21578
Derks, K., Steenhuijsen, J. L., van den Berg, H. A., Houterman, S., Cnossen, J., van Haaren, P., & De Jaeger, K. (2018). Impact of brachytherapy technique (2D versus 3D) on outcome following radiotherapy of cervical cancer. Journal of contemporary brachytherapy, 10(1), 17. https://doi.org/10.5114/jcb.2018.73955
Tuntipumiamorn, L., Lohasammakul, S., Dankulchai, P., & Nakkrasae, P. (2018). Comparison of impact of target delineation of computed tomography-and magnetic resonance imaging-guided brachytherapy on dose distribution in cervical cancer. Journal of Contemporary Brachytherapy, 10(5), 418. https://doi.org/10.5114/jcb.2018.78993
Harkenrider, M. M., Alite, F., Silva, S. R., & Small, W., Jr. (2015). Image-based brachytherapy for the treatment of cervical cancer. International Journal of Radiation Oncology Biology Physics, 92(4), 921–934. https://doi.org/10.1016/j.ijrobp.2015.03.010
Ohno, T., Noda, S. E., Okonogi, N., Murata, K., Shibuya, K., Kiyohara, H., & Wakatsuki, M. (2017). In-room computed tomography–based brachytherapy for uterine cervical cancer: results of a 5-year retrospective study. Journal of Radiation Research, 58(4), 543–551. https://doi.org/10.1093/jrr/rrw121
Rijkmans, E. C., Nout, R. A., Rutten, I. H. H. M., Ketelaars, M., Neelis, K. J., Laman, M. S., & Creutzberg, C. L. (2014). Improved survival of patients with cervical cancer treated with image-guided brachytherapy compared with conventional brachytherapy. Gynecologic Oncology, 135(2), 231–238. https://doi.org/10.1016/j.ygyno.2014.08.027
Toita, T., Ohno, T., Ikushima, H., Nishimura, T., Uno, T., Ogawa, K., & Working Group of the Japanese Group of Brachytherapy/Japan Society for Radiation Oncology (JGB/JASTRO). (2018). National survey of intracavitary brachytherapy for intact uterine cervical cancer in Japan. Journal of Radiation Research, 59(4), 469–476. https://doi.org/10.1093/jrr/rry035
Kumar, A., Kim, J., Lyndon, D., Fulham, M., & Feng, D. (2016). An ensemble of fine-tuned convolutional neural networks for medical image classification. IEEE Journal of Biomedical and Health Informatics, 21(1), 31–40. https://doi.org/10.1109/JBHI.2016.2635663
Anwar, S. M., Majid, M., Qayyum, A., Awais, M., Alnowami, M., & Khan, M. K. (2018). Medical image analysis using convolutional neural networks: a review. Journal of Medical Systems, 42(11), 226. https://doi.org/10.1007/s10916-018-1088-1
Cardenas, C. E., Anderson, B. M., Aristophanous, M., Yang, J., Rhee, D. J., McCarroll, R. E., & Fuller, C. D. (2018). Auto-delineation of oropharyngeal clinical target volumes using 3D convolutional neural networks. Physics in Medicine & Biology, 63(21), 215026. https://doi.org/10.1088/1361-6560/aae8a9
Ye, Y., Cai, Z., Huang, B., He, Y., Zeng, P., Zou, G., & Huang, B. (2020). Fully-Automated segmentation of nasopharyngeal carcinoma on dual-sequence MRI using convolutional neural networks. Frontiers in Oncology, 10, 166. https://doi.org/10.3389/fonc.2020.00166
Li, W. (2015). Automatic segmentation of liver tumor in CT images with deep convolutional neural networks. Journal of Computer and Communications, 3(11), 146. https://doi.org/10.4236/jcc.2015.311023
Hesamian, M. H., Jia, W., He, X., & Kennedy, P. (2019). Deep learning techniques for medical image segmentation: Achievements and challenges. Journal of Digital Imaging, 32(4), 582–596. https://doi.org/10.1007/s10278-019-00227-x
Ronneberger, O., Fischer, P., & Brox, T. (2015, October). U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention (pp. 234–241). Springer, Cham. https://doi.org/10.1007/978-3-319-24574-4_28
Long, J., Shelhamer, E., & Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3431–3440). https://doi.org/10.1109/CVPR.2015.7298965
Chen, C., Liu, X., Ding, M., Zheng, J., & Li, J. (2019, October). 3D dilated multi-fiber network for real-time brain tumor segmentation in MRI. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 184–192). Springer, Cham. https://doi.org/10.1007/978-3-030-32248-9_21
Wong, K. C., & Moradi, M. (2019). SegNAS3D: Network Architecture Search with Derivative-Free Global Optimization for 3D Image Segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 393–401). Springer, Cham. https://doi.org/10.1007/978-3-030-32248-9_44
Oktay, O., Schlemper, J., Folgoc, L. L., Lee, M., Heinrich, M., Misawa, K., & Glocker, B. (2018). Attention u-net: Learning where to look for the pancreas. arXiv preprint. arXiv:1804.03999.
Çiçek, Ö., Abdulkadir, A., Lienkamp, S. S., Brox, T., & Ronneberger, O. (2016, October). 3D U-Net: learning dense volumetric segmentation from sparse annotation. In International conference on medical image computing and computer-assisted intervention (pp. 424–432). Springer, Cham. https://doi.org/10.1007/978-3-319-46723-8_49
Zhang, Z., Liu, Q., & Wang, Y. (2018). Road extraction by deep residual u-net. IEEE Geoscience and Remote Sensing Letters, 15(5), 749–753. https://doi.org/10.1109/LGRS.2018.2802944
Milletari, F., Navab, N., & Ahmadi, S. A. (2016, October). V-net: Fully convolutional neural networks for volumetric medical image segmentation. In 2016 fourth international conference on 3D vision (3DV) (pp. 565–571). IEEE. https://doi.org/10.1109/3DV.2016.79
Huang, C., Han, H., Yao, Q., Zhu, S., & Zhou, S. K. (2019, October). 3D U2-Net: A 3D Universal U-Net for Multi-domain Medical Image Segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 291–299). Springer, Cham. https://doi.org/10.1007/978-3-030-32245-8_33
Xingjian, S. H. I., Chen, Z., Wang, H., Yeung, D. Y., Wong, W. K., & Woo, W. C. (2015). Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In Advances in neural information processing systems (pp. 802–810).
Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint. arXiv:1502.03167.
Lin, T. Y., Goyal, P., Girshick, R., He, K., & Dollár, P. (2017). Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision (pp. 2980–2988). https://doi.org/10.1109/ICCV.2017.324
Berman, M., Rannen Triki, A., & Blaschko, M. B. (2018). The lovász-softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 4413–4421). https://doi.org/10.1109/CVPR.2018.00464
Timofte, R., Rothe, R., & Van Gool, L. (2016). Seven ways to improve example-based single image super resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1865–1873). https://doi.org/10.1109/CVPR.2016.206
We thank the anonymous reviewers for their constructive comments. This research work was supported in part by the Ministry of Science and Technology of Taiwan, ROC. (MOST 109-2634-F-006-014 and MOST 108-2221-E-006 -173 -MY2)
About this article
Cite this article
Chang, JH., Lin, KH., Wang, TH. et al. Image Segmentation in 3D Brachytherapy Using Convolutional LSTM. J. Med. Biol. Eng. 41, 636–651 (2021). https://doi.org/10.1007/s40846-021-00624-0
- Cervical cancer
- Pelvic computed tomography image
- 3D U-net