Abstract
Lesion segmentation in medical imaging is key to evaluating treatment response. We have recently shown that reinforcement learning can be applied to radiological images for lesion localization. Furthermore, we demonstrated that reinforcement learning addresses important limitations of supervised deep learning, namely, it can eliminate the requirement for large amounts of annotated training data and can provide valuable intuition lacking in supervised approaches. However, we did not address the fundamental task of lesion/structure-of-interest segmentation. Here we introduce a method combining unsupervised deep learning clustering with reinforcement learning to segment brain lesions on MRI. We initially clustered images using unsupervised deep learning clustering to generate candidate lesion masks for each MRI image. The user then selected the best mask for each of 10 training images. We then trained a reinforcement learning algorithm to select the masks. We tested the corresponding trained deep Q network on a separate testing set of 10 images. For comparison, we also trained and tested a U-net supervised deep learning network on the same set of training/testing images. Whereas the supervised approach quickly overfits the training data and predictably performed poorly on the testing set (16% average Dice score), the unsupervised deep clustering and reinforcement learning achieved an average Dice score of 83%. We have demonstrated a proof-of-principle application of unsupervised deep clustering and reinforcement learning to segment brain tumors. The approach represents human-allied AI that requires minimal input from the radiologist without the need for hand-traced annotation.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
R. Achanta et al., SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)
V. Buhrmester, D. Münch, M. Arens, Analysis of explainers of black box deep neural networks for computer vision: a survey (2019). arXiv:1911.12116
G. Chartrand et al., Deep learning: a primer for radiologists. Radiographics 37(7), 2113–2131 (2017)
L.C.H. Da Cruz et al., Pseudoprogression and pseudoresponse: imaging challenges in the assessment of posttreatment glioma. Am. J. Neuroradiol. 32(11), 1978–1985 (2011)
F. Gaillard, Cases. Last accessed 10 Nov 2020 (2020). https://radiopaedia.org/cases/9015/studies/9774?lang=us
I.J. Goodfellow, J. Shlens, C. Szegedy, Explaining and harnessing adversarial examples (2014). arXiv:1412.6572
K. He et al., Mask r-cnn, in Proceedings of the IEEE International Conference on Computer Vision (2017), pp. 2961–2969
Publicly available web image. Glioma. Last accessed 10 Nov 2020 (2020). https://www.slideshare.net/yashika54/fig-21b-axialt1-weighted-wtd-mri
X. Liu et al., A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digit. Health 1(6), e271–e297 (2019)
M.A. Mazurowski et al., Deep learning in radiology: an overview of the concepts and a survey of the state of the art with focus on MRI. J. Magn. Reson. Imaging 49(4). 939–954 (2019)
M.P. McBee et al., Deep learning in radiology. Acad. Radiol. 25(11), 1472–1480 (2018)
S. Mohan, Advanced neuroimaging of brain tumors radiogenomics, biomarkers & response assessment. Last accessed 10 Nov 2020 (2018). https://www.pennmedicine.org/cancer
D. Preston, Glioma. Last accessed 10 Nov 2020 (2006). https://case.edu/med/neurology/NR/Glioma
A. Radbruch et al., Relevance of T2 signal changes in the assessment of progression of glioblastoma according to the response assessment in neurooncology criteria. Neuro-oncology 14(2), 222–229 (2011)
O. Ronneberger, P. Fischer, T. Brox, U-net: convolutional networks for biomedical image segmentation, in International Conference on Medical Image Computing and Computer-Assisted Intervention (Springer, 2015), pp. 234–241
L. Saba et al., The present and future of deep learning in radiology. Eur. J. Radiol. 114, 14–24 (2019)
J. Buckner, S. Kizilbash, Cancer therapy advisor: central nervous system malignancies. Last accessed 10 Nov 2020 (2020). https://www.cancertherapyadvisor.com/home/decision-support-in-medicine/oncology/central-nervous-system-malignancies/
A. Smits et al., Neurological impairment linked with cortico-subcortical infiltration of diffuse low-grade gliomas at initial diagnosis supports early brain plasticity. Front. Neurol. 6, 137 (2015)
J. Stember, H. Shalu, Deep reinforcement learning to detect brain lesions on MRI: a proof-of-concept application of reinforcement learning to medical images (2020). arXiv:2008.02708
J.N. Stember, H. Shalu, Reinforcement learning using Deep Q networks and Q learning accurately localizes brain tumors on MRI with very small training sets (2020). arXiv:2010.10763
J.N. Stember et al., Convolutional neural networks for the detection and measurement of cerebral aneurysms on magnetic resonance angiography. J. Digit. Imaging 32(5), 808–815 (2019)
J.N. Stember et al., Eye tracking for deep learning segmentation using convolutional neural networks. J. Digit. Imaging 32(4), 597–604 (2019)
R.S. Sutton, A.G. Barto, Reinforcement Learning: An Introduction (MIT Press, 2018)
X. Wang et al., Inconsistent performance of deep learning models on mammogram classification. J. Am. Coll. Radiol. (2020)
P.Y. Wen et al., Response assessment in neuro-oncology clinical trials. J. Clin. Oncol. 35(21), 2439 (2017)
J. Yang, D. Parikh, D. Batra, Joint unsupervised learning of deep representations and image clusters, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 5147–5156
J. Zhang et al., Clinical applications of contrast-enhanced perfusion MRI techniques in gliomas: recent advances and current challenges, in Contrast Media & Molecular Imaging 2017 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Stember, J.N., Shalu, H. (2023). Unsupervised Deep Clustering and Reinforcement Learning Can Accurately Segment MRI Brain Tumors with Very Small Training Sets. In: Thampi, S.M., Mukhopadhyay, J., Paprzycki, M., Li, KC. (eds) International Symposium on Intelligent Informatics. ISI 2022. Smart Innovation, Systems and Technologies, vol 333. Springer, Singapore. https://doi.org/10.1007/978-981-19-8094-7_19
Download citation
DOI: https://doi.org/10.1007/978-981-19-8094-7_19
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-8093-0
Online ISBN: 978-981-19-8094-7
eBook Packages: EngineeringEngineering (R0)