Weakly-Supervised Learning-Based Feature Localization for Confocal Laser Endomicroscopy Glioma Images
Confocal Laser Endomicroscopy (CLE) is novel handheld fluorescence imaging technology that has shown promise for rapid intraoperative diagnosis of brain tumor tissue. Currently CLE is capable of image display only and lacks an automatic system to aid the surgeon in diagnostically analyzing the images. The goal of this project was to develop a computer-aided diagnostic approach for CLE imaging of human glioma with feature localization function. Despite the tremendous progress in object detection and image segmentation methods in recent years, most of such methods require large annotated datasets for training. However, manual annotation of thousands of histopathology images by physicians is costly and time consuming. To overcome this problem, we constructed a Weakly-Supervised Learning (WSL)-based model for feature localization that trains on image-level annotations, and then localizes incidences of a class-of-interest in the test image. We developed a novel convolutional neural network for diagnostic features localization from CLE images by employing a novel multiscale activation map that is laterally inhibited and collaterally integrated. To validate our method, we compared the model output to the manual annotation performed by four neurosurgeons on test images. The model achieved 88% mean accuracy and 86% mean intersection over union on intermediate features and 87% mean accuracy and 88% mean intersection over union on restrictive fine features, while outperforming other state of the art methods tested. This system can improve accuracy and efficiency in characterization of CLE images of glioma tissue during surgery, and may augment intraoperative decision-making regarding the tumor margin and improve brain tumor resection.
KeywordsDeep learning Convolutional neural networks Weakly-supervised localization Endomicroscopy Glioma Brain tumor diagnosis Digital pathology
YY is partially supported by NSF grant #1750802. This work was supported by the Newsome Chair in Neurosurgery Research held by MCP and by funds from the Barrow Neurological Foundation. EB acknowledges SP-2044.2018.4.
- 5.Almeida, J.P., Chaichana, K.L., Rincon-Torroella, J., Quinones-Hinojosa, A.: The Value of Extent of Resection of Glioblastomas: Clinical Evidence and Current Approach (2015)Google Scholar
- 8.Izadyyazdanabadi, M., et al.: Improving utility of brain tumor confocal laser endomicroscopy: objective value assessment and diagnostic frame detection with convolutional neural networks. In: Progress in Biomedical Optics and Imaging - Proceedings of SPIE (2017)Google Scholar
- 9.Murthy, N.V., Singh, V., Sun, S., Bhattacharya, S., Chen, T., Comaniciu, D.: Cascaded deep decision networks for classification of endoscopic images. In: Styner, M.A., Angelini, E.D. (eds.) Medical Imaging 2017: Image Processing, p. 101332B (2017)Google Scholar
- 10.Qi, H., Collins, S., Noble, A.: Weakly supervised learning of placental ultrasound images with residual networks. In: Annual Conference on Medical Image Understanding and Analysis, pp. 98–108 (2017)Google Scholar
- 11.Korbar, B., et al.: Looking under the hood: deep neural network visualization to interpret whole-slide image analysis outcomes for colorectal polyps. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 821–827 (2017)Google Scholar
- 12.Gondal, W.M., Köhler, J.M., Grzeszick, R., Fink, G.A., Hirsch, M.: Weakly-supervised localization of diabetic retinopathy lesions in retinal fundus images. arXiv Prepr. arXiv1706.09634. (2017)Google Scholar
- 13.Sailem, H., Arias–Garcia, M., Bakal, C., Zisserman, A., Rittscher, J.: Discovery of rare phenotypes in cellular images using weakly supervised deep learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 49–55 (2017)Google Scholar
- 14.Feng, X., Yang, J., Laine, A.F., Angelini, E.D.: Discriminative localization in CNNs for weakly-supervised segmentation of pulmonary nodules. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 568–576 (2017)Google Scholar
- 15.Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 2921–2929 (2016)Google Scholar
- 16.Baars, B.J., Gage, N.M.: Cognition, Brain and Consciousness (2010)Google Scholar
- 17.Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)Google Scholar
- 18.Jia, Y., et al.: Caffe: Convolutional Architecture for Fast Feature Embedding. arXiv Prepr. arXiv1408.5093 (2014)Google Scholar