Advertisement

Weakly-Supervised Learning-Based Feature Localization for Confocal Laser Endomicroscopy Glioma Images

  • Mohammadhassan Izadyyazdanabadi
  • Evgenii Belykh
  • Claudio Cavallo
  • Xiaochun Zhao
  • Sirin Gandhi
  • Leandro Borba Moreira
  • Jennifer Eschbacher
  • Peter Nakaji
  • Mark C. Preul
  • Yezhou YangEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11071)

Abstract

Confocal Laser Endomicroscopy (CLE) is novel handheld fluorescence imaging technology that has shown promise for rapid intraoperative diagnosis of brain tumor tissue. Currently CLE is capable of image display only and lacks an automatic system to aid the surgeon in diagnostically analyzing the images. The goal of this project was to develop a computer-aided diagnostic approach for CLE imaging of human glioma with feature localization function. Despite the tremendous progress in object detection and image segmentation methods in recent years, most of such methods require large annotated datasets for training. However, manual annotation of thousands of histopathology images by physicians is costly and time consuming. To overcome this problem, we constructed a Weakly-Supervised Learning (WSL)-based model for feature localization that trains on image-level annotations, and then localizes incidences of a class-of-interest in the test image. We developed a novel convolutional neural network for diagnostic features localization from CLE images by employing a novel multiscale activation map that is laterally inhibited and collaterally integrated. To validate our method, we compared the model output to the manual annotation performed by four neurosurgeons on test images. The model achieved 88% mean accuracy and 86% mean intersection over union on intermediate features and 87% mean accuracy and 88% mean intersection over union on restrictive fine features, while outperforming other state of the art methods tested. This system can improve accuracy and efficiency in characterization of CLE images of glioma tissue during surgery, and may augment intraoperative decision-making regarding the tumor margin and improve brain tumor resection.

Keywords

Deep learning Convolutional neural networks Weakly-supervised localization Endomicroscopy Glioma Brain tumor diagnosis Digital pathology 

Notes

Acknowledgement

YY is partially supported by NSF grant #1750802. This work was supported by the Newsome Chair in Neurosurgery Research held by MCP and by funds from the Barrow Neurological Foundation. EB acknowledges SP-2044.2018.4.

References

  1. 1.
    Martirosyan, N.L., et al.: Prospective evaluation of the utility of intraoperative confocal laser endomicroscopy in patients with brain neoplasms using fluorescein sodium: experience with 74 cases. Neurosurg. Focus 40, E11 (2016)CrossRefGoogle Scholar
  2. 2.
    Foersch, S., et al.: Confocal laser endomicroscopy for diagnosis and histomorphologic imaging of brain tumors in vivo. PLoS ONE 7, e41760 (2012)CrossRefGoogle Scholar
  3. 3.
    Belykh, E., et al.: Intraoperative fluorescence imaging for personalized brain tumor resection: current state and future directions. Front. Surg. 3, 55 (2016)CrossRefGoogle Scholar
  4. 4.
    Eschbacher, J., et al.: In vivo intraoperative confocal microscopy for real-time histopathological imaging of brain tumors: clinical article. J. Neurosurg. 116, 854–860 (2012)CrossRefGoogle Scholar
  5. 5.
    Almeida, J.P., Chaichana, K.L., Rincon-Torroella, J., Quinones-Hinojosa, A.: The Value of Extent of Resection of Glioblastomas: Clinical Evidence and Current Approach (2015)Google Scholar
  6. 6.
    Sanai, N., Polley, M.-Y., McDermott, M.W., Parsa, A.T., Berger, M.S.: An extent of resection threshold for newly diagnosed glioblastomas: clinical article. J. Neurosurg. 115, 3–8 (2011)CrossRefGoogle Scholar
  7. 7.
    Izadyyazdanabadi, M., et al.: Convolutional neural networks: ensemble modeling, fine-tuning and unsupervised semantic localization for neurosurgical CLE images. J. Vis. Commun. Image Represent. 54, 10–20 (2018)CrossRefGoogle Scholar
  8. 8.
    Izadyyazdanabadi, M., et al.: Improving utility of brain tumor confocal laser endomicroscopy: objective value assessment and diagnostic frame detection with convolutional neural networks. In: Progress in Biomedical Optics and Imaging - Proceedings of SPIE (2017)Google Scholar
  9. 9.
    Murthy, N.V., Singh, V., Sun, S., Bhattacharya, S., Chen, T., Comaniciu, D.: Cascaded deep decision networks for classification of endoscopic images. In: Styner, M.A., Angelini, E.D. (eds.) Medical Imaging 2017: Image Processing, p. 101332B (2017)Google Scholar
  10. 10.
    Qi, H., Collins, S., Noble, A.: Weakly supervised learning of placental ultrasound images with residual networks. In: Annual Conference on Medical Image Understanding and Analysis, pp. 98–108 (2017)Google Scholar
  11. 11.
    Korbar, B., et al.: Looking under the hood: deep neural network visualization to interpret whole-slide image analysis outcomes for colorectal polyps. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 821–827 (2017)Google Scholar
  12. 12.
    Gondal, W.M., Köhler, J.M., Grzeszick, R., Fink, G.A., Hirsch, M.: Weakly-supervised localization of diabetic retinopathy lesions in retinal fundus images. arXiv Prepr. arXiv1706.09634. (2017)Google Scholar
  13. 13.
    Sailem, H., Arias–Garcia, M., Bakal, C., Zisserman, A., Rittscher, J.: Discovery of rare phenotypes in cellular images using weakly supervised deep learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 49–55 (2017)Google Scholar
  14. 14.
    Feng, X., Yang, J., Laine, A.F., Angelini, E.D.: Discriminative localization in CNNs for weakly-supervised segmentation of pulmonary nodules. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 568–576 (2017)Google Scholar
  15. 15.
    Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 2921–2929 (2016)Google Scholar
  16. 16.
    Baars, B.J., Gage, N.M.: Cognition, Brain and Consciousness (2010)Google Scholar
  17. 17.
    Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)Google Scholar
  18. 18.
    Jia, Y., et al.: Caffe: Convolutional Architecture for Fast Feature Embedding. arXiv Prepr. arXiv1408.5093 (2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Mohammadhassan Izadyyazdanabadi
    • 1
    • 2
  • Evgenii Belykh
    • 2
    • 3
  • Claudio Cavallo
    • 2
  • Xiaochun Zhao
    • 2
  • Sirin Gandhi
    • 2
  • Leandro Borba Moreira
    • 2
  • Jennifer Eschbacher
    • 2
  • Peter Nakaji
    • 2
  • Mark C. Preul
    • 2
  • Yezhou Yang
    • 1
    Email author
  1. 1.School of Computing, Informatics, and Decision System EngineeringArizona State UniversityTempeUSA
  2. 2.Department of NeurosurgeryBarrow Neurological InstitutePhoenixUSA
  3. 3.Department of NeurosurgeryIrkutsk State Medical UniversityIrkutskRussia

Personalised recommendations