Skip to main content

Advertisement

SpringerLink
Log in

Fast Real-Time Brain Tumor Detection Based on Stimulated Raman Histology and Self-Supervised Deep Learning Model

  • Published:
Journal of Imaging Informatics in Medicine Aims and scope Submit manuscript

Abstract

In intraoperative brain cancer procedures, real-time diagnosis is essential for ensuring safe and effective care. The prevailing workflow, which relies on histological staining with hematoxylin and eosin (H&E) for tissue processing, is resource-intensive, time-consuming, and requires considerable labor. Recently, an innovative approach combining stimulated Raman histology (SRH) and deep convolutional neural networks (CNN) has emerged, creating a new avenue for real-time cancer diagnosis during surgery. While this approach exhibits potential, there exists an opportunity for refinement in the domain of feature extraction. In this study, we employ coherent Raman scattering imaging method and a self-supervised deep learning model (VQVAE2) to enhance the speed of SRH image acquisition and feature representation, thereby enhancing the capability of automated real-time bedside diagnosis. Specifically, we propose the VQSRS network, which integrates vector quantization with a proxy task based on patch annotation for analysis of brain tumor subtypes. Training on images collected from the SRS microscopy system, our VQSRS demonstrates a significant speed enhancement over traditional techniques (e.g., 20–30 min). Comparative studies in dimensionality reduction clustering confirm the diagnostic capacity of VQSRS rivals that of CNN. By learning a hierarchical structure of recognizable histological features, VQSRS classifies major tissue pathological categories in brain tumors. Additionally, an external semantic segmentation method is applied for identifying tumor-infiltrated regions in SRH images. Collectively, these findings indicate that this automated real-time prediction technique holds the potential to streamline intraoperative cancer diagnosis, providing assistance to pathologists in simplifying the process.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data Availability

The data used in this study were obtained from publicly available datasets. The availability of the data can be found at https://opensrh.mlins.org/ and is openly accessible to the research community.

References

  1. Ferlay J, Colombet M, Soerjomataram I, Parkin DM, Piñeros M, Znaor A, Bray F: Cancer statistics for the year 2020: An overview. Int J Cancer. 2021, 149:778–89.

    Article  CAS  Google Scholar 

  2. Chaya N: Poor access to health Services: Ways Ethiopia is overcoming it. Res Comment. 2007, 2:1–6.

    ADS  Google Scholar 

  3. Hamilton PW, Van Diest PJ, Williams R, Gallagher AG: Do we see what we think we see? The complexities of morphological assessment. J Pathol J Pathol Soc G B Irel. 2009, 218:285–91.

    Google Scholar 

  4. Novis DA, Zarbo RJ: Interinstitutional comparison of frozen section turnaround time. Arch Pathol Lab Med. 1997, 121:559.

    CAS  PubMed  Google Scholar 

  5. Gal AA, Cagle PT: The 100-year anniversary of the description of the frozen section procedure. Jama. 2005, 294:3135–7.

    Article  CAS  PubMed  Google Scholar 

  6. Wei L, Min W: Electronic Preresonance Stimulated Raman Scattering Microscopy. J Phys Chem Lett. 2018, 9:4294–301. https://doi.org/10.1021/acs.jpclett.8b00204

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Xiong H, Qian N, Miao Y, Zhao Z, Min W: Stimulated Raman Excited Fluorescence Spectroscopy of Visible Dyes. J Phys Chem Lett. 2019, 10:3563–70. https://doi.org/10.1021/acs.jpclett.9b01289

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Xiong H, Shi L, Wei L, Shen Y, Long R, Zhao Z, Min W: Stimulated Raman excited fluorescence spectroscopy and imaging. Nat Photonics. 2019, 13:412–7.

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  9. Xiong H, Min W: Combining the best of two worlds: Stimulated Raman excited fluorescence. J Chem Phys. 2020, 153.

  10. Saar BG, Freudiger CW, Reichman J, Stanley CM, Holtom GR, Xie XS: Video-rate molecular imaging in vivo with stimulated Raman scattering. science. 2010, 330:1368–70.

  11. Ozeki Y, Umemura W, Otsuka Y, et al.: High-speed molecular spectral imaging of tissue with stimulated Raman scattering. Nat Photonics. 2012, 6:845–51.

    Article  ADS  CAS  Google Scholar 

  12. Liao C-S, Slipchenko MN, Wang P, Li J, Lee S-Y, Oglesbee RA, Cheng J-X: Microsecond scale vibrational spectroscopic imaging by multiplex stimulated Raman scattering microscopy. Light Sci Appl. 2015, 4:e265–e265.

    Article  CAS  PubMed  Google Scholar 

  13. Liao C-S, Wang P, Wang P, Li J, Lee HJ, Eakins G, Cheng J-X: Spectrometer-free vibrational imaging by retrieving stimulated Raman signal from highly scattered photons. Sci Adv. 2015, 1:e1500738. https://doi.org/10.1126/sciadv.1500738

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  14. Wakisaka Y, Suzuki Y, Iwata O, et al.: Probing the metabolic heterogeneity of live Euglena gracilis with stimulated Raman scattering microscopy. Nat Microbiol. 2016, 1:1–4.

    Article  Google Scholar 

  15. Kim D, Choi DS, Kwon J, Shim S-H, Rhee H, Cho M: Selective Suppression of Stimulated Raman Scattering with Another Competing Stimulated Raman Scattering. J Phys Chem Lett. 2017, 8:6118–23. https://doi.org/10.1021/acs.jpclett.7b02752

    Article  CAS  PubMed  Google Scholar 

  16. Bi Y, Yang C, Chen Y, et al.: Near-resonance enhanced label-free stimulated Raman scattering microscopy with spatial resolution near 130 nm. Light Sci Appl. 2018, 7:81.

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  17. Xiong H, Qian N, Zhao Z, Shi L, Miao Y, Min W: Background-free imaging of chemical bonds by a simple and robust frequency-modulated stimulated Raman scattering microscopy. Opt Express. 2020, 28:15663–77.

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  18. Xiong H, Qian N, Miao Y, Zhao Z, Chen C, Min W: Super-resolution vibrational microscopy by stimulated Raman excited fluorescence. Light Sci Appl. 2021, 10:87.

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  19. Wei L, Hu F, Shen Y, et al.: Live-cell imaging of alkyne-tagged small biomolecules by stimulated Raman scattering. Nat Methods. 2014, 11:410–2.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. Hu F, Lamprecht MR, Wei L, Morrison B, Min W: Bioorthogonal chemical imaging of metabolic activities in live mammalian hippocampal tissues with stimulated Raman scattering. Sci Rep. 2016, 6:39660.

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  21. Hu F, Wei L, Zheng C, Shen Y, Min W: Live-cell vibrational imaging of choline metabolites by stimulated Raman scattering coupled with isotope-based metabolic labeling. Analyst. 2014, 139:2312–7.

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  22. Alfonso-García A, Pfisterer SG, Riezman H, Ikonen E, Potma EO: D38-cholesterol as a Raman active probe for imaging intracellular cholesterol storage. J Biomed Opt. 2016, 21:061003–061003.

    Article  PubMed  Google Scholar 

  23. Liao C-S, Wang P, Huang CY, et al.: In Vivo and in Situ Spectroscopic Imaging by a Handheld Stimulated Raman Scattering Microscope. ACS Photonics. 2018, 5:947–54. https://doi.org/10.1021/acsphotonics.7b01214

    Article  CAS  Google Scholar 

  24. Orringer DA, Pandian B, Niknafs YS, et al.: Rapid intraoperative histology of unprocessed surgical specimens via fibre-laser-based stimulated Raman scattering microscopy. Nat Biomed Eng. 2017, 1:0027.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Yang Y, Yang Y, Liu Z, et al.: Microcalcification-Based Tumor Malignancy Evaluation in Fresh Breast Biopsies with Hyperspectral Stimulated Raman Scattering. Anal Chem. 2021, 93:6223–31. https://doi.org/10.1021/acs.analchem.1c00522

    Article  CAS  PubMed  Google Scholar 

  26. Zhang B, Xu H, Chen J, et al.: Highly specific and label-free histological identification of microcrystals in fresh human gout tissues with stimulated Raman scattering. Theranostics. 2021, 11:3074.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Wei Z, Liu X, Yan R, Sun G, Yu W, Liu Q, Guo Q: Pixel-level multimodal fusion deep networks for predicting subcellular organelle localization from label-free live-cell imaging. Front Genet. 2022, 13:1002327.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Wei Z, Liu W, Yu W, Liu X, Yan R, Liu Q, Guo Q: Multiple Parallel Fusion Network for Predicting Protein Subcellular Localization from Stimulated Raman Scattering (SRS) Microscopy Images in Living Cells. Int J Mol Sci. 2022, 23:10827.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Sun G, Liu S, Shi C, Liu X, Guo Q: 3DCNAS: A universal method for predicting the location of fluorescent organelles in living cells in three-dimensional space. Exp Cell Res. 2023, 433:113807.

    Article  CAS  PubMed  Google Scholar 

  30. Freudiger CW, Min W, Saar BG, et al.: Label-free biomedical imaging with high sensitivity by stimulated Raman scattering microscopy. Science. 2008, 322:1857–61.

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  31. Zhang J, Zhao J, Lin H, Tan Y, Cheng J-X: High-speed chemical imaging by dense-net learning of femtosecond stimulated Raman scattering. J Phys Chem Lett. 2020, 11:8573–8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Hu F, Shi L, Min W: Biological imaging of chemical bonds by stimulated Raman scattering microscopy. Nat Methods. 2019, 16:830–42.

    Article  CAS  PubMed  Google Scholar 

  33. Zhang L, Wu Y, Zheng B, et al.: Rapid histology of laryngeal squamous cell carcinoma with deep-learning based stimulated Raman scattering microscopy. Theranostics. 2019, 9:2541.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Tian F, Yang W, Mordes DA, et al.: Monitoring peripheral nerve degeneration in ALS by label-free stimulated Raman scattering imaging. Nat Commun. 2016, 7:13283.

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  35. Lu F-K, Basu S, Igras V, et al.: Label-free DNA imaging in vivo with stimulated Raman scattering microscopy. Proc Natl Acad Sci. 2015, 112:11624–9.

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  36. Fu D, Zhou J, Zhu WS, et al.: Imaging the intracellular distribution of tyrosine kinase inhibitors in living cells with quantitative hyperspectral stimulated Raman scattering. Nat Chem. 2014, 6:614–22.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Freudiger CW, Min W, Holtom GR, Xu B, Dantus M, Sunney Xie X: Highly specific label-free molecular imaging with spectrally tailored excitation-stimulated Raman scattering (STE-SRS) microscopy. Nat Photonics. 2011, 5:103–9.

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  38. Wang MC, Min W, Freudiger CW, Ruvkun G, Xie XS: RNAi screening for fat regulatory genes with SRS microscopy. Nat Methods. 2011, 8:135–8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Ji M, Orringer DA, Freudiger CW, et al.: Rapid, label-free detection of brain tumors with stimulated Raman scattering microscopy. Sci Transl Med. 2013, 5:201ra119–201ra119.

  40. Hollon TC, Pandian B, Adapa AR, et al.: Near real-time intraoperative brain tumor diagnosis using stimulated Raman histology and deep neural networks. Nat Med. 2020, 26:52–8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  41. Liu Z, Su W, Ao J, et al.: Instant diagnosis of gastroscopic biopsy via deep-learned single-shot femtosecond stimulated Raman histology. Nat Commun. 2022, 13:4050. https://doi.org/10.1038/s41467-022-31339-8

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  42. Cho K, Kim KD, Nam Y, et al.: CheSS: Chest X-Ray Pre-trained Model via Self-supervised Contrastive Learning. J Digit Imaging. 2023, 1–9.

  43. Liang X, Dai J, Zhou X, et al.: An Unsupervised Learning-Based Regional Deformable Model for Automated Multi-Organ Contour Propagation. J Digit Imaging. 2023, 1–9.

  44. Goyal P, Caron M, Lefaudeux B, et al.: Self-supervised pretraining of visual features in the wild. ArXiv Prepr ArXiv210301988. 2021.

  45. Holmberg OG, Köhler ND, Martins T, et al.: Self-supervised retinal thickness prediction enables deep learning from unlabelled data to boost classification of diabetic retinopathy. Nat Mach Intell. 2020, 2:719–26.

    Article  Google Scholar 

  46. Batson J, Royer L: Noise2self: Blind denoising by self-supervision. In: International Conference on Machine Learning. PMLR; 2019. 524–33.

  47. Jiang C, Chowdury A, Hou X, et al.: OpenSRH: optimizing brain tumor surgery using intraoperative stimulated Raman histology. Adv Neural Inf Process Syst. 2022, 35:28502–16.

    PubMed  PubMed Central  Google Scholar 

  48. Zhang D, Slipchenko MN, Cheng J-X: Highly sensitive vibrational imaging by femtosecond pulse stimulated Raman loss. J Phys Chem Lett. 2011, 2:1248–53.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  49. Cao R, Nelson SD, Davis S, et al.: Label-free intraoperative histology of bone tissue via deep-learning-assisted ultraviolet photoacoustic microscopy. Nat Biomed Eng. 2023, 7:124–34.

    Article  CAS  PubMed  Google Scholar 

  50. Harms J, Lei Y, Wang T, et al.: Paired cycle-GAN-based image correction for quantitative cone-beam computed tomography. Med Phys. 2019, 46:3998–4009.

    Article  PubMed  PubMed Central  Google Scholar 

  51. Wu H, Flierl M: Vector quantization-based regularization for autoencoders. In: Proceedings of the AAAI Conference on Artificial Intelligence. 2020. 6380–7.

  52. Razavi A, Van den Oord A, Vinyals O: Generating diverse high-fidelity images with vq-vae-2. Adv Neural Inf Process Syst. 2019, 32.

  53. Van Den Oord A, Vinyals O: Neural discrete representation learning. Adv Neural Inf Process Syst. 2017, 30.

  54. McInnes L, Healy J, Melville J: Umap: Uniform manifold approximation and projection for dimension reduction. ArXiv Prepr ArXiv180203426. 2018.

  55. Hussain SF, Ramazan M: Biclustering of human cancer microarray data using co-similarity based co-clustering. Expert Syst Appl. 2016, 55:520–31. https://doi.org/10.1016/j.eswa.2016.02.029

    Article  Google Scholar 

  56. Wu Z, Zhang Y, Zhang JZ, Xia K, Xia F: Determining Optimal Coarse-Grained Representation for Biomolecules Using Internal Cluster Validation Indexes. J Comput Chem. 2020, 41:14–20. https://doi.org/10.1002/jcc.26070

    Article  CAS  PubMed  Google Scholar 

  57. Zelig A, Kariti H, Kaplan N: KMD clustering: Robust general-purpose clustering of biological data. 2023, 2020.10.04.325233. https://doi.org/10.1101/2020.10.04.325233

  58. Cheng Y, Church GM: Biclustering of expression data. In: Ismb. 2000. 93–103.

Download references

Funding

This work was supported by NSFCs (Nos.52361145714, 21673252) and the Beijing Municipal Education Commission, China, under grant number 2019821001, and the fund of Climbing Program Foundation from Beijing Institute of Petrochemical Technology (Project No. BIPTAAI-2021007)

Author information

Authors and Affiliations

  1. Academy of Artificial Intelligence, Beijing Institute of Petrochemical Technology, Beijing, 102617, China

    Zijun Wang, Kaitai Han, Wu Liu, Zhenghui Wang, Chaojing Shi, Xi Liu, Mengyuan Huang, Guocheng Sun, Shitou Liu & Qianjin Guo

Authors
  1. Zijun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kaitai Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhenghui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chaojing Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mengyuan Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Guocheng Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Shitou Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Qianjin Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Zijun Wang constructed the model, implement the algorithm, performed experiments, and partially wrote the manuscript. Kaitai Han and Zhenghui Wang analyzed the results and partially wrote the manuscript. Wu Liu and Mengyuan Huang partially analyzed the results. Chaojing Shi, Xi Liu, Guocheng Sun, and Shitou Liu collected the data. Qianjin Guo supervised the whole study, conceptualized the algorithm, analyzed the results, and partially wrote the manuscript.

Corresponding author

Correspondence to Qianjin Guo.

Ethics declarations

Ethics Approval and Consent to Participate

This study utilized publicly available datasets, which do not involve human participants. As the data are pre-existing and anonymized, no ethics approval or consent to participate was required for this research.

Competing Interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 241 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Z., Han, K., Liu, W. et al. Fast Real-Time Brain Tumor Detection Based on Stimulated Raman Histology and Self-Supervised Deep Learning Model. J Digit Imaging. Inform. med. (2024). https://doi.org/10.1007/s10278-024-01001-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10278-024-01001-4

Keywords

Search

Navigation