Skip to main content
Log in

ImSpect: Image-driven self-supervised learning for surgical margin evaluation with mass spectrometry

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Real-time assessment of surgical margins is critical for favorable outcomes in cancer patients. The iKnife is a mass spectrometry device that has demonstrated potential for margin detection in cancer surgery. Previous studies have shown that using deep learning on iKnife data can facilitate real-time tissue characterization. However, none of the existing literature on the iKnife facilitate the use of publicly available, state-of-the-art pretrained networks or datasets that have been used in computer vision and other domains.

Methods

In a new framework we call ImSpect, we convert 1D iKnife data, captured during basal cell carcinoma (BCC) surgery, into 2D images in order to capitalize on state-of-the-art image classification networks. We also use self-supervision to leverage large amounts of unlabeled, intraoperative data to accommodate the data requirements of these networks.

Results

Through extensive ablation studies, we show that we can surpass previous benchmarks of margin evaluation in BCC surgery using iKnife data, achieving an area under the receiver operating characteristic curve (AUC) of 81%. We also depict the attention maps of the developed DL models to evaluate the biological relevance of the embedding space

Conclusions

We propose a new method for characterizing tissue at the surgical margins, using mass spectrometry data from cancer surgery.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Code availability

Code used for this study is available on Github: https://github.com/med-i-lab/ImSpect.

References

  1. Adib E, Fernandez A, Afghah F, Prevost JJ (2023) Synthetic ecg signal generation using probabilistic diffusion models. arXiv preprint arXiv:2303.02475

  2. Ahmad Z, Tabassum A, Guan L, Khan NM (2021) Ecg heartbeat classification using multimodal fusion. IEEE Access 9:100615–100626

    Article  Google Scholar 

  3. Alzubaidi L, Fadhel MA, Al-Shamma O, Zhang J, Santamaría J, Duan YR, Oleiwi S (2020) Towards a better understanding of transfer learning for medical imaging: a case study. Appl Sci 10(13):4523

    Article  Google Scholar 

  4. Ayyachamy S, Alex V, Khened M, Krishnamurthi G (2019) Medical image retrieval using resnet-18. In: Medical imaging 2019: imaging informatics for healthcare, research, and applications, pp 233–241

  5. Bai W, Chen C, Tarroni G, Duan J, Guitton F, Petersen SE, Guo Y, Matthews PM, Rueckert D (2019) Self-supervised learning for cardiac mr image segmentation by anatomical position prediction. In: International Conf. on Medical Image Computing and Computer-Assisted Intervention, pp 541–549

  6. Balog J, Sasi-Szabó L, Kinross J, Lewis MR, Muirhead LJ, Veselkov K, Mirnezami R, Dezső B, Damjanovich L, Darzi A, Nicholson TZ, Jeremy K (2013) Intraoperative tissue identification using rapid evaporative ionization mass spectrometry. Sci Transl Med 5(194):194ra93

    Article  PubMed  Google Scholar 

  7. Chen L, Bentley P, Mori K, Misawa K, Fujiwara M, Rueckert D (2019) Self-Supervised learning for medical image analysis using image context restoration. Med Image Anal 58:101539

    Article  PubMed  PubMed Central  Google Scholar 

  8. Fooladgar F, Jamzad A, Connolly L, Santilli A, Kaufmann M, Ren K, Abolmaesumi P, Rudan JF, McKay D, Fichtinger G, Mousavi P (2022) Uncertainty estimation for margin detection in cancer surgery using mass spectrometry. IJCARS 17(12):2305–2313

    Google Scholar 

  9. Hatami N, Gavet Y, Debayle J (2018) Classification of time-series images using deep convolutional neural networks. In: Tenth international conference on machine vision (ICMV 2017), pp 242–249

  10. Jamzad A, Sedghi A, Santilli AM, Janssen NN, Kaufmann M, Ren KY, Vanderbeck K, Wang A, McKay D, Rudan JF, Fichtinger G, Mousavi P (2020) Improved resection margins in surgical oncology using intraoperative mass spectrometry. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part III 23, pp 44–53

  11. Jamzad A, Sedghi A, Santilli AM, Janssen NN, Kaufmann M, Ren KY, Vanderbeck K, Wang A, Mckay D, Rudan JF, Fichtinger G, Mousavi P (2020) Improved resection margins in surgical oncology using intraoperative mass spectrometry. Medical Image Computing and Computer Assisted Intervention, MICCAI Lecture Notes in Computer Science vol 12263 Springer, Cham

  12. Janssen NN, Kaufmann M, Santilli A, Jamzad A, Vanderbeck K, Ren KYM, Ungi T, Mousavi P, Rudan JF, McKay D, Wang A, Fichtinger G (2020) Navigated tissue characterization during skin cancer surgery. Int J Comput Assist Radiol Surg 15(10):1665–1672

    Article  PubMed  Google Scholar 

  13. Jiang W, Zhang D, Ling L, Lin R (2022) Time series classification based on image transformation using feature fusion strategy. Neural Process Lett 54(5):3727–3748

    Article  Google Scholar 

  14. Khosla P, Teterwak P, Wang C, Sarna A, Tian Y, Isola P, Maschinot A, Liu C, Krishnan D (2020) Supervised contrastive learning. Adv Neural Inf Process Syst 33:18661–18673

    Google Scholar 

  15. Koundouros N, Poulogiannis G (2020) Reprogramming of fatty acid metabolism in cancer. Br J Cancer 122(1):4–22

    Article  CAS  PubMed  Google Scholar 

  16. Lohuis PJ, Joshi A, Borggreven PA, Vermeeren L, Zupan-Kajcovski B, Al-Mamgani A, Balm AJ (2016) Aggressive basal cell carcinoma of the head and neck: challenges in surgical management. Eur Arch Otorhinolaryngol 273(11):3881–3889

    Article  PubMed  Google Scholar 

  17. Morid MA, Borjali A, Del Fiol G (2021) A scoping review of transfer learning research on medical image analysis using imagenet. Comput Biol Med 128:104115

    Article  PubMed  Google Scholar 

  18. Santilli AM, Jamzad A, Janssen NN, Kaufmann M, Connolly L, Vanderbeck K, Wang A, McKay D, Rudan JF, Fichtinger G, Mousavi P (2020) Perioperative margin detection in basal cell carcinoma using a deep learning framework: a feasibility study. Int J Comput Assist Radiol Surg 15(5):887–896

    Article  PubMed  Google Scholar 

  19. Schäfer KC, Balog J, Szaniszlo T, Szalay D, Mezey G, Dénes J, Bognar L, Oertel M, Takáts Z (2011) Real time analysis of brain tissue by direct combination of ultrasonic surgical aspiration and sonic spray mass spectrometry. Anal Chem 83(20):7729–7735

    Article  PubMed  Google Scholar 

  20. Taleb A, Lippert C, Klein T, Nabi M (2021) Multimodal self-supervised learning for medical image analysis. In: International Conference on Information Processing in Medical Imaging. pp 661–673

  21. Wang Z, Oates T (2015) Imaging time-series to improve classification and imputation. arXiv preprint arXiv:1506.00327

  22. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2921–2929

Download references

Funding

This work is supported by the Natural Sciences and Engineering Research Council and the Canadian Institutes of Health Research. Gabor Fichtinger is supported by a Canada Research Chair, Tier 1. John F. Rudan is supported by a Britton Smith Chair in Surgery. Parvin Mousavi is supported by a Canadian Institute for Advanced Research AI chair and the Vector Institute. Laura Connolly is supported by the Walter C. Sumner Memorial Fellowship.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Laura Connolly.

Ethics declarations

Conflict of interest

The authors declare no conflicts of interest.

Ethics approval and consent to participate

This study was approved by Queen’s University Health Sciences Research Ethics Board. All patients provided informed verbal and written consent

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Connolly, L., Fooladgar, F., Jamzad, A. et al. ImSpect: Image-driven self-supervised learning for surgical margin evaluation with mass spectrometry. Int J CARS (2024). https://doi.org/10.1007/s11548-024-03106-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11548-024-03106-1

Keywords

Navigation