Skip to main content
SpringerLink
Log in

Künstliche Intelligenz und hyperspektrale Bildgebung zur bildgestützten Assistenz in der minimal-invasiven Chirurgie

Artificial intelligence and hyperspectral imaging for image-guided assistance in minimally invasive surgery

  • Leitthema
  • Published:
Die Chirurgie Aims and scope Submit manuscript

Zusammenfassung

Hintergrund

Intraoperative Bildgebung unterstützt Chirurgen bei minimal-invasiven Eingriffen. Die hyperspektrale Bildgebung („hyperspectral imaging“, HSI) ist ein nichtinvasives und kontaktloses optisches Verfahren mit großem diagnostischem Potenzial in der Medizin. Die Kombination mit Ansätzen der künstlichen Intelligenz (KI) für die Analyse der HSI-Daten wird in diesem Artikel intelligente HSI genannt.

Fragestellung

Wie kann intelligente HSI in der minimal-invasiven Viszeralchirurgie eingesetzt werden und wo liegen die Vorteile?

Material und Methoden

Innerhalb verschiedener klinischer Studien wurden HSI-Daten mehrerer In-vivo-Gewebearten und onkologischer Resektate mit einer HSI-Kamera aufgenommen. Verschiedene KI-Algorithmen wurden zur Identifikation und Diskriminierung von Organen, Risikostrukturen und Tumoren evaluiert.

Ergebnisse

Im Tierversuch konnten 20 verschiedene Organe mit hoher Genauigkeit (> 95 %) mittels KI differenziert werden. In-vivo konnten die Nebenschilddrüse mit einem F1-Score von 47 % und einer Sensitivität von 75 % sowie der Gallengang mit einem F1-Score von 79 % und einer Sensitivität von 90 % vom umliegenden Gewebe diskriminiert werden. Des Weiteren konnte Ex-vivo-Tumorgewebe mit einer Fläche unter der „Receiver-operating-characteristic“(ROC)-Kurve (AUC) größer als 0,91 erfolgreich erkannt werden.

Diskussion

Die vorliegenden Arbeiten zeigen die Eignung intelligenter HSI zur automatischen Gewebeerkennung. Trotz großer Fortschritte in den letzten zehn Jahren zeigt intelligentes HSI noch Einschränkungen. So sind genauere und für den Benutzer verständlichere KI-Algorithmen sowie umfangreiche standardisierte und kontinuierlich wachsende Datenbanken notwendig. Weitere klinische Studien sollten die vielseitigen Anwendungen stützen und den Einsatz im klinischen Alltag fest etablieren.

Abstract

Background

Intraoperative imaging assists surgeons during minimally invasive procedures. Hyperspectral imaging (HSI) is a noninvasive and noncontact optical technique with great diagnostic potential in medicine. The combination with artificial intelligence (AI) approaches to analyze HSI data is called intelligent HSI in this article.

Objective

What are the medical applications and advantages of intelligent HSI for minimally invasive visceral surgery?

Material and methods

Within various clinical studies HSI data from multiple in vivo tissue types and oncological resections were acquired using an HSI camera system. Different AI algorithms were evaluated for detection and discrimination of organs, risk structures and tumors.

Results

In an experimental animal study 20 different organs could be differentiated with high precision (> 95%) using AI. In vivo, the parathyroid glands could be discriminated from surrounding tissue with an F1 score of 47% and sensitivity of 75%, and the bile duct with an F1 score of 79% and sensitivity of 90%. Furthermore, ex vivo tumor tissue could be successfully detected with an area under the receiver operating characteristic (ROC) curve (AUC) larger than 0.91.

Discussion

This study demonstrates that intelligent HSI can automatically and accurately detect different tissue types. Despite great progress in the last decade intelligent HSI still has limitations. Thus, accurate AI algorithms that are easier to understand for the user and an extensive standardized and continuously growing database are needed. Further clinical studies should support the various medical applications and lead to the adoption of intelligent HSI in the clinical routine practice.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Abb. 1
Abb. 2
Abb. 3
Abb. 4
Abb. 5
Abb. 6

Literatur

  1. Ayala L, Adler TJ, Seidlitz S et al (2022) Spectral imaging enables contrast agent-free real-time ischemia monitoring in laparoscopic surgery https://doi.org/10.1101/2022.03.08.22271465

    Book  Google Scholar 

  2. Ayala LA, Wirkert SJ, Gröhl J et al (2019) Live monitoring of haemodynamic changes with multispectral image analysis. In: Zhou L, Sarikaya D, Kia SM et al (Hrsg) OR 2.0 context-aware operating theaters and machine learning in clinical neuroimaging. Springer, Cham, S 38–46

    Chapter  Google Scholar 

  3. Cervantes-Sanchez F, Maktabi M, Köhler H et al (2021) Automatic tissue segmentation of hyperspectral images in liver and head neck surgeries using machine learning. AIS. https://doi.org/10.20517/ais.2021.05

    Article  Google Scholar 

  4. Clancy NT, Jones G, Maier-Hein L et al (2020) Surgical spectral imaging. Med Image Anal 63:101699. https://doi.org/10.1016/j.media.2020.101699

    Article  PubMed  PubMed Central  Google Scholar 

  5. Collins T, Maktabi M, Barberio M et al (2021) Automatic recognition of colon and esophagogastric cancer with machine learning and hyperspectral imaging. Diagnostics 11:1810. https://doi.org/10.3390/diagnostics11101810

    Article  PubMed  PubMed Central  Google Scholar 

  6. Cooney GS, Barberio M, Diana M et al (2020) Comparison of spectral characteristics in human and pig biliary system with hyperspectral imaging (HSI). Curr Dir Biomed Eng. https://doi.org/10.1515/cdbme-2020-0012

    Article  Google Scholar 

  7. Eggert D, Bengs M, Westermann S et al (2021) In vivo detection of head and neck tumors by hyperspectral imaging combined with deep learning methods. J Biophoton. https://doi.org/10.1002/jbio.202100167

    Article  Google Scholar 

  8. Halicek M, Fabelo H, Ortega S et al (2019) In-vivo and ex-vivo tissue analysis through hyperspectral imaging techniques: revealing the invisible features of cancer. Cancers 11:756. https://doi.org/10.3390/cancers11060756

    Article  CAS  PubMed Central  Google Scholar 

  9. Holmer A, Marotz J, Wahl P et al (2018) Hyperspectral imaging in perfusion and wound diagnostics—methods and algorithms for the determination of tissue parameters. Biomed Eng 63:547–556. https://doi.org/10.1515/bmt-2017-0155

    Article  Google Scholar 

  10. Jansen-Winkeln B, Barberio M, Chalopin C et al (2021) Feedforward artificial neural network-based colorectal cancer detection using hyperspectral imaging: a step towards automatic optical biopsy. Cancers 13:967. https://doi.org/10.3390/cancers13050967

    Article  PubMed  PubMed Central  Google Scholar 

  11. Jansen-Winkeln B, Dvorak M, Köhler H et al (2022) Border line definition using hyperspectral imaging in colorectal resections. Cancers 14:1188. https://doi.org/10.3390/cancers14051188

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Kho E, Dashtbozorg B, de Boer LL et al (2019) Broadband hyperspectral imaging for breast tumor detection using spectral and spatial information. Biomed Opt Express 10:4496. https://doi.org/10.1364/BOE.10.004496

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Köhler H, Kulcke A, Maktabi M et al (2020) Laparoscopic system for simultaneous high-resolution video and rapid hyperspectral imaging in the visible and near-infrared spectral range. J Biomed Opt 25:86004. https://doi.org/10.1117/1.JBO.25.8.086004

    Article  PubMed Central  Google Scholar 

  14. Lu G, Fei B (2014) Medical hyperspectral imaging: a review. J Biomed Opt 19:10901. https://doi.org/10.1117/1.JBO.19.1.010901

    Article  CAS  PubMed Central  Google Scholar 

  15. Maktabi M, Köhler H, Ivanova M et al (2019) Tissue classification of oncologic esophageal resectates based on hyperspectral data. Int J CARS. https://doi.org/10.1007/s11548-019-02016-x

    Article  Google Scholar 

  16. Maktabi M, Köhler H, Ivanova M et al (2020) Classification of hyperspectral endocrine tissue images using support vector machines. Int J Med Robot 16:1–10. https://doi.org/10.1002/rcs.2121

    Article  PubMed  Google Scholar 

  17. Maktabi M, Tkachenko M, Kohler H et al (2021) Using physiological parameters measured by hyperspectral imaging to detect colorectal cancer. 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Mexico, S 3865–3868

    Google Scholar 

  18. Maktabi M, Wichmann Y, Köhler H et al (2022) Tumor cell identification and classification in esophageal adenocarcinoma specimens by hyperspectral imaging. Sci Rep 12:4508. https://doi.org/10.1038/s41598-022-07524-6

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Nickel F, Studier-Fischer A, Özdemir B et al (2021) Optimization of anastomotic technique and gastric conduit perfusion with hyperspectral imaging in an experimental model for minimally invasive esophagectomy https://doi.org/10.1101/2021.10.03.462901

    Book  Google Scholar 

  20. Nouri D, Lucas Y, Treuillet S (2016) Hyperspectral interventional imaging for enhanced tissue visualization and discrimination combining band selection methods. Int J CARS 11:2185–2197. https://doi.org/10.1007/s11548-016-1449-5

    Article  Google Scholar 

  21. Ortega S, Halicek M, Fabelo H et al (2020) Hyperspectral and multispectral imaging in digital and computational pathology: a systematic review. Biomed Opt Express 11:3195. https://doi.org/10.1364/BOE.386338

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Saiko G, Lombardi P, Au Y et al (2020) Hyperspectral imaging in wound care: a systematic review. Int Wound J. https://doi.org/10.1111/iwj.13474

    Article  PubMed  PubMed Central  Google Scholar 

  23. Seidlitz S, Sellner J, Odenthal J et al (2022) Robust deep learning-based semantic organ segmentation in hyperspectral images. Med Image Anal 80:102488. https://doi.org/10.1016/j.media.2022.102488

    Article  PubMed  Google Scholar 

  24. Shapey J, Xie Y, Nabavi E et al (2019) Intraoperative multispectral and hyperspectral label-free imaging: a systematic review of in vivo clinical studies. J Biophoton. https://doi.org/10.1002/jbio.201800455

    Article  Google Scholar 

  25. Studier-Fischer A, Seidlitz S, Sellner J et al (2021) Spectral organ fingerprints for intraoperative tissue classification with hyperspectral imaging https://doi.org/10.1101/2021.11.24.469943

    Book  Google Scholar 

  26. Wirkert SJ, Kenngott H, Mayer B et al (2016) Robust near real-time estimation of physiological parameters from megapixel multispectral images with inverse Monte Carlo and random forest regression. Int J CARS 11:909–917. https://doi.org/10.1007/s11548-016-1376-5

    Article  Google Scholar 

  27. Wisotzky EL, Uecker FC, Arens P et al (2018) Intraoperative hyperspectral determination of human tissue properties. J Biomed Opt 23:1. https://doi.org/10.1117/1.JBO.23.9.091409

    Article  PubMed  Google Scholar 

  28. Yang G, Ye Q, Xia J (2022) Unbox the black-box for the medical explainable AI via multi-modal and multi-centre data fusion: a mini-review, two showcases and beyond. Inf Fusion 77:29–52. https://doi.org/10.1016/j.inffus.2021.07.016

    Article  PubMed  PubMed Central  Google Scholar 

  29. Yoon J (2022) Hyperspectral imaging for clinical applications. BioChip J 16:1–12. https://doi.org/10.1007/s13206-021-00041-0

    Article  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland

    Claire Chalopin, Annekatrin Pfahl, Hannes Köhler, Marianne Maktabi, Thomas Neumuth & Andreas Melzer

  2. Klinik für Allgemein‑, Viszeral- und Transplantationschirurgie, Universitätsklinikum Heidelberg, Heidelberg, Deutschland

    Felix Nickel, Alexander Studier-Fischer & Beat Peter Müller-Stich

  3. Klinik und Poliklinik für Viszeral‑, Transplantations‑, Thorax- und Gefäßchirurgie, Universitätsklinikum Leipzig, Leipzig, Deutschland

    René Thieme, Robert Sucher, Boris Jansen-Winkeln & Ines Gockel

  4. Abteilung für Allgemein‑, Viszeral- und Onkologische Chirurgie, Klinikum St. Georg Leipzig, Leipzig, Deutschland

    Boris Jansen-Winkeln

  5. Division of Intelligent Medical Systems, Deutsches Krebsforschungszentrum, Heidelberg, Deutschland

    Silvia Seidlitz & Lena Maier-Hein

Authors
  1. Claire Chalopin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Felix Nickel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Annekatrin Pfahl
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hannes Köhler
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marianne Maktabi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. René Thieme
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Robert Sucher
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Boris Jansen-Winkeln
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Alexander Studier-Fischer
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Silvia Seidlitz
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Lena Maier-Hein
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Thomas Neumuth
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Andreas Melzer
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Beat Peter Müller-Stich
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Ines Gockel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Claire Chalopin.

Ethics declarations

Interessenkonflikt

C. Chalopin, F. Nickel, A. Pfahl, H. Köhler, M. Maktabi, R. Thieme, R. Sucher, B. Jansen-Winkeln, A. Studier-Fischer, S. Seidlitz, L. Maier-Hein, T. Neumuth, A. Melzer, B.P. Müller-Stich und I. Gockel geben an, dass kein Interessenkonflikt besteht.

Für diesen Beitrag wurden von den Autoren keine Studien an Menschen oder Tieren durchgeführt. Für die aufgeführten Studien gelten die jeweils dort angegebenen ethischen Richtlinien.

Additional information

Redaktion

I. Gockel, Leipzig

Claire Chalopin und Felix Nickel teilen sich die Erstautorenschaft.

Beat Peter Müller-Stich und Ines Gockel teilen sich die Seniorautorschaft.

figure qr

QR-Code scannen & Beitrag online lesen

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chalopin, C., Nickel, F., Pfahl, A. et al. Künstliche Intelligenz und hyperspektrale Bildgebung zur bildgestützten Assistenz in der minimal-invasiven Chirurgie. Chirurgie 93, 940–947 (2022). https://doi.org/10.1007/s00104-022-01677-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00104-022-01677-w

Schlüsselwörter

Keywords

Search

Navigation