Transferability of Deep Learning Algorithms for Malignancy Detection in Confocal Laser Endomicroscopy Images from Different Anatomical Locations of the Upper Gastrointestinal Tract

  • Marc AubrevilleEmail author
  • Miguel Goncalves
  • Christian Knipfer
  • Nicolai Oetter
  • Tobias Würfl
  • Helmut Neumann
  • Florian Stelzle
  • Christopher Bohr
  • Andreas Maier
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1024)


Squamous Cell Carcinoma (SCC) is the most common cancer type of the epithelium and is often detected at a late stage. Besides invasive diagnosis of SCC by means of biopsy and histo-pathologic assessment, Confocal Laser Endomicroscopy (CLE) has emerged as noninvasive method that was successfully used to diagnose SCC in vivo. For interpretation of CLE images, however, extensive training is required, which limits its applicability and use in clinical practice of the method. To aid diagnosis of SCC in a broader scope, automatic detection methods have been proposed. This work compares two methods with regard to their applicability in a transfer learning sense, i.e. training on one tissue type (from one clinical team) and applying the learnt classification system to another entity (different anatomy, different clinical team). Besides a previously proposed, patch-based method based on convolutional neural networks, a novel classification method on image level (based on a pre-trained Inception V.3 network with dedicated preprocessing and interpretation of class activation maps) is proposed and evaluated.

The newly presented approach improves recognition performance, yielding accuracies of 91.63% on the first data set (oral cavity) and 92.63% on a joint data set. The generalization from oral cavity to the second data set (vocal folds) lead to similar area-under-the-ROC curve values than a direct training on the vocal folds data set, indicating good generalization.


Confocal Laser Endomicroscopy Transfer learning Head and neck squamous cell carcinoma 


  1. 1.
    Aubreville, M., et al.: Patch-based carcinoma detection on confocal laser endomicroscopy images - a cross-site robustness assessment. In: Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies, BIOIMAGING, vol. 2, pp. 27–34. INSTICC, SciTePress (2018).
  2. 2.
    Aubreville, M., et al.: Automatic classification of cancerous tissue in laser endomicroscopy images of the oral cavity using deep learning. Sci. Rep. 7(1), s41598–017 (2017).
  3. 3.
    Betz, C.S., et al.: Optical diagnostic methods for early tumour diagnosis in the upper aerodigestive tract. HNO 64(1), 41–48 (2016). Scholar
  4. 4.
    Chauhan, S.S., et al.: Confocal laser endomicroscopy. Gastrointest. Endosc. 80(6), 928–938 (2014). Scholar
  5. 5.
    Cikojević, D., Glunčić, I., Pešutić-Pisac, V.: Comparison of contact endoscopy and frozen section histopathology in the intra-operative diagnosis of laryngeal pathology. J. Laryngol. Otol. 122(8), 836–839 (2008). Scholar
  6. 6.
    Esteva, A., et al.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639), 115–118 (2017). Scholar
  7. 7.
    Forastiere, A., Koch, W., Trotti, A., Sidransky, D.: Head and neck cancer. N. Engl. J. Med. 345(26), 1890–1900 (2001). Scholar
  8. 8.
    Goncalves, M., Iro, H., Dittberner, A., Agaimy, A., Bohr, C.: Value of confocal laser endomicroscopy in the diagnosis of vocal cord lesions. Eur. Rev. Med. Pharmacol. Sci. 21, 3990–3997 (2017)Google Scholar
  9. 9.
    Goncalves, M., et al.: Probe-based confocal laser endomicroscopy in detecting malignant lesions of the vocal folds. Acta Otorhinolaryngol. Ital. (2019).
  10. 10.
    Izadyyazdanabadi, M., et al.: Weakly-supervised learning-based feature localization for confocal laser endomicroscopy glioma images. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 300–308. Springer, Cham (2018). Scholar
  11. 11.
    Izadyyazdanabadi, M., et al.: Improving utility of brain tumor confocal laser endomicroscopy: objective value assessment and diagnostic frame detection with convolutional neural networks. In: Proceedings of the SPIE, vol. 10134, p. 101342J (2017).
  12. 12.
    Jaremenko, C., et al.: Classification of confocal laser endomicroscopic images of the oral cavity to distinguish pathological from healthy tissue. In: Handels, H., Deserno, T.M., Meinzer, H.-P., Tolxdorff, T. (eds.) Bildverarbeitung für die Medizin 2015. INFORMAT, pp. 479–485. Springer, Heidelberg (2015). Scholar
  13. 13.
    Knipfer, C., et al.: Raman difference spectroscopy: a non-invasive method for identification of oral squamous cell carcinoma. Biomed. Opt. Express 5(9), 3252–3265 (2014). Scholar
  14. 14.
    Lüllmann-Rauch, R., Paulsen, F.: Taschenlehrbuch Histologie, 4th edn. Thieme, Stuttgart (2012)Google Scholar
  15. 15.
    Maier, H., Dietz, A., Gewelke, U., Heller, W., Weidauer, H.: Tobacco andalcohol and the risk of head and neck cancer. Clin. Investig. 70(3–4), 320–327 (1992). Scholar
  16. 16.
    Murthy, V.N., Singh, V., Sun, S., Bhattacharya, S., Chen, T., Comaniciu, D.: Cascaded deep decision networks for classification of endoscopic images. In: Proceedings of the SPIE, vol. 10133 (2017).
  17. 17.
    Muto, M.: Squamous cell carcinoma in situ at oropharyngeal and hypopharyngeal mucosal sites. Cancer 101(6), 1375–1381 (2004). Scholar
  18. 18.
    Neumann, H., Kiesslich, R., Wallace, M.B., Neurath, M.F.: Confocal laser endomicroscopy: technical advances and clinical applications. Gastroenterology 139(2), 388–392.e2 (2010). Scholar
  19. 19.
    Neumann, H., Langner, C., Neurath, M.F., Vieth, M.: Confocal laser endomicroscopy for diagnosis of barrett’s esophagus. Front. Oncol. 2 (2012).
  20. 20.
    Neumann, H., Vieth, M., Atreya, R., Neurath, M.F., Mudter, J.: Prospective evaluation of the learning curve of confocal laser endomicroscopy in patients with IBD. Histol. Histopathol. 26(7), 867–872 (2011). Scholar
  21. 21.
    Oetter, N., et al.: Development and validation of a classification and scoring system for the diagnosis of oral squamous cell carcinomas through confocal laser endomicroscopy. J. Transl. Med. 14(1), 1–11 (2016). Scholar
  22. 22.
    Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Is object localization for free? - weakly-supervised learning with convolutional neural networks. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 685–694. IEEE (2015).
  23. 23.
    Parkin, D.M., Bray, F., Ferlay, J., Pisani, P.: Global cancer statistics, 2002. CA: Cancer J. Clin. 55(2), 74–108 (2005). Scholar
  24. 24.
    Robert Koch Institut: Zentrum für Krebsregisterdaten: Krebs in Deutschland für 2013/2014, 11th edn. Robert Koch Institut, Berlin (2017)Google Scholar
  25. 25.
    Rohen, J.W.: Histologische Differentialdiagnose, 5th edn. Schattauer, Stuttgart (1994)Google Scholar
  26. 26.
    Rohen, J.W., Lütjen-Drecoll, E.: Funktionelle Histologie, 4th edn. Schattauer, Stuttgart (2000)Google Scholar
  27. 27.
    Stoeve, M., et al.: Motion artifact detection in confocal laser endomicroscopy images. Bildverarbeitung für die Medizin 2018. INFORMAT, pp. 328–333. Springer, Heidelberg (2018). Scholar
  28. 28.
    Szegedy, C., et al.: Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), September 2015.
  29. 29.
    Vo, K., Jaremenko, C., Bohr, C., Neumann, H., Maier, A.: Automatic classification and pathological staging of confocal laser endomicroscopic images of the vocal cords. Bildverarbeitung für die Medizin 2017. INFORMAT, pp. 312–317. Springer, Heidelberg (2017). Scholar
  30. 30.
    Westra, W.H.: The pathology of HPV-related head and neck cancer: implications for the diagnostic pathologist. Semin. Diagn. Pathol. 32(1), 42–53 (2015). Scholar
  31. 31.
    Xing, F., Xie, Y., Su, H., Liu, F., Yang, L.: Deep learning in microscopy image analysis: a survey. IEEE Trans. Neural Netw. Learn. Syst. PP(99), 1–19 (2017). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Marc Aubreville
    • 1
    Email author
  • Miguel Goncalves
    • 2
  • Christian Knipfer
    • 3
    • 4
  • Nicolai Oetter
    • 4
    • 5
  • Tobias Würfl
    • 1
  • Helmut Neumann
    • 6
  • Florian Stelzle
    • 4
    • 5
  • Christopher Bohr
    • 7
  • Andreas Maier
    • 1
    • 4
  1. 1.Pattern Recognition Lab, Computer ScienceFriedrich-Alexander-Universität Erlangen-NürnbergErlangenGermany
  2. 2.Department of Otorhinolaryngology, Head and Neck SurgeryUniversity Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-NürnbergErlangenGermany
  3. 3.Department of Oral and Maxillofacial SurgeryUniversity Medical Center Hamburg-EppendorfHamburgGermany
  4. 4.Erlangen Graduate School in Advanced Optical Technologies (SAOT)Friedrich-Alexander-Universität Erlangen-NürnbergErlangenGermany
  5. 5.Department of Oral and Maxillofacial SurgeryUniversity Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-NürnbergErlangenGermany
  6. 6.First Department of Internal MedicineUniversity Medical Center Mainz, Johannes Gutenberg-Universität MainzErlangenGermany
  7. 7.Department of Otorhinolaryngology, Head and Neck SurgeryUniversität Regensburg, University HospitalRegensburgGermany

Personalised recommendations