Skip to main content
SpringerLink
Log in

Diagnostic Accuracy and Reliability of Deep Learning-Based Human Papillomavirus Status Prediction in Oropharyngeal Cancer

  • Conference paper
  • First Online:
Medical Imaging and Computer-Aided Diagnosis (MICAD 2022)

Part of the book series: Lecture Notes in Electrical Engineering ((LNEE,volume 810))

  • 87 Accesses

Abstract

Oropharyngeal cancer (OPC) patients with associated human papillomavirus (HPV) infection generally present more favorable outcomes than HPV-negative patients and, consequently, their treatment with radiation therapy may be potentially de-escalated. The diagnostic accuracy of a deep learning (DL) model to predict HPV status on computed tomography (CT) images was evaluated in this study, together with its ability to perform unsupervised heatmap-based localization of relevant regions in OPC and HPV infection, i.e., the primary tumor and lymph nodes, as a measure of its reliability. The dataset consisted of 767 patients from one internal and two public collections from The Cancer Imaging Archive and was split into training, validation and test sets using the ratio 60–20–20. Images were resampled to a resolution of 2 mm3 and a sub-volume of 96 pixels3 was automatically cropped, which spanned from the nose until the start of the lungs. Models Genesis was fine-tuned for the classification task. Grad-CAM and Score-CAM were applied to the test subjects that belonged to the internal cohort (n = 24), and the overlap and Dice coefficients between the resulting heatmaps and the planning target volumes (PTVs) were calculated. Final train/validation/test area-under-the-curve (AUC) values of 0.9/0.87/0.87, accuracies of 0.83/0.82/0.79, and F1-scores of 0.83/0.79/0.74 were achieved. The reliability analysis showed an increased focus on dental artifacts in HPV-positive patients, whereas promising overlaps and moderate Dice coefficients with the PTVs were obtained for HPV-negative cases. These findings prove the necessity of performing reliability studies before a DL model is implemented in a real clinical setting, even if there is optimal diagnostic accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Sung, H., Ferlay, J., Siegel, R.L., Laversanne, M., Soerjomataram, I., Jemal, A., Bray, F.: Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA: A Cancer Journal for Clinicians. 71, 209–249 (2021).

    Google Scholar 

  2. de Martel, C., Plummer, M., Vignat, J., Franceschi, S.: Worldwide burden of cancer attributable to HPV by site, country and HPV type. Int J Cancer. 141, 664–670 (2017).

    Article  Google Scholar 

  3. Ang, K.K., Harris, J., Wheeler, R., Weber, R., Rosenthal, D.I., Nguyen-Tân, P.F., Westra, W.H., Chung, C.H., Jordan, R.C., Lu, C., Kim, H., Axelrod, R., Silverman, C.C., Redmond, K.P., Gillison, M.L.: Human papillomavirus and survival of patients with oropharyngeal cancer. N Engl J Med. 363, 24–35 (2010).

    Article  Google Scholar 

  4. Masterson, L., Moualed, D., Liu, Z.W., Howard, J.E.F., Dwivedi, R.C., Tysome, J.R., Benson, R., Sterling, J.C., Sudhoff, H., Jani, P., Goon, P.K.C.: De-escalation treatment protocols for human papillomavirus-associated oropharyngeal squamous cell carcinoma: A systematic review and meta-analysis of current clinical trials. European Journal of Cancer. 50, 2636–2648 (2014).

    Article  Google Scholar 

  5. Fauzi, F.H., Hamzan, N.I., Rahman, N.A., Suraiya, S., Mohamad, S.: Detection of human papillomavirus in oropharyngeal squamous cell carcinoma. J Zhejiang Univ Sci B. 21, 961–976 (2020).

    Article  Google Scholar 

  6. Cantrell, S.C., Peck, B.W., Li, G., Wei, Q., Sturgis, E.M., Ginsberg, L.E.: Differences in imaging characteristics of HPV-positive and HPV-Negative oropharyngeal cancers: a blinded matched-pair analysis. AJNR Am J Neuroradiol. 34, 2005–2009 (2013).

    Article  Google Scholar 

  7. Bogowicz, M., Riesterer, O., Ikenberg, K., Stieb, S., Moch, H., Studer, G., Guckenberger, M., Tanadini-Lang, S.: Computed Tomography Radiomics Predicts HPV Status and Local Tumor Control After Definitive Radiochemotherapy in Head and Neck Squamous Cell Carcinoma. Int J Radiat Oncol Biol Phys. 99, 921–928 (2017).

    Article  Google Scholar 

  8. Lee, J.Y., Han, M., Kim, K.S., Shin, S.-J., Choi, J.W., Ha, E.J.: Discrimination of HPV status using CT texture analysis: tumour heterogeneity in oropharyngeal squamous cell carcinomas. Neuroradiology. 61, 1415–1424 (2019).

    Article  Google Scholar 

  9. Mungai, F., Verrone, G.B., Pietragalla, M., Berti, V., Addeo, G., Desideri, I., Bonasera, L., Miele, V.: CT assessment of tumor heterogeneity and the potential for the prediction of human papillomavirus status in oropharyngeal squamous cell carcinoma. Radiol Med. 124, 804–811 (2019).

    Article  Google Scholar 

  10. Lang, D.M., Peeken, J.C., Combs, S.E., Wilkens, J.J., Bartzsch, S.: Deep Learning Based HPV Status Prediction for Oropharyngeal Cancer Patients. Cancers (Basel). 13, 786 (2021).

    Google Scholar 

  11. La Greca Saint-Esteven, A., Bogowicz, M., Konukoglu, E., Riesterer, O., Balermpas, P., Guckenberger, M., Tanadini-Lang, S., van Timmeren, J.E.: A 2.5D convolutional neural network for HPV prediction in advanced oropharyngeal cancer. Comput Biol Med. 142, 105215 (2022).

    Google Scholar 

  12. Zhou, Z., Sodha, V., Pang, J., Gotway, M.B., Liang, J.: Models Genesis. Medical Image Analysis. 67, 101840 (2021).

    Article  Google Scholar 

  13. The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository | SpringerLink, https://link.springer.com/article/https://doi.org/10.1007/s10278-013-9622-7, last accessed 2021/02/10.

  14. Kwan, J.Y.Y., Su, J., Huang, S.H., Ghoraie, L.S., Xu, W., Chan, B., Yip, K.W., Giuliani, M., Bayley, A., Kim, J., Hope, A.J., Ringash, J., Cho, J., McNiven, A., Hansen, A., Goldstein, D., De Almeida, J.R., Aerts, H.J., Waldron, J.N., Haibe-Kains, B., O’Sullivan, B., Bratman, S.V., Liu, F.-F.: Data from Radiomic Biomarkers to Refine Risk Models for Distant Metastasis in Oropharyngeal Carcinoma, https://wiki.cancerimagingarchive.net/x/XAQGAg, (2019).

  15. Grossberg, A., Mohamed, A., El Halawani, H., Bennett, W., Smith, K., Nolan, T., Chamchod, S., Kantor, M., Browne, T., Hutcheson, K., Gunn, G., Garden, A., Frank, S., Rosenthal, D., Freymann, J., Fuller, C.: Data from Head and Neck Cancer CT Atlas, https://wiki.cancerimagingarchive.net/x/CoFyAQ, (2017).

  16. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. In: 2017 IEEE International Conference on Computer Vision (ICCV). pp. 618–626 (2017).

    Google Scholar 

  17. Wang, H., Wang, Z., Du, M., Yang, F., Zhang, Z., Ding, S., Mardziel, P., Hu, X.: Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). pp. 111–119. IEEE, Seattle, WA, USA (2020).

    Google Scholar 

  18. Salahuddin, Z., Woodruff, H.C., Chatterjee, A., Lambin, P.: Transparency of deep neural networks for medical image analysis: A review of interpretability methods. Computers in Biology and Medicine. 140, 105111 (2022).

    Article  Google Scholar 

  19. Magesh, P., Myloth, R., Tom, R.: An Explainable Machine Learning Model for Early Detection of Parkinson’s Disease using LIME on DaTSCAN Imagery. Computers in Biology and Medicine. 126, 104041 (2020).

    Article  Google Scholar 

  20. Eitel, F., Soehler, E., Bellmann-Strobl, J., Brandt, A.U., Ruprecht, K., Giess, R.M., Kuchling, J., Asseyer, S., Weygandt, M., Haynes, J.-D., Scheel, M., Paul, F., Ritter, K.: Uncovering convolutional neural network decisions for diagnosing multiple sclerosis on conventional MRI using layer-wise relevance propagation. Neuroimage Clin. 24, 102003 (2019).

    Article  Google Scholar 

  21. Böhle, M., Eitel, F., Weygandt, M., Ritter, K.: Layer-Wise Relevance Propagation for Explaining Deep Neural Network Decisions in MRI-Based Alzheimer’s Disease Classification. Frontiers in Aging Neuroscience. 11, (2019).

    Google Scholar 

  22. Panwar, H., Gupta, P.K., Siddiqui, M.K., Morales-Menendez, R., Bhardwaj, P., Singh, V.: A deep learning and grad-CAM based color visualization approach for fast detection of COVID-19 cases using chest X-ray and CT-Scan images. Chaos Solitons Fractals. 140, 110190 (2020).

    Article  MathSciNet  Google Scholar 

  23. Gillison, M.L., D’Souza, G., Westra, W., Sugar, E., Xiao, W., Begum, S., Viscidi, R.: Distinct Risk Factor Profiles for Human Papillomavirus Type 16–Positive and Human Papillomavirus Type 16–Negative Head and Neck Cancers. JNCI: Journal of the National Cancer Institute. 100, 407–420 (2008).

    Google Scholar 

  24. Fujima, N., Andreu-Arasa, V.C., Meibom, S.K., Mercier, G.A., Truong, M.T., Sakai, O.: Prediction of the human papillomavirus status in patients with oropharyngeal squamous cell carcinoma by FDG-PET imaging dataset using deep learning analysis: A hypothesis-generating study. European Journal of Radiology. 126, 108936 (2020).

    Article  Google Scholar 

  25. Wang, H., Zhang, Y., Bai, W., Wang, B., Wei, J., Ji, R., Xin, Y., Dong, L., Jiang, X.: Feasibility of Immunohistochemical p16 Staining in the Diagnosis of Human Papillomavirus Infection in Patients With Squamous Cell Carcinoma of the Head and Neck: A Systematic Review and Meta-Analysis. Front. Oncol. 0, (2020).

    Google Scholar 

  26. Reyes, M., Meier, R., Pereira, S., Silva, C.A., Dahlweid, F.-M., Tengg-Kobligk, H. von, Summers, R.M., Wiest, R.: On the Interpretability of Artificial Intelligence in Radiology: Challenges and Opportunities. Radiology: Artificial Intelligence. 2, e190043 (2020).

    Google Scholar 

Download references

Funding

This work was supported by the Swiss National Science Foundation (310030 173303), the EMDO foundation and the SPHN project IMAGINE.

Author information

Authors and Affiliations

  1. Department of Radiation Oncology, University Hospital Zurich and the University of Zurich, Rämistrasse 100, 8091, Zurich, Switzerland

    Agustina La Greca Saint-Esteven, Marta Bogowicz, Javier Barranco-García, Zahra Khodabakhshi, Oliver Riesterer, Panagiotis Balermpas, Matthias Guckenberger, Janita E. van Timmeren & Stephanie Tanadini-Lang

  2. Computer Vision Laboratory, Department of Information Technology and Electrical Engineering, ETH Zurich, Sternwartstrasse 7, 8092, Zurich, Switzerland

    Agustina La Greca Saint-Esteven & Ender Konukoglu

  3. IBM Research Zurich, Säumerstrasse 4, 8803, Rüschlikon, Switzerland

    Chiara Marchiori & A. Cristiano I. Malossi

  4. Department of Nuclear Medicine, University Hospital Zurich and the University of Zurich, Rämistrasse 100, 8091, Zurich, Switzerland

    Martin Hüllner

Authors
  1. Agustina La Greca Saint-Esteven
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chiara Marchiori
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marta Bogowicz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Javier Barranco-García
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zahra Khodabakhshi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ender Konukoglu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Oliver Riesterer
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Panagiotis Balermpas
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Martin Hüllner
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. A. Cristiano I. Malossi
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Matthias Guckenberger
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Janita E. van Timmeren
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Stephanie Tanadini-Lang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Agustina La Greca Saint-Esteven .

Editor information

Editors and Affiliations

  1. Department of Computer Sciences and Engineering, Shanghai Jiao Tong University, Shanghai, China

    Ruidan Su

  2. Department of Informatics, University of Leicester, Leicester, UK

    Yudong Zhang

  3. College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, Guangdong, China

    Han Liu

  4. Computational Medicine, University of Manchester, Manchester, UK

    Alejandro F Frangi

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

La Greca Saint-Esteven, A. et al. (2023). Diagnostic Accuracy and Reliability of Deep Learning-Based Human Papillomavirus Status Prediction in Oropharyngeal Cancer. In: Su, R., Zhang, Y., Liu, H., F Frangi, A. (eds) Medical Imaging and Computer-Aided Diagnosis. MICAD 2022. Lecture Notes in Electrical Engineering, vol 810. Springer, Singapore. https://doi.org/10.1007/978-981-16-6775-6_23

Download citation

  • DOI: https://doi.org/10.1007/978-981-16-6775-6_23

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-16-6774-9

  • Online ISBN: 978-981-16-6775-6

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics

Search

Navigation