Skip to main content
SpringerLink
Log in

An ultrasound-based ensemble machine learning model for the preoperative classification of pleomorphic adenoma and Warthin tumor in the parotid gland

  • Head and Neck
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objectives

The preoperative classification of pleomorphic adenomas (PMA) and Warthin tumors (WT) in the parotid gland plays an essential role in determining therapeutic strategies. This study aims to develop and validate an ultrasound-based ensemble machine learning (USEML) model, employing nonradiative and noninvasive features to differentiate PMA from WT.

Methods

A total of 203 patients with histologically confirmed PMA or WT who underwent parotidectomy from two centers were enrolled. Clinical factors, ultrasound (US) features, and radiomic features were extracted to develop three types of machine learning model: clinical models, US models, and USEML models. The diagnostic performance of the USEML model, as well as that of physicians based on experience, was evaluated and validated using receiver operating characteristic (ROC) curves in internal and external validation cohorts. DeLong’s test was used for comparisons of AUCs. SHAP values were also utilized to explain the classification model.

Results

The USEML model achieved the highest AUC of 0.891 (95% CI, 0.774–0.961), surpassing the AUCs of both the US (0.847; 95% CI, 0.720–0.932) and clinical (0.814; 95% CI, 0.682–0.908) models. The USEML model also outperformed physicians in both internal and external validation datasets (both p < 0.05). The sensitivity, specificity, negative predictive value, and positive predictive value of the USEML model and physician experience were 89.3%/75.0%, 87.5%/54.2%, 87.5%/65.6%, and 89.3%/65.0%, respectively.

Conclusions

The USEML model, incorporating clinical factors, ultrasound factors, and radiomic features, demonstrated efficient performance in distinguishing PMA from WT in the parotid gland.

Clinical relevance statement

This study developed a machine learning model for preoperative diagnosis of pleomorphic adenoma and Warthin tumor in the parotid gland based on clinical, ultrasound, and radiomic features. Furthermore, it outperformed physicians in an external validation dataset, indicating its potential for clinical application.

Key Points

• Differentiating pleomorphic adenoma (PMA) and Warthin tumor (WT) affects management decisions and is currently done by invasive biopsy.

• Integration of US-radiomic, clinical, and ultrasound findings in a machine learning model results in improved diagnostic accuracy.

• The ultrasound-based ensemble machine learning (USEML) model consistently outperforms physicians, suggesting its potential applicability in clinical settings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Abbreviations

CI:

Confidence interval

ICC:

Inter- and intraclass correlation coefficient

ML:

Machine learning

OR:

Odds ratio

PMA:

Pleomorphic adenoma

SHAP:

SHapley Additive exPlanation

US:

Ultrasound

USEML:

Ultrasound-based ensemble machine learning

WT:

Warthin tumor

References

  1. Sentani KOI, Ozasa K et al (2019) Characteristics of 5015 salivary gland neoplasms registered in the Hiroshima tumor tissue registry over a period of 39 years. J Clin Med 26:566

    Article  Google Scholar 

  2. Xu XXJ, Ling R, Ouyang S et al (2023) Single-cell transcriptomic analysis uncovers the origin and intratumoral heterogeneity of parotid pleomorphic adenoma. Int J Oral Sci 15:38

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Skalova A, Hyrcza MD, Leivo I (2022) Update from the 5th Edition of the World Health Organization Classification of Head and Neck Tumors: Salivary Glands. Head Neck Pathol 16:40-53

  4. Lee DH, Yoon TM, Lee JK, Lim SC (2019) Surgical treatment strategy in Warthin tumor of the parotid gland. Braz J Otorhinolaryngol 85:546–550

    Article  PubMed  Google Scholar 

  5. Yeung DCM, Leung HHS, Lai R et al (2023) A safety and feasibility trial of ultrasound-guided radiofrequency ablation of parotid Warthin’s tumor. Otolaryngol Head Neck Surg. https://doi.org/10.1002/ohn.417

    Article  PubMed  Google Scholar 

  6. Park S, Lee YC, Lim SJ, Kim C (2023) Malignant transformation of Warthin tumor in the cervical lymph node. Clin Nucl Med 48:342–344

    Article  PubMed  Google Scholar 

  7. Key S, Chia C, Hasan Z, Sundaresan P, Dwivedi RC, Riffat F (2022) Systematic review of prognostic factors in carcinoma ex pleomorphic adenoma. Oral Oncol 133:106052

    Article  PubMed  Google Scholar 

  8. Al-Balas H, Metwalli ZA, Eberson S, Sada DM (2021) Clinicopathological features of incidental parotid lesions. Head Face Med 17:10

    Article  PubMed  PubMed Central  Google Scholar 

  9. Huang N, Chen Y, She D, Xing Z, Chen T, Cao D (2022) Diffusion kurtosis imaging and dynamic contrast-enhanced MRI for the differentiation of parotid gland tumors. Eur Radiol 32:2748–2759

    Article  PubMed  Google Scholar 

  10. Abdel Razek AA, Ashmalla GA, Gaballa G, Nada N (2015) Pilot study of ultrasound parotid imaging reporting and data system (PIRADS): inter-observer agreement Eur J Radiol. 2533-2538

  11. Fisher R, Ronen O (2022) Cytologic diagnosis of parotid gland Warthin tumor: systematic review and meta-analysis. Head Neck 44:2277–2287

    Article  PubMed  PubMed Central  Google Scholar 

  12. Peiffer-Smadja N, Rawson TM, Ahmad R et al (2020) Machine learning for clinical decision support in infectious diseases: a narrative review of current applications. Clin Microbiol Infect 26:584–595

    Article  CAS  PubMed  Google Scholar 

  13. Handelman GS, Kok HK, Chandra RV, Razavi AH, Lee MJ, Asadi H (2018) eDoctor: machine learning and the future of medicine. J Intern Med 284:603–619

    Article  CAS  PubMed  Google Scholar 

  14. Lambin P, Rios-Velazquez E, Leijenaar R et al (2012) Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer 48:441–446

    Article  PubMed  PubMed Central  Google Scholar 

  15. Li Q, Jiang T, Zhang C et al (2022) A nomogram based on clinical information, conventional ultrasound and radiomics improves prediction of malignant parotid gland lesions. Cancer Lett 527:107–114

    Article  CAS  PubMed  Google Scholar 

  16. Al Ajmi E, Forghani B, Reinhold C, Bayat M, Forghani R (2018) Spectral multi-energy CT texture analysis with machine learning for tissue classification: an investigation using classification of benign parotid tumours as a testing paradigm. Eur Radiol 28:2604–2611

    Article  PubMed  Google Scholar 

  17. Zheng YL, Zheng YN, Li CF et al (2022) Comparison of different machine models based on multi-phase computed tomography radiomic analysis to differentiate parotid basal cell adenoma from pleomorphic adenoma. Front Oncol 12:889833

    Article  PubMed  PubMed Central  Google Scholar 

  18. Feng B, Wang Z, Cui J et al (2023) Distinguishing parotid polymorphic adenoma and Warthin tumor based on the CT radiomics nomogram: a multicenter study. Acad Radiol 30:717–726

    Article  PubMed  Google Scholar 

  19. Hu Z, Guo J, Feng J, Huang Y, Xu H, Zhou Q (2023) Value of T2-weighted-based radiomics model in distinguishing Warthin tumor from pleomorphic adenoma of the parotid. Eur Radiol 33:4453–4463

    Article  PubMed  Google Scholar 

  20. Zheng Y, Zhou D, Liu H, Wen M (2022) CT-based radiomics analysis of different machine learning models for differentiating benign and malignant parotid tumors. Eur Radiol 32:6953–6964

    Article  PubMed  Google Scholar 

  21. Shen XM, Mao L, Yang ZY et al (2022) Deep learning-assisted diagnosis of parotid gland tumors by using contrast-enhanced CT imaging. Oral Dis. https://doi.org/10.1111/odi.14474

    Article  PubMed  Google Scholar 

  22. Committeri U, Barone S, Salzano G et al (2023) Support tools in the differential diagnosis of salivary gland tumors through inflammatory biomarkers and radiomics metrics: a preliminary study. Cancers (Basel) 15

  23. Chang YJ, Huang TY, Liu YJ, Chung HW, Juan CJ (2021) Classification of parotid gland tumors by using multimodal MRI and deep learning. NMR Biomed 34:e4408

    Article  PubMed  Google Scholar 

  24. Zheng YM, Li J, Liu S et al (2021) MRI-Based radiomics nomogram for differentiation of benign and malignant lesions of the parotid gland. Eur Radiol 31:4042–4052

    Article  PubMed  Google Scholar 

  25. Collins GS, Reitsma JB, Altman DG, Moons KG (2015) Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ 350:g7594

    Article  PubMed  Google Scholar 

  26. Hernandez-Prera JC (2022) Update from the 5th Edition of the World Health Organization Classification of Head and Neck Tumors: the neck and lymph nodes, metastasis, and melanocytic tumors. Head Neck Pathol 16:110-122

  27. Mentz RJ, Hernandez AF, Berdan LG et al (2016) Good clinical practice guidance and pragmatic clinical trials: balancing the best of both worlds. Circulation 133:872–880

    Article  PubMed  PubMed Central  Google Scholar 

  28. Sultan SR, AlKharaiji M, Rajab SH (2022) Diagnosis of parotid gland tumours with contrast-enhanced ultrasound: a systematic review and meta-analysis. Med Ultrason 24:211–218

    Article  PubMed  Google Scholar 

  29. Yushkevich P A, Piven J, C HH (2006) User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. Neuroimage 31:1116–1128

  30. van Griethuysen JJM, Fedorov A, Parmar C et al (2017) Computational radiomics system to decode the radiographic phenotype. Cancer Res 77:e104–e107

    Article  PubMed  PubMed Central  Google Scholar 

  31. R V (2018) Pingouin: statistics in Python. Journal of Open Source Software 3:31

  32. Pedregosa F, Varoquaux G, Gramfort A et al (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830

  33. Wr E (1995) Reading and understanding multivariate statistics logistic regression. American Psychological Association, Washington, DC, US, pp 217–244

    Google Scholar 

  34. Qj R (1986) Induction of decision trees. Machine Learning 1:81–106

    Article  Google Scholar 

  35. L B (2001) Random forests. Machine Learning 45:5–32

  36. Cortes C, Vapnik V (1995) Support-vector networks. Machine Learning 20:273–297

    Article  Google Scholar 

  37. Chen T, Guestrin C (2016) Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining XGBoost, 785–794

  38. Lundberg S M, Lee S-I (2017) A unified approach to interpreting model predictions. Curran Associates, Inc.

  39. Mantsopoulos K, Iro H (2021) Tumour spillage of the pleomorphic adenoma of the parotid gland: a proposal for intraoperative measures. Oral Oncol 112:104986

    Article  PubMed  Google Scholar 

  40. Vernuccio F AF, Cannella R, Verro B et al (2021) Diagnostic performance of qualitative and radiomics approach to parotid gland tumors: which is the added benefit of texture analysis? Br J Radiol https://doi.org/10.1259/bjr.20210340

  41. Yu Q, Ning Y, Wang A et al (2023) Deep learning-assisted diagnosis of benign and malignant parotid tumors based on contrast-enhanced CT: a multicenter study. Eur Radiol 33:6054–6065

    Article  PubMed  Google Scholar 

  42. Matsuo H, Nishio M, Kanda T et al (2020) Diagnostic accuracy of deep-learning with anomaly detection for a small amount of imbalanced data: discriminating malignant parotid tumors in MRI. Sci Rep 10:19388

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  43. Muntean DD, Dudea SM, Baciut M et al (2023) The role of an MRI-based radiomic signature in predicting malignancy of parotid gland tumors. Cancers (Basel) 15:3319

  44. Xia X, Feng B, Wang J et al (2021) Deep learning for differentiating benign from malignant parotid lesions on MR images. Front Oncol 11:632104

    Article  PubMed  PubMed Central  Google Scholar 

  45. Gunduz E, Alcin OF, Kizilay A, Yildirim IO (2022) Deep learning model developed by multiparametric MRI in differential diagnosis of parotid gland tumors. Eur Arch Otorhinolaryngol 279:5389–5399

    Article  PubMed  Google Scholar 

  46. Zheng YM, Xu WJ, Hao DP et al (2021) A CT-based radiomics nomogram for differentiation of lympho-associated benign and malignant lesions of the parotid gland. Eur Radiol 31:2886–2895

    Article  PubMed  Google Scholar 

  47. Lu J, Li L, Zhang C, Changshui. (2022) Self-reinforcing unsupervised matching. IEEE Trans Pattern Anal Mach Intell 44:4404-4418

Download references

Funding

Project of Foshan “Fourteen Five” Medicine High-level Key Specialty Construction (FSGSP145037); Medical Research Project of Foshan Health Bureau (20230029); Foshan Self-funded Science and Technology Innovation Project (Medical Science and Technology Research 2320001006907); Natural Science Foundation of Guangdong Province (2414050003969); Bureau of Science and Technology of Ganzhou Municipality (2022-ZD1373; 2022--RC1349).

Author information

Author notes

  1. Yanping He, Bowen Zheng contributed equally to this work.

Authors and Affiliations

  1. Department of Medical Ultrasonics, The First People’s Hospital of Foshan, No. 81, Lingnan Avenue North, Foshan, 528000, China

    Yanping He, Weiwei Peng, Yongyu Chen, Lihui Yu & Weijun Huang

  2. Department of Radiology, Nanfang Hospital, Southern Medical University, 1838 Guangzhou Avenue North, Guangzhou, 510515, China

    Bowen Zheng & Genggeng Qin

  3. Medical Imaging Center, Ganzhou People’s Hospital, 16th Meiguan Avenue, Ganzhou, 34100, China

    Genggeng Qin

Authors
  1. Yanping He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bowen Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weiwei Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yongyu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lihui Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Weijun Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Genggeng Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Weijun Huang or Genggeng Qin.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Genggeng Qin.

Conflict of interest

The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.

Statistics and biometry

No complex statistical methods were necessary for this paper.

Informed consent

Written informed consent was waived by the Institutional Review Board.

Ethical approval

Institutional Review Board approval was obtained.

Study subjects or cohorts overlap

No study subject or cohort has been previously reported.

Methodology

• retrospective

• diagnostic or prognostic study

• multicenter study

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 180 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, Y., Zheng, B., Peng, W. et al. An ultrasound-based ensemble machine learning model for the preoperative classification of pleomorphic adenoma and Warthin tumor in the parotid gland. Eur Radiol (2024). https://doi.org/10.1007/s00330-024-10719-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00330-024-10719-2

Keywords

Search

Navigation