Skip to main content

Advertisement

SpringerLink
Log in

An integrated AI model to improve diagnostic accuracy of ultrasound and output known risk features in suspicious thyroid nodules

  • Ultrasound
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objectives

From the viewpoint of ultrasound (US) physicians, an ideal thyroid US computer-assisted diagnostic (CAD) system for thyroid cancer should perform well in suspicious thyroid nodules with atypical risk features and be able to output explainable results. This study aims to develop an explainable US CAD model for suspicious thyroid nodules.

Methods

A total of 2992 solid or almost-solid thyroid nodules were analyzed retrospectively. All nodules had pathological results (1070 malignancies and 1992 benignities) confirmed by ultrasound-guided fine-needle aspiration cytology and histopathology after thyroidectomy. A deep learning model (ResNet50) and a multiple risk features learning ensemble model (XGBoost) were used to train the US images of 2794 thyroid nodules. Then, an integrated AI model was generated by combining both models. The diagnostic accuracies of the three AI models (ResNet50, XGBoost, and the integrated model) were predicted in a testing set including 198 thyroid nodules and compared to the diagnostic efficacy of five ultrasonographers.

Results

The accuracy of the integrated model was 76.77%, while the mean accuracy of the ultrasonographers was 68.38%. Of the risk features, microcalcifications showed the highest contribution to the diagnosis of malignant nodules.

Conclusions

The integrated AI model in our study can improve the diagnostic accuracy of suspicious thyroid nodules and output the known risk features simultaneously, thus aiding in training young ultrasonographers by linking the explainable results to their clinical experience and advancing the acceptance of AI diagnosis for thyroid cancer in clinical practice.

Key Points

• We developed an artificial intelligence (AI) diagnosis model based on both deep learning and multiple risk feature ensemble learning methods.

• The AI diagnosis model showed higher diagnostic accuracy for suspicious thyroid nodules than ultrasonographers.

• The AI diagnosis model showed partial explainability by outputting the known risk features, thus aiding young ultrasonic doctors in increasing the diagnostic level for thyroid cancer.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Abbreviations

ACR:

The American College of Radiology

AI:

Artificial intelligence

AUC:

The area under the curve

DL:

Deep learning

FNA:

Fine-needle aspiration

ROI:

Region of interest

TI-RADS:

Thyroid Image Radiology and Data System

US CAD:

US computer-assisted diagnosis

US:

Ultrasound

References

  1. Sung H, Ferlay J, Siegel RL et al (2021) Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 71(3):209–249

    Google Scholar 

  2. Li M, Dal Maso L, Vaccarella S (2020) Global trends in thyroid cancer incidence and the impact of overdiagnosis. Lancet Diabetes Endocrinol 8(6):468–470

    Article  Google Scholar 

  3. Durante C, Grani G, Lamartina L, Filetti S, Mandel SJ, Cooper DS (2018) The diagnosis and management of thyroid nodules: a review. JAMA 319(9):914–924

    Article  Google Scholar 

  4. Itani M, Assaker R, Moshiri M, Dubinsky TJ, Dighe MK (2019) Inter-observer variability in the American College of Radiology Thyroid Imaging Reporting and Data System: in-depth analysis and areas for improvement. Ultrasound Med Biol 45(2):461–470

    Article  Google Scholar 

  5. Vaccarella S, Franceschi S, Bray F, Wild CP, Plummer M, Dal Maso L (2016) Worldwide thyroid-cancer epidemic? The increasing impact of overdiagnosis. N Engl J Med 375(7):614–617

    Article  Google Scholar 

  6. Tessler FN, Middleton WD, Grant EG et al (2017) ACR Thyroid Imaging, Reporting and Data System (TI-RADS): white paper of the ACR TI-RADS Committee. J Am Coll Radiol 14(5):587–595

    Article  Google Scholar 

  7. Hoang JK, Middleton WD, Farjat AE et al (2018) Reduction in thyroid nodule biopsies and improved accuracy with American College of Radiology Thyroid Imaging Reporting and Data System. Radiology 287(1):185–193

    Article  Google Scholar 

  8. Buda M, Wildman-Tobriner B, Hoang JK et al (2019) Management of thyroid nodules seen on US images: deep learning may match performance of radiologists. Radiology 292(3):695–701

    Article  Google Scholar 

  9. Thomas J, Ledger GA, Mamillapalli CK (2020) Use of artificial intelligence and machine learning for estimating malignancy risk of thyroid nodules. Curr Opin Endocrinol Diabetes Obes 27(5):345–350

    Article  Google Scholar 

  10. Thomas J, Haertling T (2020) AIBx, Artificial intelligence model to risk stratify thyroid nodules. Thyroid 30(6):878–884

    Article  Google Scholar 

  11. Ye FY, Lyu GR, Li SQ et al (2021) Diagnostic performance of ultrasound computer-aided diagnosis software compared with that of radiologists with different levels of expertise for thyroid malignancy: a multicenter prospective study. Ultrasound Med Biol 47(1):114–124

    Article  Google Scholar 

  12. Morid MA, Borjali A, Del Fiol G (2021) A scoping review of transfer learning research on medical image analysis using ImageNet. Comput Biol Med 128:104115

    Article  Google Scholar 

  13. Liu SF, Wang Y, Yang X (2019) Deep learning in medical ultrasound analysis: a review. Engineering 5:261–275

    Article  Google Scholar 

  14. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA. Available via https://ieeexplore.ieee.org/document/7780459. Accessed 16 Aug 2021

  15. Krizhevsky A, Sutskever I, Hinton G (2017) ImageNet classification with deep convolutional neural networks. Commun ACM 60(6):84–90

    Article  Google Scholar 

  16. Shin JH, Baek JH, Chung J et al (2016) Ultrasonography diagnosis and imaging-based management of thyroid nodules: revised Korean Society of Thyroid Radiology Consensus Statement and Recommendations. Korean J Radiol 17(3):370–395

    Article  PubMed Central  Google Scholar 

  17. Torlay L, Perrone-Bertolotti M, Thomas E, Baciu M (2017) Machine learning-XGBoost analysis of language networks to classify patients with epilepsy. Brain Inform 4(3):159–169

    Article  CAS  PubMed Central  Google Scholar 

  18. Szczepanek-Parulska E, Wolinski K, Dobruch-Sobczak K et al (2020) S-Detect software vs. EU-TIRADS classification: a dual-center validation of diagnostic performance in differentiation of thyroid nodules. J Clin Med 9(8):2495

    Article  PubMed Central  Google Scholar 

  19. Remonti LR, Kramer CK, Leitão CB, Pinto LC, Gross JL (2015) Thyroid ultrasound features and risk of carcinoma: a systematic review and meta-analysis of observational studies. Thyroid 25(5):538–550

    Article  PubMed Central  Google Scholar 

  20. Rosario PW, Mourão GF (2019) Noninvasive follicular thyroid neoplasm with papillary-like nuclear features (NIFTP): a review for clinicians. Endocr Relat Cancer 26(5):R259–R266

    Article  Google Scholar 

Download references

Acknowledgements

We thank the ultrasonic physicians who took part in the labeling process of the study, as well as the engineers Yuli She and Yi Jing who gave good suggestions from a standpoint of medical image analysis. Last, we also want to give our sincere thanks to pathological physicians who offered the histopathological results for the thyroid nodules in our study.

Funding

This study has received funding from the National Key Research and Development Program of China (No. 2017YFA0700800), the National Natural Science Foundation of China (No. 81871366), the Key Research and Development Program of Shaanxi Province of China (No. 2021SF-346).

Author information

Author notes

  1. Dong Zhang

    Present address: Institute of Artificial Intelligence and Robotics, College of Artificial Intelligence, Xi’an Jiaotong University, Xi’an, 710049, China

  2. Juan Wang and Jue Jiang contributed equally to this work and are considered to be co-first authors.

Authors and Affiliations

  1. Department of Ultrasound, the Second Affiliated Hospital, Medical School of Xi’an Jiaotong University, Xi’an, 710004, China

    Juan Wang, Jue Jiang & Qi Zhou

  2. School of Automation Science and Engineering, Faculty of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an, 710049, Shaanxi, China

    Dong Zhang

  3. The Institute of Medical Science, The University of Tokyo, Shirokanedai 4-6-1, Minato-ku, Tokyo, 108-8639, Japan

    Yao-zhong Zhang

  4. Precision Medicine Institute, Western China Science and Technology Innovation Harbor, Xi’an Jiaotong University, Xi’an, 712000, China

    Long Guo

  5. Department of Computer Science, Columbia University, 500 West 120 Street, New York, NY, 10027, USA

    Yusheng Jiang

  6. Institute of Artificial Intelligence and Robotics, College of Artificial Intelligence, Xi’an Jiaotong University, Xi’an, 710049, China

    Shaoyi Du

Authors
  1. Juan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jue Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yao-zhong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Long Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yusheng Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Shaoyi Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Qi Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Shaoyi Du or Qi Zhou.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Qi Zhou.

Conflict of interest

The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.

Statistics and biometry

One of the authors has significant statistical expertise.

Informed consent

Written informed consent was obtained from all subjects (patients) in this study.

Ethical approval

Institutional Review Board approval was obtained.

Methodology

  • Retrospective

  • Diagnostic study

  • Performed at one institution

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 518 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, J., Jiang, J., Zhang, D. et al. An integrated AI model to improve diagnostic accuracy of ultrasound and output known risk features in suspicious thyroid nodules. Eur Radiol 32, 2120–2129 (2022). https://doi.org/10.1007/s00330-021-08298-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-021-08298-7

Keywords

Search

Navigation