Skip to main content
SpringerLink
Log in

US of thyroid nodules: can AI-assisted diagnostic system compete with fine needle aspiration?

  • Head and Neck
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objectives

Artificial intelligence (AI) systems can diagnose thyroid nodules with similar or better performance than radiologists. Little is known about how this performance compares with that achieved through fine needle aspiration (FNA). This study aims to compare the diagnostic yields of FNA cytopathology alone and combined with BRAFV600E mutation analysis and an AI diagnostic system.

Methods

The ultrasound images of 637 thyroid nodules were collected in three hospitals. The diagnostic efficacies of an AI diagnostic system, FNA-based cytopathology, and BRAFV600E mutation analysis were evaluated in terms of sensitivity, specificity, accuracy, and the κ coefficient with respect to the gold standard, defined by postsurgical pathology and consistent benign outcomes from two combined FNA and mutation analysis examinations performed with a half-year interval.

Results

The malignancy threshold for the AI system was selected according to the Youden index from a retrospective cohort of 346 nodules and then applied to a prospective cohort of 291 nodules. The combination of FNA cytopathology according to the Bethesda criteria and BRAFV600E mutation analysis showed no significant difference from the AI system in terms of accuracy for either cohort in our multicenter study. In addition, for 45 included indeterminate Bethesda category III and IV nodules, the accuracy, sensitivity, and specificity of the AI system were 84.44%, 95.45%, and 73.91%, respectively.

Conclusions

The AI diagnostic system showed similar diagnostic performance to FNA cytopathology combined with BRAFV600E mutation analysis. Given its advantages in terms of operability, time efficiency, non-invasiveness, and the wide availability of ultrasonography, it provides a new alternative for thyroid nodule diagnosis.

Clinical relevance statement

Thyroid ultrasonic artificial intelligence shows statistically equivalent performance for thyroid nodule diagnosis to FNA cytopathology combined with BRAFV600E mutation analysis. It can be widely applied in hospitals and clinics to assist radiologists in thyroid nodule screening and is expected to reduce the need for relatively invasive FNA biopsies.

Key Points

• In a retrospective cohort of 346 nodules, the evaluated artificial intelligence (AI) system did not significantly differ from fine needle aspiration (FNA) cytopathology alone and combined with gene mutation analysis in accuracy.

• In a prospective multicenter cohort of 291 nodules, the accuracy of the AI diagnostic system was not significantly different from that of FNA cytopathology either alone or combined with gene mutation analysis.

• For 45 indeterminate Bethesda category III and IV nodules, the AI system did not perform significantly differently from BRAFV600E mutation analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Abbreviations

AI:

Artificial intelligence

AUC:

Area under the receiver operating characteristic curve

FNA:

Fine needle aspiration

PCR:

Polymerase chain reaction

PTC:

Papillary thyroid carcinoma

PTMC:

Papillary thyroid microcarcinoma

ROC:

Receiver operating characteristic

TI-RADS:

Thyroid Imaging Reporting and Data Systems

References

  1. Tessler FN, Middleton WD, Grant EG et al (2017) ACR Thyroid Imaging, Reporting and Data System (TI-RADS): white paper of the ACR TI-RADS Committee. J Am Coll Radiol 14:587–595

    Article  PubMed  Google Scholar 

  2. Cibas ES, Ali SZ (2017) The 2017 Bethesda System for Reporting Thyroid Cytopathology. Thyroid 27:1341–1346

    Article  PubMed  Google Scholar 

  3. Xing M (2019) Genetic-guided risk assessment and management of thyroid cancer. Endocrinol Metab Clin North Am 48:109–124

    Article  PubMed  PubMed Central  Google Scholar 

  4. Li L, Li P, Chen X et al (2021) Diagnostic value of puncture feeling combined with BRAF V600E mutation in repeat US-FNA biopsy of Bethesda III thyroid nodules. Gland Surg 10:2019–2027

    Article  PubMed  PubMed Central  Google Scholar 

  5. Rashid FA, Munkhdelger J, Fukuoka J et al (2020) Prevalence of BRAF(V600E) mutation in Asian series of papillary thyroid carcinoma-a contemporary systematic review. Gland Surg 9:1878–1900

    Article  PubMed  PubMed Central  Google Scholar 

  6. Ellison G, Donald E, McWalter G et al (2010) A comparison of ARMS and DNA sequencing for mutation analysis in clinical biopsy samples. J Exp Clin Cancer Res 29:132

    Article  PubMed  PubMed Central  Google Scholar 

  7. Zhang YZ, Xu T, Cui D et al (2015) Value of TIRADS, BSRTC and FNA-BRAF V600E mutation analysis in differentiating high-risk thyroid nodules. Sci Rep 5:16927

    Article  CAS  PubMed  PubMed Central  ADS  Google Scholar 

  8. Choi YJ, Park HS, Shim WH et al (2017) A computer-aided diagnosis system using artificial intelligence for the diagnosis and characterization of thyroid nodules on ultrasound: initial clinical assessment. Thyroid 27:546–552

    Article  PubMed  Google Scholar 

  9. Li XZS, Zhang Q, Wei X et al (2019) Diagnosis of thyroid cancer using deep convolutional neural network models applied to sonographic images: a retrospective, multicohort, diagnostic study. Lancet Oncol 20:193–201

    Article  PubMed  Google Scholar 

  10. Peng S, Liu Y, Lv W et al (2021) Deep learning-based artificial intelligence model to assist thyroid nodule diagnosis and management: a multicenter diagnostic study. Lancet Digit Health 3:e250–e259

    Article  CAS  PubMed  Google Scholar 

  11. Wang Y, Xu L, Lu W et al (2023) Clinical evaluation of malignancy diagnosis of rare thyroid carcinomas by an artificial intelligent automatic diagnosis system. Endocrine 80:93–99

    Article  CAS  PubMed  Google Scholar 

  12. Guo F, Chang W, Zhao J et al (2023) Assessment of the statistical optimization strategies and clinical evaluation of an artificial intelligence-based automated diagnostic system for thyroid nodule screening. Quant Imaging Med Surg 13(2):695–706

    Article  PubMed  PubMed Central  Google Scholar 

  13. Ma J, Wu F, Zhu J et al (2017) A pre-trained convolutional neural network based method for thyroid nodule diagnosis. Ultrasonics 73:221–230

    Article  PubMed  Google Scholar 

  14. Ma JWF, Jiang T, Zhao Q et al (2017) Ultrasound image-based thyroid nodule automatic segmentation using convolutional neural networks. Int J Comput Assist Radiol Surg 12:1895–1910

    Article  PubMed  Google Scholar 

  15. Du J, Han R, Chen C et al (2022) diagnostic efficacy of ultrasound, cytology, and BRAFV600E mutation analysis and their combined use in thyroid nodule screening for papillary thyroid microcarcinoma. Front Oncol 11:746776

    Article  PubMed  PubMed Central  Google Scholar 

  16. Vanbelle S, Albert A (2011) A bootstrap method for comparing correlated kappa coefficients. J Stat Comput Simul 78(11):1009–1015

    Article  MathSciNet  Google Scholar 

  17. Fluss R, Faraggi D, Reiser B (2005) Estimation of the Youden Index and its associated cutoff point. Biom J 47:458–472

    Article  MathSciNet  PubMed  Google Scholar 

  18. Li Y, Nakamura M, Kakudo K (2009) Targeting of the BRAF gene in papillary thyroid carcinoma (review). Oncol Rep 22(4):671–681

    CAS  PubMed  Google Scholar 

  19. Han Y, Wu JQ, Hou XJ et al (2021) Strain imaging in the evaluation of thyroid nodules: the associated factors leading to misdiagnosis. Ultrasound Med Biol 47(12):3372–3383

    Article  PubMed  Google Scholar 

  20. Zatelli MC, Trasforini G, Leoni S et al (2009) BRAF V600E mutation analysis increases diagnostic accuracy for papillary thyroid carcinoma in fine-needle aspiration biopsies. Eur J Endocrinol 161(3):467–473

    Article  CAS  PubMed  Google Scholar 

  21. Grussendorf M, Ruschenburg I, Brabant G (2022) Malignancy rates in thyroid nodules: a long-term cohort study of 17,592 patients. Eur Thyroid J 11(4):e220027

  22. Steward DL, Carty SE, Sippel RS et al (2019) Performance of a multigene genomic classifier in thyroid nodules with indeterminate cytology: a prospective blinded multicenter study. JAMA Oncol 5:204–212

    Article  PubMed  Google Scholar 

  23. Xu D, Wang Y, Wu H et al (2022) An artificial intelligence ultrasound system's ability to distinguish benign from malignant follicular-patterned lesions. Front Endocrinol (Lausanne) 13:981403

  24. Lo W-C, Cheng P-W, Wang C-T et al (2014) Pain levels associated with ultrasound-guided fine-needle aspiration biopsy for neck masses. Head Neck 36(2):252–256

    Article  PubMed  Google Scholar 

  25. Cappelli C, Pirola I, Agosti B et al (2017) Complications after fine-needle aspiration cytology: a retrospective study of 7449 consecutive thyroid nodules. Br J Oral Maxillofac Surg 55(3):266–269

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

This study has a preprint which has been published in the medRxiv: https://doi.org/10.1101/2022.04.28.22274306.

Funding

This study has received funding by the Zhejiang Basic Public Welfare Research Project (LGF22H160082).

Author information

Author notes

  1. Tianhan Zhou, Lei Xu, Jingjing Shi, and Yu Zhang equally contributed to this work

Authors and Affiliations

  1. Department of Surgical Oncology, Affiliated Hangzhou First People’s Hospital, Zhejiang University School of Medicine, Hangzhou, China

    Tianhan Zhou, Jingjing Shi, Yu Zhang, Fan Wu, Jiaoping Zhang & Dingcun Luo

  2. The Department of General Surgery, Hangzhou TCM Hospital Affiliated to Zhejiang Chinese Medical University, Hangzhou, China

    Tianhan Zhou

  3. Zhejiang Qiushi Institute for Mathematical Medicine, Hangzhou, China

    Lei Xu

  4. Department of Thyroid Surgery, Affiliated Yantai Yuhuangding Hospital, Qingdao University, Yantai, China

    Xiangfeng Lin & Haitao Zheng

  5. Department of Thyroid Surgery, The First Affiliated Hospital of Henan University, Zhengzhou, China

    Yuanyuan Wang

  6. The Fourth Clinical Medical College, Zhejiang Chinese Medical University, Hangzhou, China

    Tao Hu, Tian Tang, Siqi Zhan & Dingcun Luo

  7. Department of Pathology, Affiliated Hangzhou First People’s Hospital, Zhejiang University School of Medicine, Hangzhou, China

    Rujun Xu, Lesi Xie, Lijuan Sun, Dandan Li & Wenhua Zhang

  8. Department of Ultrasonography, Affiliated Hangzhou First People’s Hospital, Zhejiang University School of Medicine, Hangzhou, China

    Chuanghua Chen, Wei Wang, Chenke Xu & Fanlei Kong

  9. Department of Translational Medicine Research Center, Affiliated Hangzhou First People’s Hospital, Zhejiang University School of Medicine, Hangzhou, China

    Yanping Xun & Shirong Zhang

  10. Department of Endocrinology, Affiliated Hangzhou First People’s Hospital, Zhejiang University School of Medicine, Hangzhou, China

    Lingying Yu

  11. Department of Head and Neck Surgery, Cancer hospital of the University of Chinese Academy of Sciences, Hangzhou, China

    Jinwang Ding

  12. Department of General Surgery, Affiliated Zhongshan Hospital, Xiamen University, Xiamen, China

    Guoyang Wu

  13. College of Mathematical Medicine, Zhejiang Normal University, Jinhua, China

    Dexing Kong

  14. School of Mathematical Sciences, Zhejiang University, Hangzhou, China

    Dexing Kong

Authors
  1. Tianhan Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lei Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jingjing Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiangfeng Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yuanyuan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Tao Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Rujun Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Lesi Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Lijuan Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Dandan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Wenhua Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Chuanghua Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Wei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Chenke Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Fanlei Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Yanping Xun
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Lingying Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Shirong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  20. Jinwang Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  21. Fan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  22. Tian Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  23. Siqi Zhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  24. Jiaoping Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  25. Guoyang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  26. Haitao Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  27. Dexing Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  28. Dingcun Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Guoyang Wu, Haitao Zheng, Dexing Kong or Dingcun Luo.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Dingcun Luo.

Conflict of interest

This was an independent study and was not sponsored by the company that provided the AI system being evaluated. All the authors declare no conflicts of interest.

Statistics and biometry

Tianhan Zhou has significant statistical expertise.

Informed consent

Written informed consent was obtained from all subjects (patients) in this study.

Ethical approval

Institutional Review Board approval was obtained. This clinical study is registered in www.clinicaltrials.gov with the registration number of ChiCTR2200061242.

Study subjects or cohorts overlap

None study subjects or cohorts have been previously reported.

Methodology

• Retrospective

• Diagnostic study

• Multicenter study

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 257 kb)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, T., Xu, L., Shi, J. et al. US of thyroid nodules: can AI-assisted diagnostic system compete with fine needle aspiration?. Eur Radiol 34, 1324–1333 (2024). https://doi.org/10.1007/s00330-023-10132-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-023-10132-1

Keywords

Search

Navigation