Skip to main content
SpringerLink
Log in

Artificial intelligence–based prediction of cervical lymph node metastasis in papillary thyroid cancer with CT

  • Imaging Informatics and Artificial Intelligence
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objectives

To develop an artificial intelligence (AI) system for predicting cervical lymph node metastasis (CLNM) preoperatively in patients with papillary thyroid cancer (PTC) based on CT images.

Methods

This multicenter retrospective study included the preoperative CT of PTC patients who were divided into the development, internal, and external test sets. The region of interest of the primary tumor was outlined manually on the CT images by a radiologist who has eight years of experience. With the use of the CT images and lesions masks, the deep learning (DL) signature was developed by the DenseNet combined with convolutional block attention module. One-way analysis of variance and least absolute shrinkage and selection operator were used to select features, and a support vector machine was used to construct the radiomics signature. Random forest was used to combine the DL, radiomics, and clinical signature to perform the final prediction. The receiver operating characteristic curve, sensitivity, specificity, and accuracy were used by two radiologists (R1 and R2) to evaluate and compare the AI system.

Results

For the internal and external test set, the AI system achieved excellent performance with AUCs of 0.84 and 0.81, higher than the DL (p = .03, .82), radiomics (p < .001, .04), and clinical model (p < .001, .006). With the aid of the AI system, the specificities of radiologists were improved by 9% and 15% for R1 and 13% and 9% for R2, respectively.

Conclusions

The AI system can help predict CLNM in patients with PTC, and the radiologists’ performance improved with AI assistance.

Clinical relevance statement

This study developed an AI system for preoperative prediction of CLNM in PTC patients based on CT images, and the radiologists’ performance improved with AI assistance, which could improve the effectiveness of individual clinical decision-making.

Key Points

• This multicenter retrospective study showed that the preoperative CT image-based AI system has the potential for predicting the CLNM of PTC.

• The AI system was superior to the radiomics and clinical model in predicting the CLNM of PTC.

• The radiologists’ diagnostic performance improved when they received the AI system assistance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Abbreviations

ANOVA:

The analysis of variance

AUC:

Area under the ROC curve

CBAM:

Convolutional block attention module

CI:

Confidence interval

CLNM:

Cervical lymph node metastases

DL:

Deep learning

ICC:

Intra- and inter-class correlation coefficient

KNN:

K-Nearest Neighbor

LASSO:

Least absolute shrinkage and selection operator

NPV:

Negative predictive value

PPV:

Positive predictive value

PTC:

Papillary thyroid carcinoma

ROC:

Receiver operating characteristic

SVM:

Support vector machine

US:

Ultrasound

References

  1. Haugen BR, Alexander EK, Bible KC et al (2016) 2015 American thyroid association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: The American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer. Thyroid 26:1–133

    Article  PubMed  PubMed Central  Google Scholar 

  2. Gibbons J (2016) Kernels, in a nutshell. J Log Algebr Methods Program 85:921–930

    Article  Google Scholar 

  3. Schlumberger M, Leboulleux S (2021) Current practice in patients with differentiated thyroid cancer. Nat Rev Endocrinol 17:176–188

    Article  CAS  PubMed  Google Scholar 

  4. Singh Ospina N, Iniguez-Ariza NM, Castro MR (2020) Thyroid nodules: diagnostic evaluation based on thyroid cancer risk assessment. BMJ 368:l6670

    Article  PubMed  Google Scholar 

  5. O’Connell K, Yen TW, Quiroz F, Evans DB, Wang TS (2013) The utility of routine preoperative cervical ultrasonography in patients undergoing thyroidectomy for differentiated thyroid cancer. Surgery 154:697–701 (discussion 701-693)

  6. Kim E, Park JS, Son K-R, Kim J-H, Jeon SJ, Na DG (2008) Preoperative diagnosis of cervical metastatic lymph nodes in papillary thyroid carcinoma: comparison of ultrasound, computed tomography, and combined ultrasound with computed tomography. Thyroid 18:411–418

    Article  CAS  PubMed  Google Scholar 

  7. Kawada K, Taketo MM (2011) Significance and mechanism of lymph node metastasis in cancer progression. Cancer Res 71:1214–1218

    Article  CAS  PubMed  Google Scholar 

  8. Dong D, Fang MJ, Tang L et al (2020) Deep learning radiomic nomogram can predict the number of lymph node metastasis in locally advanced gastric cancer: an international multicenter study. Ann Oncol 31:912–920

    Article  CAS  PubMed  Google Scholar 

  9. Jiang Y, Wang W, Chen C et al (2019) Radiomics signature on computed tomography imaging: association with lymph node metastasis in patients with gastric cancer. Front Oncol 9:340

    Article  PubMed  PubMed Central  Google Scholar 

  10. Eraslan G, Avsec Z, Gagneur J, Theis FJ (2019) Deep learning: new computational modelling techniques for genomics. Nat Rev Genet 20:389–403

    Article  CAS  PubMed  Google Scholar 

  11. Zhou LQ, Wu XL, Huang SY et al (2020) Lymph node metastasis prediction from primary breast cancer US images using deep learning. Radiology 294:19–28

    Article  PubMed  Google Scholar 

  12. Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts H (2018) Artificial intelligence in radiology. Nat Rev Cancer 18:500–510

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Lee JH, Ha EJ, Kim D et al (2020) Application of deep learning to the diagnosis of cervical lymph node metastasis from thyroid cancer with CT: external validation and clinical utility for resident training. Eur Radiol 30:3066–3072

    Article  PubMed  Google Scholar 

  14. Yu J, Deng Y, Liu T et al (2020) Lymph node metastasis prediction of papillary thyroid carcinoma based on transfer learning radiomics. Nat Commun 11:4807

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Li F, Pan D, He Y et al (2020) Using ultrasound features and radiomics analysis to predict lymph node metastasis in patients with thyroid cancer. BMC Surg 20:315

    Article  PubMed  PubMed Central  Google Scholar 

  16. Zhang Y, Li H, Du J et al (2021) 3D multi-attention guided multi-task learning network for automatic gastric tumor segmentation and lymph node classification. IEEE Trans Med Imaging 40:1618–1631

    Article  PubMed  Google Scholar 

  17. Ma H, Liu ZX, Zhang JJ et al (2020) Construction of a convolutional neural network classifier developed by computed tomography images for pancreatic cancer diagnosis. World J Gastroenterol 26:5156–5168

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Hirota M, Mizota A, Mimura T et al (2020) Effect of color information on the diagnostic performance of glaucoma in deep learning using few fundus images. Int Ophthalmol 40:3013–3022

    Article  PubMed  Google Scholar 

  19. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 2261–2269

  20. Woo S, Park J, Lee J-Y, Kweon IS (2018) CBAM: convolutional block attention module. In: Ferrari, Hebert M, Sminchisescu C, Weiss Y, (eds) Computer Vision - ECCV 2018, PT VII. Springer International Publishing, Cham, pp 3–19

  21. Walsh SLF, Calandriello L, Silva M, Sverzellati N (2018) Deep learning for classifying fibrotic lung disease on high-resolution computed tomography: a case-cohort study. Lancet Respir Med 6:837–845

    Article  PubMed  Google Scholar 

  22. Finotello F, Mayer C, Plattner C et al (2019) Molecular and pharmacological modulators of the tumor immune contexture revealed by deconvolution of RNA-seq data. Genome Med 11:34

    Article  PubMed  PubMed Central  Google Scholar 

  23. Yang Z, Heng Y, Lin J et al (2020) Nomogram for predicting central lymph node metastasis in papillary thyroid cancer: a retrospective cohort study of two clinical centers. Cancer Res Treat 52:1010–1018

    CAS  PubMed  PubMed Central  Google Scholar 

  24. Liu J, Jia X, Gu Y et al (2021) Thyroid parenchyma microcalcifications on ultrasound for predicting lymph node metastasis in papillary thyroid carcinoma: a prospective multicenter study in China. Front Oncol 11:609075

    Article  PubMed  PubMed Central  Google Scholar 

  25. Zhou Y, Su GY, Hu H et al (2022) Radiomics from primary tumor on dual-energy CT derived iodine maps can predict cervical lymph node metastasis in papillary thyroid cancer. Acad Radiol 29(Suppl 3):S222–S231

    Article  PubMed  Google Scholar 

  26. Tong Y, Li J, Huang Y et al (2021) Ultrasound-based radiomic nomogram for predicting lateral cervical lymph node metastasis in papillary thyroid carcinoma. Acad Radiol 28:1675–1684

    Article  PubMed  Google Scholar 

  27. Li J, Wu X, Mao N et al (2021) Computed tomography-based radiomics model to predict central cervical lymph node metastases in papillary thyroid carcinoma: a multicenter study. Front Endocrinol (Lausanne) 12:741698

    Article  PubMed  Google Scholar 

  28. Li L, Zhao J, Hou L, Zhai Y, Shi J, Cui F (2019) An attention-based deep learning model for clinical named entity recognition of Chinese electronic medical records. BMC Med Inform Decis Mak 19:235

    Article  PubMed  PubMed Central  Google Scholar 

  29. Xing Z, Qiu Y, Yang Q et al (2020) Thyroid cancer neck lymph nodes metastasis: Meta-analysis of US and CT diagnosis. Eur J Radiol 129:109103

    Article  PubMed  Google Scholar 

  30. Qian X, Pei J, Zheng H et al (2021) Prospective assessment of breast cancer risk from multimodal multiview ultrasound images via clinically applicable deep learning. Nat Biomed Eng 5:522–532

    Article  PubMed  Google Scholar 

  31. Zhou W, Yang Y, Yu C et al (2021) Ensembled deep learning model outperforms human experts in diagnosing biliary atresia from sonographic gallbladder images. Nat Commun 12:1259

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Isensee F, Jaeger PF, Kohl SAA, Petersen J, Maier-Hein KH (2021) nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat Methods 18:203–211

    Article  CAS  PubMed  Google Scholar 

  33. Balagopal A, Kazemifar S, Nguyen D et al (2018) Fully automated organ segmentation in male pelvic CT images. Phys Med Biol 63:245015

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors thank Xingyue Jiang of The Affiliated Hospital of Binzhou Medical University, for assisting with the collection of the imaging data used in this study.

Funding

This study has received funding from the Taishan Scholars Project (No. ts20190991).

Author information

Authors and Affiliations

  1. School of Clinical Medicine, Weifang Medical University, Weifang, Shandong, 261042, People’s Republic of China

    Cai Wang & Zheying Song

  2. Department of Otorhinolaryngology, Head and Neck Surgery, Yantai Yuhuangding Hospital, Qingdao University, Yantai, Shandong, 264000, People’s Republic of China

    Cai Wang, Pengyi Yu, Xiao Han, Zheying Song, Guangkuo Wang & Xicheng Song

  3. Shandong Provincial Clinical Research Center for Otorhinolaryngologic Diseases, Yantai, Shandong, 264000, People’s Republic of China

    Cai Wang, Pengyi Yu, Xiao Han, Zheying Song, Guangkuo Wang & Xicheng Song

  4. Yantai Key Laboratory of Otorhinolaryngologic Diseases, Yantai Yuhuangding Hospital, Yantai, Shandong, 264000, People’s Republic of China

    Cai Wang, Pengyi Yu, Xiao Han, Zheying Song, Guangkuo Wang & Xicheng Song

  5. Big Data and Artificial Intelligence Laboratory, Yantai Yuhuangding Hospital, Qingdao University, Yantai, Shandong, 264000, People’s Republic of China

    Haicheng Zhang & Ning Mao

  6. Department of Thyroid Surgery, Yantai Yuhuangding Hospital, Qingdao University, Yantai, Shandong, 264000, People’s Republic of China

    Guibin Zheng & Haitao Zheng

  7. Department of Radiology, Yantai Yuhuangding Hospital, Qingdao University, Yantai, Shandong, 264000, People’s Republic of China

    Ning Mao

Authors
  1. Cai Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pengyi Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haicheng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiao Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zheying Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Guibin Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Guangkuo Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Haitao Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Ning Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Xicheng Song
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Ning Mao or Xicheng Song.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Xicheng Song.

Conflict of interest

The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.

Statistics and biometry

One of the authors, Haicheng Zhang, has significant statistical expertise. No complex statistical methods were necessary for this paper.

Informed consent

Written informed consent was waived by the Institutional Review Board.

Ethical approval

Institutional Review Board approval was obtained.

Study subjects or cohorts overlap

The study subjects or cohorts have never been previously reported.

Methodology

  • retrospective

  • diagnostic study

  • multicenter study

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Cai Wang and Pengyi Yu contributed equally to this work and share first authorship.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 363 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, C., Yu, P., Zhang, H. et al. Artificial intelligence–based prediction of cervical lymph node metastasis in papillary thyroid cancer with CT. Eur Radiol 33, 6828–6840 (2023). https://doi.org/10.1007/s00330-023-09700-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-023-09700-2

Keywords

Search

Navigation