Skip to main content

Advertisement

SpringerLink
Log in

Application of deep learning to the diagnosis of cervical lymph node metastasis from thyroid cancer with CT

  • Imaging Informatics and Artificial Intelligence
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Purpose

To develop a deep learning–based computer-aided diagnosis (CAD) system for use in the CT diagnosis of cervical lymph node metastasis (LNM) in patients with thyroid cancer.

Methods

A total of 995 axial CT images that included benign (n = 647) and malignant (n = 348) lymph nodes were collected from 202 patients with thyroid cancer who underwent CT for surgical planning between July 2017 and January 2018. The datasets were randomly split into training (79.0%), validation (10.5%), and test (10.5%) datasets. Eight deep convolutional neural network (CNN) models were used to classify the images into metastatic or benign lymph nodes. Pretrained networks were used on the ImageNet and the best-performing algorithm was selected. Class-specific discriminative regions were visualized with attention heatmap using a global average pooling method.

Results

The area under the ROC curve (AUROC) for the tested algorithms ranged from 0.909 to 0.953. The sensitivity, specificity, and accuracy of the best-performing algorithm were all 90.4%, respectively. Attention heatmap highlighted important subregions for further clinical review.

Conclusion

A deep learning–based CAD system could accurately classify cervical LNM in patients with thyroid cancer on preoperative CT with an AUROC of 0.953. Whether this approach has clinical utility will require evaluation in a clinical setting.

Key Points

• A deep learning–based CAD system could accurately classify cervical lymph node metastasis. The AUROC for the eight tested algorithms ranged from 0.909 to 0.953.

• Of the eight models, the ResNet50 algorithm was the best-performing model for the validation dataset with 0.953 AUROC. The sensitivity, specificity, and accuracy of the ResNet50 model were all 90.4%, respectively, in the test dataset.

• Based on its high accuracy of 90.4%, we consider that this model may be useful in a clinical setting to detect LNM on preoperative CT in patients with thyroid cancer.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Abbreviations

AUROC:

Area under the receiver operating characteristic curve

CAD:

Computer-aided diagnosis

CAM:

Class activation mapping

CNN:

Convolutional neural network

FNA:

Fine needle aspiration

LNM:

Lymph node metastasis

PTC:

Papillary thyroid carcinoma

US:

Ultrasonography

References

  1. Davies L, Welch HG (2014) Current thyroid cancer trends in the United States. JAMA Otolaryngol Head Neck Surg 140:317–322

    Article  PubMed  Google Scholar 

  2. Li N, Du XL, Reitzel LR, Xu L, Sturgis EM (2013) Impact of enhanced detection on the increase in thyroid cancer incidence in the United States: review of incidence trends by socioeconomic status within the surveillance, epidemiology, and end results registry, 1980-2008. Thyroid 23:103–110

    Article  PubMed  PubMed Central  Google Scholar 

  3. Haugen BR, Alexander EK, Bible KC et al (2016) 2015 American Thyroid Association management guidelines for adult patients with thyroid nodules and differentiated thyroid Cancer: the American Thyroid Association guidelines task force on thyroid nodules and differentiated thyroid cancer. Thyroid 26:1–133

    Article  PubMed  PubMed Central  Google Scholar 

  4. Al-Saif O, Farrar WB, Bloomston M, Porter K, Ringel MD, Kloos RT (2010) Long-term efficacy of lymph node reoperation for persistent papillary thyroid cancer. J Clin Endocrinol Metab 95:2187–2194

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Stack BC Jr, Ferris RL, Goldenberg D et al (2012) American Thyroid Association consensus review and statement regarding the anatomy, terminology, and rationale for lateral neck dissection in differentiated thyroid cancer. Thyroid 22:501–508

    Article  PubMed  Google Scholar 

  6. Durante C, Montesano T, Torlontano M et al (2013) Papillary thyroid cancer: time course of recurrences during postsurgery surveillance. J Clin Endocrinol Metab 98:636–642

    Article  CAS  PubMed  Google Scholar 

  7. Tufano RP, Clayman G, Heller KS et al (2015) Management of recurrent/persistent nodal disease in patients with differentiated thyroid cancer: a critical review of the risks and benefits of surgical intervention versus active surveillance. Thyroid 25:15–27

    Article  PubMed  Google Scholar 

  8. Yeh MW, Bauer AJ, Bernet VA et al (2015) American Thyroid Association statement on preoperative imaging for thyroid cancer surgery. Thyroid 25:3–14

    Article  PubMed  PubMed Central  Google Scholar 

  9. Wong SC, Gatt A, Stamatescu V, McDonnell MD (2016) Understanding data augmentation for classification: when to warp? https://arXiv.org/abs/1609.08764. Accessed 19 Nov 2018

  10. Shin JH, Baek JH, Chung J et al (2016) Ultrasonography diagnosis and imaging-based management of thyroid nodules: revised Korean Society of Thyroid Radiology consensus statement and recommendations. Korean J Radiol 17:370–395

    Article  PubMed  PubMed Central  Google Scholar 

  11. Suh CH, Baek JH, Choi YJ, Lee JH (2017) Performance of CT in the preoperative diagnosis of cervical lymph node metastasis in patients with papillary thyroid cancer: a systematic review and meta-analysis. AJNR Am J Neuroradiol 38:154–161

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Lee Y, Kim JH, Baek JH et al (2018) Value of CT added to ultrasonography for the diagnosis of lymph node metastasis in patients with thyroid cancer. Head Neck. https://doi.org/10.1002/hed.25202

  13. Shen D, Wu G, Suk HI (2017) Deep learning in medical image analysis. Annu Rev Biomed Eng 19:221–248

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Miotto R, Wang F, Wang S, Jiang X, Dudley JT (2017) Deep learning for healthcare: review, opportunities and challenges. Brief Bioinform

  15. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436

    Article  CAS  PubMed  Google Scholar 

  16. Choi YJ, Baek JH, Park HS et al (2017) A computer-aided diagnosis system using artificial intelligence for the diagnosis and characterization of thyroid nodules on ultrasound: initial clinical assessment. Thyroid 27:546–552

    Article  PubMed  Google Scholar 

  17. Wu MH, Chen CN, Chen KY et al (2016) Quantitative analysis of echogenicity for patients with thyroid nodules. Sci Rep 6:35632

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Jeong EY, Kim HL, Ha EJ et al (2018) Computer-aided diagnosis system for thyroid nodules on ultrasonography: diagnostic performance and reproducibility based on the experience level of operators. Eur Radiol

  19. Lee JH, Baek JH, Kim JH et al (2018) Deep learning-based computer-aided diagnosis system for localization and diagnosis of metastatic lymph nodes on ultrasound: a pilot study. Thyroid 28:1332–1338

    Article  PubMed  Google Scholar 

  20. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. https://arXiv.org/abs/1409.1556. Accessed 19 Nov 2018

  21. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognitionProceedings of the IEEE conference on computer vision and. Pattern Recogn:770–778

  22. Chollet F (2017) Xception: deep learning with depthwise separable convolutions. https://arXiv.org/abs/1610.02357. Accessed 19 Nov 2018

  23. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. https://arXiv.org/abs/1512.00567. Accessed 19 Nov 2018

  24. Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2016) Inception-v4, inception-resnet and the impact of residual connections on learning. https://arXiv.org/abs/1602.07261. Accessed 19 Nov 2018

  25. Iandola F, MoskewiczM,Karayev S, Girshick R, Darrell T, Keutzer K (2014) Densenet: Implementing efficient convnet descriptor pyramids. https://arXiv.org/abs/1404.1869. Accessed 19 Nov 2018

  26. Russakovsky O, Deng J, Su H et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115:211–252

    Article  Google Scholar 

  27. Abadi M, Barham P, Chen J et al (2016) Tensorflow: a system for large-scale machine learning. https://arXiv.org/abs/1605.08695. Accessed 19 Nov 2018

  28. Mulla MG, Knoefel WT, Gilbert J, McGregor A, Schulte KM (2012) Lateral cervical lymph node metastases in papillary thyroid cancer: a systematic review of imaging-guided and prophylactic removal of the lateral compartment. Clin Endocrinol 77:126–131

    Article  Google Scholar 

  29. Moon HJ, Kim EK, Yoon JH, Kwak JY (2012) Differences in the diagnostic performances of staging US for thyroid malignancy according to experience. Ultrasound Med Biol 38:568–573

    Article  PubMed  Google Scholar 

  30. Mulla M, Schulte KM (2012) Central cervical lymph node metastases in papillary thyroid cancer: a systematic review of imaging-guided and prophylactic removal of the central compartment. Clin Endocrinol 76:131–136

    Article  Google Scholar 

  31. Jeong HS, Baek CH, Son YI et al (2006) Integrated 18F-FDG PET/CT for the initial evaluation of cervical node level of patients with papillary thyroid carcinoma: comparison with ultrasound and contrast-enhanced CT. Clin Endocrinol 65:402–407

    Article  Google Scholar 

  32. Ahn JE, Lee JH, Yi JS et al (2008) Diagnostic accuracy of CT and ultrasonography for evaluating metastatic cervical lymph nodes in patients with thyroid cancer. World J Surg 32:1552–1558

    Article  PubMed  Google Scholar 

  33. Lee DW, Ji YB, Sung ES et al (2013) Roles of ultrasonography and computed tomography in the surgical management of cervical lymph node metastases in papillary thyroid carcinoma. Eur J Surg Oncol 39:191–196

    Article  CAS  PubMed  Google Scholar 

  34. Ha EJ, Baek JH, Na DG (2017) Risk stratification of thyroid nodules on ultrasonography: current status and perspectives. Thyroid 27:1463–1468

    Article  PubMed  Google Scholar 

  35. Esteva A, Kuprel B, Novoa RA et al (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542:115

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  36. Lee HJ, Yoon DY, Seo YL et al (2018) Intraobserver and interobserver variability in ultrasound measurements of thyroid nodules. J Ultrasound Med 37:173–178

    Article  PubMed  Google Scholar 

Download references

Funding

The authors state that this work was supported by the National Research Foundation of Korea (no. 2017R1C1B5016217).

Author information

Authors and Affiliations

  1. Division of Biomedical Informatics, Seoul National University Biomedical Informatics (SNUBI), Seoul National University College of Medicine, Seoul, 110799, Republic of Korea

    Jeong Hoon Lee & Ju Han Kim

  2. Department of Radiology, Ajou University School of Medicine, Wonchon-Dong, Yeongtong-Gu, Suwon, 443-380, South Korea

    Eun Ju Ha

Authors
  1. Jeong Hoon Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eun Ju Ha
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ju Han Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eun Ju Ha.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Eun Ju Ha.

Conflict of interest

The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.

Statistics and biometry

No complex statistical methods were necessary for this paper.

Informed consent

Written informed consent was obtained from all patients before they underwent US.

Ethical approval

This study was approved by our institutional review board.

Methodology

• retrospective

• case-control study

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(DOC 11717 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, J.H., Ha, E.J. & Kim, J.H. Application of deep learning to the diagnosis of cervical lymph node metastasis from thyroid cancer with CT. Eur Radiol 29, 5452–5457 (2019). https://doi.org/10.1007/s00330-019-06098-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-019-06098-8

Keywords

Search

Navigation