Skip to main content
SpringerLink
Log in

Comparative analysis of machine learning-based ultrasound radiomics in predicting malignancy of partially cystic thyroid nodules

  • Original Article
  • Published:
Endocrine Aims and scope Submit manuscript

Abstract

Objective

To investigate the application of machine learning (ML) model-based thyroid ultrasound radiomics in the evaluation of malignancy in partially cystic thyroid nodules (PCTNs).

Methods

One hundred and ninety-two patients with 197 nodules PCTNs from January 2020 to December 2020 were retrospectively analyzed. Radiomics features were extracted based on hand-crafted features from the ultrasound images, and machine learning methods were used to build a classification model by radiomics features. The least absolute shrinkage and selection operator regression was applied to select the features of nonzero coefficients from radiomics features. The prediction performance of the established model was mainly evaluated by the area under the curve (AUC) and accuracy, sensitivity, and specificity.

Results

Nineteen radiomics features were extracted from the original images for each nodule. Eight ML classifiers were able to differentiate malignancy in PCTNs. The AUC, accuracy, sensitivity, and specificity of k-Nearest Neighbor (KNN) model were 0.909, 82.95%, 83.33%, and 89.90%, respectively, on the test cohort. The comparative result showed statistically equivalent performance for thyroid nodule diagnosis based on image fusion and single image. In addition, the ML-Based ultrasound radiomics system showed a better AUC as compared with ACR TI-RADS model and the ultrasound features model.

Conclusion

The novel ultrasonic-based ML model has an important clinical value for predicting malignancy in PCTNs. It can provide clinicians with a preoperative non-invasive primary screening method for PCTN diagnosis to avoid unnecessary medical investment and improve treatment outcomes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Abbreviations

PCTNs:

Partially Cystic Thyroid Nodules

ML:

Machine Learning

TI-RADS:

Thyroid Imaging Reporting and Data Systems

LASSO:

The least absolute shrinkage and selection operator

AUC:

Area Under the Receiver Operating Characteristic Curve

KNN:

k-Nearest Neighbor

SVM:

Support Vector Machine;

KNN:

k-Nearest Neighbor

XGBoost:

Extreme Gradient Boosting

MLP:

Multilayer Perceptron;

LR:

Logistic Regression

PPV:

positive predictive value

NPV:

negative predictive value

References

  1. R. Wong, S.G. Farrell, M. Grossmann, Thyroid nodules: diagnosis and management. Med J. Aust. 209, 92–98 (2018).

    Article  PubMed  Google Scholar 

  2. K.D. Burman, L. Wartofsky, CLINICAL PRACTICE. Thyroid Nodules. N. Engl. J. Med 373(24), 2347–2356 (2015).

    Article  CAS  PubMed  Google Scholar 

  3. E.K. Alexander, E.S. Cibas, Diagnosis of thyroid nodules. Lancet Diabetes Endocrinol. 10(7), 533–539 (2022).

    Article  PubMed  Google Scholar 

  4. B.R. Haugen, E.K. Alexander, K.C. Bible et al. American Thyroid Association Management Guidelines for Adult Patients with Thyroid Nodules and Differentiated Thyroid Cancer: The American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer. Thyroid 2016(26), 1–133 (2015).

    Google Scholar 

  5. F.N. Tessler, W.D. Middleton, E.G. Grant et al. ACR Thyroid Imaging, Reporting and Data System (TI-RADS): White Paper of the ACR TI-RADS Committee. J. Am. Coll. Radio. 14, 587–595 (2017).

    Article  Google Scholar 

  6. T. Zhou, H. Huang, H. Dong, Z. Ni, H. Sun, T. He, C. Ma, Ultrasound-Based Risk Stratification System for the Assessment of Partially Cystic Thyroid Nodules. Endocr. Pr. 29(6), 428–435 (2023).

    Article  Google Scholar 

  7. Y.J. Lee, J.Y. Kim, D.G. Na et al. Malignancy risk of thyroid nodules with minimal cystic changes: a multicenter retrospective study. Ultrasonography 41(4), 670–677 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  8. Y. Xin, F. Liu, Y. Shi, X. Yan, L. Liu, J. Zhu, A Scoring System for Assessing the Risk of Malignant Partially Cystic Thyroid Nodules Based on Ultrasound Features. Front Oncol. 11, 731779 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  9. Y.J.B.J. Choi, H.S. Park, W.H. Shim, T.Y. Kim, Y.K. Shong, J.H. Lee, A Computer-Aided Diagnosis System Using Artificial Intelligence for the Diagnosis and Characterization of Thyroid Nodules on Ultrasound: Initial Clinical Assessment. Thyroid 27, 546–52. (2017).

    Article  PubMed  Google Scholar 

  10. X.Z.S. Li, Q. Zhang, X. Wei, Y. Pan, J. Zhao, X. Xin, C. Qin, X. Wang, J. Li, F. Yang, Y. Zhao, M. Yang, Q. Wang, Z. Zheng, X. Zheng, X. Yang, C.T. Whitlow, M.N. Gurcan, L. Zhang, X. Wang, B.C. Pasche, M. Gao, W. Zhang, K. Chen, Diagnosis of thyroid cancer using deep convolutional neural network models applied to sonographic images: a retrospective, multicohort, diagnostic study. Lancet Oncol. 20, 193–201 (2019)

    Article  PubMed  Google Scholar 

  11. S. Peng, Y. Liu, W. Lv et al. Deep learning-based artificial intelligence model to assist thyroid nodule diagnosis and management: a multicentre diagnostic study. Lancet Digit Health 3, e250–e9. (2021)

    Article  CAS  PubMed  Google Scholar 

  12. J.M. Park, Y. Choi, H.J. Kwag, Partially cystic thyroid nodules: ultrasound findings of malignancy. Korean J. Radio. 13(5), 530–535 (2012).

    Article  Google Scholar 

  13. T. Zhou, H. Huang, H. Dong, Z. Ni, H. Sun, T. He, C. Ma, Ultrasound-Based Risk Stratification System for the Assessment of Partially Cystic Thyroid Nodules. Endocr. Pr. 29(6), 428–435 (2023).

    Article  Google Scholar 

  14. X. Shi, R. Liu, L. Gao, Y. Xia, Y. Jiang, Diagnostic Value of Sonographic Features in Distinguishing Malignant Partially Cystic Thyroid Nodules: A Systematic Review and Meta-Analysis. Front Endocrinol. 12, 624409 (2021).

    Article  Google Scholar 

  15. W. Li, Q. Zhu, Y. Jiang, Q. Zhang, Z. Meng, J. Sun, J. Li, Q. Dai, Partially cystic thyroid nodules in ultrasound-guided fine needle aspiration: Prevalence of thyroid carcinoma and ultrasound features. Medicine 96(46), e8689 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  16. Y.Z. Shi, Y. Jin, L. Zheng, Partially cystic thyroid nodules on ultrasound: The associated factors for malignancy. Clin. Hemorheol. Microcirc. 74(4), 373–381 (2020)

    Article  CAS  PubMed  Google Scholar 

  17. B. Zhang, J. Tian, S. Pei, Y. Chen, X. He, Y. Dong, L. Zhang, X. Mo, W. Huang, S. Cong, S. Zhang, Machine Larning-Assisted System for Thyroid Nodule Diagnosis. Thyroid 29(6), 858–867 (2019).

    Article  PubMed  Google Scholar 

  18. H. Zhou, Y. Jin, L. Dai, M. Zhang, Y. Qiu, K. Wang, J. Tian, J. Zheng, Differential Diagnosis of Benign and Malignant Thyroid Nodules Using Deep Learning Radiomics of Thyroid Ultrasound Images. Eur. J. Radiol. 127, 108992 (2020). https://doi.org/10.1016/j.ejrad.2020.108992.

    Article  PubMed  Google Scholar 

  19. H. Wu, Z. Deng, B. Zhang, Q. Liu, J. Chen, Classifier Model Based on Machine Learning Algorithms: Application to Differential Diagnosis of Suspicious Thyroid Nodules via Sonography. AJR Am. J. Roentgenol. 207(4), 859–864 (2016).

    Article  PubMed  Google Scholar 

  20. H.A. Abu Alfeilat, A.B.A. Hassanat, O. Lasassmeh, A.S. Tarawneh, M.B. Alhasanat, H.S. Eyal Salman, V.B.S. Prasath, Effects of Distance Measure Choice on K-Nearest Neighbor Classifier Performance: A Review. Big Data 7(4), 221–248 (2019).

    Article  PubMed  Google Scholar 

  21. W.C. Lo, P.W. Cheng, C.T. Wang, S.T. Yeh, L.J. Liao, Pain levels associated with ultrasound-guided fine-needle aspiration biopsy for neck masses. Head. Neck 36(2), 252–256 (2014).

    Article  PubMed  Google Scholar 

  22. C. Cappelli, I. Pirola, B. Agosti, A. Tironi, E. Gandossi, P. Incardona, F. Marini, A. Guerini, M. Castellano, Complications after fine-needle aspiration cytology: a retrospective study of 7449 consecutive thyroid nodules. Br. J. Oral. Maxillofac. Surg. 55(3), 266–269 (2017).

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

Some of our experiments were carried out on the OnekeyAI platform. We thank OnekeyAI and their developers’ help in this scientifific research work.

Funding

The Science and Technology Program of Traditional Chinese Medicine in Zhejiang Province (2022ZA119) and the Zhejiang Medical and Health Science and Technology Plan Project (2021KY927, 2020KY482).

Author information

Author notes

  1. These authors contributed equally: Tianhan Zhou, Tao Hu

Authors and Affiliations

  1. The Department of General Surgery, Hangzhou TCM Hospital Affiliated to Zhejiang Chinese Medical University, Hangzhou, China

    Tianhan Zhou, Zhongkai Ni, Yangyang Xie, Haimin Jin & Hai Huang

  2. The Fourth Clinical Medical College, Zhejiang Chinese Medical University, Hangzhou, China

    Tao Hu

  3. The Department of Ultrasound, Hangzhou TCM Hospital Affiliated to Zhejiang Chinese Medical University, Hangzhou, China

    Chun Yao

  4. Department of Surgical Oncology, Affiliated Hangzhou First People’s Hospital, Zhejiang University School of Medicine, Hangzhou, China

    Dingcun Luo

Authors
  1. Tianhan Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tao Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhongkai Ni
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chun Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yangyang Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Haimin Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Dingcun Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Hai Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Dingcun Luo or Hai Huang.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, T., Hu, T., Ni, Z. et al. Comparative analysis of machine learning-based ultrasound radiomics in predicting malignancy of partially cystic thyroid nodules. Endocrine 83, 118–126 (2024). https://doi.org/10.1007/s12020-023-03461-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12020-023-03461-0

Keywords

Search

Navigation