Skip to main content
Log in

Comparison of diagnostic performance of two ultrasound risk stratification systems for thyroid nodules: a systematic review and meta-analysis

  • Diagnostic Imaging in Oncology
  • Published:
La radiologia medica Aims and scope Submit manuscript

Abstract

Objectives

To assume the ideal cut-off values and diagnostic performance of two thyroid imaging reporting and data systems published by the Korean Thyroid Association/Korean Society of Thyroid Radiology (Korean TI-RADS) and the American Thyroid Association (ATA TI-RADS).

Methods

Eighteen studies with 25,422 patients from PubMed, SCOPUS, Embase, Web of Science, and Cochrane Library databases up to August 2022. True and false positive and negative values with characteristics were extracted.

Results

The highest area under the receiver operating characteristic curve (AUC) was 0.893 and 0.887 for Korean and ATA TI-RADS. High suspicion was judged as the best cut-off value with the highest AUC based on optimal sensitivity and specificity. In determining the risk of malignant thyroid nodules, high suspicion in Korean and ATA TI-RADS showed sensitivity as 71.3% and 73.5%, specificity as 7.9% and 86.4%, diagnostic odds ratios as 20.0289 and 20.9076, AUC as 0.893 and 0.887. There was no significant difference when directly comparing the diagnostic accuracy of both TI-RADS.

Conclusion

The two risk stratification systems had good diagnostic performance with high AUC and no significant differences. The ideal cut-off can depend on the medical condition or thyroid nodules, because the changes of cut-off point may reciprocally alter sensitivity and specificity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Similar content being viewed by others

References

  1. Gharib H, Papini E, Paschke R, Duick DS, Valcavi R, Hegedüs L, Vitti P (2010) American association of clinical endocrinologists, Associazione Medici Endocrinologi, and European thyroid association medical guidelines for clinical practice for the diagnosis and management of thyroid nodules: executive summary of recommendations. J Endocrinlog Investig 16(3):468–475. https://doi.org/10.4158/ep.16.3.468

    Article  Google Scholar 

  2. Perros P, Boelaert K, Colley S, Evans C, Evans RM, Gerrard Ba G, Gilbert J, Harrison B, Johnson SJ, Giles TE, Moss L, Lewington V, Newbold K, Taylor J, Thakker RV, Watkinson J, Williams GR (2014) Guidelines for the management of thyroid cancer. Clin Endocrinol 81(Suppl 1):1–122. https://doi.org/10.1111/cen.12515

    Article  CAS  Google Scholar 

  3. Orhan Soylemez UP, Gunduz N (2022) Diagnostic accuracy of five different classification systems for thyroid nodules: a prospective comparative study. J Ultrasound Med 41(5):1125–1136. https://doi.org/10.1002/jum.15802

    Article  PubMed  Google Scholar 

  4. Barbosa TLM, Junior COM, Graf H, Cavalvanti T, Trippia MA, da Silveira Ugino RT, de Oliveira GL, Granella VH, de Carvalho GA (2019) ACR TI-RADS and ATA US scores are helpful for the management of thyroid nodules with indeterminate cytology. BMC Endocr Disord 19(1):112. https://doi.org/10.1186/s12902-019-0429-5

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Haugen BR (2017) 2015 American thyroid association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: what is new and what has changed? Cancer 123(3):372–381. https://doi.org/10.1002/cncr.30360

    Article  PubMed  Google Scholar 

  6. Basha MAA, Alnaggar AA, Refaat R, El-Maghraby AM, Refaat MM, Abd Elhamed ME, Abdalla A, Aly SA, Hanafy AS, Mohamed AEM, Afifi AHM, Harb O (2019) The validity and reproducibility of the thyroid imaging reporting and data system (TI-RADS) in categorization of thyroid nodules: multicentre prospective study. Eur J Radiol 117:184–192. https://doi.org/10.1016/j.ejrad.2019.06.015

    Article  PubMed  Google Scholar 

  7. Zhang WB, Xu HX, Zhang YF, Guo LH, Xu SH, Zhao CK, Liu BJ (2020) Comparisons of ACR TI-RADS, ATA guidelines, Kwak TI-RADS, and KTA/KSThR guidelines in malignancy risk stratification of thyroid nodules. Clin Hemorheol Microcirc 75(2):219–232. https://doi.org/10.3233/ch-190778

    Article  CAS  PubMed  Google Scholar 

  8. Horvath E, Majlis S, Rossi R, Franco C, Niedmann JP, Castro A, Dominguez M (2009) An ultrasonogram reporting system for thyroid nodules stratifying cancer risk for clinical management. J Clin Endocrinol Metab 94(5):1748–1751. https://doi.org/10.1210/jc.2008-1724

    Article  CAS  PubMed  Google Scholar 

  9. Park JY, Lee HJ, Jang HW, Kim HK, Yi JH, Lee W, Kim SH (2009) A proposal for a thyroid imaging reporting and data system for ultrasound features of thyroid carcinoma. Thyroid 19(11):1257–1264. https://doi.org/10.1089/thy.2008.0021

    Article  PubMed  Google Scholar 

  10. Russ G, Bigorgne C, Royer B, Rouxel A, Bienvenu-Perrard M (2011) The thyroid imaging reporting and data system (TIRADS) for ultrasound of the thyroid. J Radiol 92(7–8):701–713. https://doi.org/10.1016/j.jradio.2011.03.022

    Article  CAS  PubMed  Google Scholar 

  11. Kwak JY, Han KH, Yoon JH, Moon HJ, Son EJ, Park SH, Jung HK, Choi JS, Kim BM, Kim EK (2011) Thyroid imaging reporting and data system for US features of nodules: a step in establishing better stratification of cancer risk. Radiology 260(3):892–899. https://doi.org/10.1148/radiol.11110206

    Article  PubMed  Google Scholar 

  12. Haugen BR, Alexander EK, Bible KC, Doherty GM, Mandel SJ, Nikiforov YE, Pacini F, Randolph GW, Sawka AM, Schlumberger M, Schuff KG, Sherman SI, Sosa JA, Steward DL, Tuttle RM, Wartofsky L (2016) 2015 American thyroid association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: the American thyroid association guidelines task force on thyroid nodules and differentiated thyroid cancer. Thyroid 26(1):1–133. https://doi.org/10.1089/thy.2015.0020

    Article  PubMed  PubMed Central  Google Scholar 

  13. Shin JH, Baek JH, Chung J, Ha EJ, Kim JH, Lee YH, Lim HK, Moon WJ, Na DG, Park JS, Choi YJ, Hahn SY, Jeon SJ, Jung SL, Kim DW, Kim EK, Kwak JY, Lee CY, Lee HJ, Lee JH, Lee JH, Lee KH, Park SW, Sung JY (2016) Ultrasonography diagnosis and imaging-based management of thyroid nodules: revised Korean society of thyroid radiology consensus statement and recommendations. Korean J Radiol 17(3):370–395. https://doi.org/10.3348/kjr.2016.17.3.370

    Article  PubMed  PubMed Central  Google Scholar 

  14. Tessler FN, Middleton WD, Grant EG, Hoang JK, Berland LL, Teefey SA, Cronan JJ, Beland MD, Desser TS, Frates MC, Hammers LW, Hamper UM, Langer JE, Reading CC, Scoutt LM, Stavros AT (2017) ACR thyroid imaging, reporting and data system (TI-RADS): white paper of the ACR TI-RADS committee. J Am Coll Radiol 14(5):587–595. https://doi.org/10.1016/j.jacr.2017.01.046

    Article  PubMed  Google Scholar 

  15. Ha SM, Ahn HS, Baek JH, Ahn HY, Chung YJ, Cho BY, Park SB (2017) Validation of three scoring risk-stratification models for thyroid nodules. Thyroid 27(12):1550–1557. https://doi.org/10.1089/thy.2017.0363

    Article  PubMed  Google Scholar 

  16. Kim DH, Kim SW, Basurrah MA, Hwang SH (2022) Clinical and laboratory features of various criteria of eosinophilic chronic rhinosinusitis: a systematic review and meta-analysis. Clin Exp Otorhinolaryngol 15(3):230–246. https://doi.org/10.21053/ceo.2022.00052

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Ha EJ, Na DG, Baek JH, Sung JY, Kim JH, Kang SY (2018) US fine-needle aspiration biopsy for thyroid malignancy: diagnostic performance of seven society guidelines applied to 2000 thyroid nodules. Radiology 287(3):893–900. https://doi.org/10.1148/radiol.2018171074

    Article  PubMed  Google Scholar 

  18. Ha EJ, Na DG, Moon WJ, Lee YH, Choi N (2018) Diagnostic performance of ultrasound-based risk-stratification systems for thyroid nodules: comparison of the 2015 American thyroid association guidelines with the 2016 Korean thyroid association/Korean society of thyroid radiology and 2017 American college of radiology guidelines. Thyroid 28(11):1532–1537. https://doi.org/10.1089/thy.2018.0094

    Article  PubMed  Google Scholar 

  19. Hong HS, Lee JY (2019) Diagnostic performance of ultrasound patterns by K-TIRADS and 2015 ATA guidelines in risk stratification of thyroid nodules and follicular lesions of undetermined significance. AJR Am J Roentgenol 213(2):444–450. https://doi.org/10.2214/ajr.18.20961

    Article  PubMed  Google Scholar 

  20. Xiang P, Chu X, Chen G, Liu B, Ding W, Zeng Z, Wu X, Wang J, Xu S, Liu C (2019) Nodules with nonspecific ultrasound pattern according to the 2015 American thyroid association malignancy risk stratification system: a comparison to the thyroid imaging reporting and data system (TIRADS-Na). Medicine 98(44):e17657. https://doi.org/10.1097/md.0000000000017657

    Article  PubMed  PubMed Central  Google Scholar 

  21. Yang W, Fananapazir G, LaRoy J, Wilson M, Campbell MJ (2020) Can the American thyroid association, K-Tirads, and Acr-Tirads ultrasound classification systems be used to predict malignancy in Bethesda category IV nodules? Endocr Pract 26(9):945–952. https://doi.org/10.4158/ep-2020-0024

    Article  PubMed  Google Scholar 

  22. Yoon JH, Lee HS, Kim EK, Moon HJ, Park VY, Kwak JY (2020) Pattern-based versus score-based guidelines using ultrasound features have different strengths in risk stratification of thyroid nodules. Eur Radiol 30(7):3793–3802. https://doi.org/10.1007/s00330-020-06722-y

    Article  PubMed  Google Scholar 

  23. Zhang WB, Li JJ, Chen XY, He BL, Shen RH, Liu H, Chen J, He XF (2020) SWE combined with ACR TI-RADS categories for malignancy risk stratification of thyroid nodules with indeterminate FNA cytology. Clin Hemorheol Microcirc 76(3):381–390. https://doi.org/10.3233/ch-200893

    Article  PubMed  Google Scholar 

  24. Chen X, Kutaiba N, Pearce S, Digby S, Van Gelderen D (2022) Application of thyroid imaging reporting and data system (TIRADS) guidelines to thyroid nodules with cytopathological correlation and impact on healthcare costs. Intern Med J 52(8):1366–1373. https://doi.org/10.1111/imj.15343

    Article  PubMed  Google Scholar 

  25. Ha EJ, Shin JH, Na DG, Jung SL, Lee YH, Paik W, Hong MJ, Kim YK, Lee CY (2021) Comparison of the diagnostic performance of the modified Korean thyroid imaging reporting and data system for thyroid malignancy with three international guidelines. Ultrasonography 40(4):594–601. https://doi.org/10.14366/usg.21056

    Article  PubMed  PubMed Central  Google Scholar 

  26. Kang S, Kwon SK, Choi HS, Kim MJ, Park YJ, Park DJ, Cho SW (2021) Comparison of Korean versus American thyroid imaging reporting and data system in malignancy risk assessment of indeterminate thyroid nodules. Endocrinol Metab 36(5):1111–1120. https://doi.org/10.3803/EnM.2021.1208

    Article  Google Scholar 

  27. Na DG, Paik W, Cha J, Gwon HY, Kim SY, Yoo RE (2021) Diagnostic performance of the modified Korean thyroid imaging reporting and data system for thyroid malignancy according to nodule size: a comparison with five society guidelines. Ultrasonography 40(4):474–485. https://doi.org/10.14366/usg.20148

    Article  PubMed  Google Scholar 

  28. Qi Q, Zhou A, Guo S, Huang X, Chen S, Li Y, Xu P (2021) Explore the diagnostic efficiency of Chinese thyroid imaging reporting and data systems by comparing with the other four systems (ACR TI-RADS, Kwak-TIRADS, KSThR-TIRADS, and EU-TIRADS): a single-center study. Front Endocrinol 12:763897. https://doi.org/10.3389/fendo.2021.763897

    Article  Google Scholar 

  29. Scappaticcio L, Maiorino MI, Iorio S, Docimo G, Longo M, Grandone A, Luongo C, Cozzolino I, Piccardo A, Trimboli P, Miraglia Del Giudice E, Esposito K, Bellastella G (2021) Exploring the performance of ultrasound risk stratification systems in thyroid nodules of pediatric patients. Cancers 13(21):5304. https://doi.org/10.3390/cancers13215304

    Article  PubMed  PubMed Central  Google Scholar 

  30. Seifert P, Schenke S, Zimny M, Stahl A, Grunert M, Klemenz B, Freesmeyer M, Kreissl MC, Herrmann K, Görges R (2021) Diagnostic performance of Kwak, EU, ACR, and Korean TIRADS as Well as ATA Guidelines for the ultrasound risk stratification of non-autonomously functioning thyroid nodules in a region with long history of iodine deficiency: a German multicenter trial. Cancers (Basel) 13(17):4467. https://doi.org/10.3390/cancers13174467

    Article  CAS  PubMed  Google Scholar 

  31. Zhang Z, Lin N (2022) Clinical diagnostic value of American college of radiology thyroid imaging report and data system in different kinds of thyroid nodules. BMC Endocr Disord 22(1):145. https://doi.org/10.1186/s12902-022-01053-z

    Article  PubMed  PubMed Central  Google Scholar 

  32. Zhu H, Yang Y, Wu S, Chen K, Luo H, Huang J (2021) Diagnostic performance of US-based FNAB criteria of the 2020 Chinese guideline for malignant thyroid nodules: comparison with the 2017 American college of radiology guideline, the 2015 American thyroid association guideline, and the 2016 Korean thyroid association guideline. Quant Imaging Med Surg 11(8):3604–3618. https://doi.org/10.21037/qims-20-1365

    Article  PubMed  PubMed Central  Google Scholar 

  33. Chen Q, Lin M, Wu S (2022) Validating and comparing C-TIRADS, K-TIRADS and ACR-TIRADS in stratifying the malignancy risk of thyroid nodules. Front Endocrinol 13:899575. https://doi.org/10.3389/fendo.2022.899575

    Article  Google Scholar 

  34. Sharma P, Elfatairy K, Gandhi D, Sawhney H, Osman M, Kochar PS, Cohen S (2021) Diagnostic performance of ACR-TIRADS in differentiating benign from malignant thyroid nodules in patients undergoing fine-needle aspiration biopsy: comparative study based on five international guidelines for management of thyroid nodules. J Endocrinol Metab 11(3–4):69–75

    Article  Google Scholar 

  35. Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH (2005) Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol 58(10):982–990. https://doi.org/10.1016/j.jclinepi.2005.02.022

    Article  PubMed  Google Scholar 

  36. Hoeboer SH, van der Geest PJ, Nieboer D, Groeneveld AB (2015) The diagnostic accuracy of procalcitonin for bacteraemia: a systematic review and meta-analysis. Clin Microbiol Infect 21(5):474–481. https://doi.org/10.1016/j.cmi.2014.12.026

    Article  CAS  PubMed  Google Scholar 

  37. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, Leeflang MM, Sterne JA, Bossuyt PM (2011) QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 155(8):529–536. https://doi.org/10.7326/0003-4819-155-8-201110180-00009

    Article  PubMed  Google Scholar 

  38. Zhang J, Liu BJ, Xu HX, Xu JM, Zhang YF, Liu C, Wu J, Sun LP, Guo LH, Liu LN, Xu XH, Qu S (2015) Prospective validation of an ultrasound-based thyroid imaging reporting and data system (TI-RADS) on 3980 thyroid nodules. Int J Clin Exp Med 8(4):5911–5917

    PubMed  PubMed Central  Google Scholar 

  39. Yang R, Zou X, Zeng H, Zhao Y, Ma X (2020) Comparison of diagnostic performance of five different ultrasound TI-RADS classification guidelines for thyroid nodules. Front Oncol 10:598225. https://doi.org/10.3389/fonc.2020.598225

    Article  PubMed  PubMed Central  Google Scholar 

  40. Migda B, Migda M, Migda MS (2019) A systematic review and meta-analysis of the Kwak TIRADS for the diagnostic assessment of indeterminate thyroid nodules. Clin Radiol 74(2):123–130. https://doi.org/10.1016/j.crad.2018.09.009

    Article  CAS  PubMed  Google Scholar 

  41. Migda B, Migda M, Migda MS, Slapa RZ (2018) Use of the Kwak thyroid image reporting and data system (K-TIRADS) in differential diagnosis of thyroid nodules: systematic review and meta-analysis. Eur Radiol 28(6):2380–2388. https://doi.org/10.1007/s00330-017-5230-0

    Article  PubMed  PubMed Central  Google Scholar 

  42. Castellana M, Castellana C, Treglia G, Giorgino F, Giovanella L, Russ G, Trimboli P (2020) Performance of five ultrasound risk stratification systems in selecting thyroid nodules for FNA. J Clin Endocrinol Metab 105(5):1659–1669. https://doi.org/10.1210/clinem/dgz170

    Article  Google Scholar 

  43. Wei X, Li Y, Zhang S, Gao M (2016) Meta-analysis of thyroid imaging reporting and data system in the ultrasonographic diagnosis of 10,437 thyroid nodules. Head Neck 38(2):309–315. https://doi.org/10.1002/hed.23878

    Article  PubMed  Google Scholar 

  44. Kim PH, Suh CH, Baek JH, Chung SR, Choi YJ, Lee JH (2021) Unnecessary thyroid nodule biopsy rates under four ultrasound risk stratification systems: a systematic review and meta-analysis. Eur Radiol 31(5):2877–2885. https://doi.org/10.1007/s00330-020-07384-6

    Article  PubMed  Google Scholar 

  45. Choi YJ, Baek JH, Baek SH, Shim WH, Lee KD, Lee HS, Shong YK, Ha EJ, Lee JH (2015) Web-based malignancy risk estimation for thyroid nodules using ultrasonography characteristics: development and validation of a predictive model. Thyroid 25(12):1306–1312. https://doi.org/10.1089/thy.2015.0188

    Article  PubMed  Google Scholar 

  46. Wendl CM, Janke M, Jung W, Stroszczysnski C, Jung EM (2015) Contrast-enhanced ultrasound with perfusion analysis for the identification of malignant and benign tumours of the thyroid gland. Clin Hemorheol Microcirc 63(2):113–121. https://doi.org/10.3233/ch-151966

    Article  CAS  PubMed  Google Scholar 

  47. Zhao H, Liu X, Lei B, Cheng P, Li J, Wu Y, Ma Z (2019) Impact of thyroid nodule sizes on the diagnostic performance of Korean thyroid imaging reporting and data system and contrast-enhanced ultrasound. Clin Hemorheol Microcirc 72(3):317–326. https://doi.org/10.3233/ch-180545

    Article  PubMed  Google Scholar 

  48. Chen BD, Xu HX, Zhang YF, Liu BJ, Guo LH, Li DD, Zhao CK, Li XL, Wang D, Zhao SS (2017) The diagnostic performances of conventional strain elastography (SE), acoustic radiation force impulse (ARFI) imaging and point shear-wave speed (pSWS) measurement for non-calcified thyroid nodules. Clin Hemorheol Microcirc 65(3):259–273. https://doi.org/10.3233/ch-16178

    Article  CAS  PubMed  Google Scholar 

  49. Gao L, Xi X, Jiang Y, Yang X, Wang Y, Zhu S, Lai X, Zhang X, Zhao R, Zhang B (2019) Comparison among TIRADS (ACR TI-RADS and KWAK-TI-RADS) and 2015 ATA guidelines in the diagnostic efficiency of thyroid nodules. Endocrine 64(1):90–96. https://doi.org/10.1007/s12020-019-01843-x

    Article  CAS  PubMed  Google Scholar 

  50. Middleton WD, Teefey SA, Reading CC, Langer JE, Beland MD, Szabunio MM, Desser TS (2017) Multiinstitutional analysis of thyroid nodule risk stratification using the American college of radiology thyroid imaging reporting and data system. AJR Am J Roentgenol 208(6):1331–1341. https://doi.org/10.2214/ajr.16.17613

    Article  PubMed  Google Scholar 

  51. Yun G, Kim YK, Choi SI, Kim JH (2018) Medullary thyroid carcinoma: application of thyroid imaging reporting and data system (TI-RADS) classification. Endocrine 61(2):285–292. https://doi.org/10.1007/s12020-018-1594-4

    Article  CAS  PubMed  Google Scholar 

  52. He YP, Xu HX, Zhao CK, Sun LP, Li XL, Yue WW, Guo LH, Wang D, Ren WW, Wang Q, Qu S (2017) Cytologically indeterminate thyroid nodules: increased diagnostic performance with combination of US TI-RADS and a new scoring system. Sci Rep 7(1):6906. https://doi.org/10.1038/s41598-017-07353-y

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Funding

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (2022R1F1A1066232). This work was supported by the Soonchunhyang University Hospital Cheonan Research Fund. The sponsors had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Author information

Authors and Affiliations

Authors

Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by YJK and SHH. The first draft of the manuscript was written by HSA, GS and JEL. All authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Se Hwan Hwang.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (XLSX 13 KB)

Supplementary file2 (DOCX 17 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kang, Y.J., Ahn, H.S., Stybayeva, G. et al. Comparison of diagnostic performance of two ultrasound risk stratification systems for thyroid nodules: a systematic review and meta-analysis. Radiol med 128, 1407–1414 (2023). https://doi.org/10.1007/s11547-023-01709-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11547-023-01709-3

Keywords

Navigation