Abstract
Objectives
To assume the ideal cut-off values and diagnostic performance of two thyroid imaging reporting and data systems published by the Korean Thyroid Association/Korean Society of Thyroid Radiology (Korean TI-RADS) and the American Thyroid Association (ATA TI-RADS).
Methods
Eighteen studies with 25,422 patients from PubMed, SCOPUS, Embase, Web of Science, and Cochrane Library databases up to August 2022. True and false positive and negative values with characteristics were extracted.
Results
The highest area under the receiver operating characteristic curve (AUC) was 0.893 and 0.887 for Korean and ATA TI-RADS. High suspicion was judged as the best cut-off value with the highest AUC based on optimal sensitivity and specificity. In determining the risk of malignant thyroid nodules, high suspicion in Korean and ATA TI-RADS showed sensitivity as 71.3% and 73.5%, specificity as 7.9% and 86.4%, diagnostic odds ratios as 20.0289 and 20.9076, AUC as 0.893 and 0.887. There was no significant difference when directly comparing the diagnostic accuracy of both TI-RADS.
Conclusion
The two risk stratification systems had good diagnostic performance with high AUC and no significant differences. The ideal cut-off can depend on the medical condition or thyroid nodules, because the changes of cut-off point may reciprocally alter sensitivity and specificity.
Similar content being viewed by others
References
Gharib H, Papini E, Paschke R, Duick DS, Valcavi R, Hegedüs L, Vitti P (2010) American association of clinical endocrinologists, Associazione Medici Endocrinologi, and European thyroid association medical guidelines for clinical practice for the diagnosis and management of thyroid nodules: executive summary of recommendations. J Endocrinlog Investig 16(3):468–475. https://doi.org/10.4158/ep.16.3.468
Perros P, Boelaert K, Colley S, Evans C, Evans RM, Gerrard Ba G, Gilbert J, Harrison B, Johnson SJ, Giles TE, Moss L, Lewington V, Newbold K, Taylor J, Thakker RV, Watkinson J, Williams GR (2014) Guidelines for the management of thyroid cancer. Clin Endocrinol 81(Suppl 1):1–122. https://doi.org/10.1111/cen.12515
Orhan Soylemez UP, Gunduz N (2022) Diagnostic accuracy of five different classification systems for thyroid nodules: a prospective comparative study. J Ultrasound Med 41(5):1125–1136. https://doi.org/10.1002/jum.15802
Barbosa TLM, Junior COM, Graf H, Cavalvanti T, Trippia MA, da Silveira Ugino RT, de Oliveira GL, Granella VH, de Carvalho GA (2019) ACR TI-RADS and ATA US scores are helpful for the management of thyroid nodules with indeterminate cytology. BMC Endocr Disord 19(1):112. https://doi.org/10.1186/s12902-019-0429-5
Haugen BR (2017) 2015 American thyroid association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: what is new and what has changed? Cancer 123(3):372–381. https://doi.org/10.1002/cncr.30360
Basha MAA, Alnaggar AA, Refaat R, El-Maghraby AM, Refaat MM, Abd Elhamed ME, Abdalla A, Aly SA, Hanafy AS, Mohamed AEM, Afifi AHM, Harb O (2019) The validity and reproducibility of the thyroid imaging reporting and data system (TI-RADS) in categorization of thyroid nodules: multicentre prospective study. Eur J Radiol 117:184–192. https://doi.org/10.1016/j.ejrad.2019.06.015
Zhang WB, Xu HX, Zhang YF, Guo LH, Xu SH, Zhao CK, Liu BJ (2020) Comparisons of ACR TI-RADS, ATA guidelines, Kwak TI-RADS, and KTA/KSThR guidelines in malignancy risk stratification of thyroid nodules. Clin Hemorheol Microcirc 75(2):219–232. https://doi.org/10.3233/ch-190778
Horvath E, Majlis S, Rossi R, Franco C, Niedmann JP, Castro A, Dominguez M (2009) An ultrasonogram reporting system for thyroid nodules stratifying cancer risk for clinical management. J Clin Endocrinol Metab 94(5):1748–1751. https://doi.org/10.1210/jc.2008-1724
Park JY, Lee HJ, Jang HW, Kim HK, Yi JH, Lee W, Kim SH (2009) A proposal for a thyroid imaging reporting and data system for ultrasound features of thyroid carcinoma. Thyroid 19(11):1257–1264. https://doi.org/10.1089/thy.2008.0021
Russ G, Bigorgne C, Royer B, Rouxel A, Bienvenu-Perrard M (2011) The thyroid imaging reporting and data system (TIRADS) for ultrasound of the thyroid. J Radiol 92(7–8):701–713. https://doi.org/10.1016/j.jradio.2011.03.022
Kwak JY, Han KH, Yoon JH, Moon HJ, Son EJ, Park SH, Jung HK, Choi JS, Kim BM, Kim EK (2011) Thyroid imaging reporting and data system for US features of nodules: a step in establishing better stratification of cancer risk. Radiology 260(3):892–899. https://doi.org/10.1148/radiol.11110206
Haugen BR, Alexander EK, Bible KC, Doherty GM, Mandel SJ, Nikiforov YE, Pacini F, Randolph GW, Sawka AM, Schlumberger M, Schuff KG, Sherman SI, Sosa JA, Steward DL, Tuttle RM, Wartofsky L (2016) 2015 American thyroid association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: the American thyroid association guidelines task force on thyroid nodules and differentiated thyroid cancer. Thyroid 26(1):1–133. https://doi.org/10.1089/thy.2015.0020
Shin JH, Baek JH, Chung J, Ha EJ, Kim JH, Lee YH, Lim HK, Moon WJ, Na DG, Park JS, Choi YJ, Hahn SY, Jeon SJ, Jung SL, Kim DW, Kim EK, Kwak JY, Lee CY, Lee HJ, Lee JH, Lee JH, Lee KH, Park SW, Sung JY (2016) Ultrasonography diagnosis and imaging-based management of thyroid nodules: revised Korean society of thyroid radiology consensus statement and recommendations. Korean J Radiol 17(3):370–395. https://doi.org/10.3348/kjr.2016.17.3.370
Tessler FN, Middleton WD, Grant EG, Hoang JK, Berland LL, Teefey SA, Cronan JJ, Beland MD, Desser TS, Frates MC, Hammers LW, Hamper UM, Langer JE, Reading CC, Scoutt LM, Stavros AT (2017) ACR thyroid imaging, reporting and data system (TI-RADS): white paper of the ACR TI-RADS committee. J Am Coll Radiol 14(5):587–595. https://doi.org/10.1016/j.jacr.2017.01.046
Ha SM, Ahn HS, Baek JH, Ahn HY, Chung YJ, Cho BY, Park SB (2017) Validation of three scoring risk-stratification models for thyroid nodules. Thyroid 27(12):1550–1557. https://doi.org/10.1089/thy.2017.0363
Kim DH, Kim SW, Basurrah MA, Hwang SH (2022) Clinical and laboratory features of various criteria of eosinophilic chronic rhinosinusitis: a systematic review and meta-analysis. Clin Exp Otorhinolaryngol 15(3):230–246. https://doi.org/10.21053/ceo.2022.00052
Ha EJ, Na DG, Baek JH, Sung JY, Kim JH, Kang SY (2018) US fine-needle aspiration biopsy for thyroid malignancy: diagnostic performance of seven society guidelines applied to 2000 thyroid nodules. Radiology 287(3):893–900. https://doi.org/10.1148/radiol.2018171074
Ha EJ, Na DG, Moon WJ, Lee YH, Choi N (2018) Diagnostic performance of ultrasound-based risk-stratification systems for thyroid nodules: comparison of the 2015 American thyroid association guidelines with the 2016 Korean thyroid association/Korean society of thyroid radiology and 2017 American college of radiology guidelines. Thyroid 28(11):1532–1537. https://doi.org/10.1089/thy.2018.0094
Hong HS, Lee JY (2019) Diagnostic performance of ultrasound patterns by K-TIRADS and 2015 ATA guidelines in risk stratification of thyroid nodules and follicular lesions of undetermined significance. AJR Am J Roentgenol 213(2):444–450. https://doi.org/10.2214/ajr.18.20961
Xiang P, Chu X, Chen G, Liu B, Ding W, Zeng Z, Wu X, Wang J, Xu S, Liu C (2019) Nodules with nonspecific ultrasound pattern according to the 2015 American thyroid association malignancy risk stratification system: a comparison to the thyroid imaging reporting and data system (TIRADS-Na). Medicine 98(44):e17657. https://doi.org/10.1097/md.0000000000017657
Yang W, Fananapazir G, LaRoy J, Wilson M, Campbell MJ (2020) Can the American thyroid association, K-Tirads, and Acr-Tirads ultrasound classification systems be used to predict malignancy in Bethesda category IV nodules? Endocr Pract 26(9):945–952. https://doi.org/10.4158/ep-2020-0024
Yoon JH, Lee HS, Kim EK, Moon HJ, Park VY, Kwak JY (2020) Pattern-based versus score-based guidelines using ultrasound features have different strengths in risk stratification of thyroid nodules. Eur Radiol 30(7):3793–3802. https://doi.org/10.1007/s00330-020-06722-y
Zhang WB, Li JJ, Chen XY, He BL, Shen RH, Liu H, Chen J, He XF (2020) SWE combined with ACR TI-RADS categories for malignancy risk stratification of thyroid nodules with indeterminate FNA cytology. Clin Hemorheol Microcirc 76(3):381–390. https://doi.org/10.3233/ch-200893
Chen X, Kutaiba N, Pearce S, Digby S, Van Gelderen D (2022) Application of thyroid imaging reporting and data system (TIRADS) guidelines to thyroid nodules with cytopathological correlation and impact on healthcare costs. Intern Med J 52(8):1366–1373. https://doi.org/10.1111/imj.15343
Ha EJ, Shin JH, Na DG, Jung SL, Lee YH, Paik W, Hong MJ, Kim YK, Lee CY (2021) Comparison of the diagnostic performance of the modified Korean thyroid imaging reporting and data system for thyroid malignancy with three international guidelines. Ultrasonography 40(4):594–601. https://doi.org/10.14366/usg.21056
Kang S, Kwon SK, Choi HS, Kim MJ, Park YJ, Park DJ, Cho SW (2021) Comparison of Korean versus American thyroid imaging reporting and data system in malignancy risk assessment of indeterminate thyroid nodules. Endocrinol Metab 36(5):1111–1120. https://doi.org/10.3803/EnM.2021.1208
Na DG, Paik W, Cha J, Gwon HY, Kim SY, Yoo RE (2021) Diagnostic performance of the modified Korean thyroid imaging reporting and data system for thyroid malignancy according to nodule size: a comparison with five society guidelines. Ultrasonography 40(4):474–485. https://doi.org/10.14366/usg.20148
Qi Q, Zhou A, Guo S, Huang X, Chen S, Li Y, Xu P (2021) Explore the diagnostic efficiency of Chinese thyroid imaging reporting and data systems by comparing with the other four systems (ACR TI-RADS, Kwak-TIRADS, KSThR-TIRADS, and EU-TIRADS): a single-center study. Front Endocrinol 12:763897. https://doi.org/10.3389/fendo.2021.763897
Scappaticcio L, Maiorino MI, Iorio S, Docimo G, Longo M, Grandone A, Luongo C, Cozzolino I, Piccardo A, Trimboli P, Miraglia Del Giudice E, Esposito K, Bellastella G (2021) Exploring the performance of ultrasound risk stratification systems in thyroid nodules of pediatric patients. Cancers 13(21):5304. https://doi.org/10.3390/cancers13215304
Seifert P, Schenke S, Zimny M, Stahl A, Grunert M, Klemenz B, Freesmeyer M, Kreissl MC, Herrmann K, Görges R (2021) Diagnostic performance of Kwak, EU, ACR, and Korean TIRADS as Well as ATA Guidelines for the ultrasound risk stratification of non-autonomously functioning thyroid nodules in a region with long history of iodine deficiency: a German multicenter trial. Cancers (Basel) 13(17):4467. https://doi.org/10.3390/cancers13174467
Zhang Z, Lin N (2022) Clinical diagnostic value of American college of radiology thyroid imaging report and data system in different kinds of thyroid nodules. BMC Endocr Disord 22(1):145. https://doi.org/10.1186/s12902-022-01053-z
Zhu H, Yang Y, Wu S, Chen K, Luo H, Huang J (2021) Diagnostic performance of US-based FNAB criteria of the 2020 Chinese guideline for malignant thyroid nodules: comparison with the 2017 American college of radiology guideline, the 2015 American thyroid association guideline, and the 2016 Korean thyroid association guideline. Quant Imaging Med Surg 11(8):3604–3618. https://doi.org/10.21037/qims-20-1365
Chen Q, Lin M, Wu S (2022) Validating and comparing C-TIRADS, K-TIRADS and ACR-TIRADS in stratifying the malignancy risk of thyroid nodules. Front Endocrinol 13:899575. https://doi.org/10.3389/fendo.2022.899575
Sharma P, Elfatairy K, Gandhi D, Sawhney H, Osman M, Kochar PS, Cohen S (2021) Diagnostic performance of ACR-TIRADS in differentiating benign from malignant thyroid nodules in patients undergoing fine-needle aspiration biopsy: comparative study based on five international guidelines for management of thyroid nodules. J Endocrinol Metab 11(3–4):69–75
Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH (2005) Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol 58(10):982–990. https://doi.org/10.1016/j.jclinepi.2005.02.022
Hoeboer SH, van der Geest PJ, Nieboer D, Groeneveld AB (2015) The diagnostic accuracy of procalcitonin for bacteraemia: a systematic review and meta-analysis. Clin Microbiol Infect 21(5):474–481. https://doi.org/10.1016/j.cmi.2014.12.026
Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, Leeflang MM, Sterne JA, Bossuyt PM (2011) QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 155(8):529–536. https://doi.org/10.7326/0003-4819-155-8-201110180-00009
Zhang J, Liu BJ, Xu HX, Xu JM, Zhang YF, Liu C, Wu J, Sun LP, Guo LH, Liu LN, Xu XH, Qu S (2015) Prospective validation of an ultrasound-based thyroid imaging reporting and data system (TI-RADS) on 3980 thyroid nodules. Int J Clin Exp Med 8(4):5911–5917
Yang R, Zou X, Zeng H, Zhao Y, Ma X (2020) Comparison of diagnostic performance of five different ultrasound TI-RADS classification guidelines for thyroid nodules. Front Oncol 10:598225. https://doi.org/10.3389/fonc.2020.598225
Migda B, Migda M, Migda MS (2019) A systematic review and meta-analysis of the Kwak TIRADS for the diagnostic assessment of indeterminate thyroid nodules. Clin Radiol 74(2):123–130. https://doi.org/10.1016/j.crad.2018.09.009
Migda B, Migda M, Migda MS, Slapa RZ (2018) Use of the Kwak thyroid image reporting and data system (K-TIRADS) in differential diagnosis of thyroid nodules: systematic review and meta-analysis. Eur Radiol 28(6):2380–2388. https://doi.org/10.1007/s00330-017-5230-0
Castellana M, Castellana C, Treglia G, Giorgino F, Giovanella L, Russ G, Trimboli P (2020) Performance of five ultrasound risk stratification systems in selecting thyroid nodules for FNA. J Clin Endocrinol Metab 105(5):1659–1669. https://doi.org/10.1210/clinem/dgz170
Wei X, Li Y, Zhang S, Gao M (2016) Meta-analysis of thyroid imaging reporting and data system in the ultrasonographic diagnosis of 10,437 thyroid nodules. Head Neck 38(2):309–315. https://doi.org/10.1002/hed.23878
Kim PH, Suh CH, Baek JH, Chung SR, Choi YJ, Lee JH (2021) Unnecessary thyroid nodule biopsy rates under four ultrasound risk stratification systems: a systematic review and meta-analysis. Eur Radiol 31(5):2877–2885. https://doi.org/10.1007/s00330-020-07384-6
Choi YJ, Baek JH, Baek SH, Shim WH, Lee KD, Lee HS, Shong YK, Ha EJ, Lee JH (2015) Web-based malignancy risk estimation for thyroid nodules using ultrasonography characteristics: development and validation of a predictive model. Thyroid 25(12):1306–1312. https://doi.org/10.1089/thy.2015.0188
Wendl CM, Janke M, Jung W, Stroszczysnski C, Jung EM (2015) Contrast-enhanced ultrasound with perfusion analysis for the identification of malignant and benign tumours of the thyroid gland. Clin Hemorheol Microcirc 63(2):113–121. https://doi.org/10.3233/ch-151966
Zhao H, Liu X, Lei B, Cheng P, Li J, Wu Y, Ma Z (2019) Impact of thyroid nodule sizes on the diagnostic performance of Korean thyroid imaging reporting and data system and contrast-enhanced ultrasound. Clin Hemorheol Microcirc 72(3):317–326. https://doi.org/10.3233/ch-180545
Chen BD, Xu HX, Zhang YF, Liu BJ, Guo LH, Li DD, Zhao CK, Li XL, Wang D, Zhao SS (2017) The diagnostic performances of conventional strain elastography (SE), acoustic radiation force impulse (ARFI) imaging and point shear-wave speed (pSWS) measurement for non-calcified thyroid nodules. Clin Hemorheol Microcirc 65(3):259–273. https://doi.org/10.3233/ch-16178
Gao L, Xi X, Jiang Y, Yang X, Wang Y, Zhu S, Lai X, Zhang X, Zhao R, Zhang B (2019) Comparison among TIRADS (ACR TI-RADS and KWAK-TI-RADS) and 2015 ATA guidelines in the diagnostic efficiency of thyroid nodules. Endocrine 64(1):90–96. https://doi.org/10.1007/s12020-019-01843-x
Middleton WD, Teefey SA, Reading CC, Langer JE, Beland MD, Szabunio MM, Desser TS (2017) Multiinstitutional analysis of thyroid nodule risk stratification using the American college of radiology thyroid imaging reporting and data system. AJR Am J Roentgenol 208(6):1331–1341. https://doi.org/10.2214/ajr.16.17613
Yun G, Kim YK, Choi SI, Kim JH (2018) Medullary thyroid carcinoma: application of thyroid imaging reporting and data system (TI-RADS) classification. Endocrine 61(2):285–292. https://doi.org/10.1007/s12020-018-1594-4
He YP, Xu HX, Zhao CK, Sun LP, Li XL, Yue WW, Guo LH, Wang D, Ren WW, Wang Q, Qu S (2017) Cytologically indeterminate thyroid nodules: increased diagnostic performance with combination of US TI-RADS and a new scoring system. Sci Rep 7(1):6906. https://doi.org/10.1038/s41598-017-07353-y
Funding
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (2022R1F1A1066232). This work was supported by the Soonchunhyang University Hospital Cheonan Research Fund. The sponsors had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by YJK and SHH. The first draft of the manuscript was written by HSA, GS and JEL. All authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no relevant financial or non-financial interests to disclose.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Kang, Y.J., Ahn, H.S., Stybayeva, G. et al. Comparison of diagnostic performance of two ultrasound risk stratification systems for thyroid nodules: a systematic review and meta-analysis. Radiol med 128, 1407–1414 (2023). https://doi.org/10.1007/s11547-023-01709-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11547-023-01709-3