Abstract
Purpose
Ultrasound evaluation of thyroid nodules is the preferred technique, but it is dependent on operator interpretation, leading to inter-observer variability. The current study aimed to determine the inter-physician consensus on nodular characteristics, risk categorization in the classification systems, and the need for fine needle aspiration puncture.
Methods
Four endocrinologists from the same center blindly evaluated 100 ultrasound images of thyroid nodules from 100 different patients. The following ultrasound features were evaluated: composition, echogenicity, margins, calcifications, and microcalcifications. Nodules were also classified according to ATA, EU-TIRADS, K-TIRADS, and ACR-TIRADS classifications. Krippendorff’s alpha test was used to assess interobserver agreement.
Results
The interobserver agreement for ultrasound features was: Krippendorff’s coefficient 0.80 (0.71–0.89) for composition, 0.59 (0.47–0.72) for echogenicity, 0.73 (0.57–0.88) for margins, 0.55 (0.40–0.69) for calcifications, and 0.50 (0.34–0.67) for microcalcifications. The concordance for the classification systems was 0.7 (0.61–0.80) for ATA, 0.63 (0.54–0.73) for EU-TIRADS, 0.64 (0.55–0.73) for K-TIRADS, and 0.68 (0.60–0.77) for K-TIRADS. The concordance in the indication of fine needle aspiration puncture (FNA) was 0.86 (0.71–1), 0.80 (0.71–0.88), 0.77 0.67–0.87), and 0.73 (0.64–0.83) for systems previously described respectively.
Conclusions
Interobserver agreement was acceptable for the identification of nodules requiring cytologic study using various classification systems. However, limited concordance was observed in risk stratification and many ultrasonographic characteristics of the nodules.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
B.R. Haugen, E.K. Alexander, K.C. Bible, G.M. Doherty, S.J. Mandel, Y.E. Nikiforov et al. 2015 American Thyroid Association Management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: the American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer. Thyroid 26(1), 1–133 (2016)
S. Guth, U. Theune, J. Aberle, A.B.C. Galach, Very high prevalence of thyroid nodules detected by high frequency (13 MHz) ultrasound examination. Eur. J. Clin. Investig. 39(8), 699–706 (2009)
L H. Clinical Practice, The thyroid nodule. N. Engl. J. Med. 351(17), 1764–1771 (2004)
L.R. Remonti, C.K. Kramer, C.B. Leitao, L.C.G.J. Pinto, Thyroid ultrasound features and risk of carcinoma: a systematic review and meta-analysis of observational studies. Thyroid 25, 538–550 (2015)
E.K. Kim, C.S. Park, W.Y. Chung, K.K. Oh, D.I. Kim, J.T.Y.H. Lee, New sonographic criteria for recommending fine-needle aspiration biopsy of non-palpable solid nodules of the thyroid. Am. J. Roentgenol. 178, 687–691 (2002)
A. Persichetti, E. DI Stasio, C. Coccaro, F. Graziano, A. Bianchini, V. DI Donna et al. Inter- and intraobserver agreement in the assessment of thyroid nodule ultrasound features and classification systems: a blinded multicenter study. Thyroid 30(2), 237–242 (2020)
Y.P. Sych, V.V. Fadeev, E.P. Fisenko, M. Kalashnikova, Reproducibility and interobserver agreement of different Thyroid Imaging and Reporting Data Systems (TIRADS). Eur. Thyroid J. 10(2), 161–167 (2021)
G. Grani, L. Lamartina, V. Cantisani, M. Maranghi, P. Lucia, C. Durante, Interobserver agreement of various thyroid imaging reporting and data systems. Endocr. Connect. 7(1), 1–7 (2018)
F.N. Tessler, W.D. Middleton, E.G. Grant, J.K. Hoang, L.L. Berland, S.A. Teefey, J.J. Cronan, M.D. Beland, T.S. Desser, M.C. Frates, L.W. Hammers, U.M. Hamper, J.E. Langer, C.C. Reading, L.M. Scoutt, ACR Thyroid Imaging, Reporting and Data System (TI-RADS): white paper of the ACR TI-RADS Committee. J. Am. Coll. Radiol. 14(5), 587–595 (2017)
G. Russ, S.J. Bonnema, M.F. Erdogan, C. Durante, R.L.L. Ngu, European Thyroid Association guidelines for ultrasound malignancy risk stratification of thyroid nodules in adults: the EU-TIRADS. Eur. Thyroid J. 6(5), 225–237 (2017)
W.J. Moon, J.H. Baek, S.L. Jung, D.W. Kim, E.K. Kim, J.Y. Kim et al. Ultrasonography and the ultrasound-based management of thyroid nodules: consensus statement and recommendations. Korean J. Radio. 12, 1–14 (2011)
K. Krippendorff, Content analysis: an introduction to its methodology, 2nd edition. (Sage Publications, Thousand Oaks, CA, 2004), pp. 211–256.
J.R.K.G. Landis, The measurement of observer agreement for categorical data. Biometrics 33, 159–174 (1977)
A. Persichetti, E. Di Stasio, R. Guglielmi, G. Bizzarri, S. Taccogna, I. Misischi et al. Predictive value of malignancy of thyroid nodule ultrasound classification systems: a prospective study. J. Clin. Endocrinol. Metab. 103(4), 1359–1368 (2018)
H.K. Su, L.L. Dos Reis, M.A. Lupo, M. Milas, L.A. Orloff, J.E. Langer et al. Striving toward standardization of reporting of ultrasound features of thyroid nodules and lymph nodes: a multidisciplinary consensus statement. Thyroid 24(9), 1341–1349 (2014)
D.G. Na, J.H. Baek, J.Y. Sung, J.H. Kim, J.K. Kim, Y.J. Choi et al. Thyroid imaging reporting and data system risk stratification of thyroid nodules: categorization based on solidity and echogenicity. Thyroid 26(4), 562–572 (2016)
G. Grani, M. D’Alessandri, G. Carbotta, A. Nesca, M. Del Sordo, S. Alessandrini et al. Grey-scale analysis improves the ultrasonographic evaluation of thyroid nodules. Medicines 94(27), e1129 (2015)
K. Dobruch-Sobczak, Z. Adamczewski, M. Dedecjus, A. Lewiński, B. Migda, M. Ruchała et al. Summary of meta-analyses of studies involving TIRADS classifications (EU-TIRADS, ACR-TIRADS, and K-TIRADS) in evaluating the malignant potential of focal lesions of the thyroid gland. J. Ultrason. 22(89), e121–e129 (2022)
M.P. Curado, B. Edwards, H.R. Shin, H. Storm, J. Ferlay, M. Heanue, P. Boyle, Cancer incidence in five continents, vol 10 (Lyon, France, Iarc Scientific Publications, 2014)
J.K. Hoang, W.D. Middleton, A.E. Farjat, S.A. Teefey, N. Abinanti, F.J. Boschini et al. Interobserver variability of sonographic features used in the American College of Radiology thyroid imaging reporting and data system. Am. J. Roentgenol. 211(1), 162–167 (2018)
J. Alyami, F.F. Almutairi, S. Aldoassary, A. Albeshry, A. Almontashri, M. Abounassif et al. Interobserver variability in ultrasound assessment of thyroid nodules. Medicines 101(41), E31106 (2022)
B. Madeo, G. Brigante, A. Ansaloni, E. Taliani, S. Kaleci, M.L. Monzani et al. The added value of operator’s judgement in thyroid nodule ultrasound classification arising from histologically based comparison of different risk stratification systems. Front. Endocrinol. 11(7), 1–9 (2020)
C.S. Park, S.H. Kim, S.L. Jung, B.J. Kang, J.Y. Kim, J.J. Choi et al. Observer variability in the sonographic evaluation of thyroid nodules. J. Clin. Ultrasound 38(6), 287–293 (2010)
P.H. Kim, C.H. Suh, J.H. Baek, S.R. Chung, Y.J. Choi, J.H. Lee, Unnecessary thyroid nodule biopsy rates under four ultrasound risk stratification systems: a systematic review and meta-analysis. Eur. Radiol. 31(5), 2877–2885 (2021)
G. Azizi, K. Faust, M.L. Mayo, J. Farrell, C. Malchoff, Diagnosis of thyroid nodule with new ultrasound imaging modalities. VideoEndocrinology 7(1), 8–10 (2020)
T. Solymosi, L. Hegedűs, S.J. Bonnema, A. Frasoldati, L. Jambor, Z. Karanyi et al. Considerable interobserver variation calls for unambiguous definitions of thyroid nodule ultrasound characteristics. Eur. Thyroid J. 12(2), e220134 (2023)
C. Zhang, D. Liu, L. Huang, Y. Zhao, L.G.Y. Chen, Classification of thyroid nodules by using deep learning radiomics based on ultrasound dynamic video. J. Ultrasound. Med. 41(12), 2993–3002 (2022)
S. Sorrenti, V. Dolcetti, M. Radzina, M.I. Bellini, F. Frezza, K. Munir et al. Artificial intelligence for thyroid nodule characterization: where are we standing? Cancers 14(14), 1–15 (2022)
S. Peng, Y. Liu, W. Lv, L. Liu, Q. Zhou, H. Yang et al. Deep learning-based artificial intelligence model to assist thyroid nodule diagnosis and management: a multicentre diagnostic study. Lancet Digit. Health [Internet] 3(4), e250–e259 (2021). https://doi.org/10.1016/S2589-7500(21)00041-8.
Acknowledgements
The authors would like to thank the Surgery, Pathology, Radiology, Nuclear Medicine, and Endocrinology Departments at Hospital Universitario de Navarra for the support provided in treating patients.
Funding
This research did not receive any specific grant from any funding agency in the public, commercial, or not-for-profit sector. Publication fees were supported by the Fundación de Endocrinología, Nutrición y Diabetes de Navarra.
Author information
Authors and Affiliations
Contributions
J.d.C. was in charge of analysis, writing the article, and interpretation of data. J.G. was responsible for data collection and manuscript correction. F.J.B. was responsible for data analysis, interpretation, and revising critically. J.J.P. was responsible for follow-up patients and intellectual production. M.D.O. was responsible for the adaptation and translation of the text. M.T. and P.M. oversaw data acquisition. E.A. is the overall coordinator of the entire study. All authors discussed previous versions of the manuscript and agreed to the submission of the final version.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no competing interests.
Ethics approval
Ethical principles for medical research involving human subjects under the World Medical Association Declaration of Helsinki have been conducted. The study protocol has been approved by the ethics committee of the Government of Navarre (Spain), (28-may-2021, PI_2021/64). This study has been granted an exemption from requiring written informed consent by the ethics of the Government of Navarre (Spain).
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
de Carlos, J., Garcia, J., Basterra, F.J. et al. Interobserver variability in thyroid ultrasound. Endocrine (2024). https://doi.org/10.1007/s12020-024-03731-5
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s12020-024-03731-5