pp 1–8 | Cite as

Learnability and reproducibility of ACR Thyroid Imaging, Reporting and Data System (TI-RADS) in postgraduate freshmen

  • Dengke Teng
  • Ping Fu
  • Wenjia Li
  • Feng GuoEmail author
  • Hui WangEmail author
Original Article



The Thyroid Imaging, Reporting and Data System (TI-RADS) from the American College of Radiology (ACR) has been used since 2017 for the evaluation of thyroid nodules. The purpose of this study is to assess the learnability and reproducibility of TI-RADS in postgraduate freshmen.


This was a retrospective study involving 400 nodules with a final diagnosis following ultrasound (US) examination. The nodules were randomized into eight groups (50/group). Three postgraduate freshmen and three experts evaluated the nodules according to ACR TI-RADS without knowledge of the final diagnosis. After evaluating each group, training was carried out based on the inconsistencies of the freshmen/experts. Training was stopped after 200 nodules because the κ value showed almost perfect concordance. Three months later, the 50 nodules of Group 4 (the last evaluated group) were re-evaluated to assess the reproducibility of ACR TI-RADS.


The diagnostic accuracy of the three postgraduate freshmen increased from 60%, 48%, and 46% in Group 1 to 80%, 78%, and 72% in Group 4 (P= 0.029, 0.002, and 0.008), respectively. After training, the diagnostic accuracy of the postgraduate freshmen was close to that of the experts (84%). For the US features, the postgraduate freshmen were consistent with the experts (all κ > 0.6). When re-evaluating Group 4 three months later, the five features had substantial to almost perfect agreement for the same researcher (all κ > 0.7).


Based on experts’ consensus, ACR TI-RADS can be learned well, and its reproducibility is excellent.


Thyroid nodule TI-RADS Ultrasound Diagnosis Learning curve Education 



This study was supported by the Chinese Ultrasound Doctors Association (No. KJXX2019002), the Finance Department of Jilin Province (No. SCZSY201701), the Jilin Provincial Health and Family Planning Commission (No. 2016ZC032), and Jilin Province Science and Technology Department (No. 20170414042GH).

Supplementary information

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee (the ethics committee of the China–Japan Union Hospital of Jilin University) and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

The need for individual consent was waived by the committee.

Supplementary material


  1. 1.
    H. Gharib, E. Papini, J.R. Garber, D.S. Duick, R.M. Harrell, L. Hegedus, R. Paschke, R. Valcavi, P. Vitti, A.A.A.T.Fo.T. Nodules, American Association of Clinical Endocrinologists, American College of Endocrinology, and Associazione Medici Endocrinologi Medical Guidelines for Clinical Practice for the Diagnosis and Management of Thyroid Nodules–2016 Update. Endocr. Pract. 22(5), 622–639 (2016). PubMedCrossRefGoogle Scholar
  2. 2.
    G. Grani, L. Lamartina, V. Cantisani, M. Maranghi, P. Lucia, C. Durante, Interobserver agreement of various thyroid imaging reporting and data systems. Endocr Connect. 7(1), 1–7 (2018). PubMedCrossRefGoogle Scholar
  3. 3.
    R.L. Siegel, K.D. Miller, A. Jemal, Cancer statistics, 2018. CA: Cancer J. Clin. 68(1), 7–30 (2018). CrossRefGoogle Scholar
  4. 4.
    H. Lim, S.S. Devesa, J.A. Sosa, D. Check, C.M. Kitahara, Trends in thyroid cancer incidence and mortality in the United States, 1974–2013. JAMA 317(13), 1338–1348 (2017). PubMedCrossRefGoogle Scholar
  5. 5.
    M.E. Cabanillas, D.G. McFadden, C. Durante, Thyroid cancer. Lancet. 388(10061), 2783–2795 (2016). PubMedCrossRefGoogle Scholar
  6. 6.
    H.G. Welch, G.M. Doherty, Saving thyroids—overtreatment of small papillary cancers. N. Engl. J. Med. 379(4), 310–312 (2018). PubMedCrossRefGoogle Scholar
  7. 7.
    R.M. Tuttle, J.A. Fagin, G. Minkowitz, R.J. Wong, B. Roman, S. Patel, B. Untch, I. Ganly, A.R. Shaha, J.P. Shah, M. Pace, D. Li, A. Bach, O. Lin, A. Whiting, R. Ghossein, I. Landa, M. Sabra, L. Boucai, S. Fish, L.G.T. Morris, Natural history and tumor volume kinetics of papillary thyroid cancers during active surveillance. JAMA Otolaryngol. Head Neck Surg. 143(10), 1015–1020 (2017). PubMedPubMedCentralCrossRefGoogle Scholar
  8. 8.
    M.A. Al Dawish, A. Alwin Robert, M.A. Thabet, R. Braham, Thyroid nodule management: thyroid-stimulating hormone, ultrasound, and cytological classification system for predicting malignancy. Cancer Inform. 17, 1176935118765132 (2018). PubMedPubMedCentralCrossRefGoogle Scholar
  9. 9.
    B.R. Haugen, E.K. Alexander, K.C. Bible, G.M. Doherty, S.J. Mandel, Y.E. Nikiforov, F. Pacini, G.W. Randolph, A.M. Sawka, M. Schlumberger, K.G. Schuff, S.I. Sherman, J.A. Sosa, D.L. Steward, R.M. Tuttle, L. Wartofsky, 2015 American Thyroid Association Management Guidelines for adult patients with thyroid nodules and differentiated thyroid cancer the American Thyroid Association Guidelines Task Force on thyroid nodules and differentiated thyroid cancer. Thyroid.: Off. J. Am. Thyroid. Assoc. 26(1), 1–133 (2016). CrossRefGoogle Scholar
  10. 10.
    V. Chaudhary, S. Bano, Thyroid ultrasound. Indian J. Endocrinol. Metab. 17(2), 219–227 (2013). PubMedPubMedCentralCrossRefGoogle Scholar
  11. 11.
    F.N. Tessler, W.D. Middleton, E.G. Grant, J.K. Hoang, L.L. Berland, S.A. Teefey, J.J. Cronan, M.D. Beland, T.S. Desser, M.C. Frates, L.W. Hammers, U.M. Hamper, J.E. Langer, C.C. Reading, L.M. Scoutt, A.T. Stavros, ACR Thyroid Imaging, Reporting And Data System (TI-RADS): white paper of the ACR TI-RADS committee. J. Am. Coll. Radiol. 14(5), 587–595 (2017). PubMedCrossRefGoogle Scholar
  12. 12.
    J.H. Shin, J.H. Baek, J. Chung, E.J. Ha, J.H. Kim, Y.H. Lee, H.K. Lim, W.J. Moon, D.G. Na, J.S. Park, Y.J. Choi, S.Y. Hahn, S.J. Jeon, S.L. Jung, D.W. Kim, E.K. Kim, J.Y. Kwak, C.Y. Lee, H.J. Lee, J.H. Lee, J.H. Lee, K.H. Lee, S.W. Park, J.Y. Sung, Korean Society of Thyroid Radiology (KSThR) and Korean Society of Radiology, Ultrasonography diagnosis and imaging-based management of thyroid nodules: Revised Korean Society of Thyroid Radiology consensus statement and recommendations. Korean J. Radiol. 17(3), 370–395 (2016). PubMedPubMedCentralCrossRefGoogle Scholar
  13. 13.
    J.L. Wemeau, J.L. Sadoul, M. d’Herbomez, H. Monpeyssen, J. Tramalloni, E. Leteurtre, F. Borson-Chazot, P. Caron, B. Carnaille, J. Leger, C. Do, M. Klein, I. Raingeard, R. Desailloud, L. Leenhardt, Guidelines of the French society of endocrinology for the management of thyroid nodules. Ann. d.’Endocrinol. 72(4), 251–281 (2011). CrossRefGoogle Scholar
  14. 14.
    A.S. Griffin, J. Mitsky, U. Rawal, A.J. Bronner, F.N. Tessler, J.K. Hoang, Improved quality of thyroid ultrasound reports after implementation of the ACR thyroid imaging reporting and data system nodule lexicon and risk stratification system. J. Am. Coll. Radiol. 15(5), 743–748 (2018). PubMedCrossRefGoogle Scholar
  15. 15.
    J.K. Hoang, W.D. Middleton, A.E. Farjat, S.A. Teefey, N. Abinanti, F.J. Boschini, A.J. Bronner, N. Dahiya, B.S. Hertzberg, J.R. Newman, D. Scanga, R.C. Vogler, F.N. Tessler, Interobserver variability of sonographic features used in the American College of Radiology Thyroid Imaging Reporting and Data System. Am. J. Roentgenol. 211(1), 162–167 (2018). CrossRefGoogle Scholar
  16. 16.
    E.J. Ha, D.G. Na, J.H. Baek, J.Y. Sung, J.H. Kim, S.Y. Kang, US fine-needle aspiration biopsy for thyroid malignancy: diagnostic performance of seven society guidelines applied to 2000 thyroid nodules. Radiology 171074 (2018). PubMedCrossRefGoogle Scholar
  17. 17.
    J.K. Hoang, W.D. Middleton, A.E. Farjat, J.E. Langer, C.C. Reading, S.A. Teefey, N. Abinanti, F.J. Boschini, A.J. Bronner, N. Dahiya, B.S. Hertzberg, J.R. Newman, D. Scanga, R.C. Vogler, F.N. Tessler, Reduction in thyroid nodule biopsies and improved accuracy with American College of Radiology Thyroid Imaging Reporting and Data System. Radiology 287(1), 185–193 (2018). PubMedCrossRefGoogle Scholar
  18. 18.
    X.L. Wu, J.R. Du, H. Wang, C.X. Jin, G.Q. Sui, D.Y. Yang, Y.Q. Lin, Q. Luo, P. Fu, H.Q. Li, D.K. Teng, Comparison and preliminary discussion of the reasons for the differences in diagnostic performance and unnecessary FNA biopsies between the ACR TIRADS and 2015 ATA guidelines. Endocrine (2019). PubMedCrossRefGoogle Scholar
  19. 19.
    E.G. Grant, F.N. Tessler, J.K. Hoang, J.E. Langer, M.D. Beland, L.L. Berland, J.J. Cronan, T.S. Desser, M.C. Frates, U.M. Hamper, W.D. Middleton, C.C. Reading, L.M. Scoutt, A.T. Stavros, S.A. Teefey, Thyroid ultrasound reporting lexicon: white paper of the ACR Thyroid Imaging, Reporting and Data System (TIRADS) Committee. J. Am. Coll. Radiol. 12(12 Pt A), 1272–1279 (2015). PubMedCrossRefGoogle Scholar
  20. 20.
    J.R. Landis, G.G. Koch, The measurement of observer agreement for categorical data. Biometrics 33(1), 159–174 (1977)CrossRefGoogle Scholar
  21. 21.
    A. Lauria Pantano, E. Maddaloni, S.I. Briganti, G. Beretta Anguissola, E. Perrella, C. Taffon, A. Palermo, P. Pozzilli, S. Manfrini, A. Crescenzi, Differences between ATA, AACE/ACE/AME and ACR TI-RADS ultrasound classifications performance in identifying cytological high-risk thyroid nodules. Eur. J. Endocrinol. 178(6), 595–603 (2018). PubMedCrossRefGoogle Scholar
  22. 22.
    F.N. Tessler, W.D. Middleton, E.G. Grant, Thyroid imaging reporting and data system (TI-RADS): A user’s guide. Radiology 287(1), 29–36 (2018). PubMedCrossRefGoogle Scholar
  23. 23.
    F.D. Koseoglu Atilla, B. Ozgen Saydam, N.A. Erarslan, A.G. Diniz Unlu, H. Yilmaz Yasar, M. Ozer, B. Akinci, Does the ACR TI-RADS scoring allow us to safely avoid unnecessary thyroid biopsy? Single center analysis in a large cohort. Endocrine (2018). PubMedCrossRefGoogle Scholar
  24. 24.
    V.L. Weiss, R.F. Andreotti, K.A. Ely, Use of the thyroid imaging, reporting, and data system (TI-RADS) scoring system for the evaluation of subcentimeter thyroid nodules. Cancer Cytopathol. 126(8), 518–524 (2018). PubMedCrossRefGoogle Scholar
  25. 25.
    Z.T. Sahli, A.K. Sharma, J.K. Canner, F. Karipineni, O. Ali, S. Kawamoto, J.F. Hang, A. Mathur, S.Z. Ali, M.A. Zeiger, S. Sheth, TIRADS interobserver variability among indeterminate thyroid nodules: a single-institution study. J. Ultrasound Med. (2018). PubMedCrossRefGoogle Scholar
  26. 26.
    M. Itani, R. Assaker, M. Moshiri, T.J. Dubinsky, M.K. Dighe, Inter-observer variability in the American College of Radiology Thyroid Imaging Reporting and Data System: in-depth analysis and areas for improvement. Ultrasound Med. Biol. 45(2), 461–470 (2019). PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2020

Authors and Affiliations

  1. 1.Department of UltrasoundChina–Japan Union Hospital of Jilin UniversityChangchunChina
  2. 2.Department of UltrasoundThe First Affiliated Hospital of Nanchang UniversityNanchangChina
  3. 3.Department of Breast SurgeryChina–Japan Union Hospital of Jilin UniversityChangchunChina

Personalised recommendations