Skip to main content

Advertisement

SpringerLink
Log in

Ultrasound Liver Imaging Reporting and Data System (US LI-RADS) Visualization Score: a reliability analysis on inter-reader agreement

  • Hepatobiliary
  • Published:
Abdominal Radiology Aims and scope Submit manuscript

Abstract

Background & Aim

The American College of Radiology Ultrasound Liver Imaging Reporting and Data System (ACR US LI-RADS) Visualization Score conveys the expected level of sensitivity of screening and surveillance ultrasound exams in patients at risk for hepatocellular carcinoma (HCC). We sought to determine inter-reader agreement of the Visualization Score which is currently unknown.

Methods

Consecutive 6998 ultrasound HCC screening and surveillance studies in 3115 patients from 2017 to 2020 were retrospectively retrieved. Of these, 6154 (87.9%) studies were Visualization A (No or minimal limitations), 709 (10.1%) were Visualization B (Moderate limitations), and 135 (1.9%) were Visualization C (Severe limitations). Randomly sampled 90 studies, with 30 studies in each Visualization category, were included for analysis. Nine radiologists (3 senior attendings, 3 junior attendings and 3 body imaging fellows) blinded to the original categorization independently reviewed each study and assigned a Visualization Score. Intraclass correlation coefficient (ICC) was used to quantify inter-reader agreement.

Results

ICC among all 9 radiologists was 0.70 (95% CI 0.63–0.77). ICCs among senior attendings, junior attendings and body imaging fellows were 0.68 (CI 0.58–0.76), 0.72 (CI 0.62–0.80) and 0.76 (CI 0.68–0.83), respectively. Subgroup analysis by liver parenchyma was further performed. ICC was highest in the patient group with normal liver parenchyma (0.69, CI 0.56–0.81), followed by steatosis (0.66, CI 0.54–0.79) and cirrhosis (0.58, CI 0.43–0.73), respectively.

Conclusions

US LI-RADS Visualization Score is a reliable tool with good inter-reader agreement that can be used to indicate the expected level of sensitivity of a screening and surveillance ultrasound examination for detecting focal liver observations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Heimbach JK, Kulik LM, Finn RS, et al. AASLD guidelines for the treatment of hepatocellular carcinoma. Hepatology. 2018;67(1):358-380.

  2. Marrero JA, Kulik LM, Sirlin CB, et al. Diagnosis, Staging, and Management of Hepatocellular Carcinoma: 2018 Practice Guidance by the American Association for the Study of Liver Diseases. Hepatology. 2018;68(2):723-750.

  3. Expert Panel on Gastrointestinal I, Bashir MR, Horowitz JM, et al. ACR Appropriateness Criteria(R) Chronic Liver Disease. J Am Coll Radiol. 2020;17(5S):S70-S80.

  4. American College of Radiology. Ultrasound LI-RADS v2017. https://www.acr.org/Clinical-Resources/Reporting-and-Data-Systems/LI-RADS/Ultrasound-LI-RADS-v2017. Accessed December 15, 2020.

  5. Rodgers SK, Fetzer DT, Gabriel H, et al. Role of US LI-RADS in the LI-RADS Algorithm. Radiographics. 2019;39(3):690-708.

  6. Morgan TA, Maturen KE, Dahiya N, et al. US LI-RADS: ultrasound liver imaging reporting and data system for screening and surveillance of hepatocellular carcinoma. Abdom Radiol (NY). 2018;43(1):41-55.

  7. Kamaya A. Ultrasound Screening of Hepatocellular Carcinoma. https://www.youtube.com/watch?v=3el5A61XZfI&t=615s. Published 2020. Accessed December 15, 2020.

  8. Choi HH, Rodgers SK, Fetzer DT, et al. Ultrasound Liver Imaging Reporting and Data System (US LI-RADS): An Overview with Technical and Practical Applications. Acad Radiol. 2020.

  9. Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med. 2016;15(2):155-163.

  10. Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological assessment. 1994;6(4):284.

  11. Zou G. Sample size formulas for estimating intraclass correlation coefficients with precision and assurance. Statistics in medicine. 2012;31(29):3972-3981.

  12. Choi HH, Perez MG, Millet JD, et al. Association of advanced hepatic fibrosis and sonographic visualization score: a dual-center study using ACR US LI-RADS. Abdominal Radiology. 2019;44(4):1415-1422.

  13. Irshad A, Leddy R, Ackerman S, et al. Effects of Changes in BI-RADS Density Assessment Guidelines (Fourth Versus Fifth Edition) on Breast Density Assessment: Intra- and Interreader Agreements and Density Distribution. AJR Am J Roentgenol. 2016;207(6):1366–1371.

  14. Kamaya A, Rodgers SK. Hepatocellular Carcinoma Screening at Transplant Centers: Point-Ultrasound Should Remain First Line. AJR Am J Roentgenol. 2021;216(3):579-580.

  15. Millet JD, Kamaya A, Choi HH, et al. ACR Ultrasound Liver Reporting and Data System: Multicenter Assessment of Clinical Performance at One Year. J Am Coll Radiol. 2019;16(12):1656-1662.

  16. Pacheco EO, Torres US, Alves AMA, Bekhor D, D'Ippolito G. Bosniak classification of cystic renal masses version 2019 does not increase the interobserver agreement or the proportion of masses categorized into lower Bosniak classes for non-subspecialized readers on CT or MR. Eur J Radiol. 2020;131:109270.

  17. Davenport MS, Khalatbari S, Liu PS, et al. Repeatability of diagnostic features and scoring systems for hepatocellular carcinoma by using MR imaging. Radiology. 2014;272(1):132-142.

Download references

Funding

Thodsawit Tiyarattanachai is a Prince Mahidol Award Youth Program Scholar, Prince Mahidol Award Foundation under the Royal Patronage.

Author information

Authors and Affiliations

  1. Faculty of Medicine, Chulalongkorn University, 1873 Rama 4 Rd., Pathum Wan, Bangkok, 10330, Thailand

    Thodsawit Tiyarattanachai

  2. Body and Chest Divisions, Department of Radiology, Stanford University School of Medicine, 300 Pasteur Drive, Stanford, CA, 94305, USA

    Kristen N. Bird

  3. Department of Radiology, Stanford University School of Medicine, 300 Pasteur Drive, Room S048A, Stanford, CA, 94305, USA

    Edward C. Lo

  4. Department of Radiology, Stanford University School of Medicine, 300 Pasteur Drive, Room H-1307, Stanford, CA, 94305, USA

    Aladin T. Mariano, Amanzo A. Ho, Craig W. Ferguson, Ranjit S. Chima, Terry S. Desser & Aya Kamaya

  5. Department of Radiology, Stanford University School of Medicine, 300 Pasteur Drive, Stanford, CA, 94305, USA

    Thodsawit Tiyarattanachai & Linda Nayeli Morimoto

Authors
  1. Thodsawit Tiyarattanachai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kristen N. Bird
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Edward C. Lo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aladin T. Mariano
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Amanzo A. Ho
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Craig W. Ferguson
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ranjit S. Chima
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Terry S. Desser
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Linda Nayeli Morimoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Aya Kamaya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aya Kamaya.

Ethics declarations

Conflict of interest

Aya Kamaya—Disclosures: Book royalties from Elsevier. The other authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tiyarattanachai, T., Bird, K.N., Lo, E.C. et al. Ultrasound Liver Imaging Reporting and Data System (US LI-RADS) Visualization Score: a reliability analysis on inter-reader agreement. Abdom Radiol 46, 5134–5141 (2021). https://doi.org/10.1007/s00261-021-03067-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00261-021-03067-y

Keywords

Search

Navigation