Skip to main content
SpringerLink
Log in

Predictive accuracy and reproducibility of the O-RADS US scoring system among sonologists with different training levels

  • Images in Obstetrics and Gynecology
  • Published:
Archives of Gynecology and Obstetrics Aims and scope Submit manuscript

Abstract

Purpose

To investigate the predictive performance and reproducibility of Ovarian-Adnexal Reporting and Data System (O-RADS) ultrasound (US) system in evaluating adnexal masses between sonologists with varying levels of expertise.

Methods

This was a single-center retrospective study conducted between May 2019 and May 2020, which included 147 adnexal mases with pathological results. Four sonologists with varying experiences independently assigned an O-RADS US category to each adnexal mass twice. The intra- and inter-observer agreement was assessed using weighted kappa values. The area under the curve (AUC), sensitivity, specificity, positive and negative predictive value (PPV and NPV) were assessed for each sonologist.

Results

Of the 147 adnexal mases, 115 (78.2%) lesions were benign and 32 (21.8%) lesions were malignant. Considering O-RADS > 3 as a predictor for adnexal malignancy, the predictive accuracies of the four sonologists were excellent, with AUCs ranging from 0.831 to 0.926. The predictive accuracies of O-RADS US by experienced sonologists were significantly higher compared to inexperienced sonologists (all P values < 0.005). The O-RADS US presented high sensitivity and NPV value for each sonologist. With regard to the reproducibility of O-RADS, the intra- and inter-observer agreement among experienced sonologists performed better than inexperienced sonologists.

Conclusion

O-RADS showed difference in the predictive accuracy and reproducibility in the evaluation of adnexal masses among sonologists with different levels of expertise. Training is required for inexperienced sonologists before the generalization of O-RADS classification system in clinical practice.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Abbreviations

US:

Ultrasound

ACR:

American college of radiology

O-RADS:

Ovarian adnexal reporting and data system

IOTA:

International ovarian tumor analysis

CI:

Confidence interval

GI-RADS:

Gynecologic imaging reporting and data system

References

  1. Glanc P, Benacerraf B, Bourne T et al (2017) First international consensus report on adnexal masses: management recommendations. J Ultrasound Med 36:849–863

    Article  PubMed  Google Scholar 

  2. Fung-Kee-Fung M, Kennedy EB, Biagi J et al (2015) The optimal organization of gynecologic oncology services: a systematic review. Curr Oncol 22:e282-293

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Togashi K (2003) Ovarian cancer: the clinical role of US, CT, and MRI. Eur Radiol 13(Suppl 4):L87-104

    Article  PubMed  Google Scholar 

  4. Brown DL, Dudiak KM, Laing FC (2010) Adnexal masses: US characterization and reporting. Radiology 254:342–354

    Article  PubMed  Google Scholar 

  5. Levine D, Brown DL, Andreotti RF et al (2010) Management of asymptomatic ovarian and other adnexal cysts imaged at US: society of radiologists in ultrasound consensus conference statement. Radiology 256:943–954

    Article  PubMed  Google Scholar 

  6. Meys EM, Kaijser J, Kruitwagen RF et al (2016) Subjective assessment versus ultrasound models to diagnose ovarian cancer: a systematic review and meta-analysis. Eur J Cancer 58:17–29

    Article  CAS  PubMed  Google Scholar 

  7. Stein EB, Roseland ME, Shampain KL, Wasnik AP, Maturen KE (2021) Contemporary guidelines for adnexal mass imaging: a 2020 update. Abdom Radiol (NY) 46:2127–2139

    Article  PubMed  Google Scholar 

  8. Amor F, Alcázar JL, Vaccaro H, León M, Iturra A (2011) GI-RADS reporting system for ultrasound evaluation of adnexal masses in clinical practice: a prospective multicenter study. Ultrasound Obstet Gynecol 38:450–455

    Article  CAS  PubMed  Google Scholar 

  9. Andreotti RF, Timmerman D, Strachowski LM et al (2020) O-RADS US risk stratification and management system: a consensus guideline from the ACR ovarian-adnexal reporting and data system committee. Radiology 294:168–185

    Article  PubMed  Google Scholar 

  10. Timmerman D, Valentin L, Bourne TH, Collins WP, Verrelst H, Vergote I (2000) Terms, definitions and measurements to describe the sonographic features of adnexal tumors: a consensus opinion from the international ovarian tumor analysis (IOTA) group. Ultrasound Obstet Gynecol 16:500–505

    Article  CAS  PubMed  Google Scholar 

  11. Elder JW, Pavlik EJ, Long A et al (2014) Serial ultrasonographic evaluation of ovarian abnormalities with a morphology index. Gynecol Oncol 135:8–12

    Article  PubMed  Google Scholar 

  12. Basha MA, Refaat R, Ibrahim SA et al (2019) Gynecology imaging reporting and data system (GI-RADS): diagnostic performance and inter-reviewer agreement. Eur Radiol 29:5981–5990

    Article  PubMed  Google Scholar 

  13. Andreotti RF, Timmerman D, Benacerraf BR et al (2018) Ovarian-adnexal reporting lexicon for ultrasound: a white paper of the ACR ovarian-adnexal reporting and data system committee. J Am Coll Radiol 15:1415–1429

    Article  PubMed  Google Scholar 

  14. Kaijser J, Sayasneh A, Van Hoorde K et al (2014) Presurgical diagnosis of adnexal tumours using mathematical models and scoring systems: a systematic review and meta-analysis. Hum Reprod Update 20:449–462

    Article  PubMed  Google Scholar 

  15. Knafel A, Banas T, Nocun A et al (2016) The prospective external validation of international ovarian tumor analysis (IOTA) simple rules in the hands of level I and II examiners. Ultraschall Med 37:516–523

    CAS  PubMed  Google Scholar 

  16. Cao L, Wei M, Liu Y et al (2021) Validation of American college of radiology ovarian-adnexal reporting and data system ultrasound (O-RADS US): analysis on 1054 adnexal masses. Gynecol Oncol 162:107–112

    Article  PubMed  Google Scholar 

  17. Hiett AK, Sonek JD, Guy M, Reid TJ (2021) Performance of IOTA simple rules, simple rules risk assessment, ADNEX model and O-RADS in differentiating between benign and malignant adnexal lesions in North American women. Ultrasound Obstet Gynecol. https://doi.org/10.1002/uog.24777

    Article  Google Scholar 

  18. Lai HW, Lyu GR, Kang Z, Li LY, Zhang Y, Huang YJ (2021) Comparison of O-RADS, GI-RADS, and ADNEX for diagnosis of adnexal masses: an external validation study conducted by junior sonologists. J Ultrasound Med. https://doi.org/10.1002/jum.15834

    Article  PubMed  Google Scholar 

  19. Guo Y, Zhao B, Zhou S et al (2022) A comparison of the diagnostic performance of the O-RADS, RMI4, IOTA LR2, and IOTA SR systems by senior and junior doctors. Ultrasonography. https://doi.org/10.14366/usg.21237

    Article  PubMed  PubMed Central  Google Scholar 

  20. Basha MA, Metwally MI, Gamil SA et al (2021) Comparison of O-RADS, GI-RADS, and IOTA simple rules regarding malignancy rate, validity, and reliability for diagnosis of adnexal masses. Eur Radiol 31:674–684

    Article  PubMed  Google Scholar 

  21. Pi Y, Wilson MP, Katlariwala P et al (2021) Diagnostic accuracy and inter-observer reliability of the O-RADS scoring system among staff radiologists in a North American academic clinical setting. Abdom Radiol (NY) 46:4967–4973

    Article  PubMed  Google Scholar 

  22. Bland JM, Altman DG (1986) Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1:307–310

    Article  CAS  PubMed  Google Scholar 

  23. Zannoni L, Savelli L, Jokubkiene L et al (2013) Intra- and interobserver reproducibility of assessment of Doppler ultrasound findings in adnexal masses. Ultrasound Obstet Gynecol 42:93–101

    Article  CAS  PubMed  Google Scholar 

Download references

Funding

This project is supported by the Natural Science Foundation of Guangdong Province, China (No. 2022A1515012027).

Author information

Author notes

  1. Manli Wu and Man Zhang have contributed equally to this work and should be considered co-first authors.

Authors and Affiliations

  1. Department of Ultrasound, Third Affiliated Hospital of Sun Yat-Sen University, 600 Tianhe Road, Guangzhou, 510630, Guangdong Province, People’s Republic of China

    Manli Wu, Man Zhang, Junyan Cao, Shuangyu Wu, Ying Chen, Liping Luo, Xin Lin, Manting Su & Xinling Zhang

Authors
  1. Manli Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Man Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junyan Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shuangyu Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ying Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Liping Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Xin Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Manting Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Xinling Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

MW: project development, data collection, manuscript writing. MZ: data analysis, manuscript writing. JC: project development, data analysis. SW: data analysis. YC: data analysis. LL: data analysis. XL: data analysis. MS: data collection. XZ: project development, data management, manuscript editing.

Corresponding author

Correspondence to Xinling Zhang.

Ethics declarations

Conflict of interest

All authors have no conflicts of interest pertaining to this manuscript. In addition, all the authors declare that they have contributed to this manuscript. The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Ethical approval

Ethical approval was obtained from the Institutional Ethical Committee of xxx. Hospital. The requirement for informed consent was waived off due to the retrospective design of this study.

Consent for publication

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, M., Zhang, M., Cao, J. et al. Predictive accuracy and reproducibility of the O-RADS US scoring system among sonologists with different training levels. Arch Gynecol Obstet 308, 631–637 (2023). https://doi.org/10.1007/s00404-022-06752-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00404-022-06752-5

Keywords

Search

Navigation