Abstract
Purpose
Reliability of pre-operative testing is important for adequate surgical planning. For urethral stricture disease, preoperative planning frequently includes retrograde urethrogram (RUG). The radiographic interpretation of RUGs is often done by urologists themselves. We aimed to evaluate the reliability of RUG interpretation by urologists at our institution.
Methods
We examined the RUGs of 193 patients. These were deidentified and interpreted by three urologists, two general urologists and one reconstructive urologist. These interpretations were compared in 2 ways. Each of the general urologists was compared to the “gold standard” reconstructive urologist interpretation, and the general urologists were additionally compared to each other. We used intraclass correlation coefficient (ICC) for numerical variables and Fleiss’ Kappa or Cohen’s Kappa statistic (κ) for categorical variables to rate inter-interpreter reliability and agreement among interpretations with regards to the quantitative variables of stricture length and caliber.
Results
Level of agreement ranged from poor to moderate across all variables interpreted. Comparing general urologists to the gold standard yielded no better than moderate agreement, with the majority being poor to fair. Similarly, agreement amongst the general urologists did not reach above moderate, with the majority being poor to slight.
Conclusion
To our knowledge, this is the first analysis of inter-rater reliability of RUGs among practicing urologists. Our analysis showed clinically unacceptable reliability with regards to stricture length, location, caliber, and indicated procedures. This study suggests a need for standardized interpretation of RUGs and poses an opportunity for actionable improvement in management of strictures.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The datasets generated and analysed during the current study are not publicly available due to institutional regulations, but are available from the corresponding author on reasonable request.
References
Wessells H, Angermeier KW, Elliott SP (2016) Male urethral stricture. American Urological Association (AUA) Guideline. pp. 1–34.
Bullock TL, Brandes SB (2007) Adult anterior urethral strictures: a national practice patterns survey of board certified urologists in the United States. J Urol 177(2):685–690. https://doi.org/10.1016/j.juro.2006.09.052
Eswara JR, Song JB, Chang AJ, Rosenstein DI, Gonzalez CM, Vetter JM, Brandes SB (2014) Urethrography interpretation skills of urology and radiology residents at tertiary care medical centers. Urology 83(6):1239–1242. https://doi.org/10.1016/j.urology.2014.02.027
Kerkhof M, Hagenbeek RE, van der Kallen BFW, Lycklama à Nijeholt GJ, Dirven L, Taphoorn MJB, Vos MJ (2016) Interobserver variability in the radiological assessment of magnetic resonance imaging (MRI) including perfusion MRI in glioblastoma multiforme. Eur J Neurol 23(10):1528–1533. https://doi.org/10.1111/ene.13070
Kim JK, McCammon K, Robey C, Castillo M, Gomez O, Pua PJL, Chua ME (2022) Identifying urethral strictures using machine learning: a proof-of-concept evaluation of convolutional neural network model. World J Urol 40(12):3107–3111. https://doi.org/10.1007/s00345-022-04199-6
Koo TK, Li MY (2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 15(2):155–163. https://doi.org/10.1016/j.jcm.2016.02.012
American Urological Association (2016) The State of Urology Workforce and Practice in the United States. AUA Census Book.
Lydon S, Fitzgerald N, Gannon L, Choynowski M, O’Connor P, Devitt A, Byrne D (2021) A randomised controlled trial of SAFMEDS to improve musculoskeletal radiology interpretation. The Surgeon. https://doi.org/10.1016/j.surge.2020.11.004
Richard LJ, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33(1):159–174. https://doi.org/10.2307/2529310
Schmid AM, Raunig DL, Miller CG, Walovitch RC, Ford RW, O’Connor M, Ford RR (2021) Radiologists and clinical trials: part 1 the truth about reader disagreements. Ther Innov Regul Sci 55(6):1111–1121. https://doi.org/10.1007/s43441-021-00316-6
Tajmir SH, Lee H, Shailam R, Gale HI, Nguyen JC, Westra SJ, Do S (2018) Artificial intelligence-assisted interpretation of bone age radiographs improves accuracy and decreases variability. Skeletal Radiol 48(2):275–283. https://doi.org/10.1007/s00256-018-3033-2
Williams I, Baird M, Pearce B, Schneider M (2018) Improvement of radiographer commenting accuracy of the appendicular skeleton following a short course in plain radiography image interpretation: a pilot study. J Med Radiat Sci. https://doi.org/10.1002/jmrs.306
Funding
The authors did not receive support from any organization for the submitted work. No funding was received to assist with the preparation of this manuscript. No funding was received for conducting this study. No funds, grants, or other support was received.
Author information
Authors and Affiliations
Contributions
CR and KMcC: contributed to protocol/project development, data collection, manuscript writing. MP, TZ, VB and JL: were involved in data collection.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no relevant financial or non-financial interests to disclose. The authors have no competing interests to declare that are relevant to the content of this article. All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript. The authors have no financial or proprietary interests in any material discussed in this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Robey, C., McCammon, K., Perry, M. et al. Inter-rater reliability of retrograde urethrograms. World J Urol 41, 1163–1167 (2023). https://doi.org/10.1007/s00345-023-04323-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00345-023-04323-0