Current and future applications of machine and deep learning in urology: a review of the literature on urolithiasis, renal cell carcinoma, and bladder and prostate cancer

  • Rodrigo Suarez-IbarrolaEmail author
  • Simon Hein
  • Gerd Reis
  • Christian Gratzke
  • Arkadiusz Miernik
Topic Paper



The purpose of the study was to provide a comprehensive review of recent machine learning (ML) and deep learning (DL) applications in urological practice. Numerous studies have reported their use in the medical care of various urological disorders; however, no critical analysis has been made to date.


A detailed search of original articles was performed using the PubMed MEDLINE database to identify recent English literature relevant to ML and DL applications in the fields of urolithiasis, renal cell carcinoma (RCC), bladder cancer (BCa), and prostate cancer (PCa).


In total, 43 articles were included addressing these four subfields. The most common ML and DL application in urolithiasis is in the prediction of endourologic surgical outcomes. The main area of research involving ML and DL in RCC concerns the differentiation between benign and malignant small renal masses, Fuhrman nuclear grade prediction, and gene expression-based molecular signatures. BCa studies employ radiomics and texture feature analysis for the distinction between low- and high-grade tumors, address accurate image-based cytology, and use algorithms to predict treatment response, tumor recurrence, and patient survival. PCa studies aim at developing algorithms for Gleason score prediction, MRI computer-aided diagnosis, and surgical outcomes and biochemical recurrence prediction. Studies consistently found the superiority of these methods over traditional statistical methods.


The continuous incorporation of clinical data, further ML and DL algorithm retraining, and generalizability of models will augment the prediction accuracy and enhance individualized medicine.


Artificial intelligence Machine learning Deep learning Artificial neural network Convolutional neural network Prostate cancer Bladder cancer Renal cell carcinoma Urolithiasis 


Author contributions

Project development: RS and AM. Literature review and data extraction: RS. Manuscript drafting: RS, GR, and AM. Manuscript editing: SH, CG, and AM.


This research received no financial or other support.

Compliance with ethical standards

Conflict of interest

The authors declare no conflicts of interest.

Human and animal rights statement

This research did not involve human subjects or animals.

Ethical approval

As this is a review of the literature, no ethical approval was necessary.


  1. 1.
    Nuffield Council on Bioethics (2018) Bioethics briefing notes: artificial intelligence (AI) in healthcare and research. Accessed 21 Dec 2018
  2. 2.
    Frankish K, Ramsey WM (eds) (2014) Introduction. The Cambridge handbook of artificial intelligence. Cambridge University Press, Cambridge, pp 1–14Google Scholar
  3. 3.
    Stuart R, Norvig P (eds) (2010) Artificial intelligence—a modern approach, 3rd edn. Prentice Hall, Upper Saddle RiverGoogle Scholar
  4. 4.
    Tran BX et al (2019) Global evolution of research in artificial intelligence in health and medicine: a bibliometric study. J Clin Med 8(3):360PubMedCentralCrossRefPubMedGoogle Scholar
  5. 5.
    Goldenberg SL, Nir G, Salcudean SE (2019) A new era: artificial intelligence and machine learning in prostate cancer. Nat Rev Urol 16(7):391–403PubMedCrossRefPubMedCentralGoogle Scholar
  6. 6.
    Yu KH, Beam AL, Kohane IS (2018) Artificial intelligence in healthcare. Nat Biomed Eng 2(10):719–731PubMedCrossRefGoogle Scholar
  7. 7.
    Curran Associates Inc. (2014) Advances in neural information processing systems 26: 27th annual conference on neural information processing systems 2014, December 8–13. Curran Associates Inc., vol 1Google Scholar
  8. 8.
    Abbod MF et al (2007) Application of artificial intelligence to the management of urological cancer. J Urol 178(4 Pt 1):1150–1156PubMedCrossRefGoogle Scholar
  9. 9.
    Kadlec AO et al (2014) Nonlinear logistic regression model for outcomes after endourologic procedures: a novel predictor. Urolithiasis 42(4):323–327PubMedCrossRefGoogle Scholar
  10. 10.
    Aminsharifi A et al (2017) Artificial neural network system to predict the postoperative outcome of percutaneous nephrolithotomy. J Endourol 31(5):461–467PubMedCrossRefPubMedCentralGoogle Scholar
  11. 11.
    Choo MS et al (2018) A prediction model using machine learning algorithm for assessing stone-free status after single session shock wave lithotripsy to treat ureteral stones. J Urol 200(6):1371–1377PubMedCrossRefPubMedCentralGoogle Scholar
  12. 12.
    Mannil M et al (2018) Prediction of successful shock wave lithotripsy with CT: a phantom study using texture analysis. Abdom Radiol (NY) 43(6):1432–1438PubMedCrossRefPubMedCentralGoogle Scholar
  13. 13.
    Mannil M et al (2018) Three-dimensional texture analysis with machine learning provides incremental predictive information for successful shock wave lithotripsy in patients with kidney stones. J Urol 200(4):829–836PubMedCrossRefPubMedCentralGoogle Scholar
  14. 14.
    Seckiner I et al (2017) A neural network-based algorithm for predicting stone-free status after ESWL therapy. Int Braz J Urol 43(6):1110–1114PubMedPubMedCentralCrossRefGoogle Scholar
  15. 15.
    Langkvist M et al (2018) Computer aided detection of ureteral stones in thin slice computed tomography volumes using Convolutional Neural Networks. Comput Biol Med 97:153–160PubMedCrossRefPubMedCentralGoogle Scholar
  16. 16.
    Kazemi Y, Mirroshandel SA (2018) A novel method for predicting kidney stone type using ensemble learning. Artif Intell Med 84:117–126PubMedCrossRefPubMedCentralGoogle Scholar
  17. 17.
    Richard PO et al (2015) Renal tumor biopsy for small renal masses: a single-center 13-year experience. Eur Urol 68(6):1007–1013PubMedCrossRefGoogle Scholar
  18. 18.
    Mir MC et al (2018) Role of active surveillance for localized small renal masses. Eur Urol Oncol 1(3):177–187PubMedCrossRefGoogle Scholar
  19. 19.
    Bektas CT et al (2019) Clear cell renal cell carcinoma: machine learning-based quantitative computed tomography texture analysis for prediction of Fuhrman nuclear grade. Eur Radiol 29(3):1153–1163PubMedCrossRefGoogle Scholar
  20. 20.
    Kocak B et al (2018) Textural differences between renal cell carcinoma subtypes: machine learning-based quantitative computed tomography texture analysis with independent external validation. Eur J Radiol 107:149–157PubMedCrossRefPubMedCentralGoogle Scholar
  21. 21.
    Kanapuli G et al (2018) A decision-support tool for renal mass classification. J Digit Imaging 31(6):929–939CrossRefGoogle Scholar
  22. 22.
    Yu H et al (2017) Texture analysis as a radiomic marker for differentiating renal tumors. Abdom Radiol (NY) 42(10):2470–2478CrossRefGoogle Scholar
  23. 23.
    Yan L et al (2015) Angiomyolipoma with minimal fat: differentiation from clear cell renal cell carcinoma and papillary renal cell carcinoma by texture analysis on CT images. Acad Radiol 22(9):1115–1121PubMedCrossRefPubMedCentralGoogle Scholar
  24. 24.
    Feng Z et al (2018) Machine learning-based quantitative texture analysis of CT images of small renal masses: differentiation of angiomyolipoma without visible fat from renal cell carcinoma. Eur Radiol 28(4):1625–1633PubMedCrossRefPubMedCentralGoogle Scholar
  25. 25.
    Cui EM et al (2019) Differentiation of renal angiomyolipoma without visible fat from renal cell carcinoma by machine learning based on whole-tumor computed tomography texture features. Acta Radiol 60(11):1543–1552PubMedCrossRefPubMedCentralGoogle Scholar
  26. 26.
    Coy H et al (2019) Deep learning and radiomics: the utility of Google TensorFlow Inception in classifying clear cell renal cell carcinoma and oncocytoma on multiphasic CT. Abdom Radiol 44(6):2009–2020CrossRefGoogle Scholar
  27. 27.
    Minardi D et al (2005) Prognostic role of Fuhrman grade and vascular endothelial growth factor in pT1a clear cell carcinoma in partial nephrectomy specimens. J Urol 174(4 Pt 1):1208–1212PubMedCrossRefPubMedCentralGoogle Scholar
  28. 28.
    Holdbrook DA et al (2018) Automated renal cancer grading using nuclear pleomorphic patterns. JCO Clin Cancer Inform 2:1–12PubMedCrossRefPubMedCentralGoogle Scholar
  29. 29.
    Ding J et al (2018) CT-based radiomic model predicts high grade of clear cell renal cell carcinoma. Eur J Radiol 103:51–56PubMedPubMedCentralCrossRefGoogle Scholar
  30. 30.
    Kocak B et al (2019) Unenhanced CT texture analysis of clear cell renal cell carcinomas: a machine learning-based study for predicting histopathologic nuclear grade. AJR Am J Roentgenol 212:W1–W8CrossRefGoogle Scholar
  31. 31.
    Lin F et al (2019) CT-based machine learning model to predict the Fuhrman nuclear grade of clear cell renal cell carcinoma. Abdom Radiol 44(7):2528–2534CrossRefGoogle Scholar
  32. 32.
    Sun X et al (2019) Prediction of ISUP grading of clear cell renal cell carcinoma using support vector machine model based on CT images. Medicine (Baltimore) 98(14):e15022CrossRefGoogle Scholar
  33. 33.
    Li P et al (2018) Fifteen-gene expression based model predicts the survival of clear cell renal cell carcinoma. Medicine (Baltimore) 97(33):e11839CrossRefGoogle Scholar
  34. 34.
    Kocak B et al (2019) Radiogenomics in clear cell renal cell carcinoma: machine learning-based high-dimensional quantitative CT texture analysis in predicting PBRM1 mutation status. AJR Am J Roentgenol 212(3):W55–W63PubMedCrossRefGoogle Scholar
  35. 35.
    Xu X et al (2017) Three-dimensional texture features from intensity and high-order derivative maps for the discrimination between bladder tumors and wall tissues via MRI. Int J CARS 12(4):645–656CrossRefGoogle Scholar
  36. 36.
    Zhang X et al (2017) Radiomics assessment of bladder cancer grade using texture features from diffusion-weighted imaging. J Magn Reson Imaging 46(5):1281–1288PubMedPubMedCentralCrossRefGoogle Scholar
  37. 37.
    Eminaga O et al (2018) Diagnostic classification of cystoscopic images using deep convolutional neural networks. JCO Clin Cancer Inform 2:1–8PubMedCrossRefGoogle Scholar
  38. 38.
    Sokolov I et al (2018) Noninvasive diagnostic imaging using machine-learning analysis of nanoresolution images of cell surfaces: detection of bladder cancer. Proc Natl Acad Sci USA 115(51):12920–12925PubMedCrossRefGoogle Scholar
  39. 39.
    Brieu N et al (2019) Automated tumour budding quantification by machine learning augments TNM staging in muscle-invasive bladder cancer prognosis. Sci Rep 9(1):5174PubMedPubMedCentralCrossRefGoogle Scholar
  40. 40.
    Hasnain Z et al (2019) Machine learning models for predicting post-cystectomy recurrence and survival in bladder cancer patients. PLoS ONE 14(2):e0210976PubMedPubMedCentralCrossRefGoogle Scholar
  41. 41.
    Bartsch G Jr et al (2016) Use of artificial intelligence and machine learning algorithms with gene expression profiling to predict recurrent nonmuscle invasive urothelial carcinoma of the bladder. J Urol 195(2):493–498PubMedCrossRefGoogle Scholar
  42. 42.
    Wu E et al (2019) Deep learning approach for assessment of bladder cancer treatment response. Tomography 5(1):201–208PubMedPubMedCentralCrossRefGoogle Scholar
  43. 43.
    Cha KH et al (2018) Diagnostic accuracy of CT for prediction of bladder cancer treatment response with and without computerized decision support. Acad Radiol 26:1137–1145PubMedCrossRefGoogle Scholar
  44. 44.
    Takeuchi T et al (2019) Prediction of prostate cancer by deep learning with multilayer artificial neural network. Can Urol Assoc J 13(5):E145–E150PubMedGoogle Scholar
  45. 45.
    Zhang YD et al (2016) An imaging-based approach predicts clinical outcomes in prostate cancer through a novel support vector machine classification. Oncotarget 7(47):78140–78151PubMedPubMedCentralGoogle Scholar
  46. 46.
    Ishioka J et al (2018) Computer-aided diagnosis of prostate cancer on magnetic resonance imaging using a convolutional neural network algorithm. BJU Int 122(3):411–417PubMedCrossRefGoogle Scholar
  47. 47.
    Bonekamp D et al (2018) Radiomic machine learning for characterization of prostate lesions with MRI: comparison to ADC values. Radiology 289(1):128–137PubMedCrossRefGoogle Scholar
  48. 48.
    Arvaniti E et al (2018) Automated Gleason grading of prostate cancer tissue microarrays via deep learning. Sci Rep 8(1):12054PubMedPubMedCentralCrossRefGoogle Scholar
  49. 49.
    Donovan MJ et al (2018) Development and validation of a novel automated Gleason grade and molecular profile that define a highly predictive prostate cancer progression algorithm-based test. Prostate Cancer Prostatic Dis 21(4):594–603PubMedCrossRefGoogle Scholar
  50. 50.
    Auffenberg GB et al (2019) askMUSIC: leveraging a clinical registry to develop a new machine learning model to inform patients of prostate cancer treatments chosen by similar men. Eur Urol 75(6):901–907PubMedCrossRefPubMedCentralGoogle Scholar
  51. 51.
    Abdollahi H et al (2019) Machine learning-based radiomic models to predict intensity-modulated radiation therapy response, Gleason score and stage in prostate cancer. Radiol Med 124(6):555–567PubMedCrossRefPubMedCentralGoogle Scholar
  52. 52.
    Hung AJ et al (2018) Utilizing machine learning and automated performance metrics to evaluate robot-assisted radical prostatectomy performance and predict outcomes. J Endourol 32(5):438–444PubMedCrossRefPubMedCentralGoogle Scholar
  53. 53.
    Hung AJ et al (2019) A deep-learning model using automated performance metrics and clinical features to predict urinary continence recovery after robot-assisted radical prostatectomy. BJU Int 124(3):487–495PubMedCrossRefPubMedCentralGoogle Scholar
  54. 54.
    Wong NC et al (2019) Use of machine learning to predict early biochemical recurrence after robot-assisted prostatectomy. BJU Int 123(1):51–57PubMedCrossRefPubMedCentralGoogle Scholar
  55. 55.
    Chen J et al (2019) Current status of artificial intelligence applications in urology and their potential to influence clinical practice. BJU Int [Epub ahead of print]Google Scholar
  56. 56.
    Goldenberg SL, Nir G, Salcudean SE (2019) A new era: artificial intelligence and machine learning in prostate cancer. Nat Rev Urol 16(7):391–403PubMedCrossRefPubMedCentralGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Urology, Faculty of MedicineUniversity of Freiburg-Medical CentreFreiburgGermany
  2. 2.Department Augmented VisionGerman Research Center for Artificial IntelligenceKaiserslauternGermany

Personalised recommendations